Observer agreement for a novel visual rating scale of vermian atrophy in childhood ataxia telangiectasia.

BACKGROUND AND PURPOSE: Progressive cerebellar neurodegeneration occurs in the inherited disorder ataxia telangiectasia (A-T). Trials of disease-modifying therapies for A-T are in development, prompting the need for MRI metrics reflecting cerebellar neurodegeneration. Volumetric brain MRI analysis provides a gold standard, but implementation can be challenging when image acquisition or quality varies, or in low-resource settings. To address this we developed a 10-point scale for rating vermian atrophy (ROVA) in childhood A-T. In this study we report the observer agreement of ROVA and correlations with vermis and cerebellar volumes.

MATERIALS AND METHODS: Twelve attending-level paediatric neuroradiologists from 3 countries applied ROVA to 60 randomly-ordered midline sagittal T1-weighted images from children and young people (A-T, n=42; without A-T, n=18), blinded to age and disease status, and rated image quality using a 5-point ordinal scale. After 4-weeks observers blindly rated a second set of 30 images (A-T, n=20; without A-T, n=10) randomly selected and re-ordered from the initial set. Inter-observer agreement was calculated using Krippendorf's alpha. Intra-observer agreement was calculated using weighted kappa. Correlation between mode ROVA scores and fractional vermis and cerebellar volumes was calculated. Analyses were repeated after splitting the data into acceptable (n=45) and non-acceptable (n=15) image quality.

RESULTS: Inter-observer agreement (alpha = 0.86, 95%CI, 0.77, 0.91) and intra-observer agreement (median weighted kappa = 0.79, range 0.72, 0.89) exceeded the pre-specified values for satisfactory agreement. Intra-observer agreement was not related to years of specialist practice (Spearman's rho -0.11, p = .74). Inter-and intra-observer agreement increased after removing cases with non-acceptable image quality (alpha = 0.89, 95%CI 0.81, 0.92; median weighted kappa = 0.80, range 0.71, 0.90, respectively). Significant correlation was found between mode ROVA score and both fractional vermis volume (Spearman's rho = 0.82, 95%CI 0.71, 0.89, p<0.001) and fractional cerebellar volume (Spearman's rho = 0.80, 95%CI 0.69, 0.88, p<0.001).

CONCLUSIONS: The ROVA scale demonstrates high observer agreement and significant correlation with fractional vermis and cerebellar volumes when applied to a dataset of mixed image quality by paediatric neuroradiologists. This novel scale may be useful to provide a semi-quantitative measure of cerebellar degeneration in treatment trials for childhood A-T.