Author(s)
Adrian Williamson, MD
Samira Ibrahim, DO
Wei Fang, PhD
Matthew Kabalan, MD
Steven W. Coutras, MD, FRACS
Michele M. Carr, MD, DDS, PhD, FRCSC
Affiliation(s)
West Virginia University; West Virginia Clinical and Translational Science Institute (WVCTSI); SUNY University at Buffalo, Dept of Otolaryngology;
Abstract:
<u>Introduction:</u> The surgical management of Obstructive Sleep Apnea (OSA) can be guided by Drug Induced Sleep Endoscopy (DISE), but there is no universally accepted scoring system for DISE. The purpose of this study is to evaluate the intra-rater test-retest and the inter-rater reliability of a novel scoring system for DISE in children.
<br><u>Methods:</u> 30 pediatric DISE videos were reviewed and scored twice by 5 raters with a range of experience levels. The videos were scored using the VOTE (Velum, Oropharynx, Tongue base, Epiglottis) system and a novel scoring system. The raters were also asked to recommend surgical intervention based on the DISE exam alone. Intra-rater test-retest analysis of the responses was conducted using weighted kappa statistic and percentage agreement. Inter-rater reliability analysis of responses was evaluated using Krippendorff’s alpha reliability coefficient (KA). Using a proportional odds model, a comparison of the weighted kappa statistic for the VOTE and the novel scoring systems was conducted.
<br><u>Results:</u> For the novel pediatric DISE scoring system, the test-retest weighted kappa coefficient was 0.62 and 0.87 and the percentage agreement was 64% and 82% at the first and third quartile respectively. KA was 0.55 and 0.71 at the first and third quartile. For the VOTE system, the test-retest weighted kappa coefficient was 0.50 and 1.00 and the percentage agreement was 75% and 100% at the first and third quartile respectively. KA was .36 and .77 at the first and third quartile. There was no significant difference regarding the weighted kappa statistics between the novel DISE scoring system and the VOTE system (p = 0.15). Responses to the surgical intervention survey analysis yielded a test-retest weighted kappa coefficient of 0.40 and 1.00 and a percentage agreement 71% to 100% at the first and third quartile respectively. KA was 0.40 and 0.61 at the first and third quartile respectively.
<br><u>Conclusion:</u> Our novel scoring system demonstrated test-retest and inter-rater reliability similar to the VOTE system. There was a high degree of agreement for surgical decisions made based on DISE.