Multi-modular AI Approach to Streamline Autism Diagnosis in Young Children
Halim Abbas, Ford Garberson, Stuart Liu-Mayo, Eric Glover & Dennis P. Wall Scientific Reports volume 10, Article number: 5014 (2020) Cite this article 2 Altmetric | Metricsdetails Abstract Autism has become a pressing healthcare challenge. The instruments used to aid diagnosis are time and labor expensive and require trained clinicians to administer, leading to long wait times for at-risk children. We present a multi-modular, machine learning-based assessment of autism comprising three complementary modules for a unified outcome of diagnostic-grade reliability: A 4-minute, parent-report questionnaire delivered via a mobile app, a list of key behaviors identified from 2-minute, semi-structured home videos of children, and a 2-minute questionnaire presented to the clinician at the time of clinical assessment. We demonstrate the assessment reliability in a blinded, multi-site clinical study on children 18-72 months of age (n = 375) in the United States. It outperforms baseline screeners administered to children by 0.35 (90% CI: 0.26 to 0.43) in AUC and 0.69 (90% CI: 0.58 to 0.81) in specificity when operating at 90% sensitivity. Compared to the baseline screeners evaluated on children less than 48 months of age, our assessment outperforms the most accurate by 0.18 (90% CI: 0.08 to 0.29 at 90%) in AUC and 0.30 (90% CI: 0.11 to 0.50) in specificity when operating at 90% sensitivity. Read full publication here.
ASD symptoms in toddlers and preschoolers: An examination of sex differences
Rosmary Ros‐Demarize, Catherine Bradley, Stephen M. Kanne, Zachary Warren, Andrea Boan, Clara Lajonchere, Justine Park, Laura Arnstein Carpenter Abstract Although considerable work has documented higher prevalence rates of autism spectrum disorder (ASD) in boys, fewer studies have focused on sex differences within samples of young children at‐risk for ASD. This study examined sex differences in ASD symptom domains and ASD screening outcomes among toddlers (18–35 months) and preschoolers (36–72 months) with ASD‐related concerns. Participants included 480 children between 18 and 72 months evaluated by university‐based ASD specialty clinics. Results revealed significant sex differences in severity of social communication (SC) deficits across age groups. Within the toddler group, girls diagnosed with ASD displayed greater SC deficits according to standardized observation and clinician severity ratings. Within the preschool group, girls diagnosed with ASD were rated by parents as having more severe SC deficits, but these differences were not corroborated by standardized observations or clinician ratings. No sex differences emerged for severity of restricted repetitive behaviors (RRBs) for either age group. Across the entire referred sample, boys and girls did not differ in terms of scores on commonly used screening instruments. Importantly, results suggest that two of the most commonly used ASD screeners (i.e., Modified‐Checklist for Autism in Toddlers‐Revised with Follow‐up and Social Communication Questionnaire ) may underidentify RRBs in toddler and preschool‐aged girls as screening scores were only influenced by severity of SC deficits. Greater SC deficits in young girls with ASD along with its impact on screening status suggests greater attention be placed on the under‐identification of ASD in girls as well as current screening measures’ ability to tap into the topography of ASD symptoms across genders. Autism Res 2020, 13: 157–166. © 2019 International Society for Autism Research, Wiley Periodicals, Inc. Read full publication here.
In-Home Speech and Language Screening for Young Children: A Proof-of-Concept Study Using Interactive Mobile Storytime
Du Y1,2, Abbas H2, Taraman S1,2,3, Segar S2, Bischoff N2. Author information 1University of California, Irvine, CA, USA.2Cognoa Inc., Palo Alto, CA, USA.3Children’s Hospital of Orange County, Orange, CA, USA. Abstract Early identification and intervention of speech and language delays in children contribute to better communication and literacy skills for school readiness and are protective against behavioral and mental health problems. Through collaboration between the data science and clinical teams at Cognoa, we designed Storytime, an interactive storytelling experience on a mobile device using a virtual avatar to mediate speech and language screening for children ages 4 to 6 years old. Our proof-of-concept study collects Storytime session footage from 71 pairs of parents and children including 57 typically developing children and 14 children with a current or prior history of communication impairments. Initial findings suggest that participating children verbally engaged with the video avatar without significant differences in performance across age, gender, and experimental location, leading to promising implications for using Storytime as a future tracking tool with automated feature analyses to detect speech and language delays. See full publication here.
When Are We Sure? Predictors of Clinician Certainty in the Diagnosis of Autism Spectrum Disorder.
Journal of Autism and Developmental Disorders. Abstract Differential diagnosis of autism spectrum disorder (ASD) is challenging, and uncertainty regarding a child’s diagnosis may result in under-identification or prolonged diagnostic pathways. The current study examined diagnostic certainty, or how sure clinicians were that their diagnosis was accurate, among 478 toddler and preschool-aged children referred for possible ASD to academic medical specialty clinics. Overall, 60 percent of diagnoses were made with complete certainty. Clinicians were more certain when positively identifying ASD than ruling it out. Children presenting with a moderate (vs high or low) level of observable ASD symptoms were less likely to have a certain diagnosis. Further, clinicians rated less diagnostic certainty for older children, those with public insurance, and those with higher IQ and adaptive behavior abilities. Access full publication here.
Effect of Wearable Digital Intervention for Improving Socialization in Children With Autism Spectrum Disorder: A Randomized Clinical Trial
JAMA Pediatr. 2019;173(5):446-454. doi:10.1001/jamapediatrics.2019.0285 Catalin Voss, MS1; Jessey Schwartz, BA2; Jena Daniels, BS2; et al Abstract Importance Autism behavioral therapy is effective but expensive and difficult to access. While mobile technology–based therapy can alleviate wait-lists and scale for increasing demand, few clinical trials exist to support its use for autism spectrum disorder (ASD) care. Objective To evaluate the efficacy of Superpower Glass, an artificial intelligence–driven wearable behavioral intervention for improving social outcomes of children with ASD. Design, Setting, and Participants A randomized clinical trial in which participants received the Superpower Glass intervention plus standard of care applied behavioral analysis therapy and control participants received only applied behavioral analysis therapy. Assessments were completed at the Stanford University Medical School, and enrolled participants used the Superpower Glass intervention in their homes. Children aged 6 to 12 years with a formal ASD diagnosis who were currently receiving applied behavioral analysis therapy were included. Families were recruited between June 2016 and December 2017. The first participant was enrolled on November 1, 2016, and the last appointment was completed on April 11, 2018. Data analysis was conducted between April and October 2018. Interventions The Superpower Glass intervention, deployed via Google Glass (worn by the child) and a smartphone app, promotes facial engagement and emotion recognition by detecting facial expressions and providing reinforcing social cues. Families were asked to conduct 20-minute sessions at home 4 times per week for 6 weeks. Main Outcomes and Measures Four socialization measures were assessed using an intention-to-treat analysis with a Bonferroni test correction. Results Overall, 71 children (63 boys [89%]; mean [SD] age, 8.38 [2.46] years) diagnosed with ASD were enrolled (40 [56.3%] were randomized to treatment, and 31 (43.7%) were randomized to control). Children receiving the intervention showed significant improvements on the Vineland Adaptive Behaviors Scale socialization subscale compared with treatment as usual controls (mean [SD] treatment impact, 4.58 [1.62]; P = .005). Positive mean treatment effects were also found for the other 3 primary measures but not to a significance threshold of P = .0125. Conclusions and Relevance The observed 4.58-point average gain on the Vineland Adaptive Behaviors Scale socialization subscale is comparable with gains observed with standard of care therapy. To our knowledge, this is the first randomized clinical trial to demonstrate efficacy of a wearable digital intervention to improve social behavior of children with ASD. The intervention reinforces facial engagement and emotion recognition, suggesting either or both could be a mechanism of action driving the observed improvement. This study underscores the potential of digital home therapy to augment the standard of care. Read full publication here.
Mobile detection of autism through machine learning on home video: A development and prospective validation study.
Plos Medicine. Published: November 27, 2018. Abstract Background The standard approaches to diagnosing autism spectrum disorder (ASD) evaluate between 20 and 100 behaviors and take several hours to complete. This has in part contributed to long wait times for a diagnosis and subsequent delays in access to therapy. We hypothesize that the use of machine learning analysis on home video can speed the diagnosis without compromising accuracy. We have analyzed item-level records from 2 standard diagnostic instruments to construct machine learning classifiers optimized for sparsity, interpretability, and accuracy. In the present study, we prospectively test whether the features from these optimized models can be extracted by blinded nonexpert raters from 3-minute home videos of children with and without ASD to arrive at a rapid and accurate machine learning autism classification. Methods and findings We created a mobile web portal for video raters to assess 30 behavioral features (e.g., eye contact, social smile) that are used by 8 independent machine learning models for identifying ASD, each with >94% accuracy in cross-validation testing and subsequent independent validation from previous work. We then collected 116 short home videos of children with autism (mean age = 4 years 10 months, SD = 2 years 3 months) and 46 videos of typically developing children (mean age = 2 years 11 months, SD = 1 year 2 months). Three raters blind to the diagnosis independently measured each of the 30 features from the 8 models, with a median time to completion of 4 minutes. Although several models (consisting of alternating decision trees, support vector machine [SVM], logistic regression (LR), radial kernel, and linear SVM) performed well, a sparse 5-feature LR classifier (LR5) yielded the highest accuracy (area under the curve [AUC]: 92% [95% CI 88%–97%]) across all ages tested. We used a prospectively collected independent validation set of 66 videos (33 ASD and 33 non-ASD) and 3 independent rater measurements to validate the outcome, achieving lower but comparable accuracy (AUC: 89% [95% CI 81%–95%]). Finally, we applied LR to the 162-video-feature matrix to construct an 8-feature model, which achieved 0.93 AUC (95% CI 0.90–0.97) on the held-out test set and 0.86 on the validation set of 66 videos. Validation on children with an existing diagnosis limited the ability to generalize the performance to undiagnosed populations. Conclusions These results support the hypothesis that feature tagging of home videos for machine learning classification of autism can yield accurate outcomes in short time frames, using mobile devices. Further work will be needed to confirm that this approach can accelerate autism diagnosis at scale. PLOS Medicine. Read full publication here.
Exploratory study examining the at-home feasibility of a wearable tool for social-affective learning in children with autism.
npj Digital Medicine. Published: 02 August 2018. Abstract Although standard behavioral interventions for autism spectrum disorder (ASD) are effective therapies for social deficits, they face criticism for being time-intensive and overdependent on specialists. Earlier starting age of therapy is a strong predictor of later success, but waitlists for therapies can be 18 months long. To address these complications, we developed Superpower Glass, a machine-learning-assisted software system that runs on Google Glass and an Android smartphone, designed for use during social interactions. This pilot exploratory study examines our prototype tool’s potential for social-affective learning for children with autism. We sent our tool home with 14 families and assessed changes from intake to conclusion through the Social Responsiveness Scale (SRS-2), a facial affect recognition task (EGG), and qualitative parent reports. A repeated-measures one-way ANOVA demonstrated a decrease in SRS-2 total scores by an average 7.14 points (F(1,13) = 33.20, p = <.001, higher scores indicate higher ASD severity). EGG scores also increased by an average 9.55 correct responses (F(1,10) = 11.89, p = <.01). Parents reported increased eye contact and greater social acuity. This feasibility study supports using mobile technologies for potential therapeutic purposes. Npj Digital Medicine. Access publication here.
Screening in toddlers and preschoolers at risk for autism spectrum disorder: Evaluating a novel mobile‐health screening tool.
Wiley Online Library. Abstract There are many available tools with varying levels of accuracy designed to screen for Autism Spectrum Disorder (ASD) in young children, both in the general population and specifically among those referred for developmental concerns. With burgeoning waitlists for comprehensive diagnostic ASD assessments, finding accurate methods and tools for advancing diagnostic triage becomes increasingly important. The current study compares the efficacy of four oft used paper and pencil measures, the Modified Checklist for Autism in Toddlers Revised with Follow‐up, the Social Responsiveness Scale, Second Edition, and the Social Communication Questionnaire, and the Child Behavior Checklist to a novel mobile‐health screening tool developed by Cognoa, Inc. (Cognoa) in a group of children 18–72 months of age. The Cognoa tool may have potential benefits as it integrates a series of parent‐report questions with remote clinical ratings of brief video segments uploaded via parent’s smartphones to calculate level of ASD risk. Participants were referred to one of three tertiary care diagnostic centers for ASD‐related concerns (n = 230) and received a best estimate ASD diagnosis. Analysis and comparison of psychometric properties indicated potential advantages for Cognoa within this clinical sample across age ranges not often covered by another single measure/tool. Autism Res 2018, 11: 1038–1049. © 2018 International Society for Autism Research, Wiley Periodicals, Inc. Access full publication here.
Machine learning approach for early detection of autism by combining questionnaire and home video screening.
JAMIA. Abstract Background Existing screening tools for early detection of autism are expensive, cumbersome, time- intensive, and sometimes fall short in predictive value. In this work, we sought to apply Machine Learning (ML) to gold standard clinical data obtained across thousands of children at-risk for autism spectrum disorder to create a low-cost, quick, and easy to apply autism screening tool. Methods Two algorithms are trained to identify autism, one based on short, structured parent-reported questionnaires and the other on tagging key behaviors from short, semi-structured home videos of children. A combination algorithm is then used to combine the results into a single assessment of higher accuracy. To overcome the scarcity, sparsity, and imbalance of training data, we apply novel feature selection, feature engineering, and feature encoding techniques. We allow for inconclusive determination where appropriate in order to boost screening accuracy when conclusive. The performance is then validated in a controlled clinical study. Results A multi-center clinical study of n = 162 children is performed to ascertain the performance of these algorithms and their combination. We demonstrate a significant accuracy improvement over standard screening tools in measurements of AUC, sensitivity, and specificity. Conclusion These findings suggest that a mobile, machine learning process is a reliable method for detection of autism outside of clinical settings. A variety of confounding factors in the clinical analysis are discussed along with the solutions engineered into the algorithms. Final results are statistically limited and will benefit from future clinical studies to extend the sample size. Access full publication here.
Sparsifying machine learning models identify stable subsets of predictive features for behavioral detection of autism.
Molecular Autism. Abstract Background Autism spectrum disorder (ASD) diagnosis can be delayed due in part to the time required for administration of standard exams, such as the Autism Diagnostic Observation Schedule (ADOS). Shorter and potentially mobilized approaches would help to alleviate bottlenecks in the healthcare system. Previous work using machine learning suggested that a subset of the behaviors measured by ADOS can achieve clinically acceptable levels of accuracy. Here we expand on this initial work to build sparse models that have higher potential to generalize to the clinical population. Methods We assembled a collection of score sheets for two ADOS modules, one for children with phrased speech (Module 2; 1319 ASD cases, 70 controls) and the other for children with verbal fluency (Module 3; 2870 ASD cases, 273 controls). We used sparsity/parsimony enforcing regularization techniques in a nested cross validation grid search to select features for 17 unique supervised learning models, encoding missing values as additional indicator features. We augmented our feature sets with gender and age to train minimal and interpretable classifiers capable of robust detection of ASD from non-ASD. Results By applying 17 unique supervised learning methods across 5 classification families tuned for sparse use of features and to be within 1 standard error of the optimal model, we find reduced sets of 10 and 5 features used in a majority of models. We tested the performance of the most interpretable of these sparse models, including Logistic Regression with L2 regularization or Linear SVM with L1 regularization. We obtained an area under the ROC curve of 0.95 for ADOS Module 3 and 0.93 for ADOS Module 2 with less than or equal to 10 features. Conclusions The resulting models provide improved stability over previous machine learning efforts to minimize the time complexity of autism detection due to regularization and a small parameter space. These robustness techniques yield classifiers that are sparse, interpretable and that have potential to generalize to alternative modes of autism screening, diagnosis and monitoring, possibly including analysis of short home videos. Read full publication here.
