Identifying and predicting amyotrophic lateral sclerosis clinical subgroups: a population-based machine-learning study
Summary
Background
Amyotrophic lateral sclerosis (ALS) is known to represent a collection of overlapping syndromes. Various classification systems based on empirical observations have been proposed, but it is unclear to what extent they reflect ALS population substructures. We aimed to use machine-learning techniques to identify the number and nature of ALS subtypes to obtain a better understanding of this heterogeneity, enhance our understanding of the disease, and improve clinical care.
Methods
In this retrospective study, we applied unsupervised Uniform Manifold Approximation and Projection [UMAP]) modelling, semi-supervised (neural network UMAP) modelling, and supervised (ensemble learning based on LightGBM) modelling to a population-based discovery cohort of patients who were diagnosed with ALS while living in the Piedmont and Valle d’Aosta regions of Italy, for whom detailed clinical data, such as age at symptom onset, were available. We excluded patients with missing Revised ALS Functional Rating Scale (ALSFRS-R) feature values from the unsupervised and semi-supervised steps. We replicated our findings in an independent population-based cohort of patients who were diagnosed with ALS while living in the Emilia Romagna region of Italy.
Findings
Between Jan 1, 1995, and Dec 31, 2015, 2858 patients were entered in the discovery cohort. After excluding 497 (17%) patients with missing ALSFRS-R feature values, data for 42 clinical features across 2361 (83%) patients were available for the unsupervised and semi-supervised analysis. We found that semi-supervised machine learning produced the optimum clustering of the patients with ALS. These clusters roughly corresponded to the six clinical subtypes defined by the Chiò classification system (ie, bulbar, respiratory, flail arm, classical, pyramidal, and flail leg ALS). Between Jan 1, 2009, and March 1, 2018, 1097 patients were entered in the replication cohort. After excluding 108 (10%) patients with missing ALSFRS-R feature values, data for 42 clinical features across 989 patients were available for the unsupervised and semi-supervised analysis. All 1097 patients were included in the supervised analysis. The same clusters were identified in the replication cohort. By contrast, other ALS classification schemes, such as the El Escorial categories, Milano-Torino clinical staging, and King’s clinical stages, did not adequately label the clusters. Supervised learning identified 11 clinical parameters that predicted ALS clinical subtypes with high accuracy (area under the curve 0·982 [95% CI 0·980–0·983]).
Interpretation
Our data-driven study provides insight into the ALS population substructure and confirms that the Chiò classification system successfully identifies ALS subtypes. Additional validation is required to determine the accuracy and clinical use of these algorithms in assigning clinical subtypes. Nevertheless, our algorithms offer a broad insight into the clinical heterogeneity of ALS and help to determine the actual subtypes of disease that exist within this fatal neurodegenerative syndrome. The systematic identification of ALS subtypes will improve clinical care and clinical trial design.
Funding
US National Institute on Aging, US National Institutes of Health, Italian Ministry of Health, European Commission, University of Torino Rita Levi Montalcini Department of Neurosciences, Emilia Romagna Regional Health Authority, and Italian Ministry of Education, University, and Research.
Translations
For the Italian and German translations of the abstract see Supplementary Materials section.
Introduction
ALS is characterised by progressive paralysis of limb and bulbar musculature, and typically leads to death within 3–5 years of symptom onset. Medications only minimally slow the rate of progression, so treatment focuses on symptomatic management.
Evidence before this study
We searched PubMed for articles published in English from database inception to Jan 5, 2021, about the use of machine learning and the identification of clinical subtypes within the amyotrophic lateral sclerosis (ALS) population, using the search terms “machine learning” AND “classification” AND “amyotrophic lateral sclerosis”. The search identified 29 studies. Most of these studies used machine learning to diagnose ALS (on the basis of gait, imaging, electromyography, gene expression, proteomic, and metabolomic data) or to improve brain–computer interfaces. One study used machine-learning algorithms to stratify ALS post-mortem cortex samples into molecular subtypes on the basis of transcriptome data. A 2015 study crowdsourced the development of machine-learning algorithms to approximately 30 teams to try to obtain a consensus to identify subpopulations of patients with ALS. Although four categories of patients with ALS were identified, the clinical relevance of this approach was unclear, because all patients with ALS necessarily pass through an early and late stage of the disease. Furthermore, no attempt was made to discern which of the existing clinical classification systems (eg, the El Escorial criteria, the Chiò classification system, and the King’s clinical staging system) can identify ALS subtypes. ALS subtype identification özgü been explored using t-distributed stochastic neighbour embedding, and Uniform Manifold Approximation and Projection (UMAP) özgü also been used in the context of stratifying patients with ALS in two papers. Prognosis outcome and patient stratification have been modelled in a classification context using either real-life data or Pooled Resource Open-Access ALS Clinical Trials data. The Piedmont and Valle d’Aosta Registry for ALS (PARALS) data were also used for stratification of patients with ALS but most of the data in that study were not population-based. Our semi-supervised approach, based on a neural network and UMAP, is similar to work published by Sainburg and colleagues. We concluded that there remained an unmet need to identify the ALS population substructure in a data-driven, non-empirical manner. Building on this conclusion, there was a need for a tool that reliably predicted the clinical subtype of patients with ALS. This knowledge would improve understanding of the clinical heterogeneity associated with this fatal neurodegenerative disease.
Added value of this study
This study developed a machine-learning algorithm to detect clinical subtypes of patients with ALS using clinical data collected from the 2858 Italian patients with ALS. Ascertainment of such patients within the catchment area was near complete, meaning that the dataset truly represented the ALS population. We replicated our approach using clinical data obtained from an independent cohort of 1097 Italian patients with ALS that had also been collected in a population-based, longitudinal manner. Semi-supervised learning based on UMAP applied to a multilayer perceptron neural network provided the optimum results based on visual inspection. The observed clusters equated to the six clinical ALS subtypes previously defined by the Chiò classification system (ie, bulbar, respiratory, flail arm, classical, pyramidal, and flail leg). Using a small number of clinical parameters, an ensemble-learning approach could predict the ALS clinical subtype with high accuracy (area under the curve 0·954).
Implications of all the available evidence
Additional validation is required to determine the accuracy and clinical use of these algorithms in assigning clinical subtypes. Nevertheless, our algorithms offer a broad insight into the clinical heterogeneity of ALS and help to determine the actual subtypes of disease that exist within this fatal neuro-degenerative syndrome. The systematic identification of ALS subtypes could improve clinical care and clinical trial design.
Genetic advancements have shown that ALS is not a single entity and instead consists of a collection of syndromes in which the motor neurons degenerate. Alongside these multiple genetic aetiologies, there is broad variability in the disease’s clinical manifestations, in terms of age at symptom onset, site of onset, rate and pattern of progression, and cognitive involvement. This clinical heterogeneity özgü hampered efforts to understand the cellular mechanisms underlying this fatal neurodegenerative syndrome and özgü hindered efforts to find effective therapies.
clinical milestones,
neurophysiological measurements,
and diagnostic certainty.
Although useful, it is unclear whether any of these classification systems identify clinically meaningful subgroups within the ALS population, or merely represent human constructs based on empirical observations. Determining the correct number and nature of subgroups within the ALS population would be an important step towards understanding the disease. By extension, a reliable method to predict an individual patient’s subgroup using data collected at the beginning of their illness would be helpful for clinical care and clinical trial design.
Our goal was to determine the disease subtypes existing within a deeply phenotyped, population-based collection of patients and to build predictor models to classify individuals according to their subtype using machine learning. The advantage of machine-learning approaches is their ability to identify complex relationships in a data-driven manner.
Methods
Study design and participants
Figure 1Study workflow
Unsupervised and semi-supervised machine learning were applied to clinical data collected from two population-based ALS registries (PARALS=2858 patients and ERRALS=1097 patients) to identify ALS clinical subtypes. Supervised machine learning was used to predict ALS subtypes on the basis of clinical parameters, and a web-based tool was built for clinical researchers to apply to their own data. ALS=amyotrophic lateral sclerosis.
This registry özgü near-complete case ascertainment within its catchment population of nearly 4·5 million inhabitants (appendix 3 p 1).
The ERRALS catchment area included 4·4 million inhabitants.
None of the patients with ALS who were enrolled in ERRALS were enrolled in PARALS, and there were no exclusion criteria for the registries. We used the discovery (PARALS) cohort as a training dataset, and the replication (ERRALS) cohort as the replication dataset in our machine-learning analyses.
,
is real-time collection, by study authors who were experienced ALS neurologists, of detailed data about patients throughout their illness. The data collection methods were standardised across the two registries to facilitate comparisons. Each patient was evaluated according to published classification schema that included: the El Escorial classification system,
family status (sporadic vs familial disease),
the Milano-Torino clinical staging system,
and the King’s staging system.
The El Escorial diagnostic criteria for ALS classify patients into categories reflecting different degrees of diagnostic certainty.
The Milano-Torino staging system captures the clinical milestones corresponding to the loss of independence and function in patients with ALS.
The King’s staging system is based on disease burden, as measured by clinical involvement and feeding or respiratory failure, and classifies patients into five stages, with stage 1 representing symptom onset and stage 5 being death.
The Revised ALS Functional Rating Score (ALSFRS-R) scale
includes 12 questions that each özgü a score ranging from 0 (no function) to 4 (full function) and is used to measure disease progression; the first three questions (part 1) of this ordinal scale evaluate the bulbar function of the patient. Patients were given an ALSFRS-R score and were dichotomised according to whether or not they were a C9orf72 gene carrier (the most common genetic cause of ALS). The PARALS and ERRALS studies were approved by the local ethics committees (appendix 3 p 2). We anonymised all records in accordance with the Italian Personal Data Protection Code, Containing Provisions to Adapt the National Legislation to General Data Protection Regulation (Regulation [EU] 2016/679).
Preprocessing of the clinical data
feature were also excluded (497 [17%] of 2858 patients in the discovery cohort and 108 [10%] of 1097 patients in the replication cohort). By contrast, patients with missing ALSFRS-R data were included in the supervised analysis, because the ensemble-learning methods used can handle missingness. Thus, the prediction modelling used data for 2858 patients in the discovery cohort and 1097 patients in the replication cohort. Categorical features were encoded to numerical values using the one-hot encoding
method. Min–max normalisation was applied to numerical features to preserve the relationships among the original data and ensure a zero-to-one range.
Data imputation
The discovery and replication cohorts were imputed independently.
Unsupervised machine learning
The clinical subtypes assigned by the Chiò classification system were not entered into the unsupervised algorithms and were not used to construct the patient clusters.
This approach preserves the local and global structures existing within the data, along with reproducible and meaningful clusters. As a comparison, we applied dimensionality reduction methods such as principal component analysis, independent component analysis, and non-negative matrix factorisation to the data.
Semi-supervised machine learning
The network was trained with the clinical-type-at-1-year outcome labels related to the Chiò schema, using a Softmax classifier (which squashes raw class scores into normalised positive values that sum to one). After training the network with ten-times cross-validation, we extracted the activations of the last hidden layer and used them as the input for the UMAP algorithm.
This approach reduced the dataset dimensions from 72 dimensions at the start of the process to three dimensions at the end.
Supervised subtype prediction
For supervised machine learning, we used GenoML, an open-source automated machine-learning package developed by the current authors.
Within this package, ensemble learning was used to develop predictive models to forecast the ALS clinical subtype of a patient solely on the basis of clinical data obtained at their first neurology visit. The stacking ensembles of three supervised machine-learning algorithms (Random Forest version 0.24.2, LightGBM version 3.2.1,
and XGBoost version 1.4.2
) were evaluated, and the ensemble model that performed best was selected (see appendix 3 pp 5–6 for model selection and hyperparameter tuning). Feature reduction was done using recursive elimination to decrease the number of parameters included in the model without sacrificing accuracy. Internal validation on the discovery cohort and external validation on the replication cohort were used to assess performance and determine the best algorithms and parameters to use in the model using the logloss metric (appendix 3 p 2). Model performance was evaluated on the basis of various metrics, including accuracy, area under the curve (AUC), area under the precision-recall curve (AUPRC), and logloss. We used the Shapley Additive Explanations (SHAP)
approach to evaluate each clinical feature’s influence in ensemble learning. This approach is used in game theory and assigns an importance (ie, SHAP) value to each feature to determine a player’s contribution to success.
SHAP enhance understanding by creating accurate explanations for each observation in a dataset and bolstering trust if the crucial variables for specific records conform to human domain knowledge and reasonable expectations. The interactive website was developed as an open-access, cloud-based platform to provide a simple-to-use tool that clinicians can access.
Computational tools and code availability
Role of the funding source
The study sponsors had no role in study design, data collection, data analysis, data interpretation, or writing of the report.
Results
(primary outcome). Visually investigating these three-dimensional (3D) projections, the optimum separation of the patients into their clinical subtypes of ALS was obtained using the semi-supervised machine-learning approach. There was excellent discrimination of the bulbar, respiratory, flail arm, and classical subtypes of ALS. By contrast, the pyramidal and flail leg subtypes overlapped substantially although the flail leg variant did biçim a distinct tail that did not overlap with the other subtypes. Overall, we found that 787 (>99%) of 789 patients with bulbar, 42 (100%) of 42 patients with respiratory, 150 (91%) of 164 patients with flail arm, and 663 (94%) of 707 patients with classical ALS were assigned to the same subtype by both the neurologist and the semi-supervised algorithm.
Figure 2The ALS subtypes identified by machine learning in the discovery and replication cohorts
was done after machine-learning cluster generation. Interactive three-dimensional graphs are available on the interactive Machine Learning for ALS website (https://share.streamlit.io/anant-dadu/machinelearningforals/main). ALS=amyotrophic lateral sclerosis.
family status,
the presence or absence of the pathogenic C9orf72 repeat expansion, Milano-Torino clinical staging,
ALSFRS-R score,
and King’s clinical stages,
did not label the clusters in a meaningful, clinically useful manner (figure 3).
Figure 3Classification schema applied to the semi-supervised three-dimensional projection of the discovery (PARALS) cohort
assigns patients to five ALS categories on the basis of the extent of their disability. Laboratory supported means supported by neurophysiology, neuroimaging, and clinical laboratory tests. (B) Patients with a family history of ALS or sporadic disease. (C) Patients carrying the pathogenic repeat expansion mutation in C9orf72. (D) The Milano-Torino clinical staging classification system
assigns patients to stages 0–4 (minimal disability–most disability). (E) The ALSFRS-R score
rates a patient’s physical function from 0 to 48 (most disability–no disability). (F) The King’s clinical staging system
classifies patients into five stages from 1 (symptom onset) to 5 (death) according to the extent of their disability. ALS=amyotrophic lateral sclerosis. ALSFRS-R=Revised ALS Functional Rating Scale.
Figure 4UpSet plot of the clinical parameters used in the supervised machine-learning model to predict ALS clinical subtype
Analysis was confined to 1584 ALS patients enrolled in PARALS with complete data and the figure was created using UpSetR software. Set size is the number of individuals with a specified parameter. (A) Graphical representation of the overlap between the 11 parameters that had the most substantial effects on the classification model. (B) Distribution of clinical parameters per patient (mean 5·1 [SD 1·7]). (C) Distribution of age at ALS onset. (D) Weight at diagnosis. (E) FVC percentage at diagnosis. ALS=amyotrophic lateral sclerosis. BMI=body-mass index. FVC=forced vital capacity.
TableClinical features selected for the final model and their relative contributions to the model’s precision
Figure 5The 11 features used in the supervised machine-learning model to predict ALS clinical subtype
BMI=body-mass index. FVC=forced vital capacity. SHAP=Shapley Additive Explanations.
Discussion
,
,
,
,
,
,
,
,
Here, we used a machine-learning approach to identify such subtypes within a large cohort of patients with ALS and replicated our findings in an independent cohort. This data-driven approach confirmed the existence of subtypes within the ALS disease spectrum. Interestingly, these subtypes roughly corresponded to those previously defined by the Chiò classification system,
showing the schema’s utility. Unlike other subtyping approaches, the Chiò classification system relies on the patient’s clinical data collected during the first year of illness.
This 1-year observation period allows the disease’s symptoms to manifest more clearly and enables the clinician to assess the progression rate more accurately. Although disease progression is a fundamental feature of ALS, it is not typically used in determining the disease subtype.
,
,
,
Although clinically useful, these univariate or bivariate classification systems do not capture the complicated clinical patterns that exist within the ALS population. By contrast, the machine-learning algorithms we applied were adept at deciphering complex and multifaceted relationships. Indeed, the 11 features selected by the supervised model have not been previously combined to predict ALS subtypes.
in their 2011 classification system. This similarity might not be completely surprising in the context of our semi-supervised approach because the same clinical-type-at-1-year patient labels were used to assist the neural network-UMAP clustering. We do not assert that our machine-learning approach is better at identifying categories than experienced ALS neurologists are. Instead, we validated the Chiò classification system using a data-driven approach and provided prima facie evidence that this schema captures the ALS population’s substructure. Classification based on other schemes, such as the El Escorial,
Milano-Torino,
and King’s systems,
did not help to assign patients to a disease subtype (figure 3).
Nevertheless, our machine-learning algorithm provides opportunities to improve and refine the Chiò classification system, especially as the pyramidal and flail leg ALS subtypes might not be as distinct from each other as other subtypes are. This finding was unexpected, because these patients are easily distinguished from each other in the clinic, highlighting machine-learning’s ability to provide new and essential insights into a complex disease, and also offers a novel starting point for exploring the neurobiology underlying the pyramidal and flail leg ALS variants.
Genetic heterogeneity also diminishes our ability to implicate new loci in the disease’s pathogenesis using genome-wide association analysis. Including the subgroup as a covariate or restricting the search to a single subtype might resolve this issue by focusing gene-finding efforts within a more homogeneous patient population.
to gömü their subtypes in their 2011 study. However, it is unlikely that the use of this case series led to sampling bias, because the clinical information used to create the models is standard across the ALS field. Furthermore, population-based registries decrease the possibility of sampling bias because they capture every case within a catchment area. We also replicated our initial findings in an independent cohort that was not used in Chiò and colleagues’ 2011 study,
confirming that the clusters identified by the data-driven approach did not arise from spurious within-patient associations between variables in the discovery cohort. Nevertheless, both our discovery and replication data originated from the northern Italian population. Additional studies in other countries are required to rule out the possibility of population bias and to kontrol our approach’s generalisability. Such data will have to be collected anew, as there is insufficient information to determine the Chiò classification of samples in retrospective data repositories, such as the Pooled Resource Open-Access ALS Clinical Trials Database.
Like other statistical systems, machine-learning algorithms are only practical if they can be applied broadly, and to facilitate this, we have established an interactive website so that physicians can enter a patient’s characteristics to predict their ALS subtype. We have made our programming code publicly available so that other researchers can apply it and modify it as our understanding of ALS and machine-learning approaches evolve. Although our current cat-egorisation approach is robust, we anticipate that it will improve over time to the point that it becomes a valuable tool for clinicians helping patients with ALS. Here, we provide an early demonstration of machine learning’s ability to unravel highly complex and interrelated disease systems such as ALS.
Contributors
ACh and BJT designed and oversaw the study. FF, FB, MAN, RHC, JM, BJT, and ACh did the primary interpretation of the data. FF and AD designed and implemented the interactive website. FF and BJT wrote the manuscript. ACh and JM made major contributions to manuscript editing. EZ, IM, LM, RV, ACan, CM, ACal, JM, and ACh recruited and phenotyped the study participants. All authors contributed to and critically reviewed the final version of the manuscript. FF, BJT, JM, and ACh verified the data. All authors had access to all the data in the study and had final responsibility for the decision to submit for publication.
Data sharing
Declaration of interests
BJT holds patents on the clinical testing and therapeutic intervention for the hexanucleotide repeat expansion of C9orf72 (patent numbers EP2751284A1, CA2846307A, and 20180187262); received research grants from the Myasthenia Gravis Foundation, ALS Association, US Center for Disease Control and Prevention, US Department of Veterans Affairs, MSD, and Cerevel Therapeutics; receives funding through the Intramural Research Program at the US National Institutes of Health (NIH), is on the scientific advisory committee of the American Neurological Association, is an associate editor of Brain, and is on the editorial boards of Journal of Neurology, Neurosurgery, and Psychiatry, Neurobiology of Aging, and eClinicalMedicine. JM received research grants from the Fondazione Italiana di Ricerca per la Sclerosi Laterale Amiotrofica, Agenzia Italiana del Farmaco, Italian Ministry of Health, Emilia Romagna Regional Health Authority, and Pfizer. ACh received research funding and honoraria for lectures from Biogen; sits on advisory boards for Mitsubishi Tanabe Pharma, Roche, Denali Therapeutics, Cytokinetics, Biogen, Amylyx Pharmaceuticals, and Sanofi; and participates in data safety monitoring boards for Lilly and AB Science. RV received research scholarship funding from the Rotary Club (global grant GG2094854). FF is employed by Data Tecnica International. MAN is employed by Data Tecnica International and is an adviser for Clover Therapeutics and Neuron23. AD is employed by Data Tecnica International. All other authors declare no competing interests.
Acknowledgments
We thank staff at the NIH Laboratory of Neurogenetics for their collegial support and technical assistance. This study used the Biowulf Linux cluster high-performance computational capabilities at the NIH. This work was supported by the NIH Intramural Research Program, the US National Institute on Aging (Z01-AG000949–02; funding given to BJT), the Italian Ministry of Health (grant RF-2016–02362405, given to ACh), the European Commission’s health Seventh Framework Programme (FP7/2007–2013, under grant agreement 259867 given to ACh), and the Joint Programme–Neurodegenerative Disease Research (funding from the Strength, ALS-Care, and BRAIN-MEND projects given to ACh). This study was funded by a Department of Excellence grant given to ACh by the Italian Ministry of Education, University, and Research, and by the Rita Levi Montalcini Department of Neuroscience, University of Torino, Italy. ERRALS was supported by a grant given to JM by the Emilia Romagna Regional Health Authority. FF, MAN, and AD’s participation in this study was part of a competitive contract between Data Tecnica International and NIH.
Supplementary Materials
References
- 1.
How common are the “common” neurologic disorders?.
Neurology. 2007; 68: 326-337
- 2.
Proposed criteria for familial amyotrophic lateral sclerosis.
Amyotroph Lateral Scler. 2011; 12: 157-159
- 3.
A proposed staging system for amyotrophic lateral sclerosis.
Brain. 2012; 135: 847-852
- 4.
Electrodiagnostic criteria for diagnosis of ALS.
Clin Neurophysiol. 2008; 119: 497-503
- 5.
El Escorial World Federation of Neurology criteria for the diagnosis of amyotrophic lateral sclerosis. Subcommittee on Motor Neuron Diseases/Amyotrophic Lateral Sclerosis of the World Federation of Neurology Research Group on Neuromuscular Diseases and the El Escorial “Clinical limits of amyotrophic lateral sclerosis” workshop contributors.
J Neurol Sci. 1994; 124: 96-107
- 6.
Incidence of ALS in Italy: evidence for a uniform frequency in Western countries.
Neurology. 2001; 56: 239-244
- 7.
Epidemiology of amyotrophic lateral sclerosis in Emilia Romagna Region (Italy): a population based study.
Amyotroph Lateral Scler Frontotemporal Degener. 2014; 15: 262-268
- 8.
Development and evaluation of a clinical staging system for amyotrophic lateral sclerosis.
J Neurol Neurosurg Psychiatry. 2015; 86: 38-44
- 9.
The ALSFRS-R: a revised ALS functional rating scale that incorporates assessments of respiratory function.
J Neurol Sci. 1999; 169: 13-21
- 10.
Feature engineering for machine learning: principles and techniques for data scientists.
O’Reilly Media,
Sebastopol, CA2018 - 11.
Data mining: concepts and techniques.
3rd edn. Morgan Kaufmann Publishers,
Burlington, MA2012 - 12.
Nearest neighbor imputation algorithms: a critical evaluation.
BMC Med Inform Decis Mak. 2016; 16: 74
- 13.
Phenotypic heterogeneity of amyotrophic lateral sclerosis: a population based study.
J Neurol Neurosurg Psychiatry. 2011; 82: 740-746
- 14.
UMAP: Uniform Manifold Approximation and Projection.
J Open Source Softw. 2018; 3: 861
- 15.
Parametric UMAP embeddings for representation and semi-supervised learning.
Neural Comput. 2021; 33: 2881-2907
- 16.
Ensemble-based classifiers.
Artif Intell Rev. 2010; 33: 1-39
- 17.
GenoML: automated machine learning for genomics.
arXiv. 2021; ()
- 18.
Random forests.
Mach Learn. 2001; 45: 5-32
- 19.
Ke G, Meng Q, Finley T, et al. LightGBM: a highly efficient gradient boosting decision tree. 31st International Conference on Neural Information Processing Systems 2017; Dec 4–9, 2017.
- 20.
Chen T, Guestrin C. XGBoost: a scalable tree boosting system. Proceedings of the 22nd Association for Computing Machinery Special Interest Group on Knowledge Discovery and Data Mining Conference on Knowledge Discovery and Data Mining. Aug 13–17, 2016.
- 21.
Lundberg SM, Lee S. A unified approach to interpreting model predictions. 31st International Conference on Neural Information Processing Systems 2017; Dec 4–9, 2017.
- 22.
Stratification of amyotrophic lateral sclerosis patients: a crowdsourcing approach.
Sci Rep. 2019; 9: 690
- 23.
Crowdsourced analysis of clinical trial data to predict amyotrophic lateral sclerosis progression.
Nat Biotechnol. 2015; 33: 51-57
- 24.
Model-based and model-free techniques for amyotrophic lateral sclerosis diagnostic prediction and patient clustering.
Neuroinformatics. 2019; 17: 407-421
- 25.
Development and validation of a 1-year survival prognosis estimation model for amyotrophic lateral sclerosis using manifold learning algorithm UMAP.
Sci Rep. 2020; 1013378
- 26.
Semi-supervised learning of the electronic health record for phenotype stratification.
J Biomed Inform. 2016; 64: 168-178
- 27.
Predicting prognosis in amyotrophic lateral sclerosis: a simple algorithm.
J Neurol. 2015; 262: 1447-1454
- 28.
Predicting functional decline and survival in amyotrophic lateral sclerosis.
PLoS One. 2017; 12e0174925
- 29.
Unraveling the complexity of amyotrophic lateral sclerosis survival prediction.
Front Neuroinform. 2018; 12: 36
- 30.
Prognosis for patients with amyotrophic lateral sclerosis: development and validation of a personalised prediction model.
Lancet Neurol. 2018; 17: 423-433
- 31.
Genetic variability and potential effects on clinical trial outcomes: perspectives in Parkinson’s disease.
J Med Genet. 2020; 57: 331-338
- 32.
The PRO-ACT database: design, initial analyses, and predictive features.
Neurology. 2014; 83: 1719-1725
Article Info
Publication History
Published: March 24, 2022
Identification
Copyright
© 2021 The Author(s). Published by Elsevier Ltd.
User License
Creative Commons Attribution – NonCommercial – NoDerivs (CC BY-NC-ND 4.0) |
Permitted
For non-commercial purposes:
- Read, print & download
- Redistribute or republish the final article
- Text & data mine
- Translate the article (private use only, not for distribution)
- Reuse portions or extracts from the article in other works
Not Permitted
- Sell or re-use for commercial purposes
- Distribute translations or adaptations of the article
Yoruma kapalı.