Chronic kidney disease (CKD) is a major public health problem worldwide. Although estimates of CKD prevalence vary widely within and between countries, it is undeniable that CKD patients face the risk of serious consequences such as end-stage renal disease (ESRD) or cardiovascular disease, which is a growing global health burden.[1,2] While significant progress has been made in prevention and treatment in recent decades, more efforts are needed to reverse the situation.
Machine learning (ML) is a kind of artificial intelligence (AI). The core of it is algorithmic methods, which enable the machine to solve problems without specific computer programming. The wide application of ML in the medical field helps to promote medical innovation, reduce medical costs, and improve medical quality. However, related research on solving clinical problems through ML in nephrology still needs to increase. Understanding the purpose and method of ML application and the current situation of its application in nephrology is a prerequisite for correctly addressing and overcoming these challenges.
Overview of ML
ML helps computers possess the same ability to learn, identify, and judge as human beings. It is mainly about the development and deployment of algorithms and usually uses statistical tools to determine behavior. ML technology can be broadly divided into supervised learning, unsupervised learning, and reinforcement learning according to different modeling needs.
Supervised learning, such as logistic regression (LR), naive Bayesian classification, support vector machine (SVM), and random forest (RF), is the most common form of ML used in medical research. Each instance of supervised learning consists of an input object (usually a vector) and the desired output value (also known as a supervised signal). The application of supervised learning is very extensive. However, there are still limits in the application of this method in complex optimal control problems.
If the learning sample given does not contain category information, it is unsupervised learning. Similar to k-means clustering, unsupervised learning optimally divides samples into different categories based on the characteristics of training data without corresponding labels. In addition, unsupervised learning can capture intrinsic morphometric patterns in histology sections, which may play a key role in pathological diagnosis. In the future, unsupervised learning is likely to narrow the gap between human intelligence and AI.
Reinforcement learning describes and solves the problem of agents maximizing returns or achieving specific goals through rewarding learning strategies in interaction with the environment. The Markov decision process, a common model of reinforcement learning, can capture the uncertainty associated with treatment outcomes and the randomness of the underlying processes, particularly well-suited for modeling sequential decision-making issues such as optimal dosing strategies for chronic diseases, to find the best drug dosage sequence.
What is deep learning?
In machine learning, deep learning (DL) stands out. DL is a specific type of ML method that is very suitable for processing large quantities of data as input without the need for a clear feature selection step. It can be trained to find complex patterns in big data with a high degree of precision.
Deep neural networks
Deep neural networks (DNNs) are the basis of DL. The signatures of high-dimensional DNNs capture thousands of subtle properties of histologic images that can be used to predict other endpoints, including disease status and physiological outcomes. Recent work has shown that DNNs have achieved human expert-level performance in natural and biomedical image classification tasks.[7,8] The ability to generate assumptions, the adaptability to heterogeneous dataset analysis, and the rapid-diffusion open-source DL programs allow DL to play an important role in promoting medical development.
Convolutional neural networks
With the development of image processing, convolutional neural networks (CNNs) have gained traction in histology dataset classification. Construction of CNNs imitates the visual perception mechanism of organisms and can conduct supervised and unsupervised learning. These DL networks used in the diagnosis and treatment of a variety of diseases can also be retrained through population-specific datasets.[6,11–15] CNNs have been proven to exceed human performance in visual target recognition.
Transfer learning can be used to analyze smaller datasets without significantly affecting the performance of the model. The merit is that instead of designing and training a new network, it is based on the trained network model. On the basis of this model, the parameters and knowledge are transferred. It takes only a small number of computing resources and training time to finish new tasks. When there are similar tasks to be completed, the relevant model of pre-training can be used to carry out transfer learning.
ML in nephrology
Clinical big data are a valuable asset. With the continuous expansion of digital data in all aspects of health care and the development of AI technology, ML can be combined with clinical big data for disease diagnosis, prognosis, and other risk prediction. In the past few years, many different ML methods have been proposed and applied in medicine and computational biology. In addition to the diagnosis and treatment of clinical diseases by electronic health records (EHRs), they have also performed well in the field of medical image analysis and genomics.[7,17–19]
Unlike the relatively mature application of big data and ML in the field of cardiovascular disease, the lack of evidence and the limitation of the scope of research in kidney disease have led to the fact that nephrology has not yet benefited significantly from the clinical application of big data and ML. In recent years, the use of precision medicine has made great progress in the field of nephrology. The Nephrotic Syndrome Study Network (NEPTUNE) is implementing the concept of precision medicine to develop new disease definitions through a comprehensive, multilevel analysis of disease processes in observational cohort studies. The combination of ML and big data will be an important factor in promoting precision medicine. Although it is only in its infancy, there are bright prospects and a future in the study of renal pathology and the risk prediction of kidney disease.
The application of ML in biological image analysis is powerful and rapid and has been proven to be a reliable method for the analysis of malignant tumors such as cancer.[22–27] In nephrology, biological image analysis by ML can be used in the diagnosis of renal pathology, which is the gold standard for the diagnosis of renal diseases. This diagnostic process, in turn, affects a series of treatment options and prognoses. Prevailing methods for glomerular assessment remain manual, labor-intensive, and non-standardized. Recently, to save manpower and time, and especially to improve the accuracy of diagnosis without bias, efforts have been made to automate the quantification of glomerular injury.
Segmentation of glomeruli and tubules
Segmentation is the basis of automatic pathological diagnosis that accurately identifies the structures of glomeruli and tubules from renal pathological images. In the early days, an unsupervised semi-automated workflow was depicted by Sarder et al and Ginley et al for localization and segmentation of glomerular features. Although the localization accuracy reached 87%, the degree of glomerular injury was not reported, and the sample size was limited (15 fields with 148 glomeruli). Marée et al used a supervised learning method for the identification of glomeruli and established a model with 95% precision and 81% recall.
With the continuous development of CNNs in the field of image processing, Pedraza et al first proposed the use of CNNs for transfer learning to identify and segment the glomerulus from the background. However, due to the lack of consideration of the location of glomeruli in large tissue areas where there are multiple glomeruli, their use in automated workflows was limited. To speed up and improve this process, Bukowy et al proposed a new and robust method for glomerular localization by CNNs. Using color deconvoluted and normalized grayscale images, a glomerular localizer composed of two serially arranged ML classifiers for automatic identification of glomeruli within whole kidney sections was developed. The average precisions of used rat and human kidneys were 96.94% and 80.20%, and the average recall rates were 96.79% and 81.67%, respectively. The model laid the foundation for the automatic scoring of glomerular injury. Using this method, Kannan et al demonstrated the ability of DL to assess complex tissue structures from digital human kidney biopsies. This CNN model can accurately discriminate non-glomerular images from normal or partially sclerosed and globally sclerosed images (sensitivity, 0.558; specificity, 0.999).
Furthermore, it was proven that the usage of the ML algorithm can not only clearly segment the glomeruli from the kidney image but also distinguish the renal tubule. Sheehan et al used SVM classification to extract the features of renal tubules in mice (true positive rate, 92%; false-positive rate, 10%). Using 200 cores on the Vermont Advanced Compute Cluster, the glomerular segmentation pipeline can segment the full-sized mouse kidney section in approximately 40 min, allowing analysis of more glomeruli than manual completion. Additionally, the usage of SVM identifies changes in glomeruli and tubules in knock-out and wild-type genotypic mice, and scores pathological changes of mesangial matrix expansion and the degree of vacuolation of tubules. The ability of ML to identify and quantify the specific features of renal tissue structures, specifically, to facilitate segmentation of renal tissue and accurate pathological scoring and to help discover new histopathological features, has been demonstrated. Table 1 summarizes the specific research contents and ML methods of several recent articles mentioned above.
Combination with clinical indicators
Kolachalama et al first applied the analytic technique based on ML and renal pathological images in clinics. Patient-specifific trichromatic images are trained and modeled by CNNs as input, and clinical indicators, including chronic kidney disease (CKD) stage, serum creatinine and nephrotic-range proteinuria at the time of biopsy, and 1-, 3-, and 5-year renal survival, are set as output in the study. Six CNN models were trained. Comparing the performance of CNN with that of an experienced renal pathologist to predict the CKD stage, the kappa-values were 0.519 and 0.051, respectively. The area under curve (AUC) was 0.912 (CNN) versus 0.840 (pathologist-estimated fifibrosis score [PEFS]) for creatinine models and 0.867 (CNN) versus 0.702 (PEFS) For proteinuria models. AUC values of the CNN models for 1-, 3-, and 5-year renal survival were 0.878, 0.875, and 0.904, respectively, whereas the AUC values for PEFS model were 0.811, 0.800, and 0.786, respectively. All the results demonstrate the effectiveness and clinical feasibility of using DN structures in renal pathology. These predictions may provide added value for biopsy results and, along with other clinical assessments, provide more accurate care management and follow-up strategies for biopsy patients. However, no prospective validation has been performed yet.
With the introduction of the Economic and Clinical Health Information Technology act of 2009, the use of EHRs has increased dramatically. EHR data contain important information about the evolution of diseases. Processed by ML and widely used for modeling, EHRs can potentially help to diagnose and predict the progress of renal diseases and renal function damage. According to the data in EHRs, it is possible to achieve a more comprehensive understanding of the health status of patients and accurate prediction of the risk of obtaining specific diseases. In recent years, in the era of big data and technological innovation, using various ML algorithms to analyze standardized health data collected routinely and carrying out large-scale observational research is crucial and popular. The prediction model established by EHR data is expected to promote individualized diagnosis and treatment of renal diseases and improve the quality of medical treatment.
Diagnosis and prognosis of kidney diseases
It has long been found that ML is more likely to predict eGFR accurately.Table 2 lists the specific research contents and results of the following articles. Early studies found that taking advantage of AI is more likely to accurately predict the progress of various CKD. Tangri et al, using routine laboratory data such as eGFR and albuminuria in EHRs, established a highly accurate proportional hazards model (COX) for CKD patients with renal failure and implemented an external validation (C-statistic, 0.921).
In addition, it has been proven that the use of temporal information in the model improves the predictive ability of renal deterioration. The inclusion of temporal medical history information better predicts the loss of renal function and identifies high-risk patients with short-term renal failure.
With the continuous advancement of AI technology, various more complex and advanced ML algorithms have been explored and applied in modeling. Norouzi et al used 10-year clinical records of newly diagnosed CKD patients and the adaptive neurofuzzy inference system to predict the renal failure timeframe of CKD. The model accurately estimates eGFR changes for all sequential periods (normalized mean absolute error < 5%). In addition to using a variety of supervised learning algorithms for modeling, Perott et al used the unsupervised learning method latent Dirichlet allocation to process longitudinal clinical data and establish an accurate predictive model for the progression of CKD phase III to phase IV (C-statistic, 0.849).
Currently, numerous limitations remain. Although high performing CKD risk prediction models have been increasingly established, the actual effects of the models still need further exploration. After modeling, it is necessary to better calibrate and externally validate the results and verify the impacts on the outcome assessment before incorporating them into the guiding principles. The research progress of two kinds of specific CKDs, diabetic kidney diseases (DKDs), and immunoglobulin A nephropathy (IgAN), are discussed as follows. The applications of ML in DKD and IgAN are listed in Table 3.
In the field of diabetes research, many studies have used various types of ML methods, such as artificial neural networks (ANNs) or decision trees (DTs), to predict the incidence of diabetes and determine the risk factors.[44–47] Diabetic complications have a considerable impact on quality of life and mortality, so many scientists are also working on the application of ML in the diagnosis and treatment of diabetic complications. Based on CNNs, Gulshan et al created a data-driven model for the detection of diabetic retinopathy using more than 100,000 clinical images. The high sensitivity and specificity of the model for the diagnosis of diabetic retinopathy make it a milestone of AI application in medicine.
DKD is also one of the most serious microvascular complications of diabetes and the most important cause of ESRD. The early non-invasive diagnosis of diabetic nephropathy (DN) is one of the current research hotspots. However, AI technology has not been widely used in this field.
Prediction of the onset of DKD
Scientists have modeled the prediction of the occurrence of DKD. Cho et al collected the medical data of 4321 diabetic patients and followed up for 10 years to develop a new visualization system by SVM classification. The model predicts the onset of DKD 2 to 3 months before the actual diagnosis with high prediction performance (AUC, 0.969). The research was the first to use data mining technology and an advanced ML algorithm to predict DKD. Although the result was not accurate or complete enough, it provided scientists with a new idea for achieving effective and appropriate treatment strategies in the early stage of the disease. Rodriguez-Romero et al used the data of 10,251 patients screened from the Action to Control Cardiovascular Risk in Diabetes (ACCORD) trial and six kinds of ML algorithms to model DKD risk prediction. The study found that the RF (sensitivity >0.72, accuracy >0.73) and LR (sensitivity >0.76, accuracy >0.8) models showed the best predictive performance in both the training and testing databases, and among the included features, the decline in eGFR was the most important factor reflecting the development of DKD. However, a lack of external validation may cause the model to be only applicable to patients in the ACCORD trial. Recently, Ravizza et al used a LR-based Roche/IBM algorithm and real-world data (RWD) model for the onset of CKD in diabetic patients (AUC, 79.37%) and compared the performance with the results of multiple published large randomized controlled trials (RCTs). A total of 522,416 patients with diabetes from the IBM Explorys Database were screened for modeling and cross-validation, and 82,912 diabetic patients’ data from the Indiana Network for Patient Care database were screened for external validation. This demonstrates that the Roche/IBM algorithm based on RWD is consistent or even more accurate with research using RCT data. This evidence may change the current medical decision-making pattern that is mainly based on clinical RCTs.
In addition to using only EHR data, genotype was also a crucial feature of patients for modeling. Leung et al selected genotypes based on published studies and conducted structured clinical assessments of the included patients and incorporated the selected genotypes and phenotypic features into the ML program for modeling. The results reveal that the accuracy of the model based on a genotype-phenotype combination is higher than that of the model with genotype or phenotype only. Among the algorithms used, the SVM and RF algorithms show better performance (accuracy >0.9). With the advancement of technology and further research, the prediction of the onset of DKD in advance and even its application in clinical practice is no longer infeasible.
Identification of DN and non-diabetic renal disease
The Kidney Disease Outcomes Quality Initiative published in 2007, proposed the diagnostic criteria of DN and non-diabetic kidney disease (NDRD). However, further evaluation found that the diagnostic efficacy of the guidelines for Chinese patients was not satisfactory enough. Therefore, 929 patients with type 2 diabetes who underwent renal biopsy from 2005 to 2017 were screened by Liu et al [unpublished]. Based on the clinical test and renal pathological diagnosis results, differential diagnosis models of DN and NDRD based on SVM and RF were established. The sensitivity of the two models was 84.23% and 84.80%, respectively, and the specificity was 89.85% and 90.58%, respectively. Compared with other studies using the binary LR method,[54,55] the efficiency of the two models was higher.
Currently, only single-center retrospective studies in the field have been carried out by the ML method. A multicenter prospective study with large mass data needs to be conducted and verified externally. Although it is only the beginning of relevant studies, a new idea for the diagnosis of DN by AI technology without renal biopsy is provided.
IgAN is a major cause of renal failure. Early identification of patients with renal failure is useful for the prognosis and treatment of the disease. Multiple types of ML algorithms can effectively help predict the progression of ESRD in patients with IgAN.
Prediction of ESRD progression based on ANN
As early as 1998, a predictive model (sensitivity, 86.4%; specificity, 87.5%) was established based on ANN for the progression of ESRD in patients with IgAN. This proves that modeling by ML algorithms more accurately predicts the progression of renal function damage in patients with IgAN than experienced renal specialists. However, the model is based on the information of only 54 patients and without validation. With the development of data preservation, processing and ML technology, a well-trained ANN model was developed into an online clinical decision support system (CDSS) used for quantitative risk assessment and time prediction of ESRD in IgAN patients in 2016 (www.IgAN.net). The model was trained and tested based on the clinical data of 1040 IgAN patients confirmed by biopsy from different population cohorts. The performance of the model was satisfactory in different races (AUC > 90%). However, due to the inclusion of few Asians, further validation is necessary for its application.
Prediction of ESRD progression based on RF
Recently, Han et al predicted the progression of ESRD in IgAN patients by six ML algorithms (LR, RF, SVM, DT, ANNs, and k-nearest neighbor). Comparing the effectiveness of six models, the RF model can best predict the progress of ESRD (sensitivity, 80.6%; specificity, 95.29%). In the study of Liu et al, the AUC of the RF model reached 97.29% after considering the results of C3 staining according to the contribution of predictors. The algorithm is used in a variety of progressive diseases to help clinicians stage and manage patients.
Prediction of ESRD progression based on gradient boosting machine
In addition to using parallel integration methods such as RF, the serial integration method is also used for models such as gradient boosting machines (GBMs). Chen et al constructed an accurate model for predicting the progression of ESRD in IgAN patients (C-statistic, 0.84) by XGBoost. Additionally, based on stepwise Cox regression, a scoring scale model (SSM) for risk stratification was constructed to identify specific patient groups with the same risk of progress. This XGBoost prediction model and the SSM were incorporated into the Nanjing IgAN Risk Stratification System, which is available online. ML provides a more favorable tool for strengthening the individualized treatment and management of IgAN patients.
However, current studies only focus on the prediction of ESRD in patients with IgAN, and there is no research modeling for the risk prediction of suffering IgAN, which may be the next direction of future efforts.
Acute kidney injury
Regardless of whether it is in high-income or low-income countries, the seven-day mortality rate for acute kidney injury (AKI) is 10% to 12%. Efforts to improve the clinical outcomes of AKI have focused on early diagnosis and customized treatment. Early evaluation of AKI reduces mortality and improves renal prognosis. The model established by ML methods can not only realize early dynamic monitoring based on the real objective data of all patients but also save the time and energy of physicians. With the determination of AKI clinical practice guidelines and the significant growth of the application of EHRs in the field of big data, a large quantity of EHR data and ML algorithms have begun to play an important role in the clinical research of AKI. It is now an important tool for AKI diagnosis and prediction. The establishment of a CDSS based on a self-learning predictive model may be used in in-hospital AKI monitoring in future clinical practice. The application of ML in AKI is listed in Table 4.
Early assessment of AKI
Early assessment of AKI can reduce mortality and improve renal prognosis. GBM is a common method for predicting the onset of AKI at present.[65–68] Lee et al[66,67] modeled the prediction of AKI after liver transplantation and cardiac surgery by several ML algorithms. GBM showed the best performance in both studies, and the AUC of liver transplantation-related research was 90%, while the other was 78%. Additionally, Huang et al established a hazard prediction model for AKI after percutaneous coronary intervention (PCI) based on GBM. The study included a large quantity of data from 947,091 patients who underwent PCI surgery to establish a baseline model. In addition, temporal validation was conducted with data from 970,869 hospitalized patients. The AUC of the GBM model was 78.5% better than that of the baseline LR model (AUC, 75.3%). Therefore, advanced algorithms and massive data have the potential to provide accurate risk estimation.
With the cooperation of the University College London and DeepMind, the era of early prediction of AKI and practical application in clinical diagnosis and treatment by the ML method is coming. Using data from 703,782 patients from multiple centers and stratified by a time window every six hours, a recurrent neural network-based AKI risk prediction model (AUC, 0.921) was established. AKI events were predicted within a 48-h window. However, the area under the precision-recall curve was just 29.7%, which represents a ratio of two false alerts for every true alert.
Prediction of AKI death risk
The prediction of the death risk in patients with AKI was also modeled. Using the medical information mart for intensive care III database, Lin et al developed a mortality prediction model (AUC, 0.866, accuracy, 0.728) for AKI patients based on RF. However, the model slightly overestimates the mortality of patients with a low risk of death and underestimates the mortality of patients with high mortality.
With the further deepening of the research, ML-assisted monitoring may bring useful results to AKI and reduce the resulting morbidity and mortality. This inspiring prospect should be tested in further research.
ML can be widely used in dialysis prescription and monitoring, complication management, death prediction, and so on.[71–73] There is also great potential for application in child dialysis. The application of ML in dialysis is listed in Table 5. There have been several positive research results in dialytic treatments. Scientists organized the first scientific conference on the application status and future development of AI in dialysis at the Hospital Universitari Bellvitge (L’Hospitalet, Barcelona). In the conference, the experience of AI in dialysis was reviewed, and the obstacles, challenges, and future applications in this field were discussed. All the results indicate a wide application and good prospect for AI in renal dialysis. In the future, AI may significantly change clinical practice in terms of hemodialysis.
The adequacy of dialysis is difficult to determine, and its definition is constantly changing. Urea concentration in blood is a key factor in determining the dose for hemodialysis. Direct dialysate quantification (DDQ) is the most direct and accurate method for determining the removal rate of urea nitrogen (UN) in dialysis, but it is difficult to collect all the dialysate used for quantitative analysis. With the progress of AI technology, neural networks (NNs) can be used to predict changes in solute concentration during and after hemodialysis. The NN model predicted the total UN removal rate with the measured DDQ value, Akl et al found no significant difference between them (prediction error, 10.9%). There was also no significant difference between the predicted time and the actual time interval (the prediction error, 8.3%). This proves that the NN model is sensitive enough. However, due to the few patients included in this model, the available database needs to be further expanded. Fernandez et al[76,77] compared the ANN model with the follow-up methods of hemodialysis (such as Smye, Daugirdas, and urea removal ratio), and the ANN method is better for the estimation of equilibrated urea and can be used as a useful tool for dialysis data analysis of kidney diseases. This approach can be easily extended to other solutions, taking the NN model a step forward.
CKD anemia is one of the most common complications in dialyzed ESRD patients. The use of erythropoiesis-stimulating agents (ESAs) improves the treatment of CKD anemia, but owing to the risk of death and serious cardiovascular events, the treatment dose should vary from person to person. However, there are still many difficulties in choosing the best therapeutic dose. Patient status, pharmacokinetics, and pharmacodynamics of ESA are key factors for correct long-term dose-response prediction to avoid long-term sustained elevation and decreased hemoglobin (Hb) levels. Based on the clinical data of darbepoetin treatment from multicenter dialysis patients, Barbieri et al established an anemia control model (ACM) based on ANNs. It was also deployed to three pilot clinics for clinical testing to determine the impact of ACM on anemia management in daily clinical practice. After the introduction of ACM, the dosage of darbepoetin decreased by 27%, the fluctuation of Hb decreased significantly, and the rate of blood transfusion decreased. This model is helpful for improving the anemia outcome of hemodialysis patients and the positive effect of individualized ESA dose on Hb variability. However, this method mainly focuses on response prediction and does not clearly solve the problem of dose selection, and the dose selection process is more challenging than commonly recognized. Anemia management tools should not only be used to predict the occurrence and development of disease but also be used to better identify the cause of anemia in patients, benefiting doctors in choosing the most appropriate treatment path for patients.
Hemodialysis patients may die as a result of many complications during or after hemodialysis. The mortality rate is high in the first year of dialysis.[76,81] Accurate estimation of post-dialysis mortality can help patients and clinicians make decisions as early as possible. Akbilgic et al used the RF algorithm to construct four different mortality prediction models with 27,615 patients within 30, 90, 180, and 365 days after dialysis transition. The C-statistics of the models were 0.7185, 0.7446, 0.7504, and 0.7488, respectively, which shows good internal effectiveness and replication. The model accurately predicted the short-term post-dialysis mortality of patients after ESRD events, but there is still a lack of external verification.
Of the causes of death, sudden cardiac death is a serious cardiovascular complication in dialysis patients. The ML method has been revealed to be helpful in the prediction of sudden cardiac death. Based on the information of patients before and after dialysis, the RF prediction model was constructed by Goldstein et al. The C-statistic of the model is 0.799. Because the ability to predict sudden death decreases with the increase in the predicted time range, the data collected were more conducive to assessing short-term risks than long-term risks.
Challenges and future prospects
The remaining challenges
While the application of ML in health care and other areas is surprising, it is still in its infancy in the field of kidney disease. Challenges face both ML and its application in nephrology.
Challenges for ML
In the development of machine learning, it is still necessary to constantly improve ML algorithms to meet the challenges in conventional clinical practice. Due to the limitations of the development of ML, the inherent logic behind most ML models is similar to a black box, which is still difficult to explain to doctors.
Additionally, the ethics of ML must be considered. Although some guidelines have emerged, such as Singapore Model Artificial Intelligence Governance Framework, to guide private sector organizations on how to use AI on an ethical basis, the development of clear clinical guidelines still lags far behind the progress of AI technology. It is necessary to develop clear guidelines as soon as possible to standardize its clinical application. Although ML models perform well with more training data, a balance needs to be found between privacy and regulatory requirements with the use of large and diverse datasets to improve the accuracy of ML models.[84,85]
Challenges for nephrology
In addition to the limitation of the development of the ML algorithm, there are also many challenges in the application of ML in nephrology.
Challenges for clinical data collection and processing
In the process of data collection, it may be difficult to collect enough electronic medical record information, and different medical institutions lack unified standards and irregular, incomplete, or missing data, result in low data quality and difficulty in extracting meaningful information from the data. It is necessary to ensure the as much integrity and authenticity of the data as possible. The most commonly mentioned limitation in various studies is the lack of external verification. Only a few studies have carried out external verification of the model to prove the accuracy and practicability of the model. This is also related to the difficulty of data acquisition. At present, the data of each unit are basically non-shared, and the data are very scattered.
Medical data are large, confusing and complex. Data must be properly mapped and pre-processed before they are used for modeling. This step is the basis for the model because the accuracy is highly dependent on the reliability of the data in terms of clinical reflection. Regardless of how much effort is invested in improving the algorithm, any inaccuracy in the label will seriously limit the accuracy that the ML algorithms can achieve. Therefore, it is a very considerable challenge to obtain high-quality data and make it available.
Additionally, we have to consider the cost in the modeling process. It takes considerable manpower and material resources to collect data manually, to store data by computers, or to process data by large-scale computing power. Before modeling, the investment of funds is not a small challenge.
Challenge for renal pathological diagnosis
The published research is only at the stage of accurate identification of glomeruli. The sample size of the study is generally small, and the pictures used for modeling are basically from pathological sections of the kidneys of mice.[32,34] Although AI has developed rapidly, due to the complexity of pathological manifestations of various renal diseases and their close relationship with clinical indicators, the automated pathological diagnosis of specific renal diseases based on images has not yet been published. It is not possible to completely replace pathologists for the renal pathological diagnosis of all kinds of kidney diseases through comprehensive patient information. This needs to be supported by a large quantity of data and confirmed by prospective studies. The DN framework and image processing operations across different clinical practices and image datasets need to be further validated to verify this technique in the distribution and spectrum of all lesions encountered in typical renal pathology services.
Although most nephrologists are not familiar with the basic principles of medical AI now, in the future, through the collaboration of nephrologists and AI researchers, it is possible to use AI technology to build a big database for CKD research and establish a high-efficiency model that can be widely used in the diagnosis and treatment of CKD.
Objective and accurate AI techniques may be truly used in clinical renal pathological diagnosis to assist pathologists in discovering subtle differences between diseases that are indistinguishable to the naked eye. Even on this basis, an app may be developed to help patients identify renal biopsy results and predict renal prognosis.
Through the improvement of clinical data preservation, processing technology and the establishment of a resource sharing platform, kidney disease risk models based on multicenter massive data will be more reliable. When the performance of the model is high enough, it may one day be able to replace renal biopsy to achieve non-invasive diagnosis. The future is full of unknowns, but everything seems to be inevitable.
AI is playing an increasingly important role in many medical fields, helping doctors in most steps of patient management. In nephrology, AI has been used to predict the risk and progression of disease, as well as for hemodialysis prescriptions and follow-up. Although it is just scratching the surface now, with the progress of technology, the accumulation of data and the increase in investment, ML technology will make a major breakthrough in precision medicine in the field of nephrology.
This work was supported by grants from the National Natural Science Foundation of China (Nos. 61971441, 61671479, and 81804056), and the National Key R&D Program of China (No. 2016YFC1305500).
Conflicts of interest
1. Webster AC, Nagler EV, Morton RL, Masson P. Chronic kidney disease. Lancet
2017; 389:1238–1252. doi: 10.1016/S0140-6736(16)32064-5.
2. Glassock RJ, Warnock DG, Delanaye P. The global burden of chronic kidney disease: estimates, variability and pitfalls. Nat Rev Nephrol
2017; 13:104–114. doi: 10.1038/nrneph.2016.163.
3. Williams AM, Liu Y, Regner KR, Jotterand F, Liu P, Liang M. Artificial intelligence, physiological genomics, and precision medicine. Physiol Genomics
2018; 50:237–243. doi: 10.1152/physiolgenomics.00119.2017.
4. Goldenberg SL, Nir G, Salcudean SE. A new era: artificial intelligence and machine learning
in prostate cancer. Nat Rev Urol
2019; 16:391–403. doi: 10. 1038/s41585-019-0193-3.
5. Zhong C, Han J, Borowsky A, Parvin B, Wang Y, Chang H. When machine vision meets histology: a comparative evaluation of model architecture for classification of histology sections. Med Image Anal
2017; 35:530–543. doi: 10.1016/j.media.2016. 08.010.
6. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature
2015; 521:436–444. doi: 10.1038/nature14539.
7. Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, et al. A survey on deep learning in medical image analysis. Med Image Anal
2017; 42:60–88. doi: 10.1016/j.media.2017.07.005.
8. Lee JG, Jun S, Cho YW, Lee H, Kim GB, Seo JB, et al. Deep learning in medical imaging: general overview. Korean J Radiol
2017; 18:570–584. doi: 10.3348/kjr.2017.18.4.570.
9. Naylor CD. On the prospects for a (deep) learning health care system. JAMA
2018; 320:1099–1100. doi: 10.1001/jama.2018.11103.
10. Esteva A, Kuprel B, Novoa RA, Novoa RA, Ko J, Swetter SM, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature
2017; 542:115–118. doi: 10.1038/nature21056.
11. Li CX, Shen CB, Xue K, Shen X, Jing Y, Wang ZY, et al. Artificial intelligence in dermatology: past, present, and future. Chin Med J
2019; 132:2017–2020. doi: 10.1097/CM9.0000000000000372.
12. Faust O, Hagiwara Y, Hong TJ, Lih OS, Acharya UR. Deep learning for healthcare applications based on physiological signals: a review. Comput Methods Programs Biomed
2018; 161:1–13. doi: 10.1016/j.cmpb.2018.04.005.
13. Jha S, Topol EJ. Adapting to artificial intelligence: radiologists and pathologists as information specialists. JAMA
2016; 316:2353–2354. doi: 10.1001/jama.2016.17438.
14. Kantarjian H, Yu PP. Artificial intelligence, big data, and cancer. JAMA Oncol
2015; 1:573–574. doi: 10.1001/jamaoncol.2015.1203.
15. Wainberg M, Merico D, Delong A, Frey BJ. Deep learning in biomedicine. Nat Biotechnol
2018; 36:829–838. doi: 10.1038/nbt.4233.
16. Stead WW. Clinical implications and challenges of artificial intelligence and deep learning. JAMA
2018; 320:1107–1108. doi: 10.1001/jama.2018.11029.
17. Shen D, Wu G, Suk HI. Deep learning in medical image analysis. Annu Rev Biomed Eng
2017; 19:221–248. doi: 10.1146/annurev-bioeng-071516-044442.
18. Suzuki K. Overview of deep learning in medical imaging. Radiol Phys Technol
2017; 10:257–273. doi: 10.1007/s12194-017-0406-5.
19. Angermueller C, Pärnamaa T, Parts L, Stegle O. Deep learning for computational biology. Mol Systems Biol
2016; 12:878doi: 10.15252/msb.20156651.
20. Yang C, Kong G, Wang L, Zhang L, Zhao MH. Big data in nephrology
: are we ready for the change. Nephrology (Carlton)
2019; 24:1097–1102. doi: 10.1111/nep.13636.
21. Gadegbeku CA, Gipson DS, Holzman LB, Ojo AO, Song PX, Barisoni L, et al. Design of the nephrotic syndrome study network (NEPTUNE) to evaluate primary glomerular nephropathy by a multidisciplinary approach. Kidney Int
2013; 83:749–756. doi: 10.1038/ki.2012.428.
22. Xua J, Luo X, Wang G, Gilmore H, Madabhushi A. A deep convolutional neural network for segmenting and classifying epithelial and stromal regions in histopathological images. Neurocomputing
2016; 191:214–223. doi: 10.1016/j. neucom.2016.01.034.
23. Cruz-Roa A, Gilmore H, Basavanhally A, Feldman M, Ganesan S, Shih NNC, et al. Accurate and reproducible invasive breast cancer detection in whole-slide images: a deep learning approach for quantifying tumor extent. Sci Rep
2017; 7:46450doi: 10.1038/srep46450.
24. Han Z, Wei B, Zheng Y, Yin Y, Li K, Li S, et al. Breast cancer multi- classification from histopathological images with structured deep learning model. Sci Rep
2017; 7:4172doi: 10.1038/s41598-017-04075-z.
25. Saha M, Chakraborty C, Arun I, Ahmed R, Chatterjee S. An advanced deep learning approach for Ki-67 stained hotspot detection and proliferation rate scoring for prognostic evaluation of breast cancer. Sci Rep
2017; 7:3213doi: 10.1038/s41598-017-03405-5.
26. Turkki R, Linder N, Kovanen PE, Pellinen T, Lundin J. Antibody-supervised deep learning for quantification of tumor-infiltrating immune cells in hematoxylin and eosin stained breast cancer samples. J Pathol Inform
2016; 7:38doi: 10.4103/2153-3539.189703.
27. Vandenberghe ME, Scott MLJ, Scorer PW, Pellinen T, Lundin J. Relevance of deep learning to facilitate the diagnosis of HER2 status in breast cancer. Sci Rep
2017; 7:45938doi: 10.1038/srep45938.
28. Sarder P, Ginley B, Tomaszewski JE. Automated renal histopathology: Digital extraction and quantification of renal pathology. SPIE Medical Imaging 2016;97910F:1–12. doi: 10.1117/12.2254517.
29. Ginley B, Tomaszewski JE, Sarder P. Automatic computational labeling of glomerular textural boundaries. SPIE Medical Imaging 2017;101400G. doi: 10.1117/12.2254517.
30. Marée R, Dallongeville S, Olivo-Marin J-C, Meas-Yedid V. An approach for detection of glomeruli in multisite digital pathology. IEEE ISBI 2016;1033–1036. doi: 10.1109/ISBI.2016.7493442.
31. Pedraza A, Gallego J, Lopez S, Gonzalez L, Laurinavicius A, Bueno G. Glomerulus classification with convolutional neural networks. Annual Conference on Medical Image Understanding and Analysis 2017;839–849. doi: 10.1007/978-3-319-60964-5_73.
32. Bukowy JD, Dayton A, Cloutier D, Manis AD, Staruschenko A, Lombard JH, et al. Region-based convolutional neural nets for localization of glomeruli in trichrome-stained whole kidney sections. J Am Soc Nephrol
2018; 29:2081–2088. doi: 10.1681/ASN.2017111210.
33. Kannan S, Morgan LA, Liang B, Cheung MG, Lin CQ, Mun D, et al. Segmentation of glomeruli within trichrome images using deep learning. Kidney Int Rep
2019; 4:955–962. doi: 10.1016/j.ekir.2019.04.008.
34. Sheehan S, Mawe S, Cianciolo RE, Korstanje R, Mahoney JM. Detection and classification of novel renal histologic phenotypes using deep neural networks. Am J Pathol
2019; 189:1786–1796. doi: 10.1016/j. ajpath.2019.05.019.
35. Kolachalama VB, Singh P, Lin CQ, Mun D, Belghasem ME, Henderson JM, et al. Association of pathological fibrosis with renal survival using deep neural networks. Kidney Int Rep
2018; 3:464–475. doi: 10.1016/j.ekir.2017.11.002.
36. Rashidian S, Hajagos J, Moffitt RA, Wang F, Noel KM, Gupta RR, et al. Deep learning on electronic health records to improve disease coding accuracy. AMIA Jt Summits Transl Sci Proc
37. Bagshaw SM, Goldstein SL, Ronco C, Kellum JA. Acute kidney injury in the era of big data: the 15th consensus conference of the Acute Dialysis Quality Initiative (ADQI). Can J Kidney Health Dis
2016; 3:1–5. doi: 10.1186/s40697-016-0103-z.
38. Marshall MR, Song Q, Ma TM, MacDonell SG, Kasabov NK. Evolving connectionist system versus algebraic formulas for prediction of renal function from serum creatinine. Kidney Int
2005; 67:1944–1954. doi: 10.1111/j.1523-1755.2005.00293.x.
39. Tangri N, Stevens LA, Griffith J, Tighiouart H, Djurdjev O, Naimark D, et al. A predictive model for progression of chronic kidney disease to kidney failure. JAMA
2011; 305:1553–1559. doi: 10.1001/jama.2011.451.
40. Singh A, Nadkarni G, Gottesman O, Ellis SB, Bottinger EP, Guttag JV. Incorporating temporal EHR data in predictive models for risk stratification of renal function deterioration. J Biomed Inform
2015; 53:220–228. doi: 10.1016/j.jbi.2014.11.005.
41. Norouzi J, Yadollahpour A, Mirbagheri SA, Mazdeh MM, Hosseini SA. Predicting renal failure progression in chronic kidney disease using integrated intelligent fuzzy expert system. Comput Math Methods Med
2016; 2016:6080814doi: 10.1155/2016/6080814.
42. Perotte A, Ranganath R, Hirsch JS, Blei D, Elhadad N. Risk prediction for chronic kidney disease progression using heterogeneous electronic health record data and time series analysis. J Am Med Inform Assoc
2015; 22:872–880. doi: 10.1093/jamia/ocv024.
43. Echouffo-Tcheugui JB, Kengne AP. Risk models to predict chronic kidney disease and its progression: a systematic review. PLoS Med
2012; 9:e1001344doi: 10.1371/journal.pmed.1001344.
44. Xie Z, Nikolayeva O, Luo J, Li D. Building risk prediction models for type 2 diabetes using machine learning
techniques. Prev Chronic Dis
2019; 16:E130doi: 10.5888/pcd16.190109.
45. Xiong XL, Zhang RX, Bi Y, Zhou WH, Yu Y, Zhu DL. Machine learning
models in type 2 diabetes risk prediction: results from a cross-sectional retrospective study in Chinese adults. Curr Med Sci
2019; 39:582–588. doi: 10.1007/s11596-019-2077-4.
46. Perveen S, Shahbaz M, Keshavjee K, Guergachi A. Prognostic modeling and prevention of diabetes using machine learning
technique. Sci Rep
2019; 9:13805doi: 10.1038/s41598-019-49563-6.
47. Lai H, Huang H, Keshavjee K, Guergachi A, Gao X. Predictive models for diabetes mellitus using machine learning
techniques. BMC Endocr Disord
2019; 19:101doi: 10.1186/s12902-019-0436-6.
48. Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA
2016; 316:2402–2410. doi: 10.1001/jama.2016.17216.
49. Cho BH, Yu H, Kim KW, Kim TH, Kim IY, Kim SI. Application of irregular and unbalanced data to predict diabetic nephropathy using visualization and feature selection methods. Artif Intell Med
2008; 42:37–53. doi: 10.1016/j.artmed.2007. 09.005.
50. Rodriguez-Romero V, Bergstrom RF, Decker BS, Lahu G, Vakilynejad M, Bies RR. Prediction of nephropathy in type 2 diabetes: an analysis of the ACCORD trial applying machine learning
techniques. Clin Transl Sci
2019; 12:519–528. doi: 10.1111/cts.12647.
51. Ravizza S, Huschto T, Adamov A, Bohm L, Busser A, Flother FF, et al. Predicting the early risk of chronic kidney disease in patients with diabetes using real-world data. Nat Med
2019; 25:57–59. doi: 10.1038/s41591-018-0239-8.
52. Leung RK, Wang Y, Ma RC, Luk AO, Lam V, Ng M, et al. Using a multi-staged strategy based on machine learning
and mathematical modeling to predict genotype- phenotype risk patterns in diabetic kidney disease: a prospective case-control cohort analysis. BMC Nephrol
2013; 14:162doi: 10.1186/1471-2369-14-162.
53. Liu XM, Dong ZY, Zhang WG, Liu MY, Zhou JH, Wang Q, et al. Validation of the 2007 kidney disease outcomes quality initiative clinical practice guideline for the diagnosis of diabetic nephropathy and nondiabetic renal disease in Chinese patients. Diabetes Res Clin Pract
2019; 147:81–86. doi: 10.1016/j.diabres.2018.11.008.
54. Zhou J, Chen X, Xie Y, Li J, Yamanaka N, Tong X, et al. A differential diagnostic model of diabetic nephropathy and non-diabetic renal diseases. Nephrol Dial Transplant
2018; 23:1940–1945. doi: 10.1093/ndt/gfm897.
55. Liu MY, Chen XM, Sun XF, Zhou JH, Zhang XG, Zhu HY, et al. Validation of a differential diagnostic model of diabetic nephropathy and non-diabetic renal diseases and the establishment of a new diagnostic model. J Diabetes
2014; 6:519–526. doi: 10.1111/1753-0407.12150.
56. Geddes CC, Fox JG, Allison ME, Boulton-Jones JM, Simpson K. An artificial neural network can select patients at high risk of developing progressive IgA nephropathy more accurately than experienced nephrologists. Nephrol Dial Transplant
1998; 13:67–71. doi: 10.1093/ndt/13.1.67.
57. Pesce F, Diciolla M, Binetti G, Naso D, Ostuni VC, Di Noia T, et al. Clinical decision support system for end-stage kidney disease risk estimation in IgA nephropathy patients. Nephrol Dial Transplant
2016; 31:80–86. doi: 10.1093/ndt/gfv232.
58. Han X, Zheng X, Wang Y, Sun X, Xiao Y, Tang Y, et al. Random forest can accurately predict the development of end-stage renal disease in immunoglobulin a nephropathy patients. Ann Transl Med
2019; 7:234doi: 10.21037/atm.2018.12. 11.
59. Liu Y, Zhang Y, Liu D, Tan X, Tang X, Zhang F, et al. Prediction of ESRD in IgA nephropathy patients from an Asian cohort: a random forest model. Kidney Blood Press Res
2018; 43:1852–1864. doi: 10.1159/000495818.
60. Chen T, Li X, Li Y, Xia E, Qin Y, Liang S, et al. Prediction and risk stratification of kidney outcomes in IgA nephropathy. Am J Kidney Dis
2019; 74:300–309. doi: 10.1053/j.ajkd.2019. 02.016.
61. Mehta RL, Burdmann EA, Cerda J, Feehally J, Finkelstein F, Garcia-Garcia G, et al. Recognition and management of acute kidney injury in the International Society of Nephrology
0by25 Global Snapshot: a multinational cross-sectional study. Lancet
2016; 387:2017–2025. doi: 10. 1016/S0140-6736(16)30240-9.
62. Molitoris BA. Beyond biomarkers: machine learning
in diagnosing acute kidney injury. Mayo Clin Proc
2019; 94:748–750. doi: 10.1016/j.mayocp.2019.03.017.
63. Sutherland SM, Goldstein SL, Bagshaw SM. Acute kidney injury and big data. Contrib Nephrol
2018; 193:55–67. doi: 10.1159/000484963.
64. Laszczynska O, Severo M, Azevedo A. Electronic medical record-based predictive model for acute kidney injury in an acute care hospital. Stud Health Technol Inform
2016; 228:810–812. doi: 10.3233/978-1-61499-678-1-810.
65. Koyner JL, Carey KA, Edelson DP, Churpek MM. The development of a machine learning
inpatient acute kidney injury prediction model. Crit Care Med
2018; 46:1070–1077. doi: 10.1097/CCM. 0000000000003123.
66. Lee HC, Yoon SB, Yang SM, Kim WH, Ryu HG, Jung CW, et al. Prediction of acute kidney injury after liver transplantation: machine learning
approaches vs. logistic regression model. J Clin Med
2018; 7:428doi: 10.3390/jcm7110428.
67. Lee HC, Yoon HK, Nam K, Cho YJ, Kim TK, Kim WH, et al. Derivation and validation of machine learning
approaches to predict acute kidney injury after cardiac surgery. J Clin Med
2018; 7:322doi: 10.3390/jcm7100322.
68. Huang C, Murugiah K, Mahajan S, Li SX, Dhruva SS, Haimovich JS, et al. Enhancing the prediction of acute kidney injury risk after percutaneous coronary intervention using machine learning
techniques: a retrospective cohort study. PLoS Med
2018; 15:e1002703doi: 10.1371/journal.pmed.1002703.
69. Tomasev N, Glorot X, Rae JW, Zielinski M, Askham H, Saraiva A, et al. A clinically applicable approach to continuous prediction of future acute kidney injury. Nature
2019; 572:116–119. doi: 10.1038/s41586-019-1390-1.
70. Lin K, Hu Y, Kong G. Predicting in-hospital mortality of patients with acute kidney injury in the ICU using random forest model. Int J Med Inform
2019; 125:55–61. doi: 10.1016/j.ijmedinf.2019.02.002.
71. Akl AI, Sobh MA, Enab YM, Tattersall J. Artificial intelligence: a new approach for prescription and monitoring of hemodialysis therapy. Am J Kidney Dis
2001; 38:1277–1283. doi: 10.1053/ajkd.2001.29225.
72. Brier ME, Gaweda AE. Artificial intelligence for optimal anemia management in end-stage renal disease. Kidney Int
2016; 90:259–261. doi: 10.1016/j.kint.2016.05.018.
73. Akbilgic O, Obi Y, Potukuchi PK, Karabayir I, Nguyen DV, Soohoo M, et al. Machine learning
to identify dialysis patients at high death risk. Kidney Int Rep
2019; 4:1219–1229. doi: 10.1016/j.ekir.2019.06.009.
74. Hayes W, Allinovi M. Beyond playing games: nephrologist vs machine in pediatric dialysis prescribing. Pediatr Nephrol
2018; 33:1625–1627. doi: 10.1007/s00467-018-4021-4.
75. Hueso M, Vellido A, Montero N, Barbieri C, Ramos R, Angoso M, et al. Artificial intelligence for the artificial kidney: pointers to the future of a personalized hemodialysis therapy. Kidney Dis (Basel)
2018; 4:1–9. doi: 10.1159/000486394.
76. Fernandez EA, Valtuille R, Presedo JM, Willshaw P. Comparison of different methods for hemodialysis evaluation by means of ROC curves: from artificial intelligence to current methods. Clin Nephrol
2005; 64:205–213. doi: 10.5414/cnp64205.
77. Fernandez EA, Valtuille R, Willshaw P, Perazzo CA. Using artificial intelligence to predict the equilibrated postdialysis blood urea concentration. Blood Purif
2001; 19:271–285. doi: 10. 1159/000046955.
78. Singh AK, Szczech L, Tang KL, Barnhart H, Sapp S, Wolfson M, et al. Correction of anemia with epoetin alfa in chronic kidney disease. N Engl J Med
2006; 355:2085–2098. doi: 10.1056/NEJMoa065485.
79. Barbieri C, Mari F, Stopper A, Gatti E, Escandell-Montero P, Martinez-Martinez JM, et al. A new machine learning
approach for predicting the response to anemia treatment in a large cohort of End Stage Renal Disease patients undergoing dialysis. Comput Biol Med
2015; 61:56–61. doi: 10.1016/j.compbiomed.2015.03.019.
80. Barbieri C, Molina M, Ponce P, Tothova M, Cattinelli I, Ion Titapiccolo J, et al. An international observational study suggests that artificial intelligence for clinical decision support optimizes anemia management in hemodialysis patients. Kidney Int
2016; 90:422–429. doi: 10.1016/j.kint.2016.03.036.
81. Arif FM, Sumida K, Molnar MZ, Potukuchi PK, Lu JL, Hassan F, et al. Early mortality associated with inpatient versus outpatient hemodialysis initiation in a large cohort of US Veterans with incident end-stage renal disease. Nephron
2017; 137:15–22. doi: 10.1159/000473704.
82. Goldstein BA, Chang TI, Mitani AA, Assimes TL, Winkelmayer WC. Near-term prediction of sudden cardiac death in older hemodialysis patients using electronic health records. Clin J Am Soc Nephrol
2014; 9:82–91. doi: 10.2215/CJN.03050313.
83. Ngiam KY, Khor IW. Big data and machine learning
algorithms for health-care delivery. Lancet Oncol
2019; 20:e262–e273. doi: 10.1016/S1470-2045(19)30149-4.
84. Rajkomar A, Dean J, Kohane I. Machine learning
in medicine. N Engl J Med
2019; 380:1347–1358. doi: 10.1056/NEJMra1814259.
85. Powles J, Hodson H. Google DeepMind and healthcare in an age of algorithms. Health Technol (Berl)
2017; 7:351–367. doi: 10.1007/s12553-017-0179-1.
86. Zeng XX, Liu J, Ma L, Fu P. Big Data Research in Chronic Kidney Disease. Chin Med J
2018; 131:2647–2650. doi: 10.4103/0366-6999.245275.
87. van Grinsven M, van Ginneken B, Hoyng C, Theelen T, Sanchez C. Fast convolutional neural network training using selective data sampling: application to hemorrhage detection in color fundus images. IEEE Trans Med Imag
2016; 35:1273–1284. doi: 10.1109/TMI.2016.2526689.