Exploratory Machine-Learning Analysis of Acute Hepatitis of Unknown Aetiology in Children: A Retrospective Multicentre Study from Indonesia

Acute hepatitis of unknown aetiology (AHUA) was reported internationally in 2022 as acute non-A-E hepatitis with markedly increased aminotransferases in children after exclusion of established causes [1]. By 26 May 2022, the World Health Organization had received reports of 650 probable cases from 33 countries [2], while Indonesia reported 18 suspected cases across several provinces [3]. Early clinical series showed heterogeneous presentations and, in a minority of children, severe liver injury requiring intensive care or liver transplantation [4, 5].

The aetiology remains incompletely resolved. Clinical updates and paediatric hepatology guidance emphasise the need for careful exclusion of established causes and uniform documentation of virology, biochemical abnormalities, symptoms, and outcomes [6, 7]. Molecular investigations have reported associations with adeno-associated virus 2, adenovirus, helper-virus coinfection, and host susceptibility, but these findings do not provide a simple retrospective classification rule [8-10].

Machine-learning analysis of rare clinical outcomes is methodologically difficult because small event counts limit the stability of training, validation, and interpretation [11, 12]. In clinical prediction research, small samples, incomplete-data handling, and inadequate validation are common sources of bias [13], while probability calibration is essential if predicted probabilities are interpreted as risks [14]. Moreover, retrospective in silico evaluation should not be conflated with live clinical evaluation of an AI decision-support system [15]. Recent studies provide useful methodological examples of explainability and imbalance handling [16, 17], but they are not clinical evidence for acute hepatitis.

This study does not claim a new learning algorithm or a validated clinical decision-support system. Its contribution is an exploratory retrospective investigation that: (1) analyses a multicentre Indonesian paediatric cohort with expert-assigned labels; (2) compares six classifiers using leakage-free evaluation; (3) examines the instability created by only five target cases using repeated cross-validation and site-held-out robustness analysis; (4) evaluates out-of-fold probability reliability; and (5) uses SHapley Additive exPlanations (SHAP) only as a post-hoc description of model behaviour. Reporting was revised with reference to TRIPOD+AI and PROBAST+AI principles [18, 19].

3.1 Study design and data source

This retrospective multicentre study used 289 records of patients with hepatitis or suspected hepatitis collected from Fatmawati General Hospital, Sulianti Saroso Infectious Disease Hospital, H. Adam Malik General Hospital, Dr Soetomo General Academic Hospital, and Dr M. Djamil Hospital. Records were identified using hepatitis-related ICD-10 codes. The original extraction criteria included age below 18 years and jaundice or an AST/ALT increase exceeding three times the local upper reference limit. Records with liver cancer, hepatic cirrhosis, hepatic coma, drug-induced hepatitis, or materially incomplete diagnostic documentation were excluded. The available variables covered demographics, symptoms, physical findings, hospitalization, complete blood counts, aminotransferases, bilirubin fractions, albumin, renal function, and electrolytes. Appendix Table A1 defines every extracted field, its coding or unit, and its modelling role. Appendix Table A2 reports the outcome distribution by hospital.

3.2 Outcome definition and ground truth

Ground-truth annotation was performed directly by the authoring health-expert team. The team comprised health-domain specialists from the National Research and Innovation Agency of Indonesia (BRIN) together with clinical representatives from each of the five participating hospitals, all of whom had relevant expertise in evaluating paediatric hepatitis and related clinical presentations. Labels were assigned retrospectively from the available medical records by considering the recorded final diagnosis together with demographic, clinical, and laboratory findings. Three final categories were used: probable AHUA, hepatitis, and non-hepatitis. The distribution of clinical outcome classes in the original cohort is presented in Table 1. The five records in the target category therefore represent expert-assigned clinical labels rather than labels generated by the machine-learning procedure.

The labels were established before model development and remained unchanged throughout preprocessing, over-sampling, training, and evaluation. The WHO outbreak definition [1] was used as clinical context rather than a claim that every retrospectively labelled record satisfied every prospective surveillance element. The study did not employ a prospectively predefined blinded duplicate-rating protocol, and formal inter-rater agreement statistics were not recorded; this is acknowledged as an important limitation because only five records belong to the target class.

Table 1. Outcome-class distribution in the original cohort

3.3 Data preparation and leakage control

Patient identifier, hospital, and final diagnosis were excluded from the predictor matrix. Four variables with more than 80% missingness (band neutrophils, segmented neutrophils, prothrombin time, and globulin) were removed, leaving 59 candidate predictors. Non-numeric laboratory tokens were converted to missing values. Within each training fold, numeric values were median-imputed, and categorical values were mode-imputed. Categorical levels were encoded from the training data only; unseen test categories were retained as unknown levels. Scaling was applied only to algorithms that required it.

3.4 Class imbalance and model comparison

The original study centred on XGBoost. In the corrected exploratory comparison, XGBoost was retained alongside five alternative classifiers. At boosting iteration t, a tree f_t is added by minimising the regularised objective [20]:

$\begin{gathered}\mathcal{L}^{(\mathrm{t})}=\Sigma_{\mathrm{i}} \ell\left(y_{\mathrm{i}}, \hat{y}_{\mathrm{i}}^{(\mathrm{t}-1)}+f_{\mathrm{t}}\left(x_{\mathrm{i}}\right)\right)+\Omega\left(f_{\mathrm{t}}\right), \\ \Omega\left(f_{\mathrm{t}}\right)=\gamma T+1 / 2 \lambda \Sigma_{\mathrm{i}} w_{\mathrm{i}}^2\end{gathered}$   (1)

For the three-class outcome, the data-fitting term was multiclass logarithmic loss, where $y_{i c}$ is the indicator for class $c$ and $p_{i c}$ is the corresponding predicted probability:

$\operatorname{LogLoss}=-(1 / N) \sum_{\mathrm{i}} \sum y_{\mathrm{i}} \log \left(p_{\mathrm{i}}\right)$   (2)

Class imbalance was handled with SMOTENC [21]. For numerical components, a synthetic observation is generated between minority sample $x_i$ and one of its nearest minority neighbours $x_z$:

$x_{\mathrm{ne}} w=x_{\mathrm{i}}+\delta\left(x_{\mathrm{z}}-x_i\right), 0 \leq \delta \leq 1$   (3)

Categorical components were handled by SMOTENC rather than interpolated as continuous values. The controlled comparison additionally included Random Forest and Gradient Boosting as tree-ensemble alternatives, an RBF support-vector machine and logistic regression as nonlinear-margin and linear baselines, and distance-weighted K-Nearest Neighbors (KNN). KNN used Euclidean distance:

$d\left(x_{\mathrm{i}}, x_{\mathrm{j}}\right)=\sqrt{ }\left[\Sigma_{\mathrm{m}}\left(x_{\mathrm{im}}-x_{\mathrm{jm}}\right)^2\right]$    (4)

Table 2. Model configurations used in the controlled comparison

Note: XGBoost = Extreme Gradient Boosting, KNN = K-Nearest Neighbors, SVM = Support Vector Machine.

SMOTENC was applied only to the training portion of each fold, with k_neighbors = 2, sampling_strategy = "not majority", and fold-specific random states. Each training fold contained four probable unknown cases. SMOTENC increased this class from four to 130 or 131 records, producing 126–127 synthetic unknown-class observations per fold; the non-hepatitis class received 33–35 synthetic observations. Test folds contained original patients only. Six classifiers were evaluated on the same folds and preprocessing outputs. The fixed configurations in Table 2 were chosen to provide a controlled comparison rather than an exhaustive search on a very small minority class.

3.5 Validation and evaluation

Primary evaluation used stratified five-fold cross-validation with shuffling and random_state = 42. Because there were exactly five probable unknown cases, each test fold contained only one target case. Out-of-fold predictions were pooled so that every patient contributed once to the primary confusion matrix. The five individual folds were also reported to show variability. To examine partition sensitivity, the complete five-fold procedure was repeated five times using different shuffle seeds. These repeated analyses were used descriptively; model ordering was not treated as evidence of superiority. A leave-one-hospital-out site-held-out robustness analysis was performed separately and was explicitly not interpreted as external validation [23]. Probability reliability for the three tree-based models was assessed from out-of-fold predicted probabilities using multiclass Brier score and multiclass log loss [14]. Exploratory one-vs-rest reliability plots for Random Forest are shown in Figure 1.

944c68dba6496e4b81114f2628125018.png

Because conventional accuracy is dominated by the two common classes, balanced accuracy and macro F1 were treated as principal discrimination summaries. Class-specific precision $P_c$ and sensitivity $R_c$ were also reported. For probability reliability, the multiclass Brier score was calculated as $B S=\frac{1}{N} \sum_{i=1}^N \sum_{c=1}^C\left(p_{i c}-y_{i c}\right)^2$, where $p_{i c}$ is the predicted probability and $y_{i c}$ is the one-hot outcome indicator; lower values indicate smaller squared probability error. With only five target events, target-class calibration was considered too sparse for reliable calibration slope or intercept estimation.

$P_c=\frac{T P_c}{T P_c+F P_c}, \quad R_c=\frac{T P_c}{T P_c+F N_c}$   (5)

Balanced Accuracy $=\frac{1}{C} \sum_{c=1}^C R_C$   (6)

Macro F1 $=\frac{1}{C} \sum_{c=1}^C \frac{2 P_c R_c}{P_c+R_c}$   (7)

3.6 Post-hoc model-behavior analysis

For post-hoc model-behavior analysis, TreeSHAP values were calculated from a Random Forest refitted to the complete cleaned dataset. One-hot feature contributions were aggregated to their source variables. Because the fitted models had 0% sensitivity for the target class, SHAP values were not interpreted as explanations of probable unknown hepatitis and were not presented as clinical biomarkers. They were used only to describe how the fitted model behaved across the full three-class dataset, which was dominated by hepatitis and non-hepatitis records.

TreeSHAP represents an individual model output as an additive feature-attribution model, where φ₀ is the baseline output, and φⱼ is the contribution assigned to feature j [22]:

$g\left(z^{\prime}\right)=\varphi_0+\Sigma_{\mathrm{j}=1}{ }^{\mathrm{M}} \varphi_{\mathrm{j}} z_{\mathrm{j}}^{\prime}$   (8)

The second-round analyses reinforce that this study is an exploratory investigation of modelling limitations rather than a demonstration of screening capability. Overall discrimination was acceptable for the two common classes, but no model identified any of the five probable unknown cases. The primary fold-level Random Forest accuracy ranged from 79.31% to 91.38% and macro F1 from 53.83% to 61.40%, while repeated five-fold analysis showed that the highest macro-F1 model was not invariant across partition seeds. Thus, small differences in aggregate model metrics should not be interpreted as a reliable ranking.

The main explanation is event scarcity. Probable unknown hepatitis represented only 1.73% of the cohort, leaving four genuine target cases in each training fold. SMOTENC generated more than 120 synthetic target observations from that narrow empirical base, but over-sampling cannot replace genuine clinical diversity [11, 12]. Probability reliability was also inadequate for the target class: all three tree models had a worse target-class Brier score than the prevalence-only baseline, and Random Forest assigned a mean target probability of only 3.40% to the five actual target cases compared with 6.13% to non-target cases. These outputs therefore cannot be interpreted as clinically meaningful risk estimates [14].

The SHAP analysis must be interpreted within the same constraints. Because target sensitivity was 0%, the global SHAP profile primarily reflects separation of hepatitis and non-hepatitis in this retrospective cohort; it does not reveal biomarkers of AHUA. The current contribution is therefore methodological: transparent expert-label description, leakage control, rare-event validation diagnostics, probability-reliability assessment, and explicit separation between retrospective in-silico analysis and future clinical evaluation. Recent studies remain methodological examples [16, 17], while clinical interpretation is grounded in paediatric hepatitis literature [4-10].

5.1 Limitations and future work

The main limitation is the availability of only five probable unknown cases. The retrospective expert annotation was performed by BRIN health specialists and knowledgeable hospital representatives, but a prospectively predefined blinded duplicate-rating protocol and formal inter-rater agreement were not available. No independent external dataset or prospective clinical evaluation was performed. The site-held-out robustness analysis cannot establish external validity because all sites were within one national healthcare context and target events were sparse. SHAP was post-hoc, and probability calibration for the target class could not be estimated reliably. Future work should prospectively recruit substantially more target cases, use predefined adjudication criteria with duplicate review where feasible, perform independent external validation, assess calibration and clinically justified thresholds, and only then evaluate an AI-assisted workflow in a live clinical environment in accordance with early clinical AI evaluation and deployment guidance [15, 24]. Figure 4 illustrates such a future integration pathway; it was not implemented or evaluated in the present study.

image023.png

The authors thank the participating hospitals for their institutional and clinical support. This work was supported by the National Research and Innovation Agency of Indonesia (BRIN) under Research Grant No. B-231/III.9/PR.03.08/1/2023 and by Telkom University.

The data and code supporting the findings of this study are openly available in GitHub at https://github.com/hilalnuha/hepa_hybrid.