The detection of Pulmonary Hypertension (PH) from the computer analysis of digitized heart sounds is a low-cost and non-invasive solution for early PH detection and screening. We present an extensive cross-domain evaluation methodology with varying animals (humans and porcine animals) and varying auscultation technologies (phonocardiography and seisomocardiography) evaluated across four methods. We introduce PH-ELM, a resource efficient PH detection model based on the extreme learning machine that is smaller (300× fewer parameters), energy efficient (532× less watts of power), faster (36× faster to train, 44× faster at inference), and more accurate on outof-distribution testing (improves median accuracy by 0.07 area under the ROC curve (auROC)) in comparison to a previously best performing deep network. We make four observations from our analysis: (a) digital auscultation is a promising technology for the detection of pulmonary hypertension; (b) seismocardiography (SCG) signals and phonocardiography (PCG) signals are interchangeable to train PH detectors; (c) porcine heart sounds in the training data can be used to evaluate PH from human heart sounds (the PH-ELM model preserves 88 to 95% of the best indistribution baseline performance); (d) predictive performance of PH detection can be mostly preserved with as few as 10 heartbeats and capturing up to approximately 200 heartbeats per subject can improve performance. The results of this work may guide future methods for data collection, predictive modeling, and model evaluation towards the practical implementation of PH detectors.

Rank metrics quantify machine learning model prediction errors in the setting of binary classification with a given dataset, but imbalances in the class distribution or model predictions can significantly influence the interpretation. We analyze and interpret the area under the receiver operating characteristic curve (auROC) and an area under the precision recall curve, or average precision score (AP), in the presence of two kinds of imbalance: dataset class imbalance and model prediction imbalance. We show that the auROC can have a symmetric but not necessarily uni-modal response to dataset imbalance, but it is insensitive to model prediction imbalance. The AP has an asymmetric response that we show is also not uni-modal. The AP and auROC scores can be challenging to interpret due to these properties. We propose a novel precision recall space, the PR*, and associated average precision score, called AP*, to address the identified shortcomings of the AP and auROC. The proposed AP* metric has a symmetric response to class imbalance, like the auROC, a sensitivity to model prediction imbalance, like the AP, and it adds both a symmetry to model prediction imbalance and unimodal response to class imbalance that makes it more reliable for model selection than the AP or auROC. For model selection, the AP* gives a higher score to the model that tends to make most of its errors in the majority class.