Enabling proactive and personalized health management for all with AI is complex. Proactive and personalized health services require high levels of personalization and continuous risk monitoring, creating heightened risks to data privacy and data security. Personalization of health management invariably requires analysis of a person’s health data, which may become personal identifiable even after removing sensitive personal data (e.g., a patient with a rare disease living in a small village is easily identifiable). The increased availability of genetic data makes the need for strict data privacy protection even more critical.

To provide more seamless and faster patient experiences at reduced cost, many hospitals have started to automate tasks. Given the high-risk nature of health decisions, the use of AI for automating health decision-making requires allowing for human agency and oversight and explanations of automated decisions that are understandable to auditors, doctors, patients (GDPR). 

To provide safe patient experiences, AI models must be robust and safe. AI models must be robust enough that prediction accuracy is maintained when there are changes in the data. In February 2022, Statnews published the findings of a research study conducted with the Massachusetts Institute of Technology that found small shifts in data fed into well-known health care algorithms, including the Epic Sepsis Model, can cause their accuracy to degrade over time.

To promote patient safety it is vital that AI models include health domain knowledge and are interpretable. Most AI models identify correlations/associations rather than causations. Given an explainable AI solution, a domain expert should validate which correlations are actual causations. For example, a study by Cooper et al. painfully showed how a black-box neural network indicated that having asthma lowers a patient’s risk of dying from pneumonia. Why the incongruous results? The model learned that asthma patients when they exhibit symptoms of pneumonia are more likely to be admitted to the Emergency Department, thus lowering their mortality risk. Involving domain experts during AI model development and adding interpretability as a selection criterion could have prevented this correlation from being seen by the AI model as a causation.

To enable the patient to take up an active role in health decision-making and health management requires transparent, interpretable and explainable AI solutions. Black-box models are not interpretable. Also, post-hoc explanations are often not authentic, so they cannot be used to enable the patient to make a shared health decision.

To promote health equity is also challenging. Health disparities can be exacerbated through biases in the data used to train an AI algorithm, and in the algorithm’s intent. For AI to be used effectively to promote health equity, existing biases in healthcare services and systems based on race, ethnicity, age, and gender, that are encoded in data used to train algorithms, must be addressed (WHO, Guide on Ethics and Governance of Artificial Intelligence for Health, June, 2021).  Obermeyer et al. (2019) found that a widely used commercial prediction algorithm resulted in significant racial bias in predicting outcomes. A particular challenge in the application of AI to cardiovascular medicine is in the intrinsic and extrinsic biases that affect quality control, and thus clinical outcomes. The bulk of current evidence-based medicine in cardiology might not apply to women and minority groups, because they are historically under-represented (Tat et al., The Lancet, 2020. A systematic review study of COVID-19 prediction models uncovered widespread and serious shortcomings, particularly with respect to potential biases (Wijnants et al., BMJ, 2021). With the push to quickly adopt AI to assist in the fight against COVID-19 frenzy, the risk of producing biased prediction models due to unrepresentative datasets and other limitations in model development is higher than ever (Roosli et al., JAMIA, 2021).

Regular AI has a lack of respect for data privacy, human agency, explainability, transparency, fairness,  which result in lack of trust and no adoption of the AI solution supporting health management decisions.

Trustworthy AI supports hyper-personalized health management decisions that are fair, transparent and explainable by design. Trustworthy AI is ethical (fair, non-discriminating), lawful and robust (see Ethics Guidelines for Trustworthy AI, 2019). It enables fair and consistent decisions that are explainable and justifiable to all stakeholders.

The WHO also published a report on trustworthy AI for Health (2021): ‘’Guide on Ethics and Governance of Artificial Intelligence for Health’’.

Trustworthy AI tools contribute to the United Nations Sustainable Development Goal 3 (WHO, 2021): Achieve universal health coverage, including financial risk protection , access to quality essential health-care services and access to safe, effective, quality and affordable essential medicines and vaccines for all.