AI

hidden

Setting Expectations for AI Models in Medicine

Kylia Williams

Artificial intelligence is a hot topic in every field, and these algorithms are being widely used in scientific research. Particularly in my field of genetics and genomics, machine learning methods are invaluable for gleaning insights from large amounts of highly dimensional data. But there are many things to consider before applying AI and ML in a clinical setting, when real people are on the other end of the predictive model. It is important to set expectations for what AI can and cannot accomplish and what is needed for a broad application of AI in medicine in the future. In the session “Hype or Hope? Artificial Intelligence and Machine Learning in Imaging”, presenters gave a great overview of the applications of AI, its limitations, and the advancements that are needed for a wide application of AI in medicine.

Dr. Geoffrey Rubin described many different scenarios in which AI can be deployed. Specifically, he talked about how AI can be used in predictive analytics to make test selection and imaging more efficient, in image reconstruction to reduce noise, in image segmentation to identify regions of interest and provide quantitative analysis, and in interpretation to derive unique characteristics that cannot be measured directly, identify abnormalities, and create reports. In addition, Dr. Tessa Cook explained in greater depth how AI can be used as clinical decision support to incorporate diverse data types and aid in proper test selection. Dr. Damini Dey also discussed how AI can improve diagnosis and prediction, characterize disease, and personalize therapy. Overall, it is important to determine where AI can provide the greatest value while introducing the least amount of risk.

However, there are many limitations to AI and ML models. First, as Dr. David Ouyang noted, because these models are trained by humans, they can only perform tasks that a human could theoretically do. AI just performs these tasks faster, more consistently, and at a larger scale. He noted that these models are not effective unless trained on broad underlying datasets, and that unless explicitly programmed, they do not accurately weight rare significant events. AI models can easily become uninterpretable black boxes, keeping experts from recognizing where they are failing. Dr. David Playford emphasized that due to these and other limitations, AI models are not yet clinically accurate in all areas.

There are many steps that must be taken before AI models can achieve wide use in clinical settings. Dr. Ouyang suggests standardized baselines and open access to measure advancements among tools. Dr. Cook implements a “trust and value” checklist to assess how each tool was trained and tested, as well as what it can and cannot do, before using it for clinical decision support. Dr. Playford advocates for randomized trials to establish proof-of-concept and compare outcomes to the current standard of care. Most importantly, steps must be taken to reduce bias in AI models, which can negatively impact the care of underrepresented populations. Multidisciplinary collaborative teams can ensure that the data aligns with the clinical question being tackled, diverse yet consistent training datasets are being used, and methods such as transfer learning are implemented to produce more accurate predictions on previously unseen datasets. While AI can be an important tool in clinical decision making, it is ultimately the responsibility of each physician to ensure that AI tools are serving their patients as effectively as possible.

“The views, opinions and positions expressed within this blog are those of the author(s) alone and do not represent those of the American Heart Association. The accuracy, completeness and validity of any statements made within this article are not guaranteed. We accept no liability for any errors, omissions or representations. The copyright of this content belongs to the author and any liability with regards to infringement of intellectual property rights remains with them. The Early Career Voice blog is not intended to provide medical advice or treatment. Only your healthcare provider can provide that. The American Heart Association recommends that you consult your healthcare provider regarding your personal health matters. If you think you are having a heart attack, stroke or another emergency, please call 911 immediately.”
hidden

Buzzword Alert! Artificial Intelligence – Just the Hype Man or a Genuine Showstopper?

Baljash Cheema, MD, MSCI

Conversations of the utility and promise of machine learning (ML) and artificial intelligence (AI) permeate all fields of medicine, and cardiology is no exception. A quick search shows that 69 posters containing the keywords “machine learning” made it into AHA’s Scientific Sessions 2020. But is it for real? Will we really see a future in with ML/AI factors into all aspects of clinical care and in fact, re-write the script on how we care for patients?

Below is some of the discussion points and imperatives that stood out to me today from the “Hope or Hype? Artificial intelligence and Machine Learning in Imaging.” session at #AHA20 featuring thought leaders Drs. Marielle Scherrer-Crosbie, Alex Bratt, David Ouyang, Tessa Cook, Damini Dey, David Playford, and Geoffrey Rubin.

  1. While awe-inspiring in its ability to make inferences and predictions human beings often cannot themselves, we must be aware that ML/AI algorithms can recreate and reinforce the bias pre-existing in our society. We must fight this by knowing it is a possibility, screening for it, and training algorithms on datasets that are truly representative. As much of the political landscape and national conversation right now centers on structural racism and bias in America, it’s is prudent to understand how the models we create can perpetuate this.
  1. Separate low hanging fruit from the unrealistic (at the moment) and consider the unrealistic tasks in the realm of discovery science. A quick rule of thumb provided by Dr. Ouyang, summarizing the words of Dr. Andrew Ng, first determines if it is possible for humans to do a task relatively quickly. If it is, we can probably automate it with AI now or in the near future.

  1. Scrutinize our data. How much do we trust it? High-quality data for ML/AI means broad, accurate, and plentiful. We need robust training labels, as free from subjectively as possible.
  2. How open is our data for inspection? Fields in computer science are far further along than medicine in deploying and improving ML/AI models because of open data sets and shared code, allowing groups to verify, tinker, and re-create to move the needle forward. Medical AI has not been so forthcoming.

  1. As new technology is rapidly evolving and making it into the clinical space, we need to be responsible for mistakes. This means we need to assess not only our model performance before deploying but also the consequences of using the model in real life. This may require RCTs and to consider ML/AI algorithms like we consider new therapeutics.

  1. What we really want is the AI running in the background saying “Hey, this task was automated and is now solved for you. Proceed as you see fit.” Humans and machines are in this shared space. The more we can integrate ML/AI to help us with tasks we are already doing, the better our results will be.

So where does this leave us? Most in our field believe ML/AI will play an important role in our future. Ideally, we will do it in a way that will make sure human intelligence is always paired with artificial intelligence to create a product neither of the two could be alone, will ensure our algorithms are free of bias and openly shared to allow for continuous improvement.

 

“The views, opinions and positions expressed within this blog are those of the author(s) alone and do not represent those of the American Heart Association. The accuracy, completeness and validity of any statements made within this article are not guaranteed. We accept no liability for any errors, omissions or representations. The copyright of this content belongs to the author and any liability with regards to infringement of intellectual property rights remains with them. The Early Career Voice blog is not intended to provide medical advice or treatment. Only your healthcare provider can provide that. The American Heart Association recommends that you consult your healthcare provider regarding your personal health matters. If you think you are having a heart attack, stroke or another emergency, please call 911 immediately.”
hidden

Can artificial intelligence save our lives?

Noora Alhajri, MD, MPH

The role of artificial intelligence (AI) in our life is advancing rapidly and is making strides in the early detection of diseases. The consumer market is composed of wearable health devices that enables continuous ambulatory monitoring of vital signs during daily life (at rest or physical activity), or in a clinical environment with the advantage of minimizing interference with normal human activities1. These devices can record a wide spectrum of vital signs, including: heart rate and rhythm, blood pressure, respiratory rate, blood oxygen saturation, blood glucose, skin perspiration, body temperature, in addition to motion evaluation. However, there is a lot of controversies whether these health devices are reliable and secure tools for early detection of arrhythmia in the general population2.

Atrial fibrillation (afib) is the most common arrhythmia currently affecting over 5 million individuals in the US and it’s expected to reach almost 15 million people by 2050. Afib is associated with an increased risk of stroke, heart failure, mortality, and represents a growing economic burden3. Afib represents a diagnostic challenge, it is often asymptomatic and is often diagnosed when a stroke occurs. Afib represents also a long term challenge and often involves hospitalization for cardioversion, cardiac ablation, trans-esophageal echo, anti-arrhythmic treatment, and permanent pacemaker placement. However, if afib is detected, the risk of stroke can be reduced by 75% with proper medical management and treatment3.

Physicians need fast and accurate technologies to detect cardiac events and assess the efficacy of treatment. A reliable, convenient and cost-effective tool for non-invasive afib detection is desirable. Several studies assessed the efficacy and feasibility of wearable technologies in detecting arrhythmias. The Cleveland Clinic conducted a clinical research where 50 healthy volunteers were enrolled. They tested 5 different wearable heart rate monitors including: (Apple Watch, Garmin Forerunner, TomTom Spark Cardio, and a chest monitor) across different types and intensities of exercises (treadmill, stationary bike and elliptical). The study found that the chest strap monitor was the most accurate in tracking the heart rate across different types and intensities of exercises4.

Apple and Stanford’s Apple Heart Study enrolled more than 419,297 Apple Watch and iPhone owners. Among these users, 2,161 (roughly 0.5%) received a notification of an irregular pulse. Of those who received the notifications, only about 450 participants scheduled a telemedicine consultation and returned a BioTelemetry ECG monitoring patch. When the Apple Watch notification and ECG patch were compared simultaneously, researchers found 71% positive predictive value, and about 84% of the cases were experiencing Afib at the time of the alert. Additionally, 34% of participants whose initial notification prompted an ECG patch delivery were later diagnosed with Afib. This finding shows that Apple watch detected afib in about one-third of the cases which is “good” for a screening tool considering the “intermittent nature of afib and that it may not occur for a whole week” says Dr. Christopher Granger, a professor of medicine at Duke University who participated on the steering committee for the Apple Heart study5.

These studies are observational studies and are not outcome-driven. They are not randomized and are not placebo-controlled. There are potentials for false negatives, where the Apple watch fails to detect the afib and false-positive where it detects arrhythmia that does not exist. Unfortunately, patients who are false negative don’t consult the physician about their symptoms of palpitations and shortness of breath since it provides false security. While patients with false-positive are sent unnecessarily to the clinic that could lead to further unnecessary tests and anxiety for the patient.

Is the Apple Watch ready to be used as a default screening tool to monitor the heart rate and rhythm in the general population and by physicians with patients with or at high risk for Afib is still unclear and warrant further studies. In conclusion, physicians should be cautious when using data from consumer devices to treat and diagnose patients.

The views, opinions and positions expressed within this blog are those of the author(s) alone and do not represent those of the American Heart Association. The accuracy, completeness and validity of any statements made within this article are not guaranteed. We accept no liability for any errors, omissions or representations. The copyright of this content belongs to the author and any liability with regards to infringement of intellectual property rights remains with them. The Early Career Voice blog is not intended to provide medical advice or treatment. Only your healthcare provider can provide that. The American Heart Association recommends that you consult your healthcare provider regarding your personal health matters. If you think you are having a heart attack, stroke or another emergency, please call 911 immediately.

References:

  1. Cheung, Christopher C., Krahn, Andrew D., Andrade, Jason G. The Emerging Role of Wearable Technologies in Detection of Arrhythmia. Canadian Journal of Cardiology. 2018;34(8):1083-1087. doi:10.1016/j.cjca.2018.05.003
  2. Dias D, Paulo Silva Cunha J. Wearable Health Devices-Vital Sign Monitoring, Systems and Technologies. Sensors (Basel). 2018;18(8):2414. Published 2018 Jul 25. doi:10.3390/s18082414
  3. Chugh, S., Sumeet, Havmoeller, J., Rasmus, Narayanan, F., Kumar, et al. Worldwide Epidemiology of Atrial Fibrillation: A Global Burden of Disease 2010 Study. Circulation. 2014;129(8):837-847. doi:10.1161/CIRCULATIONAHA.113.005119
  4. Wrist-Worn Heart Rate Monitors Less Accurate Than Standard Chest Strap. Medical Design Technology. http://search.proquest.com/docview/1875621494/. Published March 9, 2017.
  5. Turakhia, Mintu P., Desai, Manisha, Hedlin, Haley, et al. Rationale and design of a large-scale, app-based study to identify cardiac arrhythmias using a smartwatch: The Apple Heart Study. American Heart Journal. 2019;207:66-75. doi:10.1016/j.ahj.2018.09.002

 

 
hidden

Can artificial intelligence save our lives?

Noora Alhajri, MD, MPH

The role of artificial intelligence (AI) in our life is advancing rapidly and is making strides in early detection of diseases. The consumer market is composed of wearable health devices that enables continuous ambulatory monitoring of vital signs during daily life (at rest or physical activity), or in a clinical environment with the advantage of minimizing interference with normal human activities1. These devices can record a wide spectrum of vital signs including: heart rate and rhythm, blood pressure, respiratory rate, blood oxygen saturation, blood glucose, skin perspiration, body temperature, in addition to motion evaluation. However, there is a lot of controversy whether these health devices are reliable and secure tools for early detection of arrhythmia in the general population2.

Atrial fibrillation (afib) is the most common arrhythmia currently affecting over 5 million individuals in the US and it’s expected to reach almost 15 million people by 2050. Afib is associated with an increased risk of stroke, heart failure, mortality and represents a growing economic burden3. Afib represents a diagnostic challenge, it is often asymptomatic and is often diagnosed when a stroke occurs. Afib represents also a long term challenge and often involves hospitalization for cardioversion, cardiac ablation, trans-esophageal echo, anti-arrhythmic treatment, and permanent pacemaker placement. However, if afib is detected, the risk of stroke can be reduced by 75% with proper medical management and treatment3.

Physicians need fast and accurate technologies to detect cardiac events and assess the efficacy of treatment. A reliable, convenient and cost-effective tool for non-invasive afib detection is desirable. Several studies assessed the efficacy and feasibility of wearable technologies in detecting arrhythmias. The Cleveland clinic conducted a clinical research where 50 healthy volunteers were enrolled. They tested 5 different wearable heart rate monitors including: (Apple Watch, Garmin Forerunner, TomTom Spark Cardio, and a chest monitor) across different types and intensities of exercises (treadmill, stationary bike and elliptical). The study found that chest strap monitor was the most accurate in tracking the heart rate across different types and intensities of exercises4.

The Apple and Stanford’s apple Heart Study enrolled more than 419,297 Apple Watch and iPhone owners. Among these users 2,161 (roughly 0.5%) received a notification of an irregular pulse. Of those who received the notifications, only about 450 participants scheduled a telemedicine consultation and returned a BioTelemetry ECG monitoring patch. When the Apple Watch notification and ECG patch were compared simultaneously, researchers found 71% positive predictive value and about 84% of the cases were experiencing Afib at the time of the alert. Additionally, in 34% of participants whose initial notification prompted an ECG patch delivery were later diagnosed with Afib. This finding shows that Apple watch detected afib in about one third of the cases which is “good” for a screening tool considering the “intermittent nature of afib and that it may not occur for a whole week” says Dr. Christopher Granger, a professor of medicine at Duke University who participated on the steering committee for the Apple Heart study5.

These studies are observational studies and are not outcome driven. They are not randomized and are not placebo controlled. There are potentials for false negatives, where the Apple watch fails to detect the afib and false positive where it detects arrhythmia that does not exist. Unfortunately, patients who are false negative don’t consult the physician about their symptoms of palpitations and shortness of breath since it provides false security. While patients with false positive are sent unnecessarily to the clinic that could lead to further unnecessarily test and anxiety for the patient.

Is the Apple Watch ready to be used as a default screening tool to monitor the heart rate and rhythm in the general population and by physicians with patients with or at high risk for Afib is still unclear and warrant further studies.  In conclusion, physicians should be cautious when using data from consumer devices to treat and diagnose patients.

 

References:

  1. Cheung, Christopher C., Krahn, Andrew D., Andrade, Jason G. The Emerging Role of Wearable Technologies in Detection of Arrhythmia. Canadian Journal of Cardiology. 2018;34(8):1083-1087. doi:10.1016/j.cjca.2018.05.003
  2. Dias D, Paulo Silva Cunha J. Wearable Health Devices-Vital Sign Monitoring, Systems and Technologies. Sensors (Basel). 2018;18(8):2414. Published 2018 Jul 25. doi:10.3390/s18082414
  3. Chugh, S., Sumeet, Havmoeller, J., Rasmus, Narayanan, F., Kumar, et al. Worldwide Epidemiology of Atrial Fibrillation: A Global Burden of Disease 2010 Study. Circulation. 2014;129(8):837-847. doi:10.1161/CIRCULATIONAHA.113.005119
  4. Wrist-Worn Heart Rate Monitors Less Accurate Than Standard Chest Strap. Medical Design Technology. http://search.proquest.com/docview/1875621494/. Published March 9, 2017.
  5. Turakhia, Mintu P., Desai, Manisha, Hedlin, Haley, et al. Rationale and design of a large-scale, app-based study to identify cardiac arrhythmias using a smartwatch: The Apple Heart Study. American Heart Journal. 2019;207:66-75. doi:10.1016/j.ahj.2018.09.002

 

 

The views, opinions and positions expressed within this blog are those of the author(s) alone and do not represent those of the American Heart Association. The accuracy, completeness and validity of any statements made within this article are not guaranteed. We accept no liability for any errors, omissions or representations. The copyright of this content belongs to the author and any liability with regards to infringement of intellectual property rights remains with them. The Early Career Voice blog is not intended to provide medical advice or treatment. Only your healthcare provider can provide that. The American Heart Association recommends that you consult your healthcare provider regarding your personal health matters. If you think you are having a heart attack, stroke or another emergency, please call 911 immediately.
hidden

Tech in Cardiology

John Chen, MD

Tech in Cardiology

On a recent flight from San Francisco, I found myself sitting in a dreaded middle seat.  To my left was a programmer typing way in Python, and to my right was an oncologist flipping through a slide set on chemotherapy trials.  While this may sound like the beginning of a bad joke, I remember this moment because it got me thinking about the influence of tech on medicine.  The purpose of my trip, by the way, was to interview for a fellowship position in cardiology, a specialty with arguably some of the most impressive tech.

 

Wearables

Not to discount advances in medical devices (e.g. leadless pacemakers, bioprosthetic valves), the emergence of consumer-facing wearable devices is as trendy as ever.  Google recently collaborated with AHA to build its fitness app (Google Fit), which uses algorithms to quantify physical activity in terms of “heart points.”1  The Apple Health app now incorporates EKG capabilities, allowing patients to record episodes of arrhythmias—something I have certainly witnessed in cardiology clinic.2

 

Big data

Big data is an increasingly prominent component of clinical research, and a number of joint ventures with medical and tech leaders have emerged.  One Brave Idea3 is a research collaboration between AHA and Verily (Alphabet’s life sciences division) which uses genomics to study coronary artery disease.  Meanwhile, Verily’s Project Baseline4 is a massive longitudinal observational study—a modern version of the Framingham Heart Study.

 

Artificial intelligence

AI could eventually play a prominent role in medical diagnosis and decision-making.  The Stanford Machine Learning Group5 has developed a neural network that outperforms cardiologists in diagnosing arrhythmias on EKG—a significant improvement on existing algorithms which are often unreliable.  AI also carries vast potential in radiologic interpretation.  Already, Veril is using machine learning to interpret retinal images not only to detect diabetic retinopathy and macular edema but also to extrapolate information about cardiovascular risk.6

 

EMR

Electronic medical records represent an obvious space for tech innovation.  Fast Healthcare Interoperability Resources (FHIR) are making it easier to share health information across our disjointed EMR systems.  Providers are now able to push health data directly to patients’ iPhones using Apple Health Records.7  One can only speculate whether we will see a legacy software giant compete directly in the EMR space.

 

Cardiology and the rest of medicine has long excelled at basic science and translational research, but digital tech is increasingly creeping in.  We are in a tech zeitgeist, and this is good for both patients and providers.

 

References:

  1. https://www.heart.org/en/news/2018/08/21/google-just-launched-heart-points-here-are-5-things-you-need-to-know
  2. https://www.apple.com/healthcare/site/docs/Apple_Watch_Arrhythmia_Detection.pdf
  3. https://www.onebraveidea.org/
  4. https://verily.com/projects/precision-medicine/baseline-study/
  5. https://stanfordmlgroup.github.io/projects/ecg/
  6. https://blog.verily.com/2018/02/eyes-window-into-heart-health.htm
  7. https://www.apple.com/healthcare/health-records/