Hi everyone,
I have a project I’ve been working on for some time. I’m trying to see if it’s possible to use ECG data (and just ECG data, no other inputs) to predict atrial fibrillation events with the help of machine learning. Ideally it would be great to be able to predict them at least 30-60 minutes ahead of time. Medical AI is tricky because both precision and recall need to be pretty high to actually be useful in a clinical setting, if the recall isn’t high enough no one will trust the model to replace skilled nurse check-ins, and if the precision isn’t high enough it will lead to too many unnecessary alarms and overburden staff.
I have a fairly robust dataset, a few hundred thousand hours of ECG data and around 3000 distinct A fib events. My intent is to take a time series segment (probably around 30-60 minutes), feed it to some kind of classifier, and then output a prediction of the likelihood that an event will occur. I’ve tried a number of different approaches and so far have had minimal success, with my top-performing classifier only giving about 60% balanced class accuracy. Obviously this is far below the threshold of utility.
I’m starting to wonder if I’ve been approaching the problem too simply - a number of different issues can cause Afib and lumping them all together as the “positive” class may dilute the signals I’m trying to detect. So I’m thinking perhaps I should see if the events cluster in ways that reflect the underlying physiological differences and then use a multiclass approach that predicts one of the causes instead.
I’ve been reading a bit and it seems like using KNN with a dynamic time warping metric might be a good way to do this but I have no experience using this time of unsupervised clustering approach. I’m also unclear how to deal with the fact that I don’t actually know how many clusters there will be in the data, everything I’ve read so far suggests that you need to tell KNN first how many clusters there will be.
Any help would be appreciated!
IMHO your time windows are too long for the models to learn. You might try shortening them a lot, and then labeling all those within N minutes from the event as positive, see what happens. Depends also on what the sampling rate is, if your are feedimg raw ECG to models.
As for the clusters, yes, you could try hierarchical clustering with time warping distances, but it will take lots of time.
Simple question: how do medical experts recognize the upcoming event? Can they do it based on several minutes of ECG several minutes before the event?
Thanks! I was kind of shooting in the dark on the size of the windows. I can try shortening but the sample rate is only one per minute, so I wanted to make sure there was enough signal for the model to extract useful data.
As for your second question, right now it’s not really possible for a human to predict that far out - there’s ECG instabilities that start to show up but usually only a few minutes before the event at most. My hypothesis was that subtle signals that might not be obvious to a human could be machine detectable, but it’s entirely possible that isn’t true.
If you don’t know how many of the 3000 events are detectable by an expert, how do you know your 60% classifier isn’t better than an expert already?
It might be. It could end up in that unfortunate area of “statistically significant but not clinically meaningful,” meaning it might do what it’s supposed to but not well enough to merit actually using it in the real world.
(minor point, DTW is not a metric, it is only a measure)
What you are trying to do is precursor discovery [a].
-
It is unlikely that the 30 to 60 minue lead time will work for you, it seems too ambitions.
-
DTW is a great shape measure, it it is unlikely that it will generalize from person to person. In fact, if you change the location of the leads (assuming you are using 12-lead or similar ), it is unlikely to generalize for a single person, one day to another.
-
You may wish to consider trying to solve this problem in the feature space, not the shape space. The catch22 feature set would be a great starting point [b].
[a] https://www.cs.ucr.edu/~eamonn/WeaklyLabeledTimeSeries.pdf
-
As both a medical doctor and someone completing a PhD in ML applied to healthcare, I’m a little sceptical about the desired outcome. For paroxysmal atrial fibrillation (pAF), I really doubt you would get a clear signal from the available data predicting onset 30-60 minutes before the event itself - there are far too many complex, stochastic factors at play, and it is unlikely to have such a long lead time. At best, you might stratify into higher risk periods of devolving into AF.
But even if you succeeded perfectly, I’m not convinced that predicting onset of AF half an hour in advance is of much use to a clinician. I’m struggling to think of how I might use this information to alter management of my patient (as an emergency physician), just because of the nuances of current AF management. I only mention this because the task as described is already a very tricky one, and I worry the resulting value added might be disproportionately small relative to the effort expended… I could be wrong though, just my two cents!