- A new study from Stanford Medicine finds that fitness wearables can measure heart rate with 95 percent accuracy, but fail to accurately count calories burned. Devices produced error rates between 23 percent and 93 percent, indicating that the wearables may not be very useful for making decisions about diet and exercise.
The research team devised the study to explore the reliability of popular wearables when measuring important health data such as heart rate (HR) and energy expenditure (EE), a term synonymous with calories burned.
More and more people rely on their fitness wearables to educate them on their health and assist in their health-centered decisions. If devices continue to deliver poor data, patients may be misled when relying on their wearables to make self-care choices.
“People are basing life decisions on the data provided by these devices,” said Euan Ashley, DPhil, and professor of cardiovascular medicine, of genetics and biomedical data science at Stanford. “But consumer devices aren’t held to the same standards as medical-grade devices, and it’s hard for doctors to know what to make of heart-rate data and other data from a patient’s wearable device,” he said.
In order to measure the accuracy of fitness wearables, the team provided a diverse user population with seven different models of devices. These devices included popular items such as the Apple Watch, Basis Peak, Fitbit Surge, Microsoft Band, Mio Alpha 2, PulseOn and the Samsung Gear S2.
Users then were assigned various physical tasks including walking, cycling, sitting, and standing. At the same time, the participants were measured with a medical-grade electrocardiograph (ECG) to see how the wearables stacked up to clinically accurate data.
The lowest error in measuring HR was observed for the cycling task at rates of 1.8 percent while the highest error was observed for the walking task at 5.5 percent. Of the seven devices in regards to any physical activity, the Apple Watch achieved the lowest error in HR at 2 percent while the Samsung Gear S2 had the highest HR error with 6.8 percent error.
For EE, the lowest relative error rates across all devices were achieved for the walking (31.8 percent error) and running (31 percent error) tasks. The highest error rates when recording EE occurred during the sitting tasks with 52.4 percent error. Median error rates across tasks varied from the 27.4 percent mark for the Fitbit Surge to 92.6 percent for the PulseOn.
“The heart rate measurements performed far better than we expected,but the energy expenditure measures were way off the mark,” Ashley said. “The magnitude of just how bad they were surprised me.”
Following the completion of their study, Ashley and his team created an online forum where other users can share their fitness wearable data, and look through the data gathered in the study.
The next step for the researchers will involve a similar study, where individuals will generate wearable data during their daily routines outside of a clinical lab. By doing so, the team hopes that they can further inform the public on the reliability of fitness wearables.