Garmin Smartwatch Calorie Accuracy Falls Short in New Study

A new study published in PLOS One has revealed that Garmin smartwatches exhibit the poorest calorie-counting accuracy among four leading brands tested. Researchers from Florida International University found that the Garmin Forerunner 955 model had an average error rate of 77.6% when compared to a lab-grade device measuring actual calorie expenditure. This significant inaccuracy raises questions for the millions of users who rely on these wearables for fitness and health tracking.

The scientific team evaluated the Apple Watch Series 8, Fitbit Sense 2, Samsung Galaxy Watch 5, and the Garmin Forerunner 955. To determine accuracy, participants completed a standardized 10-minute interval cycling session. During the exercise, each individual wore all four smartwatches simultaneously, with their positions rotated to mitigate any bias from wrist placement. The devices' calorie readings were then benchmarked against a COSMED K5, a sophisticated machine used in laboratories to precisely measure calorie burn by analyzing a person's respiration.

The findings placed Garmin at the bottom of the accuracy ranking, with an average calorie miss of 68.61 kilocalories. The Samsung Galaxy Watch 5 followed with a 66.5% error and an average miss of 56.76 kcal. The Apple Watch Series 8 demonstrated better performance, showing a 35.7% error and an average miss of 21.60 kcal. The Fitbit Sense 2 appeared to be the most accurate, with a reported 31.7% error and a mere 3.14 kcal average miss. However, this impressive figure for the Fitbit Sense 2 came with a significant caveat: researchers had to discard seven readings that were exceptionally inaccurate, some exceeding four times the true value.

Understanding the Implications for Fitness Tracking

The study's authors emphasize that while none of the consumer-grade devices tested achieved perfect accuracy, the discrepancies are particularly concerning for users aiming for precise calorie management. For instance, Garmin’s average miss of 68.61 kcal suggests a confidence interval between 53 and 84 kcal, a considerably wider range than other brands. This means a user might significantly over- or under-estimate their calorie expenditure, potentially impacting weight management or training plans.

Furthermore, the research indicated that the accuracy of calorie tracking diminishes as a user's body fat percentage increases. This trend was observed across all devices, but the error margin grew more rapidly for Garmin and Fitbit compared to Apple. This correlation suggests that individuals who might benefit most from accurate calorie tracking—those managing weight or working towards fitness goals—could be using the least reliable data. The study also noted that skin tone did not demonstrably affect the accuracy of the readings across the tested devices, a factor sometimes cited in other biometric technology research.

While the study focused on specific models launched in 2022, the underlying algorithms and sensor technology may still be present in newer iterations. The researchers concluded that current consumer smartwatches are not sufficiently reliable for precise weight management or scientific research purposes. However, they suggested that the trend of a user's own readings over time might still offer valuable insights into their activity levels and physiological responses, even if the absolute numbers are not perfectly accurate. This nuance is crucial for users who have established personal baselines with their devices.

The methodology involved 58 adults performing a consistent exercise protocol, ensuring a controlled environment for comparison. Despite rigorous testing, some devices occasionally failed to produce a reading altogether, with Fitbit and Samsung experiencing more instances of this than Apple. After excluding unusable data points, the number of valid readings varied, but the core findings regarding error margins remained consistent. The full dataset from this PLOS One study is available for public review, allowing for further scrutiny and analysis by the scientific community and the public.