AI food tracking apps need improvement to address accuracy and cultural diversity, says study

A new University of Sydney study has found that improved artificial intelligence (AI) training is required when developing nutrition apps that are used to track food intake or manage weight.

Researchers initially screened 800 apps before selecting 18 for further evaluation. These 18 apps, which included both AI-integrated and manual food-logging nutrition apps, were then assessed for their ability to recognize ingredients and estimate energy content.

The findings are published in the journal Nutrients.

Dr. Juliana Chen, lead author of the study and accredited practicing dietitian, lecturer and researcher in the Discipline of Nutrition and Dietetics at the University of Sydney, suggests that while AI-integrated apps offer convenience over manual food-logging, they should be used carefully.

“When patients or the public use apps to track food intake or manage weight, the process can often feel burdensome,” Dr. Chen said. “Adding AI features like food image recognition could make the process much easier for everyone.

“However, it is important to always double-check that the portion size detected matches what you ate. Some apps only identify the food, while others also estimate portion size and energy intake. So, for those undergoing weight loss, it is crucial to verify that the app’s estimates align with what you have eaten.”

A key part of the study was checking how accurate and adaptable these apps were across three different diet plans—Western, Asian, and recommended (based on the Australian Dietary Guidelines)—to ensure a range of cultural food preferences were considered.

Under Dr. Chen’s supervision, Master of Nutrition and Dietetics students Xinyi Li, Annabelle Yin and Ha Young Choi found that manual food-logging apps overestimated energy intake for the Western diet by an average of 1,040 kilojoules, while they underestimated energy intake for the Asian diet and the recommended diet by an average of 1,520 kilojoules and 944 kilojoules respectively.

In contrast, AI-integrated food apps often had difficulty accurately identifying energy content for mixed Asian dishes, for example, the calories for beef pho were overestimated by 49%, while pearl milk tea had calorie underestimations of up to 76%.

“Nutrition apps with AI-integration are generally better at detecting individual Western foods when they are separated on a plate,” said Dr. Chen who is also from the Charles Perkins Center. “However, they often struggle with mixed dishes, such as spaghetti bolognese or hamburgers.

“This issue is more common with Asian dishes, which usually contain a variety of mixed components that may not be found in the respective apps database, leading to possible errors when calculating the energy amount of a particular meal.”

Moving forward, the study recommends several steps for the improvement of nutrition apps. This includes ensuring that the educational content and advice provided by the apps are evidence-based and trustworthy, which can be achieved through collaboration with nutrition experts.

“To enhance the credibility and accuracy of nutrition apps, creators should engage dietitians in their development, train AI models with diverse food images—particularly for mixed and culturally varied dishes—expand food composition databases and educate users on capturing high-quality food images for better recognition accuracy,” said Dr. Chen.

“If you’re monitoring your health, such as managing high blood pressure or tracking your sodium intake, it’s important to compare your food choices with nutrition labels or consult with an accredited practicing dietitian. A dietitian’s expertise is invaluable in these cases, as they can provide more accurate estimates of how much energy your body is consuming and what it requires most to achieve a holistically healthy diet.”

This assessment was conducted using the Mobile App Rating Scale (MARS) and the App Behavior Change Scale (ABACUS).

Following the evaluation, “Noom” received an average score of 4.44 out of 5 on the MARS scale, meaning it was rated very highly in terms of engagement, functionality, aesthetics, and information quality. It also received a perfect 21/21 ABACUS rating for incorporating many features that promoted behavioral change, goal setting, tracking and educational content.

Among the other AI-powered apps, “MyFitnessPal” and “Fastic” successfully recognized a sample of 22 images of various foods and beverages, achieving success rates of 97% and 92%, respectively.