Apps that identify plants can be as accurate as 4%

Smartphone apps that identify plants from photographs can be as accurate as 4%, which could put people looking for food at risk and also lead to endangered plants being wrongly labeled as weeds and eradicated.

Julie Peacock of the University of Leeds, UK, and her colleagues evaluated six of the most popular apps: Google Lens, iNaturalist, Leaf Snap, Pl@ntNet, Plant Snap and Seek. They attempted to identify 38 species of plants in their natural habitat, at four locations in Ireland, with each application. The team found that some apps performed extremely poorly, while even the best ones didn’t achieve 90% accuracy.

“There are many reasons why it’s important for apps to be accurate or for people to know that these apps are a guide, but certainly not perfect,” Peacock says. For example, people might mistakenly identify important native species as invasive and remove them from their gardens, or consume potentially dangerous wild plants, thinking it to be a harmless variety.

But Peacock doesn’t think people shouldn’t use these apps, as long as they understand the limitations. “They have huge potential for people to start engaging more with plants,” she says.

The apps use artificial intelligence algorithms trained on a large number of captioned plant photographs. During training, the AI ​​learns to not only recognize training photos, but also to spot similarities between them and new photographs, which allows them to identify plants.

Generally, the apps were all better at identifying flowers than leaves, which the researchers say is due to their greater variety of shapes and colors providing the AI ​​with more clues. But that wasn’t always the case. The iNaturalist app was only able to correctly identify 3.6% of the flowers and 6.8% of the leaves. Plant Snap correctly identified 35.7% of flowers and 17.1% of leaves. The highest accuracy was achieved by Pl@ntNet at 88.2%.

Alexis Joly of Inria in Montpellier, France, who is one of the researchers behind the non-profit Pl@ntNet project, said the app’s success was down to its datasets, which are sourced and categorized by botanists, scientists, and knowledgeable hobbyists, along with algorithms that attempt to balance common species biases and instead rank several likely candidates for each search.

“It is sometimes a thankless task because people would rather see one result with 100% confidence, even if it is not the correct one, rather than three possible species at 33% each, but which represent reality compared to the picture. taken,” he said. “But it looks like our strategy is paying off.”

Stephen Harris of the University of Oxford says Peacock’s concerns are valid and that he too has encountered problems with such apps and instead relies on a good reference book. The problem lies with images downloaded from the Internet which are often mislabeled, he says.

“People tend to take pictures of similar things. So you’ll get some plants that are really obvious and everybody wants to take a picture, whereas if you get some kind of really interesting plant but just happens to be a rambling little thing that doesn’t have very attractive flowers or what whatever, you won’t get a lot of footage out of it,” says Harris. “It’s very unlikely that people will be scrambling around in ponds, removing weeds from ponds and taking pictures of them.”

Google declined a request for an interview, while other app makers did not respond.