Camera traps capture a wealth of data, offering insights into animal behaviour, population dynamics, and habitat use by wildlife. However, the sheer volume of photos can be daunting, as each must be carefully reviewed to identify the species captured.
To address this challenge, the Tasmanian Land Conservancy created WildTracker, enlisting the help of citizen scientists. Still, we know the workload can be overwhelming for landholders! That’s where artificial intelligence (AI) steps in to lighten the load.
We are delighted to introduce Stickybeak, a custom AI-powered infrastructure integrated into WildTracker.
Stickybeak begins by detecting whether an animal is present in a camera trap photo, filtering out any that are caused by false triggers (e.g. shadows or grass blowing in the wind), as well as images of people or vehicles. It then predicts which species are present in the photos.
Importantly, citizen scientists remain central to the process. You can accept or reject the AI’s classification or flag a photo for expert review. In doing so, your lovely human brain’s oversight will help enhance the AI model’s accuracy.
Behind the Scenes
Stickybeak has been designed so that any effective wildlife classification model can be ‘plugged in’, ensuring the program remains adaptable in what is a rapidly developing field of technology.
Currently at the heart of StickyBeak is an impressive model developed by researchers from the DEEP (Dynamics of Eco-Evolutionary Patterns) lab at the University of Tasmania. The Mega-efficient Wildlife Classification (MEWC) model uses cutting-edge computer vision techniques to classify wildlife photos. Currently, it includes parameters to classify 80 species (or classes, e.g. snakes or insects), with a planned 2025 update to add another 20 or so species.
What is Computer Vision?
Computer vision is a field of AI focused on teaching computers to interpret and understand the visual world. By using algorithms like convolutional neural networks (CNNs), computers can analyse images, recognise patterns, and identify objects, similar to how humans process visual information. Applications range from facial recognition systems to medical imaging – and now, wildlife monitoring.
How the MEWC Model Works
The research team at the University of Tasmania leveraged cutting-edge computer vision techniques, inspired by the visual cortex in animals. These techniques involve deep-learning, a subset of machine learning that has revolutionised computer vision. CNNs are a specific kind of deep-learning algorithm designed specifically for processing structured grid data (like pixels), making them powerful for image analysis and enabling computers to learn from vast amounts of training data. These networks mimic the human brain’s visual processing, allowing for highly accurate object recognition, classification, and detection.
Training the model involved feeding it millions of labelled camera trap images where species have been identified by experts. The model learns from these examples, improving its ability to classify new, unseen images accurately. This training process uses a technique called supervised learning, where the model iteratively adjusts its parameters to minimise prediction errors.
The model’s workflow includes four key steps:
1. Detection: After pre-processing the images (e.g. resizing), MEWC employs another machine learning model, the open source MegaDetector, developed by Microsoft’s AI Earth team. MegaDetector passes the image through a CNN, generating a heatmap indicating likely animals, people or vehicles being present in various regions of the image. If no features are detected, the images are placed in a ‘blank’ folder. Blank images submitted to WildTracker are moved to a cold cloud storage server where they will still exist but won’t clog up our database. The output from MegaDetector depends on a threshold in the model’s confidence level, set to balance between capturing animals and ignoring rocks or lumps of wood. Getting this threshold right, is perhaps one of the trickiest parts of the workflow.
2. Snipping: Features are then extracted from the larger image and resized once more, ready to be classified. The snips below show examples of what the AI “sees” and highlight that these models do not have a sense of scale like we do when classifying species.
3. Prediction: MEWC offers a choice of CNN models to classify each image snip, predicting the species present. It assigns a probability score to each possible species and ranks them accordingly. For example, an object might be predicted with 60% confidence as a pademelon, 30% confidence as a wallaby, 5% confidence as a bettong and so on. This step’s accuracy depends greatly on the quality of training data. Fortunately, MEWC has seen A LOT of Tasmanian animals in different postures, lighting, and coat colours.
4. Annotation: Species classifications and confidence levels are written into the image metadata, enabling WildTracker to display these details alongside bounding boxes from MegaDetector.
As AI technology continues to evolve, the MEWC model and others like it will become even more powerful and accurate. By integrating AI into the WildTracker citizen science program, we have a unique opportunity for human feedback to refine models. WildTracker participants will be able to either accept or reject the AI classification or refer it to an expert (one of us ecologists at TLC) for validation.
If all this wasn’t nerdy enough for you, the detailed architecture of MEWC and how it is being deployed can be found in the open access preprint.