A ground-breaking new tool that uses facial recognition technology to identify individual whales and dolphins in the wild has been developed by researchers at the University of Hawai’i at Manoa Hawai’i Institute of Marine Biology. The tool, which can identify 24 species of marine mammals, could be a game changer for the management and conservation of marine mammals in Hawaii.
The research, which has been published in the journal “Methods in Ecology and Evolution”, is based on a state-of-the-art technique used in human facial recognition and was initially created for a Kaggle competition organised by Happywhale.com. The competition challenged engineers to develop a tool that could individually identify whales and dolphins using an algorithm. The algorithm developed by the researchers can recognise the same individuals over time and enables researchers to study their behaviour and movement patterns. It can also help estimate population size and trends, which are crucial to conserving Hawaiian whales and dolphins. Not only that, but the tool will speed up the information-gathering process during surveys. Once researchers have taken a picture of an individual in the field, they can run the photos through the algorithm back in the lab to quickly identify the individual and help gather important data.
Fig. 1.
Sixteen photo–id images of cetaceans, each row contains four images of the same individual, showing the challenges and opportunities for a multi–species photo–id model. The model must learn to recognize individuals from a variety of angles, under different lighting conditions, and in different social situations. Additionally, it must transfer learning from one species, for example false killer whale Pseudorca crassidens(top row), to a similar species, for example killer whale Orcinus orca (second row), while distinguishing them from less similar species like humpback whale Megaptera novaeangliae (third row) and very different species, like southern right whale Eubalaena australis (bottom row).
The study was undertaken by six researchers from around the world who shared their valuable image data, representing six continents and 24 individual species, to advance cetacean research and conservation. The research was funded by the National Oceanic and Atmospheric Administration Quest Fellowship
Fig. 2.
A model of the training pipeline for multi–species photo–id of cetaceans. The top row consists of preprocessing steps, shown in example by an image of a common dolphin Delphinus delphis. Crops generated by the four object detection models are shown, as well as two examples of images generated by the data augmentation step. The bottom row shows the training steps of the image classification network, from the backbone to the neck to the classification head. See Section 2 for a description of each component in the pipeline.
This new tool provides a non-invasive means to observe whale and dolphin behaviour and could prove to be a valuable tool for researchers in the field of marine mammal ecology. With its high predictive performance for classifying many species of cetaceans, the tool is expected to perform well when applied to other multi-species catalogues in similar survey settings, such as camera trap surveys.
This tool is freely available as code in a GitHub repository, and as a graphical user interface via Happywhale.com. This level of accessibility will make photo-id research more accessible to grassroots organisations and more and will accelerate ecological and management-focused research. With this tool, researchers can better understand marine mammal behaviour and ecology both in Hawaii, and hopefully around the world.
Text adapted from: Patton, P. et al., 2023. A deep learning approach to photo–identification demonstrates high performance on two dozen cetacean species. Methods in Ecology and Evolution, 00, 1– 15. https://doi.org/10.1111/2041-210X.14167
and Mauinews.com
