New AI Tool Can Take Cattle's Temperature With Only a Photo

What if you could look into a cow's face and know if it had a fever? A new tool from the Artificial Intelligence and Computer Vision Lab at the University of Arkansas uses artificial intelligence and thermal cameras to estimate the body temperature of cattle.  

The system, called CattleFever, is the first step toward automated tools that ranchers could use to monitor the health of their herd. 

Trong Thang Pham, a doctoral student at the U of A, was the primary researcher on the project. The AICV lab is led by Ngan Le, an associate professor of electrical engineering and computer science, who researches medical imaging, computer vision and robotics. 

Today, cattle's temperature is measured rectally, a process that can stress the animal. The CattleFever system could both improve animal welfare and reduce the labor needed to track a herd's health. The technology could help ranchers detect diseases before symptoms appear, leading to earlier treatment and preventing outbreaks. 

BUILDING THE DATASET 

To build CattleFever, the researchers first needed data. A number of datasets exist for dogs, cats, horses and sheep. The existing dataset of cattle, called CattleEyeView, only includes overhead photos of the animals and was built for tracking herds. The existing animal datasets were also mainly RGB photos of the animals, and for CattleFever, the researchers also needed thermal images. 

The researchers had to build their own dataset with short videos and thermal images of thousands of calves. Each calf was held in a pen, then 20 seconds of both video footage and thermal images were recorded. Each animal's temperature was also recorded with a rectal thermometer to create a benchmark, or ground truth. 

The calf data was collected at the Arkansas Agricultural Experiment Station's Savoy Research Complex. 

The RGB photos of the calves, which clearly show the animal's features, then needed to be linked to the thermal image on the computer. The researchers marked the photos with 13 landmarks, such as eyes, ears, muzzle and mouth. The team manually annotated 600 frames, which were then used to train an AI tool that automatically labeled the remaining 4,000 frames in a dataset, named CattleFace-RGBT. The resulting landmark-detection tool can automatically localize a calf's face and identify its key facial features across RGB and thermal modalities. 

TESTING THE TOOL 

Once they had the dataset, could a computer determine the temperature of the calf from the images alone? 

Through extensive ablation studies examining different facial-landmark combinations, the researchers realized the temperature of the animal's eyes and nostrils were closest to the reading of the rectal thermometer. Using facial landmarks, the computer focused on the temperature of the thermal images from those spots. 

The researchers then examined the data with various machine-learning approaches to determine the animal's body temperature from those surface readings. The most accurate results came from a random forest regression, a machine learning approach to predict results from complex data. In a random forest regression, many decision trees are created, each one trained on a different portion of the data. Then all the results of these decision trees are averaged together, which helps reduce noise. 

The CattleFever system was able to automatically determine an animal temperature within 1 degree of the reading from a thermometer. 

INTO THE FIELDS? 

The researchers demonstrated that cattle's temperature could be accurately read from a thermal image, but all the images were taken with the animal directly facing the camera. 

"We probably need to take more photos of them in the real-world settings, such as running around, to capture their motion in the field," Pham said. 

The next challenge is to handle cattle faces captured from diverse angles and natural poses, essentially teaching the computer to recognize and interpret a cow's face in real-world field environments, rather than only when the cow is positioned directly in front of the camera inside a pen. 

The U of A researchers have publicly shared their CattleFace-RGBT dataset, so other researchers can build on their work and help develop a system that ranchers could use. 

"If we find something new, we share that with the world. That's the spirit," Pham said. 

The results of the CattleFever research were published in the journal Smart Agricultural Technology. Pham was the first author, and Le was the senior author. The other authors were Ethan Coffman from Le's AICV Lab and Beth Kegley, Jeremy G. Powell and Jiangchao Zhao of the Department of Animal Science in the Dale Bumpers College of Agricultural, Food and Life Sciences.