Computer vision-based automated cow lameness estimation using machine learning and synthetic data

Abstract

Lameness in cattle is a commonly encountered condition stemming from pain in one or more hoofs or limbs, which not only affects the animal’s movement, but also their productivity as well. Lameness can cause discomfort for cattle, reducing their quality of life, and can also lead to reduced milk production or reproductive issues. Farms with cattle exhibiting lameness can experience increased veterinary costs and economic loss. Therefore, it is important to monitor the lameness levels of cattle so that they can be treated as soon as possible.

This dissertation discusses lameness in dairy cows, including various scoring systems and methods used to estimate the lameness scores of cows. The dissertation then delves into new computer vision-based algorithms that were developed to extract features such as the back arch, head, and hoof movement from cows in video recordings. These algorithms make use of You Only Look Once object detectors to detect different parts of the cow’s body to track the relevant features over time, which were then used in machine learning algorithms including neural networks and support vector machines to estimate the lameness score of the cows.

The dissertation demonstrates that the lack of training data featuring lame cows is a problem and proposes a lameness simulator created in Unreal Engine 5 to simulate cows with different lameness scores as they move in a straight line past a stationary camera.

The simulations validate various machine learning algorithms explained in the dissertation, and the trained machine learning models achieved an acceptable balanced accuracy for the estimated lameness scores. For the two-class lameness estimation, the best balanced accuracy score achieved was 92.65%. For the three-class lameness estimation, a balanced accuracy score 71.9% was achieved when using features from the proposed cow lameness simulator. For the four-class lameness estimation, the best balanced accuracy achieved when using features from the proposed cow lameness simulator was 65.09%. The estimation accuracy significantly drops with an increase in the number of possible lameness levels, due to the highly overlapping nature of the features between the classes. In all cases, the results were on par or better than those in published papers, although direct comparison is difficult due to the different possible scoring systems.