Perspectives on the use of unmanned aerial systems to monitor cattle

Abstract

The use of unmanned aerial systems (UASs) in agriculture has been growing steadily in the last decade, but their use to monitor and count cattle has been very limited. This article analyses the reasons for this apparent lack of progress, considering both the technical challenges and the difficulties in defining target users who would benefit from a UAS-based system for monitoring cattle. Such an analysis is combined with the findings reported in several investigations dedicated to counting and monitoring wildlife to draw a comprehensive picture on the current situation, to suggest possible solutions to technical issues and to delineate applications that could be useful to both cattle farmers and governments. The text concludes by showing that there are unexplored viable uses for UAS in livestock monitoring, especially in countries like Brazil, where extensive stockbreeding prevails.

Keywords

Livestock image processing drones UAS UAV counting

Introduction

Monitoring livestock population is an essential part of the farm management. However, this may not be a trivial task, especially in very large properties adopting extensive stockbreeding, which is very common in countries like Brazil. In this context, aerial surveys arise as a potential solution. Satellite images are not well suited for this task, because most sensors do not have enough spatial resolution to resolve individual animals – sensors such as GeoEye-1 and WorldView could theoretically deliver enough resolution (Xue et al., 2017), but even in this case cattle would be represented by only a few pixels, and the cost of the images is still very high. Furthermore, cloud contamination can obscure features of interest (Anderson and Gaston, 2013).

Using manned aircraft for surveying cattle farms, although technically feasible, has a number of drawbacks associated: operation costs are high, elevated noise levels associated with most aircraft can disturb animals (Chrétien et al., 2015; Christie et al., 2016), accidents can cause loss of human life (Chabot and Bird, 2015; Chrétien et al., 2015) and aircraft are not always suited for the installation of image sensors. Given the limitations associated with satellites and manned aircraft, the use of unmanned aerial system (UAS) appears as a more viable option to tackle the cattle monitoring issue (Zhang and Kovacs, 2012).

Most UASs are lightweight, low-cost aircraft platforms consisting of an aircraft component (unmanned aerial vehicle (UAV) also known as drone), sensor payloads and a ground control station (Anderson and Gaston, 2013; Watts et al., 2012). Basically, there are two types of drones that can be used in agricultural applications: rotary, which are very portable but have limited sensor payload capabilities; and fixed-wing platforms, which tend to be faster and have better payload capabilities, but are usually less portable and more expensive to acquire and operate. One of the main advantages of UAS is that they come in such a wide variety of configurations and capabilities (Hogan et al., 2017) that technical requirements attached to any given application are likely to be met by some available system. Although extrinsic factors such as costs involved and government regulations may discourage UAS use (Watts et al., 2012), prices continue to fall and many regulatory barriers are being removed, so adoption levels are expected to increase in the near future (Hogan et al., 2017).

UASs have been used in agricultural applications for some time, especially in the context of precision agriculture (Beloev, 2016; Hunt et al., 2014; Zhang and Kovacs, 2012). Currently, the only country to adopt them in large scale is Japan. This is facilitated by the small average farm size, in which case small electric rotary wing systems are more cost-effective (Freeman and Freeland, 2015). However, in the last few years, there has been a steep growth in countries like the United States, where agricultural applications already account for 19% of the whole UAS market (Hogan et al., 2017).

The use of UAS for monitoring livestock, and cattle in particular, has been limited. A few academic investigations on this subject have been dedicated to animal detection and counting (Chamoso et al., 2014; Longmore et al., 2017), cattle round-up (Jung and Ariyur, 2017), feeding behaviour (Nyamuryekunge et al., 2016) and health monitoring (Webb et al., 2017). There have been also some patents deposited (Horton and Vorpahl, 2017a, 2017b; Trumbull and Myrtle, 2017). The reasons for this apparent lack of progress on the subject range from technical difficulties to regulation limitations, as discussed in the next section.

In contrast, the use of drones to monitor wildlife is steadily increasing (Chabot and Bird, 2015; Christie et al., 2016; Gonzalez et al., 2016; Linchant et al., 2015). In the specific case of terrestrial mammals, there are investigations dedicated to deer (Chrétien et al., 2015, 2016; Franke et al., 2012; Israel, 2011; Witczuk et al., 2017), elk (Chrétien et al., 2015), hippopotamus (Lhoest et al., 2015), rhinoceros (Mulero-Pázmány et al., 2014),elephants (Vermeulen et al., 2013) and so on. Many of the conditions and challenges faced by those studies are applicable to the problem of cattle monitoring. Because of that, part of the reasoning presented throughout the text draws elements from those studies.

This article provides a comprehensive discussion on the main technical, practical and regulatory challenges and barriers that still prevent the use of UAS for cattle monitoring, especially in the case of animal counting. Possible solutions are proposed, always focusing on small rotary and fixed-wing aircraft, and on automatic methods to detect and count animals. Some possible clients to a UAS-based monitoring system are also identified, and respective potential benefits are highlighted.

UAS and sensors relevant for cattle monitoring

In this text, the acronym UAS is used whenever referring to the whole system, while UAV refers to the aircraft itself.

UAV sizes range from a few grams (e.g. AV Nano Hummingbird) to hundreds of kilograms (e.g. NASA Ikhana) (Watts et al., 2012), and their maximum payload is usually proportional to their weight. For agricultural application, intermediate sizes usually offer the best cost/benefit ratio, but the best choice will ultimately depend on the sensors needed for the intended application and on budgetary constraints (Anderson and Gaston, 2013). Despite their lower endurance, electrical UAVs are usually preferred over those using fossil fuel, as they are more practical, silent and stable (Linchant et al., 2015).

UAV types and characteristics

Rotary UAVs are small helicopter-like aircraft that have four (quadcopter) to eight (octocopter) sets of rotating blades, arranged either around a central body or along two opposing arms (Anderson and Gaston, 2013). This kind of UAV has a few advantages: they can hover over fixed targets, can stay airborne even when one of the rotors loses power, exhibit less vibration than fixed-wing systems and are easier to operate and tend to be cheaper. On the other hand, they are slow, can carry only a limited payload, can cover only relatively small areas (Anderson and Gaston, 2013) and are unstable with winds above 25 km/h (Miller et al., 2017).

Fixed-wing UAVs are airplane-like aircrafts that can typically travel up to a few kilometres from the launch site, although regulations usually require that a visual line of sight from the operator be maintained permanently (Anderson and Gaston, 2013). Fixed-wing aircraft are typically hand-launched and land on their belly, thus minimizing components required for launch and recovery (Linchant et al., 2015). These UAVs are usually faster, use less power, are better suited to strong wind conditions and can cover larger areas and carry more payload than the rotary ones (Miller et al., 2017). However, they tend to be more expensive, more difficult to operate, less manoeuvrable and images captured tend to exhibit stronger smearing effects, both due to aircraft speed and associated vibrations.

Although UAVs can be flown manually, professional uses rely primarily on integrated flight systems that enable better precision, stability and replicability. This is achieved by the inclusion of GPS-enabled autopilots, inertial measurement units, battery monitor systems and emergency landing systems. Flights can now be planned and executed through tablet or smartphone applications, and autopilot systems can be altered mid-flight (Hogan et al., 2017). All technical aspects of UAS are being continuously improved, making them easier to use, less prone to mechanical failure and more robust to user error.

Sensor types and characteristics

There is a wide variety of sensors that can be attached to a UAS, with weight being the only major practical constraint. With the constant miniaturization of sensors and reduction of costs associated, it is now possible to deploy sophisticated instruments using lightweight, low-cost UAS, although deploying multiple sensors simultaneously is still largely unfeasible (Anderson and Gaston, 2013). There are a few sensors (mostly imaging devices) that can provide useful information for livestock monitoring:

RGB: These are conventional cameras that capture images using three components of the visible spectrum (red, green and blue) and are mainly used for the creation of true colour orthomosaics (Hogan et al., 2017). These sensors tend to be cheap, and the resulting images are a close depiction of the way humans would perceive the scene.

Thermal: This type of camera detects variations in heat using the long-wavelength infrared band. They can be used to detect livestock (Longmore et al., 2017) or wild animals (Miller et al., 2017; Witczuk et al., 2017), as those usually have higher temperatures than their surroundings. Another advantage of thermal cameras is that they can be used during night-time (Linchant et al., 2015). On the other hand, they tend to have much lower spatial resolution than other types of sensors (Chabot and Bird, 2015).

Multispectral: These cameras capture images at specific wavelengths (four to ten), some of which located in the visible band, and the remaining ones located either in the near-infrared band (more used for vegetation detection) or in the thermal band (animals). Some features of the objects of interest may be more prominent in specific wavelengths, which can be explored to improve the detection process. Also, bands may be combined in order to reveal other types of relevant information (e.g. normalized difference vegetation index). Multispectral sensors may be used to identify and count animals, as different species may have different spectral signatures (Terletzky et al., 2012). These sensors tend to have a lower spatial resolution than the RGB ones (Chabot and Bird, 2015).

Hyperspectral: As in the case of multispectral cameras, this kind of sensor captures images at specific wavelengths, but in this case having a much higher spectral resolution (hundreds of wavelengths). These sensors tend to be very expensive, and the spectral resolution provided is not needed for animal detection and counting; however, they may be useful to detect subtler traits, such as breed and the presence of disease.

Video cameras: This kind of sensor yields a more manageable single output file (Chabot and Bird, 2015) and is especially suitable to detect movement and track individual subjects (Fang et al., 2016). In the case of cattle counting, however, high-resolution still images are more appropriate.

Using UAS for cattle monitoring – barriers and possible solutions

There are several factors that need to be considered regarding the use of UAS for cattle monitoring. Those factors are here divided into six classes: aircraft, sensor, environmental, operational, image capture and processing and specific factors. This division is an expansion of the classification suggested by Anderson and Gaston (2013).

Aircraft factors

Payload and battery capacity

As discussed earlier, lightweight UAVs have limited payload capacity. This affects not only the sensors that can be deployed but also constrains the size of the batteries and, as a result, limits flying time (Anderson and Gaston, 2013). Considering that UAS-based cattle monitoring would be most useful in large properties, these limitations become very relevant. There are a few solutions for this problem, although all of them also have some shortcomings associated:

Fly higher: Given that flying time and speed are limited, larger areas can be covered by increasing flying altitude. Most UAVs can reach altitudes of at least a few hundred metres, but regulations may impose some limitations (for details, see the ‘Operational factors’ section). Higher altitudes also mean that the objects of interest (animals) will be represented by fewer pixels, making their identification more challenging (for details, see the ‘Image capture and processing’ section). Winds also tend to be more intense, increasing vibration (degrading image quality) and probability of system loss (for details, see the ‘Environmental factors’ section). Additionally, since the aircraft will be farther away from the operation centre, loss of communication becomes more likely. With the exception of regulation limitations, all of the problems mentioned are manageable with careful planning, monitoring and testing.

Go bigger: Larger UAVs usually have higher payload limits, allowing the use of larger batteries that increase flying times. Larger UAVs also tend to be faster and fly higher (regulations may hamper this advantage), further increasing the maximum coverage attainable in each mission. These aircraft also tend to be expensive, require special launching conditions, and may also require a specific pilot licence to be operated; in Brazil, such licence is required for UAVs weighing more than 25 kg (for details, see the ‘Operational factors’ section).

Fly in formation: Theoretically, large areas could be covered through several missions. The problem with this approach for cattle monitoring is that animals can move in the time between missions (for details, see the ‘Specific factors’ section), which would make counting unreliable. One possible way to mitigate this problem would be flying several UAVs in formation, capturing the images simultaneously. This option entails two problems. First, the cost of acquiring many UASs may be too high compared with the resulting benefit. Second, although several studies have shown the feasibility of flying UAVs in formation (Sharma and Kumar, 2015; Wang et al., 2017), there are still many technical issues to be resolved (Andre et al., 2014).

Explore solar energy: Installing solar panels in UAVs could greatly increase flight times. There have been some initiatives to mount solar panels in fixed-wing UAVs (Anderson and Gaston, 2013; Sri et al., 2016), and there are some companies, such as Alta Devices (Sunnyvale, California, USA), that are investing in this technology.Although considerable progress has been made for large UAVs, the application of this technology to small UAVs is still too incipient to be used in practical applications.

As technology evolves, some of the problems related to payload capacity are minimized, and more suitable solutions arise. It is also worth noting that governments are beginning to employ powerful UAS for surveillance and inspections; as a side task, those missions could gather data suitable for animal population estimation, which is of interest for both producers and the government.

Landing

Landing UAVs in rough terrain, which is common in cattle areas, may be challenging, especially in the case of fixed-wing aircraft. If not planned properly, gliding to a landing position may cause damage to aircraft and sensors. To minimize damage in case of unsuccessful landing, it is recommended to keep aircraft and payload weight substantially below nominal limits (Anderson and Gaston, 2013). As fixed-wing UAVs become more accessible, compact and transportable (Colefax et al., 2017), risks are reduced and landing requirements become less stringent.

Costs

Small UAS capable of handling most agricultural applications in general cost less than US$10.000, and prices are still falling (Hogan et al., 2017). However, cattle farms often are large, in which case more advanced UAS, capable of covering larger areas and gathering more information in less time, may be necessary (Freeman and Freeland, 2015). Additionally, low-end UAVs are usually more prone to mechanical failure, which may cause damage not only to the aircraft but also to the sensors (Mulero-Pázmány et al., 2014). Considering that the threat of crashes is always present (see the ‘Environmental factors’ section), costs may become too high in comparison with the potential gain. Other potential sources of costs include the software needed to process and interpret the images (Miller et al., 2017), the computational infrastructure to store the data collected, the training required to operate the whole system and the man-hours spent to set up and carry on the surveys. Ultimately, costs and benefits will strongly depend on the characteristics of the properties and on the intended uses, so a careful economical and technical analysis is recommended before deciding whether UASs are advantageous or not. It is important to consider that the UAS industry is far from mature, with technologies evolving rapidly, costs steadily falling and regulations changing towards a better balance between safety and usability. As a result, viability studies should be carried out regularly in order to take the latest changes into account. It is also worth pointing out that aerial surveys using UAS can be offered as a service by third-party companies, greatly reducing risks and costs if surveys do not need to be carried out too frequently.

Sensor factors

The small payload capacity of lightweight UAVs limits the size and amount of sensors to be deployed. As sensors are miniaturized, the payload issue becomes less limiting. However, in the specific case of imaging sensors, there is a trade-off between miniaturization and data quality (Roy and Miller, 2017). This has to be taken into account for cattle monitoring, because depending on the image resolution, optical distortions may render animal identification unfeasible.

In general, only a single sensor can be deployed in each flight. Thus, if the intention is to employ multiple sensors, they have to be deployed in separate flights. However, since animals may move between flights, it may be difficult to combine the information gathered by each sensor. In such a context, techniques that rely on multiple data sources to perform animal identification and counting may not be reliable.

Automating the image analysis is essential for the viability of UAS for livestock monitoring (see the ‘Image capture and processing’ section). Thus, it is important to consider that techniques to analyse images in the visible range have received much more attention and are considerably more mature than techniques exploring other bands of the spectrum (Longmore et al., 2017). Thus, the use of different types of sensors may require that some effort be spent on the development of algorithms suitable for the task at hand (for details, see the ‘Image capture and processing’ section).

Cost is also an important factor when choosing the sensors to be deployed. Although there are many lightweight, low-cost RGB sensors available, they do not always deliver the optical quality required by some applications. Also, more specialized sensors such as thermal, multispectral and hyperspectral tend to be considerably more expensive than the RGB ones. As a result, it is always important to consider the trade-off between the potential gain in accuracy and the cost associated with more sophisticated sensors.

Environmental factors

The threat of crashes and equipment loss is always present, as there are many hazards that can cause a UAV to crash: high winds, birds of prey, power lines, trees, signal loss and so on. Although careful planning and monitoring can greatly reduce the risk, incidents are sometimes unavoidable, potentially causing damage to aircraft and sensors. Also, in many cases, the crashed equipment can be retrieved with ease, conditions such as rough terrain and dense canopy cover may make it difficult, or even prevent aircraft recovery (Anderson and Gaston, 2013). Since equipment damage and loss are relatively common, it is recommended that spare parts, and even extra UAVs, be always available for replacement, especially when there are time constraints for completing the survey.

Besides increasing the risk of crashes, high-speed winds may cause yaw, pitch and roll movements and affect the speed of the aircraft. They also require more work from stabilization mechanisms, thus increasing energy consumption and reducing mission endurance (Chrétien et al., 2015). Due to their small size and weight, rotary UAVs are particularly vulnerable to wind, but the impact on the quality of the images will depend largely on the equipment being used – UAVs with more blades seem more robust to adverse weather conditions (Goebel et al., 2015). In any case, angular movements may alter the overlapping between images and deflect sensors from nadir, thus damaging the mosaicking process and introducing a variety of distortions. Correcting such complex distortions is not a trivial task and may require the use of reference mosaics (Chrétien et al., 2015). To reduce the effects of angular motion caused by aircraft movement, some authors recommended that sensors be mounted in three-axis gyrostabilized gimbals (Chrétien et al., 2015). However, some movement of the camera relative to the ground will inevitably happen as a result of sensor error and response latency (Harvey et al., 2016).

Operational factors

UAVs may be challenging to pilot and the learning curve may be steep, even if GPS-enabled navigation is included (Anderson and Gaston, 2013). Take-off and landing, in particular, require some proficiency to avoid incidents. Rotary aircraft are in general easier to operate due to the vertical take-off and landing. Large aircraft, on the other hand, may require special training and a pilot licence – in Brazil, any UAV weighing more than 25 kg can only be operated by a licensed pilot (National Civil Aviation Agency (ANAC), 2017).

Many investigations on the use of UAS in agriculture mention the difficulty in integrating all different expertises to properly extract and explore relevant information from the data collected. In particular, farmers might lack the necessary skills to extract reliable information from UAS images, and remote sensing scientists may be unfamiliar with the field and crop conditions (Zhang and Kovacs, 2012). However, in the case of animal counting, the problem is well posed and does not depend strongly on field characteristics and conditions. Additionally, automatic methods for scene analysis are very feasible (see the ‘Image capture and processing’ section; Barbedo, 2012), so no complex analysis or interpretations are needed.

The strict regulatory rules that are still in force in most countries around the globe are among the most important barriers for the adoption of UAS, no matter the application (Freeman and Freeland, 2015). Motivations behind these restrictions include safety of people and security against the misuse of UAS (Watts et al., 2012). As the technology evolves and the uses of UAS are better understood, regulations tend to achieve a better balance between security and practical use viability. A summary of the main rules in force in the United States, European Union (EU) and Brazil is presented below. It is important to emphasize, however, that regulations are constantly changing, so it is always recommended to check the most current documents on the subject.

In the United States, the Federal Aviation Administration (FAA) regulates the use of UAS. The following rules should be observed when using UAS for cattle monitoring (Federal Aviation Administration, 2018): a remote pilot certification should be obtained, which requires the person to be at least 16 years old, pass an aeronautical knowledge test at an FAA-approved knowledge testing centre, and undergo a Transportation Safety Administration security screening; the aircraft must be registered upon payment of a US$5 fee; a special exemption is needed for aircraft weighing more than 25 kg (payload included); fly in uncontrolled (class G) airspace; keep the aircraft within visual line of sight; fly at or below 120 m; fly at or under 160 km/h; yield right of way to manned aircraft; do not fly directly over people; and do not fly from a moving vehicle, unless in sparsely populated areas.

In the EU, until the end of 2017 the regulation of unmanned aerial systems (UAS)” with maximum take-off mass of 150 kg was the competence of each member state. In 22 December 2017, a political agreement extended the EU competence to cover the regulation of all civil unmanned aerial systems (UAS), regardless of their maximum take-off masses (European Aviation Safety Agency (EASA), 2018). By mid-2018, the final text for the unified regulations was still being discussed, but the probable rules to be enforced had already been defined, as described next. In order to fit the Open category, which has less stringent rules, the aircraft should weigh less than 25 kg, only be flown within visual line of sight and not be flown higher than 120 m above ground level. Most operations for cattle counting are likely to fit the A3 subcategory (Fly Far from People), which has some specific rules: fly in an area where it is reasonably expected that no uninvolved people will be endangered; keep a safety distance from urban areas; pilot should undergo online training and testing; UAV should possess lost-link management, selectable height limit and frangibility; the minimum age to operate the aircraft should be defined by each EU member state (EASA, 2018).

In Brazil, UAS use is regulated by the National Civil Aviation Agency. The main rules currently in force are the following (ANAC, 2017): a pilot licence is needed to operate aircraft over 25 kg, UAVs must maintain a distance of at least 30 m (horizontally) from people, unless authorized otherwise; special authorization from the airspace authority is needed if the aircraft is expected to reach altitudes above 120 m and/or if the aircraft is expected to leave visual line of sight in any moment; fully autonomous flights, in which the pilot cannot intervene at any moment, are not allowed.

As it can be seen, many of the rules are common to most countries. Among those, arguably the one that has the greatest impact in cattle monitoring is the need to keep a visual line of sight at all times. Considering that many cattle farms are extensive, broad surveys may be rendered unfeasible by this constraint. Although exemptions to this requirement may be granted, the process is complex and tends to be slow. It is also worth noting that this situation is common to most countries (Chrétien et al., 2016; Colefax et al., 2017; Watts et al., 2012).

Image capture and processing

Images captured during UAV flights are subject to a series of geometric distortions that need to be corrected (Zhang and Kovacs, 2012). This action allows the images to be properly combined into a single picture through mosaicking, so the scene can be interpreted correctly. This whole process is not trivial (Lisein et al., 2013) and has the potential to introduce errors as the one depicted in Figure 1, especially if cheaper cameras prone to pincushion and barrel effects are employed (Miller et al., 2017). In order to partially counteract possible errors, high levels of image overlapping are adopted, which increases the amount of data to be stored and processed. This leads to another problem: building image mosaics is a computationally intensive task, especially if many images are present (Zhang and Kovacs, 2012). Thus, if time is an important factor, considerable computational power needs to be available for a timely response. There are many other problems that are common to all remotely sensed images: instrument calibration, atmospheric correction, vignetting correction, line-shift correction, band-to-band registration and frame mosaicking (Zhang and Kovacs, 2012). However, which of those need to be addressed and which correction techniques must be applied will depend on the application. In some instances, relevant information will clearly come through most distortions, obviating the need for sophisticated and computationally expensive correction tools. This fact also stresses the importance of conducting thorough investigations to clearly outline the requirements of each specific application.

Figure 1.

Part of the animal is deleted by the mosaicking process.

Most mosaicking algorithms rely on distinctive features to properly build the mosaics. Therefore, if the scene being analysed is predominantly homogeneous, errors may occur. Vegetation areas, and pastures in particular, may not have enough variations or reference objects to properly guide the mosaicking process. Also, wind may change the details found in pictures of the same areas taken at different moments, further stressing the ability of mosaicking algorithms to find the correspondence between images. The robustness to relatively featureless and varying scenes is highly dependent on the algorithm being used, so some tests may be necessary to find the best option.

Another problem that may occur is the loss of images due to data corruption. If the images are not stored in the aircraft itself, communication link failures may also cause loss of information. This type of problem may occur even if very reliable equipment is employed. Using redundant systems is not a good option due to payload limitations and costs involved. A practical way to avoid that some areas be completely devoid of data associated is to ensure that all images are captured with at least 50% overlap between them, thus guaranteeing that every point on the ground is imaged at least twice, at the expense of more data to be processed.

A direct conclusion that can be drawn from the problems discussed earlier is that the fewer the number of image captures needed to cover a given area, the less damaging will be those issues. In other words, images should be captured from the highest possible altitudes with minimum overlap between them. The minimum level of overlapping will depend directly on the characteristics of the terrain, on the robustness of the mosaicking algorithm and on how critical it is to avoid areas with missing data. The ideal height is the maximum altitude above which the sensors of choice no longer deliver enough resolution for robust identification of the objects of interest (Longmore et al., 2017), as long as legal limits are observed. Although the best possible set-up can only be attained by carefully studying the specific characteristics of each survey, it should be possible to derive some general guidelines that are a reasonably good fit in most cases.

Counting animals manually is prohibitively time-consuming (Longmore et al., 2017) and prone to psychological and cognitive phenomena that may lead to bias and optical illusions (Barbedo, 2016). As a result, using automatic methods to analyse the images and extract relevant information is crucial. The automatic identification and counting of animals using remote sensing images is not trivial, even if the images are completely free of distortions and animals do not move. Pixel values are very sensitive to changes in illumination, camera conditions and object orientation (Chamoso et al., 2014). Also, shadows cast both by target objects and other structures in the scene may hamper the detection and segmentation process. Thus, finding a stable pattern that fully characterizes the objects of interest is unfeasible. Instead, automatic methods should be flexible enough to deal with a high degree of condition variations. Some authors indicate that using object-based image analysis may counteract some of the problems associated with pixel value variability (Chrétien et al., 2016). Another alternative that is quickly gaining momentum is using deep learning techniques, which try to model high-level abstractions in data by learning patterns from large-scale unlabelled data. In the specific case of remote sensing data, this kind of technique has the potential to achieve good results without explicitly taking into account the many factors that influence the classification (Goodfellow et al., 2016). It is important to emphasize, however, that this type of technique requires large image databases to be properly trained, which may be difficult to obtain.

Among deep learning tools, arguably the most commonly used are the convolutional neural networks (CNNs) (Krizhevsky et al., 2012). This kind of neural network requires fewer artificial neurons than conventional feedforward neural networks, being particularly suitable for image recognition. CNNs usually require a very large number of samples to be trained; however, in many real-world applications, it is expensive or unfeasible to collect the training data needed by the models (Pan and Yang, 2010). Thus, many authors are applying the concept of transfer learning to reuse pretrained networks (e.g. GoogLeNet and AlexNet), in which case predictions are done on examples that are not from the same distribution as the training data (Bengio, 2012). The conjunction of deep learning and transfer learning, together with the development of graphics processing units, has provided a powerful tool for animal recognition and counting (Chamoso et al., 2014). Other techniques used in image analysis for animal detection include the k-nearest neighbours and support vector machine classifiers (Gemert et al., 2015; Smit, 2016), Iterative Self-Organizing Analysis Technique (ISODATA) (Terletzky and Ramsey, 2016), mathematical morphology (Fang et al., 2016), deformable part-based model (Gemert et al., 2015), pixel intensity thresholding (Gonzalez et al., 2016), template matching (Gonzalez et al., 2016) and background subtraction (Weinstein, 2017).

As mentioned in the ‘Sensor types and characteristics’ section, thermal sensors have good potential for cattle monitoring. Many of the observations above also apply to this kind of sensor, but there are also some specific considerations that should be made. To be effective, object detection using thermal sensors requires that the difference in temperature between the object and its surroundings be detectable; the larger such a difference, the more likely is the correct detection. Thus, it is recommended that image captures be carried out when ground temperatures are lower (night and early morning), especially during warmer seasons (Witczuk et al., 2017). It is also important to consider that heat emissions vary during the day due to both physiological and environmental aspects (Chrétien et al., 2016), which may cause confusion with other elements in the image.

Specific factors

As mentioned earlier, animals move over time. Thus, unless the whole area to be surveyed is imaged in a single snapshot, which is only possible with satellites, animals will shift positions as images are captured. However, as discussed earlier, satellite images are not very suitable for livestock monitoring. As a result, some animals can appear in more than one image (Witczuk et al., 2017), others may not appear at all, and others may have only part of their bodies imaged. This, together with all technical aspects involved in the capture and processing of the images (see the ‘Image capture and processing’ section), makes it very difficult to obtain the exact number of animals in a given area, either by counting manually or by automatic methods. However, if good approximations are to be obtained, some correction factor to compensate for all sources of errors must be developed and applied whenever appropriate (Chrétien et al., 2015). This will require a research effort in which all factors, from animal movement patterns to the impact of mosaicking errors, will have to be investigated.

The time of the day in which images are captured also play an important role. Animals tend to seek shelter under trees when the temperature is high, so planning missions for the hottest periods of the day is not adequate. This recommendation is even more important in the case of thermal images, because ground objects such as rocks may become very hot and be wrongly detected as an animal (Chrétien et al., 2016). It is also important to consider that shadows increase in size the closer to sunrise or sunset. There are other physiological processes that regulate the way animals move, all of which must be taken into account in order to select the best times for the flights. It is worth noting that the behaviour of cows does not seem to be affected by the presence of drones (Longmore et al., 2017; Nyamuryekunge et al., 2016).

Even if missions are carefully planned to maximize the number of animals visible from above, areas containing trees will most certainly have at least a few animals hidden beneath the canopies. This fact, which in most cases is unavoidable, has to be taken into consideration for accurate estimates. Statistical models have been shown to improve the ability to estimate the distribution of organisms (Martin et al., 2012). If properly fed with relevant information (canopy cover, weather, topography, etc.), this type of tool may be very useful in the context of cattle counting.

Detection using RGB images rely on colour differences between animals and surroundings. In Brazil, the most common cattle breed is the Nelore, which is generally of white colour, providing a good contrast with the ground. Other breeds, however, may have colours similar to the surrounding vegetation. This can significantly increase the level of difficulty of the automatic detection task, to the point where it is necessary to develop specific algorithms to deal with those particular conditions (Linchant et al., 2015). Another possible solution would be using sensors that explore other parts of the spectrum (multispectral, thermal). Another factor that may cause problems to automated counting tools is that cattle sometimes tend to group together (Figure 2), so some rules of object separation must be applied in order to avoid error.

Figure 2.

Animals grouped together require specific segmentation.

Finally, it is worth mentioning that, in the case of small properties and confined cattle, other monitoring techniques, particularly those based on radio-frequency identification (Adrion et al., 2017), may be more reliable and cost-effective than UAS.

Conclusion

Currently, the vast potential of UAS as a tool to help cattle farmers to manage their properties cannot be fully realized due to technical and practical issues that still need better solutions and especially due to regulations that are still overly stringent. However, market pressure and technological evolution will probably lead to a relaxation of the rules, a process that has already started in many countries.

This work aimed at identifying the main barriers to the use of UAS in cattle farming, proposing some solutions for current problems and presenting some perspectives on future developments. The main conclusion that can be drawn from the literature and from the authors’ personal observations is that there is a sense of inevitability when it comes to UAS applied to agricultural tasks: as technical, practical and regulatory problems are overcome, it is likely that the use of UAS will surge. A study carried out in the United States indicated that many cattle farmers especially the younger, well-educated ones are willing to adopt the UAS technology as soon as it becomes a viable option (Allmon, 2013). When this happens, it is important that expectations be kept at realistic levels to avoid the well-known hype-disillusionment cycle, which delays or even prevents a technology from becoming widespread (Freeman and Freeland, 2015).

The UAS technology is still in its infancy. It is also evolving so rapidly that scientists and legislators are having trouble keeping up with the pace of technical advancements, which is necessary for realizing the full potential of this technology. With plenty of scientific questions still unanswered, this represents an excellent opportunity for researchers to explore. In turn, early adopters among farmers may gain competitive advantages and become familiarized with the technology before it becomes widespread.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Jayme Garcia Arnal Barbedo

References

Adrion

Kapun

Holland

. (2017) Novel approach to determine the influence of pig and cattle ears on the performance of passive UHF-RFID ear tags. Computers and Electronics in Agriculture 140:168–179.

Allmon

(2013) Will cattle producers be willing to adopt electronic cattle monitoring systems? Master’s Thesis , Oklahoma State University, USA.

Anderson

Gaston

(2013) Lightweight unmanned aerial vehicles will revolutionize spatial ecology. Frontiers in Ecology and the Environment 11:138–146.

Andre

Hummel

Schoellig

. (2014) Application-driven design of aerial communication networks. IEEE Communications Magazine 52:1298–1137.

Barbedo

JGA

(2012) A review on methods for automatic counting of objects in digital images. IEEE Latin America Transactions 10:2112–2124.

Barbedo

JGA

(2016) A review on the main challenges in automatic plant disease identification based on visible range images. Biosystems Engineering 144:52–60.

Beloev

(2016) A review on current and emerging application possibilities for unmanned aerial vehicles. Acta Technologica Agriculturae 19:70–76.

Benjio

(2012) Deep learning of representations for unsupervised and transfer learning. In: Guyon

Dror

Lemaire

(eds) Proceedings of ICML Workshop on Unsupervised and Transfer Learning. Vol. 27. Bellevue, Washington, USA: PMLR, pp. 17–36.

Chabot

Bird

(2015) Wildlife research and management methods in the 21st century: where do unmanned aircraft fit in? Journal of Unmanned Vehicle Systems 3:137–155.

10.

Chamoso

Raveane

Parra

. (2014) UAVs applied to the counting and monitoring of animals. Advances in Intelligent Systems and Computing 291:71–80.

11.

Chrétien

Théau

Ménard

(2015) Wildlife multispecies remote sensing using visible and thermal infrared imagery acquired from an unmanned aerial vehicle (UAV). In: International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Toronto, Canada, 30 August–02 September 2015, pp. 241–248.

12.

Chrétien

Théau

Ménard

(2016) Visible and thermal infrared remote sensing for the detection of white-tailed deer using an unmanned aerial system. Tools and Technology 40:181–191.

13.

Christie

Gilbert

Brown

. (2016) Unmanned aircraft systems in wildlife research: current and future applications of a transformative technology. Frontiers in Ecology and the Environment 14:241–251.

14.

Colefax

Butcher

Kelaher

(2017) The potential for unmanned aerial vehicles (UAVs) to conduct marine fauna surveys in place of manned aircraft. ICES Journal of Marine Science 75:1–8. DOI: 10.1093/icesjms/fsx100.

15.

European Aviation Safety Agency (EASA) (2018) Opinion no 01/2018 – introduction of a regulatory framework for the operation of unmanned aircraft systems in the ‘open’ and ‘specific’ categories. Available at: https://www.faa.gov/uas/getting_started/part_107/ (accessed 17 May 2018).

16.

Fang

Abdoola

. (2016) Motion based animal detection in aerial videos. Procedia Computer Science 92:13–17.

17.

Federal Aviation Administration (FAA) (2018) Fly under the small UAS rule. Available at: https://www.faa.gov/uas/getting_started/part_107/ (accessed 17 May 2018).

18.

Franke

Goll

Hohmann

. (2012) Aerial ungulate surveys with a combination of infrared and high-resolution natural colour images. Animal Biodiversity and Conservation 35:285–293.

19.

Freeman

Freeland

(2015) Agricultural UAVs in the US: potential, policy, and hype. Remote Sensing Applications: Society and Environment 2:35–43.

20.

Gemert

Verschoor

Mettes

. (2015) Nature conservation drones for automatic localization and counting of animals. Lecture Notes in Computer Science 8925:255–270.

21.

Goebel

Perryman

Hinke

. (2015) A small unmanned aerial system for estimating abundance and size of Antarctic predators. Polar Biology 38:619–630.

22.

Gonzalez

Montes

Puig

. (2016) Unmanned aerial vehicles (UAVs) and artificial intelligence revolutionizing wildlife monitoring and conservation. Sensors 16:97.

23.

Goodfellow

Bengio

Courville

(2016) Deep Learning, 1st ed. Cambridge: MIT Press.

24.

Harvey

Roskilly

Buse

. (2016) Determining position, velocity and acceleration of free-ranging animals with a low-cost unmanned aerial system. Journal of Experimental Biology 219:2687–2692.

25.

Hogan

Kelly

Stark

. (2017) Unmanned aerial systems for agriculture and natural resources. California Agriculture 71:5–14.

26.

Horton

Vorpahl

(2017a) Agricultural drone for use in livestock feeding. U.S. Patent Application 20170086429. Available at: https://patents.google.com/patent/US20170086429 (accessed 13 June 2018).

27.

Horton

Vorpahl

(2017b) Agricultural drone for use in livestock monitoring. U.S. Patent Application 20170086428. Available at: https://patents.google.com/patent/WO2017053135A1/en (accessed 13 June 2018).

28.

Hunt

Jr Daughtry

CST

Mirsky

. (2014) Remote sensing with simulated unmanned aircraft imagery for precision agriculture applications. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 7:4566–4571.

29.

Israel

(2011) A UAV-based roe deer fawn detection system. In: International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Munich, Germany, 5–7 October 2011, pp. 51–55.

30.

Jung

Ariyur

(2017) Strategic cattle roundup using multiple quadrotor UAVs. International Journal of Aeronautical and Space Sciences 18:315–326.

31.

Krizhevsky

Sutskever

Hinton

(2012) ImageNet classification with deep convolutional neural networks. In: Proceedings of the Annual Conference on Neural Information Processing Systems, Lake Tahoe, Nevada, USA, 3–8 December 2012, pp. 1106–1114.

32.

Lhoest

Linchant

Quevauvillers

. (2015) How many hippos (HOMHIP): algorithm for automatic counts of animals with infra-red thermal imagery from UAV. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XL-3/W3:355–362.

33.

Linchant

Lisein

Semeki

. (2015) Are unmanned aircraft systems (UASs) the future of wildlife monitoring? A review of accomplishments and challenges. Mammal Review 45:239–252.

34.

Lisein

Linchant

Lejeune

. (2013) Aerial surveys using an unmanned aerial system (UAS): comparison of different methods for estimating the surface area of sampling strips. Tropical Conservation Science 6:506–520.

35.

Longmore

Collins

Pfeifer

. (2017) Adapting astronomical source detection software to help detect animals in thermal images obtained by unmanned aerial systems. International Journal of Remote Sensing 38:2623–2638.

36.

Martin

Edwards

Burgess

. (2012) Estimating distribution of hidden objects with drones: from tennis balls to manatees. PLoS ONE 7:e38882.

37.

Miller

Adkins

Tully

(2017) Providing aerial images through UAVs. Fact Sheet FS-1056. Available at: https://drum.lib.umd.edu/handle/1903/19168 (accessed 13 June 2018).

38.

Mulero-Pázmány

Stolper

Essen

. (2014) Remotely piloted aircraft systems as a rhinoceros anti-poaching tool in Africa. PLoS ONE 9:e83873.

39.

National Civil Aviation Agency (ANAC) (2017) Regulamento brasileiro de aviao civil especial – RBAC – e n 94. Available at: http://www.anac.gov.br (accessed 17 May 2018).

40.

Nyamuryekunge

Cibils

Estell

. (2016) Use of an unmanned aerial vehicle – mounted video camera to assess feeding behavior of Raramuri Criollo cows. Rangeland Ecology & Management 69:386–389.

41.

Pan

Yang

(2010) A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering 22:1345–1359.

42.

Roy

Miller

(2017) Miniaturization of image sensors: the role of innovations in complementary technologies in overcoming technological trade-offs associated with product innovation. Journal of Engineering and Technology Management 44:58–69.

43.

Sharma

Kumar

(2015) A cooperative network framework for multi-UAV guided ground ad hoc networks. Journal of Intelligent & Robotic Systems 77:629–652.

44.

Smit

(2016) Automatic animal detection using unmanned aerial vehicles in natural environments. Master’s Thesis, University of Groningen, The Netherlands.

45.

Sri

KRB

Aneesh

Bhanu

. (2016) Design analysis of solar-powered unmanned aerial vehicle. Journal of Aerospace Technology and Management 8:397–407.

46.

Terletzky

Ramsey

(2016) Comparison of three techniques to identify and count individual animals in aerial imagery. Journal of Signal and Information Processing 7:123–135.

47.

Terletzky

Ramsey

Neale

CMU

(2012) Spectral characteristics of domestic and wild mammals. GIScience & Remote Sensing 49:597–608.

48.

Trumbull

Myrtle

(2017) Unmanned livestock monitoring system and methods of use. U.S. Patent Application 20170202185. Available at: https://patents.google.com/patent/WO2017127188A1/en (accessed 13 June 2018).

49.

Vermeulen

Lejeune

Lisein

. (2013) Unmanned aerial survey of elephants. PLoS ONE 8:e54700.

50.

Wang

Jiang

Han

. (2017) Taking drones to the next level: cooperative distributed unmanned-aerial-vehicular networks for small and mini drones. IEEE Vehicular Technology Magazine 12:73–82.

51.

Watts

Ambrosia

Hinkley

(2012) Unmanned aircraft systems in remote sensing and scientific research: classification and considerations of use. Remote Sensing 4:1671–1692.

52.

Webb

Mehlhorn

Smartt

(2017) Developing protocols for using a UAV to monitor herd health. ASABE Annual International Meeting, Spokane, Washington, USA: ASABE.

53.

Weinstein

(2017) A computer vision for animal ecology. Journal of Animal Ecology 87(3):533–545.

54.

Witczuk

Pagacz

Zmarz

. (2017) Exploring the feasibility of unmanned aerial vehicles and thermal imaging for ungulate surveys in forests – preliminary results. International Journal of Remote Sensing. DOI: 10.1080/01431161.2017.1390621.

55.

Xue

Wang

Skidmore

(2017) Automatic counting of large mammals from very high resolution panchromatic satellite imagery. Remote Sensing 9:878.

56.

Zhang

Kovacs

(2012) The application of small unmanned aerial systems for precision agriculture: a review. Precision Agriculture 13:693–712.