Autonomous Agriculture - Integration of Robotics and Mobile Machine Technologies

Darcy Cook, VP of Engineering/GM, JCA Technologies

The case for autonomous machines in agriculture has been written and talked about many times, and it has become clear in the last few years that there is significant growth of activity in development and deployment of these machines. The momentum can be clearly seen with the launch and announcement of many new early phase machines, and with most of the larger OEMs also working behind the scenes on innovations that will increasingly become apparent in the years to come.

It is no longer a question of if autonomous agricultural machines will become a near-term reality, but rather how quickly will this become prevalent across different applications, what will this look like, and what is the impact on the market. These are complex questions that will continue to take shape with the market, but to get a better insight into these, it is useful to understand the underlying technologies that are enabling this change.

At JCA our journey towards autonomous agricultural machines came first through the development of agricultural implement machine controls across many different applications, growing towards more advanced connected precision agriculture applications, and leading towards autonomous machines. Our first autonomous machine development started in 2016, when there were very few organizations working in this area, and so we have been part of the rapid progression of technology as this community has grown. Over this time, we have worked on several different autonomous machines across widely different applications, and have seen patterns in the underlying technology that will have enormous implications on the agricultural market as a whole. Our perspective on building towards autonomy, coming from a background focused on the end applications and machines that are involved in the large variety of agricultural jobs, has provided some insights about the technology worlds that are coming together to enable these autonomous machines in agriculture.

Merging of Technology Paradigms

There is a lot of depth to the technology that is utilized by autonomous machines, but some important insights can be drawn from understanding that there are three major technology paradigms that are merging in the development of autonomous machines. These are:

  • Off-Highway Mobile Machine Controls – Off-highway mobile machine controls has developed over more than 30 years since the introduction of electronics and software systems to machine control systems. The products from this technology paradigm consist of ECUs and displays that are real-time microcontroller-based systems, with primary functions that are typically reading simple sensors (pressure, temperature, angle positions), driving hydraulic valves, and communicating with other modules over CAN/J1939. Many of the products in this market are application (and industry) independent, with mature software development tools that are used to define the machine-specific functions. As a result, leading companies focused primarily in this area typically have divorced their engineering teams from applications as they are trying to reach a cross-industry market, with development focused on reducing product costs and increasing reliability with a march towards increased volumes and commoditized products.
  • Precision Ag Technologies – Precision ag technologies have developed over more than 20 years and are built on the core technology of GNSS/GPS based guidance. This enabling technology could increase the accuracy of input applications (e.g. seeds in planting, pesticides/herbicides in spraying), as well as use detailed yield data in harvesting that allowed for further optimization through agronomic analysis. This has resulted in functions such as auto-steer, overlap/section control, as-applied mapping, and prescription application. Precision ag technologies have also provided a core piece of data that has resulted in the fast growth of the farm management software industry that provides more value to producers through advanced analytics and operational organization. Precision ag technologies are much more application specific than mobile machine controls, with products focused specifically on planting/seeding, spraying, and harvesting solutions. While many of the functions coming from these technologies are now commonly deployed and mature, there is a high amount of new development towards connected machines and facilitating data to interface with farm management information systems (FMIS).
  • Robotics Technologies – Robotics technologies consist of a wide array of technologies that are critical enablers of autonomous machines. These include sensing and perceptions systems (e.g. LIDAR, cameras) combined with algorithms for object detection and recognition (such as machine learning vision systems), advanced processing platforms built on multi-core GPU and CPU systems, multi-modal communication systems for machine-to-machine and machine-to-user remote connectivity, guidance/navigation/controls systems that can be applied to many new and complex use cases, and many other technologies. The complexity of these technologies is significantly greater than the other two technology paradigms, requiring both new methods and software systems to manage these across multiple machines and platforms. Technologies in this area have been developed mostly from low volume and research focused organization (e.g. universities), and from other emerging industries (e.g. autonomous cars). Many of these technologies have not yet been in scaled production systems in any industry, with much of the effort in the technology development being put towards evolving these technologies from research systems to scalable production systems.

Each of these technology paradigms have strengths and weaknesses and drawing from each of these to create a new technology paradigm for autonomous agricultural systems is needed for truly scalable autonomous machines. Autonomous agricultural machines require the reliability, ruggedness, and functional safety capabilities of mobile machine controls, the task management and integration with FMIS from precision ag, and the advanced technologies and systems to manage complexity that robotics technologies bring.

What it Means to Merge Technology Paradigms

At JCA, we have worked through the hard problems involved in autonomous agricultural machine development and have taken an approach of both working with our customers to solve the end application problems while also investing in the technology to build a common platform that can be used to build autonomous machines across any application. JCA’s Autonomous Framework (AFW) consists of technologies that are not unique to the specific machine, but are common across any autonomous agricultural machine. The AFW was developed so that the OEM would not have to reinvent the wheel, thereby lowering the barriers for OEMs to develop unique machines for their applications. We have been drawing from the three technology paradigms to create a cohesive platform that addresses all of the varied systems needs. On this journey we came to key conclusions that are not initially obvious, some of which are shared in here.

System is more than the Sum of Its Parts

Autonomous agricultural machines require systems that have a significant increase in complexity over traditional machine control systems. Managing this complexity requires a systematic approach where the design of the overall system architecture require much more careful consideration than for simpler systems. A strong system architecture can be more valuable than the use of all the best components within a poor system organization. The concept is similar to people working in organizations/companies. Good people can be effective in almost any structure within a small organization as they can effectively communicate with everyone else without too much overhead; here the organization structure is not of great importance. However, in a large organization the system is more complex and a poor organization structure could prevent effective operation even with good people in the company. The system architecture (in this case the org structure) becomes a much more critical factor for success than only having good people as it allows (or prevents) effective communication and operations; in this case both good people and a strong organization are key.

Similarly, mobile machine controls and precision ag systems have consisted of a relatively small number of core functions that have been integrated into a few ECUs within a given system. With most of the functions handled within the ECUs themselves, interfaces (including communication and message definition) between components of the systems can be well defined and relatively straightforward, allowing for potential for interchangeability of components. This drives component-focused thinking where the system designer’s goal is to determine the best set of components that meet the function, quality, and cost needs of the system. The component-focused thinking has driven standards to define common messaging (e.g. J1939, ISOBUS) allowing for increased choice of different components that could be used for the same functions creating more interchangeability. While this works well when the number of components/functions in a system are relatively small, the problem becomes unmanageable with increased system complexity. This is due to a well-known principle of network complexity that the number of links or interfaces in a system grows at a nonlinear rate (technically is geometric growth). For example, a system with 2 major components that need information from each other has 1 interface to be defined, a system with 4 components has 6 interfaces, and a system with 10 components has 45 interfaces to be defined (for N components, the number of interfaces can be calculated as N(N-1)/2). Defining detailed universal interfaces between components can quickly become an unmanageable task for complex systems. In autonomous machine systems these “components” may be physical electronics platforms, but they might also be software processes, cloud systems, different machines, or any processing element that needs information from another element. The “interfaces” refer to information passing between these components and may be physical communication interfaces, messaging definition, or software APIs. Robotics technologies have dealt with this problem through architectural changes in software systems that better facilitate interfacing with these complex interconnected systems (which can be seen in approaches used in ROS/ROS2 for example), but this drives a more holistic system-focus. System-focused thinking is required to manage this problem, where the system architecture is part of the solution to manage complexity. This still (of course) requires definition of interfaces between components in a system, but also requires components that are designed to work together within a particular system architecture that have more complex dependencies. In practical terms, it means that machines that have a mix-match of many different components that were not developed with a common architecture in mind will suffer from many more integration challenges. These integration challenges will grow geometrically, and can result in these becoming overwhelming to making progress in development and support of a system.

Complexity Drives New Methods and Architectures

The methods used for development of mobile machine controls and precision ag systems have been refined over several decades to establish some staples of good practice that include methods such as model-based design, use of finite state machines, and rigorous software development and verification methods driven from functional safety requirements such as coding standards, static analysis methods, and code coverage reporting. These methods require tight control and rigor over all parts of software in a system, and while they have been shown to produce increased reliability that is needed for safety systems, they also come with a significant overhead. Scaling those same methods to autonomous machines across the board would mean that an army of engineers would be required for every system developed, and this just becomes impractical. This is another area where methods coming from the robotics industries can be drawn from to determine how to deal with this problem. Robotics has struggled with increased complexity for a long time and has developed both architectures and methods that facilitate rapid development, but with maintaining the robustness and reliability required for mission critical systems. One example of this, which is not unique to robotics, is to draw from open-source software for mission critical systems, but within defined architectures such as ROS2. The use of open-source software comes with the challenge of software that is developed without known processes, but has shown to achieve its robustness and reliability not through controlled processes, but rather heavy community contribution, review, and use. This approach has been adopted by other mission critical industries, with examples such as the use of ROS by NASA in the Mars rover project as described here.

Other examples of different architectures applied in robotics is the use and preference of behavior trees over finite state machines. The finite state machine has been considered a bedrock of software design for decades and has been used to bring modularity in designs, but suffers from overwhelming complexity in state transitions as a system grows. Behavior trees have been adopted widely within robotics (link) and have shown to provide improved modularity and testability for complex systems as shown here.

These are just a few examples of changes in the methods and architectures that robotics brings to agricultural machines control, and will begin to shape autonomous agricultural machine development down a different path than what has been done before. However, the answer is not to throw the baby out with the bathwater by fully adopting robotics systems while ignoring the decades of work in mobile machine controls and precision ag systems. The methods developed around embedded software development, functional safety, ISOBUS (as examples) exist for a reason and provide strong benefits when used in the right way. Rather it is to draw from both of these worlds selectively, accepting that the path forward will draw from strengths in both areas. As an example, this could mean updating functional safety standards to consider these approaches through the use of behavior trees for system risk and hazard analysis, or updating Performance Level requirements to consider the use of open-source software with requirements around proven use.

Can’t Be All Things to All Industries – Closer Industry Alignment

Another implication of the merging of these technology paradigms is that technologies used for autonomous agricultural machines will become more industry specific. Just as can be seen with precision agriculture technologies, complex autonomous agricultural machine solutions are closely-coupled to the unique job at hand, which means that the technologies for this industry will look quite different from other off-highway industries (e.g. construction, mining), and will need to integrate task management concepts developed in precision ag. This also means the eventual integration of farm management systems with autonomous machines, as these will need to be part of the same workflow from the end-user’s perspective. The approach from the mobile machine controls industry to come up with solutions that meet the needs of many different industries will not work for autonomous systems, and any solutions with this aim may result in some technology related to guidance and navigation, but will fall short on integration with the actual job the machine is meant to do.

Built-In Adaptability

Autonomous agricultural machines are needing to be built with technology that is changing at a rapid pace. This has a challenge for agricultural equipment that is intended to be used for 20 years, as many of the underlying technologies will change at paces much faster than this. With machine control systems in the past, this was manageable as embedded systems were closed and locked down. So, while technology for new machines continued to move, at least a locked down configuration could be produced for a long period of time to support legacy equipment. The technologies for autonomous machines will interact beyond the machine alone, communicating with cloud systems, using smartphones and tablets for user interfaces, and relying on open-source software that is changing at a fast pace. This means systems need to continue to evolve through their lifetime, and need to be built with adaptability in mind. Adaptability means the ability to easily and regularly deploy software updates without specialized tools, and with a seamless process that is not a hinderance to the end-user. Learn more here. This also requires a different approach to supporting customers as the responsibility for the function of the machine should not be looked at as its condition at sale plus warranty period, but over its entire lifetime. With a much heavier dependence on software systems this needs to consider different business models to support these machines, but also opportunities to provide increased value over the lifetime of a machine.

Conclusion: The Impact to the OEM

The technology changes described here are happening so that increases in operational efficiencies and agricultural production can continue while addressing constraints in skilled labor and available crop production land, as well as increased pressure for the reduction of chemicals towards more sustainable farming practices

The integration of robotics technologies with precision ag and mobile machine controls technologies will have far-reaching market implications for the agricultural equipment market. This technology development is happening under-the-hood and out of view of producers, and even many OEMs, but the implications of these changes will completely change the agricultural equipment industry over the next 20 years. These will drive changes such as a shift towards higher integration between machines and farm management systems, a shift in focus from individual machines to cross-machine workflows, a push towards integration of drivetrains and implements (more self-propelled machines), and new service and support systems driven by business model innovations that can provide a higher level of support for machines through the entire life of the machine. All of these changes mean significant opportunity for OEMs to engage their customers in new ways, but require a high level of adaptability and a close interaction with technology partners that can navigate this change. The system-focused approach and high complexity of systems will drive closer collaboration between technology partners and OEMs as the technology becomes an increasingly large part of the system, and the skillset for developing and maintaining it also grows. This drives the need for OEMs to closely understand their competitive advantages and core competencies and find partners that can complement those effectively.

If you would like to discuss how the topics of this article apply to you, please contact us at info@jcatechnologies.com. We are excited to discuss all opportunities.

Contact Us

Have a vision for making agricultural machinery work smarter? An idea that you've wanted to build but haven't known how to begin?

There's never been a better time to take that first step. Let JCA Technologies outline a solution for you.