By Darcy Cook – VP Engineering, JCA Electronics

There has been fast growth in the development of autonomous agricultural machines over the past few years. Most machines that have been promoted publicly are either concept machines developed by OEMs and research organizations, or machines launched by start-ups looking to be the first broad solution in a new market. In addition to this, many major OEMs are developing innovative autonomous machines behind closed doors, with the intention of major product releases in the next few years. All of these efforts are working towards developing the technologies, applications, and market for autonomous machines in agricultural applications. As more of the work behind the scenes becomes public over the next few years, the scale and impact of the technological shift towards autonomous machines will become increasingly clear.

Many organizations have developed and demonstrated (either publicly or privately) proof-of-concept autonomous machines that provide a confidence in the core technology required for autonomy. This is shifting the thinking of OEMs from asking “Can it be done?” to now answering the question of “How can we make it scale?”. The shift is one from a research mindset, where technological proof-of-concept demonstration is the goal, to a production-readiness mindset where robustness, reliability, ease-of-use, cost, and safety are critical factors. Traditionally, the transition from proof-of-concept to production systems for operator-driven machines has often been a component-focused process led by OEM procurement groups. While component reliability is as important as it has ever been, the complexity of autonomous systems means that now a robust system architecture is the most critical consideration when developing a scalable autonomous machine. This paper provides an overview of the subsystems and technologies needed in agricultural autonomous systems, the challenges that are faced by autonomous machine manufacturers in moving from proof-of-concept systems to robust and scalable field-ready systems, and why a cohesive system architecture is key.

Challenge of Scalability for Autonomous Machine Technologies

In today’s agricultural equipment, operators perform a wide variety of complex tasks that integrate different functions of the system. These include assessment of surroundings, driving the machine, operating the implement functions and adjusting to environmental conditions, and general operation monitoring. These are all tasks that must be automated in one way or another for autonomous machines. This requires a step-change in technological advancement from traditional systems to manage the complexity of these tasks.

Autonomous agricultural systems have only recently become possible because of advances in a wide variety of technology areas including connectivity (IoT), robotics, guidance systems, sensors, and machine learning that serve as the underlying technology that have the capability to perform the complex tasks people currently perform. Many of these technologies are new to the agricultural machine industry and have often originated in other industries that have different interface requirements and environmental constraints. Proof-of-concept autonomous machines have often been developed using existing components repurposed from other industries and/or early dev platforms with new technologies. These are effective in demonstrating that the core functionality needed for autonomy is possible, but unless OEMs plan to ship each machine with a team of engineers that follow it in the field, there needs to be additional effort to design the machine for scalable production.

Scalable production of autonomous systems requires both:

• Components that integrate these new underlying technologies in form factors that are designed for both the environmental conditions and interfaces appropriate for agricultural machines, and
• A robust system architecture that facilities complex interactions between autonomous subsystem functions to provide robust and reliable system interactions

Importance of a Robust System Architecture

The focus on a robust system architecture is of greater importance than it has ever been for traditional machine control systems because of the increased complexity of autonomous systems. This has now become the most critical factor in developing a scalable autonomous machine and it requires a shift in thinking for those involved in moving from proof-of-concept machines to production. To clarify the importance of architecture, it is useful to consider the subsystem functions that make up an autonomous agricultural machine, and also the typical approach many organizations have used for proof-of concept machine development.

Autonomous Machine Subsystems

All autonomous agricultural machines need to perform a set of common critical functions, but these can vary significantly based on the machine’s purpose, use cases, and implementation. JCA has developed a model where these functions are categorized into seven subsystems as follows:

• Perception – Vision and sensing system needed to detect and recognize objects, driving conditions, and job-specific environmental conditions. Perception systems often include advanced sensing technologies, such as lidar, radar, 3D and 2D cameras, as well as advanced processing and software platforms for executing machine learning and computer vision algorithms.
• Power and drivetrain – Systems needed to control the steering, propulsion, and power systems of the machine. This consists of engine and power management that is specific to the size, shape, and movement capability of the machine.
• Localization and mapping – Systems used for detecting positioning and orientation, and tracking. This includes accurate GNSS systems for positioning (such as RTK-GPS) as well as inertial measurement units (IMU) for roll, pitch, and yaw orientation sensing. This can also include map management and adaption for position tracking.
• Mission management – Systems to manage missions, which consist of work areas (fields), paths travelling by the machine, and the jobs executed by the machine. Mission management includes planning, deployment, monitoring, execution, and analysis. Guidance and navigation algorithms are included in mission management.
• Human-machine interfacing (HMI) – Systems that provide a user interface for monitoring of the machine, missions, maps, diagnostics and configuration, alarms, and safety systems. The HMI systems often include a remote tablet or computer used to plan, deploy, and monitor missions, a local interface for manual machine control (which may also be a tablet, or physical remote), and safety interfaces (e.g. E-Stop).
• Communication and data management – Systems that manage communication and data between each of the subsystems, including on-machine communication and logging, local machine-user communication, multi-machine communication within a work area, as well as remote system communication. This spans a wide set of technology areas for embedded system communication and storage to cloud-based data management systems.
• Safety – Safety systems include alarm management, integration with perception for object detection, E-stop, and functional safety controls. Safety design should be incorporated throughout all other subsystems as appropriate within a defined safety architecture.

Depending on the function and goals of the autonomous machine, each of these subsystems can be implemented differently. However, in all cases there is a significant depth of technology required to perform each of the core functions defined by the subsystems, and there are a variety of complex interactions required.

Typical Proof-of-Concept Development Approach

Robust and reliable operation requires a careful architecture design to ensure that these complex interactions can be managed, and the full capabilities required by each subsystem can be realized. However, this is rarely the case for proof-of-concept systems, as they typically evolve by adding components to demonstrate core functionality on a priority basis.

It is difficult to develop a robust architecture for research-based systems as much of the full functionality is not known when the system development begins. It is common for a build-as-needed approach to be used, where initial functionality is developed and demonstrated, then the next set of functions are added, without a cohesive architecture in mind from the start.

For example, a common path in the development of proof-of-concept autonomous machines is as follows:

1. A mechanical platform for the autonomous machine is developed that include the drivetrain and power systems with the goal of moving a machine. This is typically implemented with technology that is appropriate for agricultural machines as the technology needed is common with operator-driven system.
2. Localization and guidance capabilities are typically added to allow a machine to follow a simple path. This is usually built from existing precision ag technology designed for autosteer type functionality.
3. Some mapping and rudimentary mission management are added next. This will often require an HMI that may be tablet-based or remote connected. The communication systems must then be expanded from on-machine to remote monitoring. IoT technology is integrated at this point.
4. Coordination of the job with the path is considered at this point, requiring adapting mission management to execute the machine job based on machine position and movement. The technology used here is highly dependent on the main machine function and whether the machine has a fixed function or is intended as a universal power platform. Integrating the machine job for autonomous systems is not trivial as the operator has been the main interface between the drivetrain (tractor) and the implement for traditional systems, and now this needs to be automated.
5. Perception systems focused on obstruction detection will typically follow. This is highly dependent on the movement and shape of the machine, as well as the intended application (job). Integration of advanced sensors may be needed, with the ability to tie this to guidance systems, safety systems, and job controls.
6. Perception systems will then evolve in adding perception systems to detect job-specific functions to add more intelligent operation capabilities.
7. Basic safety systems with E-stop are added late in the process to allow user interaction to stop the machine.
8. Throughout development, communication and data systems are adapted to interconnect functions as they are needed.

This build-as-needed approach is the most efficient way to make progress towards demonstrating that functions can be accomplished without having to tackle all complex problems at the beginning, but comes with performance, reliability, and cost limitations. By the time the system builds up to include each of the key subsystems, the proof-of-concept system is likely to have major bottlenecks that hinder performance. The system organization starts to resemble a rat’s nest and becomes very difficult to diagnose and troubleshoot, and prone to reliability issues. Expanding this to a system that can support multiple machines, with a smooth user experience is very difficult because the underlying infrastructure is not designed to support it.

The system is likely made up of a group of independent hardware components and software systems that were chosen for speed of development and availability meeting near-term needs rather than adaptability, robustness, and reliability. Changing individual components in the system may improve reliability and/or cost of that component, but it is unlikely to result in a robust and cost-effective system. Transitioning to a production-ready system requires significant architectural design considerations.

Architecture Considerations for Autonomous Agricultural Systems

The architectural design of the system defines:

• How the subsystem functions will be distributed across processing and storage platforms, sensors, actuators, and other components
• What software systems will be used throughout the system, how these will interact with each other, and how they will each be managed, configured, and updated
• What are the communication channels (methods and technologies) that will be used to connect hardware and software within the system in consideration of performance, availability, remote access, security, and safety

The system architecture for autonomous machines extends across each of the core machine functions, and beyond the machine to cloud systems, remote HMI (tablet and/or computers), communication systems, and localization base stations. Expertise across multiple technology domains is needed to understand the benefits and limitations of approaches chosen. Key considerations within the architecture design are:

• Safety Integration – Safety needs to be considered early in the design to ensure critical safe operation is addressed throughout the system.
• Diagnostics – With many complex components in the system, the ability to troubleshoot effectively and have the system self-diagnose and report errors is essential.
• Adaptability – Autonomous systems are likely to continue to evolve fairly rapidly, so a rigid architecture that is difficult to make changes to will become a limiting factor.
• Technical Debt – Large amounts of custom technology development will also mean a large amount of support of the technology. Maximizing reuse of established, shared, and evolving technology is beneficial to reduce support and maintenance in the future.
• Environmental Considerations – All systems need to operate in rugged agricultural conditions and need to be capable of surviving wide temperature swings, high vibration, vehicle electrical transients, and other environmental conditions.

Developing a system architecture for an autonomous machine from the ground up is a daunting task and requires experience with autonomous machines and agricultural machine applications, familiarity with common use cases, capabilities across many technology areas, and a set of core components designed specifically for autonomous agricultural machines.

JCA Autonomous Framework (AFW)

The complexity of development of an effective and scalable autonomous machine architecture presents a barrier to many OEMs in the development of autonomous machines. With the goal of lowering this barrier for OEMs, JCA Electronics has developed a cohesive Autonomous Framework (AFW) for agricultural autonomous systems. JCA has built on its experience in development of many highly-automated and autonomous machines for a wide variety of different agricultural applications to define an AFW that is adaptable to any autonomous machine. The AFW consists of a defined and adaptable system architecture as well as core technologies to implement the common components of each autonomous subsystem, while allowing for customization for each unique autonomous machine function.

Key characteristics of the JCA AFW are:

• Ruggedized processing platforms designed specifically for agricultural autonomous machines that integrate many of the key required functions
• High-end processing (edge computing) capabilities for advanced algorithms and high data processing (e.g. machine learning algorithms, perception systems, etc.).
• Distributed processing architecture that is easily scalable and adaptable for different system needs
• Localization, guidance, navigation, and mapping systems for accurate positioning in 3D space, and moving and monitoring of the machine
• An HMI architecture that uses tablets as the key user interface as a coordinate portal for all user interactions
• Communication and data management architecture that allows for operation in locations without Internet connectivity, as well as remote system coordination and access when connected to cloud systems
• Adaptable job interface for custom machine function controls that integrate to mission planning and the HMI interface
• Mission planning that encompasses planning, deployment, execution, monitoring, and analysis of machine missions (fields, paths, and jobs), and expands to multi-machine coordination
• A defined safety system adapted specifically for autonomous machines that includes system wide alarm management coordination and system watchdog functionality to allow for a functionally safe system

The diagram below provides a high-level overview of the areas encompassed in the JCA AFW.

Figure 1: JCA Autonomous Framework (AFW) Subsystems

The JCA AFW not only consists of key components that are ruggedized, reliable, and designed for autonomous purpose, but also defines a system architecture that addresses the challenges often encountered by proof-of-concept autonomous machines. The core components are integrated with a software systems that provide for a cohesive, efficient, and scalable system.

Figure 2: Autonomous Machine Block Diagram using JCA AFW

Applying the JCA AFW for Custom Applications

The JCA AFW was developed based on the realization that much of the technology infrastructure needed for agricultural autonomous machines is independent of the specific form and function of the machine. While this autonomous technology infrastructure is critical to the function of the system, it is largely outside the area of interest for OEMs that want to focus on the end application, or the core function of the machine. The JCA AFW provides a framework that defines a robust and adaptable architecture that machine manufactures can customize to realize their unique and innovative autonomous machines.

There is, of course, still engineering work that is required to develop the unique autonomous machine envisioned by the OEM. This development work, however, is now targeting specifically the end application rather than core infrastructure. These are the portions of the system that provide competitive advantage to OEMs and are close to the end application and therefore offer more tangible value to the OEM directly. The diagram below provides a high-level overview of the work that is required within each subsystem to customize a machine.

Figure 3: Development Areas Needed for Custom OEM Machine building on the JCA AFW

The engineering development work for the custom portions of the system can be performed by the OEM engineering team, JCA’s application engineering team, or a combination of both based on the internal engineering capabilities of the OEM.

Figure 4: Development of an Autonomous Machine using the JCA AFW

Use of the JCA AFW lowers the barriers for OEMs to develop unique autonomous machines based on their vision of autonomy in agriculture. The consideration in the JCA AFW of components specifically targeting the working environment and operation of agricultural autonomous machines as well as a robust architecture provides an efficient path for OEMs to move from proof-of-concept machines towards scalable production-ready autonomous machines.

About JCA Electronics

JCA Electronics provides advanced technology solutions for off-highway mobile machines, primarily in agricultural, construction, and mining industries. JCA has engineering and manufacturing operations based in Canada and has been providing customized electrical, electronics, and software systems for OEMs since 2002. For more information see www.jcaelectronics.ca or contact us at info@jcaelectronics.ca.