Perception and Localization Solutions for Autonomous Off-highway Vehicles

The rapid growth of the off-highway vehicle market, particularly in emerging economies, creates some potentially enormous opportunities for off-highway vehicles. As we know, autonomous technologies have created great leaps in the off-highway segment, especially in the construction, mining, and agriculture sectors. Perception accuracy for object verification and prediction in worksites, mines, and agricultural areas is extremely critical in autonomous off-highway vehicles.

Global market trends for off-road vehicles

The global off-road vehicle market was valued at $12,365.18 million in 2020 and is projected to reach $22,618.45 million by 2030, registering a CAGR of 7.3%. Due to operational circumstances — off-road vehicles are designed to operate on rugged terrain — end users expect better output efficiency and optimum performance in high-end driving activities. Off-road vehicle demand is rising quickly because additional safety features may be incorporated into new off-road vehicles due to stricter regulatory oversight and new testing criteria, increasing the market’s potential for growth. New measures being taken by governments could boost the market for off-road vehicles. (Straits Research)

Use Cases of Autonomous Off-highway Vehicles

• Construction and Mining- Vehicles like autonomous dozers, excavator load carriers, and haul trucks are supposed to be used for excavating and grading operations in construction sites and mining. In many cases, these are often used to perform coordinated actions and the remapping in 4.0 visualization in these scenarios brings many complexities in their autonomous operation. GPS trackers and solution use cases can also aid drivers to navigate these uneven terrains and increase safety and efficiency.
• Agriculture- Vehicles such as trucks and tractors are not only supporting the reduction of manpower but also enhancing safety and precision in their operations. Multiple machines, manned or unmanned can coordinate and perform operations in a field to accomplish the task in a very cost-effective and efficient manner.

Challenges Associated with the Use Cases

• Mapping and Localization in an Unstructured Environment
  The impact of weather on perception sensors, poor and corrupted GPS signals, the dilemma in representing drivable regions, and errors in odometers cause difficulty in mapping and localization of the environment
  
  
• Extended Reach and Sensors in Moving Parts
  In excavators and cranes, the operating area is invisible for sensors mounted in and around the driver cameras, thus the operational area is hidden.
  
  
• Combining the static sensors and the moving sensors
  To provide the terrain views of the surroundings, these static images of point-cloud data need to be stitched with the data from the arm of the vehicle. The challenge is to fix the resolution and update the terrain map in order to reduce the complexity of sensor fusion.
  
  
• Handling the wide field-of-view sensor arrangement
  A huge amount of terrain data is to be stored and processed before their maneuvering, which requires higher storage for computing platforms.
  
  
• Need for Extrapolation and Interpolation of Data
  Vehicles like excavators experience a varied field of view, from where the data is received. To fit those data into the required map, usually we need extrapolation or interpolation of data.

Challenges in the Development of Solutions

• Data Set Requirements
  During the development phase, AI-based solutions require massive amounts of data, which in real-time from the field is difficult and time-consuming.
  
  
• Object Detection and Classification
  A camera is frequently unreliable for object detection and classification under certain operating conditions, hence the solution must use the point cloud data from LiDAR.
  
  
• Selection Simulation Environment
  Simulators are selected based on complexity, availability of the selected vehicle model, feasibility for the development or customization of vehicle models, and models for working terrains.
  
  
• Procurement of Vehicle Model or Vehicle Modeling
  Customized applications require specific vehicle models. If unavailable, they can be modeled using tools like Blender.
  
  
• Model for Working Environment for the Vehicle and of Use-cases
  The working environment shall support essential features of operation by vehicle. For example, for modeling the digging action of an excavator, the environment (terrain), shall be diggable. Modeling of the use cases must follow the lines of the test cases.
• Rigging of Vehicle Implements
  By modifying or adding to the model’s bones and joints, rigging is done. Construing the manner of the vehicle pieces and the vehicle modeling requires good experience.
• Perception Sensor Selection
  All-weather, day-and-night operations demand complex perception strategies and careful attention when using sensors. It is commonly known that cameras are used for pacification applications and that LiDAR supports pacification to a moderate extent.
• Sensor Modeling
  Sensors can be modeled as per the requirement. Resolution and scanning patterns are to be replicated. customization or remodeling of LiDAR would be required for MEM’s type of sensors. Modeling of the use cases must follow the lines of the test cases. The implementation of vehicle maneuvering and its interaction with the surroundings can be aided by ROS integration and animation scripting.
• Storage of Sensor Data
  Depending on the module being tested, the sensor data can be formatted into the necessary form. The level of noise supplied to the ground truth data must be appropriate. Given the extensive quantity of test cases, the storage need presents a hurdle.

Solution Concepts: Perception, Localization, and Mapping

Perception

In the case of environment perception, there exist some camera-based solutions like the following -

• Object Detection and Classification (Rocks)
• Semantic Segmentation (Rocks)
• Drivable Path Detection

For many use cases, LiDAR-centric solutions are necessary.

Tata Elxsi’s solutions cover:

• Object Detection and Classification
• Object Tracking
• Point Cloud Mapping

Localization

Localization is an important step to proceed with for autonomous navigation.

Importance of Localization in Automation & Autonomous Operation

• Automation solutions
• Implement control: Farming

Mapping

HD Map Generation; Sensor layer creation process

Illustration for Point Cloud Optimization

What is the scope for the reduction of the size of map data?

• Map need not store every perception of data
• Only the selected key feature points are to be stored

Illustration for Map-Generation Speed

How can Map-generation run with low compute resources?

Localization and Mapping across other vehicle systems

Remote Applications

Cloud Assisted Applications

Remote Application Use-cases

The remote applications have components for machine management for single and multiple vehicles.

Remote Application Use Cases

Conclusion

The perception and localization solutions for autonomous off-highway vehicles play a crucial role in enabling the safe and efficient operation of these vehicles. By incorporating various technologies such as LiDAR, camera, GPS, and other sensors, these solutions provide vehicles with the ability to accurately perceive their surroundings and determine their location in real time.

With the increasing demand for autonomous off-highway vehicles in industries such as agriculture, mining, and construction, there is a growing need for advanced perception and localization solutions. As the technology continues to evolve, we can expect to see further improvements in the accuracy, reliability, and cost-effectiveness of these solutions, leading to the widespread adoption of autonomous off-highway vehicles.

Tata Elxsi’s solutions in the area of autonomous driving, operator-assisted systems, etc. can address the challenges faced in this regard and improve the efficiency of operation & productivity.

About the Author

Chanjal Prakash is a seasoned marketing professional with around 10 years of experience in the industry. He has a strong passion for crafting creative and effective marketing campaigns that deliver measurable results.

In Tata Elxsi, he is responsible for marketing activities for Off-highway and automotive industry.

Connect with the Author on LinkedIn