Plant diseases cause an annual global crop loss of 20-40%, leading to estimated economic losses of 30-50 billion dollars. Tomatoes are susceptible to more than 200 diseases. Breeding disease-resistant cultivars is more cost-effective and environmentally sustainable than the frequent use of pesticides. Traditional breeding methods for disease resistance, relying on direct visual observation to measure disease-related traits, are time-consuming, inaccurate, expensive, and require specific knowledge of tomato diseases. High-throughput disease phenotyping is essential to reduce labor costs, improve measurement accuracy, and expedite the release of new varieties, thereby more effectively identifying disease-resistant crops. Precision agriculture efforts have primarily focused on detecting diseases on individual tomato leaves under controlled laboratory conditions, neglecting the assessment of disease severity of the entire plant in the field. To address this, we created a synthetic dataset using existing field and individual leaf datasets, leveraging a game engine to minimize additional data labeling. Consequently, we developed a customized unsupervised domain-adaptive tomato disease segmentation algorithm that monitors the entire tomato plant and determines disease severity based on the proportion of affected leaf areas. The system-derived disease percentages show a high correlation with manually labeled data, evidenced by a correlation coefficient of 0.91. Our research demonstrates the feasibility of using ground robots equipped with deep-learning algorithms to monitor tomato disease severity under field conditions, potentially accelerating the automation and standardization of whole-plant disease severity monitoring in tomatoes. This high-throughput disease phenotyping system can also be adapted to analyze diseases in other crops with similar foliar diseases, such as maize, soybeans, and cotton.

Reforestation is pivotal in mitigating climate change, preserving biodiversity, and safeguarding ecosystems. Precision reforestation implies efficiently using materials and human resources to conduct this intensive task. In this sense, using unmanned aerial vehicles (UAVs), also known as drones, improves the accuracy of dispensed seeds while increasing coverage and reducing labor. However, drone-based reforestation still presents technological challenges that need to be addressed. An important challenge is the presence of disturbances during flights due to environmental conditions, primarily unexpected wind, and the unavoidable loss of mass presented by the vehicle caused by the dispensing task. This paper evaluates the use of an observer-based controller to reject the particular disturbances caused by the sowing activity. Through meticulous experimentation and analysis, the study demonstrates the observer’s adeptness in mitigating external disturbances, thereby enhancing the precision and stability of UAV operations. This technological advancement holds promise for diverse practical applications and has implications for environmental conservation efforts, particularly reforestation. The obtained experimental results confirm the viability of the proposed controller and observer framework, highlighting its potential to improve the robustness of environmental monitoring, conservation, and sustainable resource management practices.

The regular inspection of the power transmission lines is essential for the uninterrupted transmission of electrical energy to demand points. This quickly requires actions with economically, efficiently, and safely. Therefore, the transmission line inspection robots are inevitable solution as an alternative to existing line inspection methods. This study present design and control of a transmission line inspection robot (I-Robot). Since the I-Robot exhibits nonlinear behaviour and has multiple inputs and multiple outputs, a model reference-based neural controller is determined to achieve nonlinear control. The robot design process consists of 4 stages which are kinematic modelling, dynamic modelling, actuator modelling and controller design. To meet inspection requirements, the conceptual design of the I-Robot is performed, and the kinematic model are calculated in terms of the transformation matrices. According to the design requirements and system constraints, the dynamic model of the I-Robot is created. To provide desired motions and trajectory tracking, the actuator models are determined. Then, the I-Robot is prototyped. According to the dynamics of joint, robot and constraints, the system identification is performed to create reference model. During the system identification, the logged data are used the train the reference model. Finally, the desired trajectory for the driving cycles is created by manual excitation of the I-Robot. During the manual excitation, the logged data are used to train the neural network-based controller. Eventually, the I-Robot is assessed under the test scenarios in term of the trajectory tracking performance as regression value and mean squared errors.

Multi-arm harvesting robots offer a promising solution to the labor shortage in fruit harvesting, due to their ability to improve harvesting efficiency. However, multi-arm harvesters necessitate additional visual sensors to acquire distribution information of fruits within larger working spaces. Greater demands are consequently imposed on graphics computation, leading to increased costs in computing hardware of robot system. To balance the graphics computing cost and reduce energy consumption, distributed graphics computation frameworks for multi-arm robot vision system are proposed in this study. First, a host-edge framework is proposed to assign the tasks of image inference and depth alignment to host computer and edge computing modules through a decentralized mode of local connection. Moreover, to increase the endurance time of robot in application, the edge computing modules are reduced and the fifth generation mobile communication is integrated into robot graphics computing system to transfer on-board image processing to a remote computing server with MQTT protocol. To verify the effectiveness of the proposed framework, comprehensive experiments were performed, demonstrating that, compared with traditional computing framework, the proposed local distributed framework reduced 35.6% average time consumption, and over 20 FPS average processing speed can be achieve. The remote distributed framework has reduced the computational power consumption of the on-board system by approximately 23.1% while ensuring the performance is not lower than the local distributed framework. Finally, by discussing the two frameworks in terms of stability and cost, we present the commercial viability for the application of multi-arm harvesting robot.

In shallow water environments, traditional technologies face challenges in retrieving unexploded ordnance (UXO). We present an innovative and safe alternative robotic platform to replace traditional divers in this dangerous task. A waterproof eighteen-DOF legged robot MR. Crab has been developed specifically for the purpose of locating and recovering UXO. Utilizing walking legs for both locomotion and manipulation, the robot demonstrates the capability to pick up inert UXO weighing at 2.78kg (half robot’s mass) with one pair of walking legs and 10.5kg (184% of robot’s mass) when utilizing all three pairs of legs. To enhance our understanding of the gripping capacity, static analysis was conducted to predict the maximum weight of graspable UXO with respect to UXO radii. For easier control, a user-friendly GUI had been built for remote teleoperation. Additionally, an automatic gripping algorithm was developed to efficiently place the robot in the optimal gripping position and then grab the inert UXO.

This article presents the first ever fully autonomous UAV (Unmanned Aerial Vehicle) mission to perform gas measurements after a real blast in an underground mine. The demonstration mission was deployed around 40 minutes after the blast took place, and as such realistic gas levels were measured. We also present multiple field robotics experiments in different mines detailing the development process. The presented novel autonomy stack, denoted as the Routine Inspection Autonomy (RIA) framework, combines a risk-aware 3D path planning D + ∗ , with 3D LiDAR-based global relocalization on a known map, and it is integrated on a custom hardware and a sensing stack with an onboard gas sensing device. In the presented framework, the autonomous UAV can be deployed in incredibly harsh conditions (dust, significant deformations of the map) shortly after blasting to perform inspections of lingering gases that present a significant safety risk to workers. We also present a change detection framework that can extract and visualize the areas that were changed in the blasting procedure, a critical parameter for planning the extraction of materials, and for updating existing mine maps. As will be demonstrated, the RIA stack can enable robust autonomy in harsh conditions, and provides reliable and safe navigation behavior for autonomous Routine Inspection missions.

Over the last few decades, the agricultural industry has witnessed significant advancements in autonomous systems, primarily aimed at improving efficiency while reducing environmental impact. The critical role of complete coverage path planning cannot be overstated in this context. It involves determining an optimal path for tasks such as harvesting, mowing, and spraying, taking into account various factors like land topography, operational requirements, and robot characteristics. Our previous approach introduced a tree-based exploration method to generate potential solutions, coupled with an optimization process to select the best ones, considering field complexities and robot characteristics. Yet, despite its strengths, it had certain limitations, notably in computational time and number of examined driving directions. In this paper, we present a novel hybrid method that combines the comprehensive coverage benefits of our original approach with the computational efficiency of the Fields2Cover algorithm. Besides combining our previous approach and Fieds2Cover strengths for optimizing coverage area, overlaps, non-working path length and overall travel time, it significantly improves the computation process, enhances the flexibility of trajectory generation. It also takes into account the working trajectory inclinations for more advanced optimization to address soil erosion and energy consumption. In an effort to support this innovative approach, we have also created and made available a public dataset. This dataset includes both 2D and 3D representations of thirty agricultural fields located in France. This resource not only illustrates the effectiveness of our approach but also provides an invaluable data for future research in complete coverage path planning within the context of modern agriculture.

The continuous and close combination of artificial intelligence technology and agriculture promotes the rapid development of smart agriculture, among which the agricultural robot navigation line recognition algorithm based on deep learning has achieved great success in detection accuracy and detection speed. However, there are still many problems, such as the large size of the algorithm is difficult to deploy in hardware equipment, and the accuracy and speed of crop row detection in real farmland environment are low. In order to solve the above problems, this paper proposed a navigation line extraction algorithm for corn spraying robot based on Light-YOLOv8s network. Firstly, the Convolution (Conv) module and C2f module of YOLOv8s network are replaced with Depthwise Convolution (DWConv) module and PP-LCNet module respectively to reduce the parameters (Params) and giga floating-point operations per second (GFLOPs) of the network, so as to achieve the purpose of network lightweight. Secondly, in order to reduce the precision loss caused by network lightweight, the spatial pyramid pooling fast (SPPF) module in the backbone network is changed to atrous spatial pyramid pooling faster (ASPPF) module to improve the accuracy of network feature extraction. Meanwhile, normalization-based attention module (NAM) is introduced into the network to improve the network’s attention to corn plants. Then the corn plant was located by using the midpoint of the corn plant detection box. Finally, the least square method is used to extract the maize crop row line, and the middle line of the maize crop row line is the navigation line of the maize spraying robot. According to the experimental results, the Params of Light-YOLOv8s network decreased by 29.24%, 86.64% and 55.38%, respectively, compared with YOLOv5s network, YOLOv7 network and YOLOv8s network. GFLOPs dropped 26.79%, 88.77%, and 58.74%, respectively, while accuracy lost only 1%, 0.6%, and 2.2%. It shows that the Light-YOLOv8s network proposed in this paper greatly reduces the size of the model, solves the problems such as the difficulty of deployment caused by the large size of the existing algorithm, and also greatly reduces the accuracy loss of the model, and solves the problems such as the reduced accuracy of the algorithm caused by the lightweight network. When the corn spraying robot works in the real farmland environment, the navigation line extraction algorithm proposed in this paper not only ensures the real-time navigation of the corn spraying robot, but also ensures the accuracy of the navigation, and makes a contribution to the development of agricultural robot visual navigation technology.

Plastering is dominated manually, exhibiting low levels of automation and inconsistent finished quality. A comprehensive review of literature indicates that extant plastering robots demonstrate a subpar performance when tasked with rectifying defects in transition area. The limitations encompass a lack of capacity to independently evaluate the quality of work or perform remedial plastering procedures. To address this issue, this research describes the system design of the Puttybot and a paradigm of plastering to solve the stated problems. The Puttybot consists of a mobile chassis, lift platform, and a macro/micro manipulator. The force-controlled scraper parameters have been calibrated to dynamically modify their rigidity in response to the applied putty. This strategy utilizes Convolutional Neural Networks to identify plastering defects and executes the plastering operation with force feedback. This paradigm’s effectiveness was validated during an autonomous plastering trial wherein a large-scale wall was processed without human involvement.

Agricultural robots offer a viable solution to the critical challenges of productivity and sustainability of modern agriculture. The widespread deployment of agricultural robotic fleets, however, is still hindered by the overall system’s complexity, requiring the integration of several non-trivial components for the operation of each robot but also the orchestration of robots working with each other and human workers. This paper proposes a topological map as the unifying representation and computational model to facilitate the smooth deployment of robotic fleets in agriculture. This topological abstraction of the system state results in an efficient representation of large-scale environments, but also offers the scalable and efficient operation of the entire fleet and allows for ex-situ modelling and analysis of operations. The practical use of the proposed framework is demonstrated in a horticultural use case with a fleet of robots supporting the work of human fruit pickers. The critical components of the system are analysed and evaluated in deployment in both realistic digital twin and real-life soft fruit farms of different scales, demonstrating the scalability and effectiveness of the proposed framework. The presented framework is general and should be easy to adopt in other multi-robot/multi-human scenarios such as warehouse logistics, cleaning and maintenance of public spaces.

A document by Simone Fontana. Click on the document to view its contents.

Projections for future air mobility envisage intensely utilised airspace that does not simply scale up from existing systems with centralised air traffic control. This paper considers the implementation and test of a software and hardware framework for decentralised control of aerial vehicles within intensely used airspace. Up to 10 rotary wing vehicles of maximum all up mass of 1 kg are flown in an outdoor volume with length scale of 100 m with GPS and WiFi connectivity. Flight control is implemented using a Pixhawk 4 flight controller running the PX4 firmware with guidance algorithms run on a separate onboard companion computer. Deconfliction is implemented using a simple elastic repulsion model with a guidance update rate of 10 Hz. Traffic structures are constructed from a path of directed waypoints and associated cross sectional geometry. Junctions are implemented when two paths converge into one or when one path diverges into two. Agents engage with structures through execution of flow, merge and swirl velocity rules. Calibration experiments showed that the worst case latency in agents sharing position information was of the order of 0 .5 s made up from delays due to finite guidance update rate, WiFi processing and centralised message processing. A choice of vehicle cruise speed of 2 m/s and conflict radius of 2 .5 m provided an acceptable compromise between experiment time efficiency (speed) and spatial efficiency (resolution) within the test volume. Results from recirculating junction experiments show that peak deconfliction activity occurs at the junction node, however biased distribution of agents within a corridor means the peak intensity is pushed ahead of the node. Use of meshed helical junction structures significantly reduces the intensity of conflict at the expense of reduced junction time efficiency.

This paper fuses ideas from Reinforcement Learning (RL), Learning from Demonstration (LfD), and Ensemble Learning into a single paradigm. Knowledge from a mixture of control algorithms (experts) are used to constrain the action space of the agent, enabling faster RL refining of a control policy, by avoiding unnecessary explorative actions. Domain-specific knowledge of each expert is exploited. However, the resulting policy is robust against errors of individual experts, since it is refined by a RL reward function without copying any particular demonstration. Our method has the potential to supplement existing RLfD methods when multiple algorithmic approaches are available to function as experts. We illustrate our method in the context of a Visual Servoing (VS) task, in which a 7-dof robot arm is controlled to maintain a desired pose relative to a target object. We explore four methods for bounding the actions of the RL agent during training. These methods include using a hypercube and convex hull with modified loss functions, ignoring actions outside the convex hull, and projecting actions onto the convex hull. We compare the training progress of each method with and without using the expert demonstrators. Our experiments show that using the convex hull with a modified loss function significantly improves training progress. Furthermore, we demonstrate faster VS error convergence while maintaining higher manipulability of the arm, compared to classical image-based VS, position-based VS, and hybrid-decoupled VS.

A document by SIVABALAKRISHNAN R. Click on the document to view its contents.

Earthquakes, fire, and floods often cause structural collapses of buildings. The inspection of damaged buildings poses a high risk for emergency forces or is even impossible, though. We present three recent selected missions of the Robotics Task Force of the German Rescue Robotics Center, where both ground and aerial robots were used to explore destroyed buildings. We describe and reflect the missions as well as the lessons learned that have resulted from them. In order to make robots from research laboratories fit for real operations, realistic test environments were set up for outdoor and indoor use and tested in regular exercises by researchers and emergency forces. Based on this experience, the robots and their control software were significantly improved. Furthermore, top teams of researchers and first responders were formed, each with realistic assessments of the operational and practical suitability of robotic systems.

Simultaneous localization and mapping (SLAM) technology is ubiquitously employed in ground robots, unmanned aerial vehicles, and autonomous cars. This paper presents LTA-OM: an efficient, robust, and accurate LiDAR SLAM system. Employing FAST-LIO2 and Stable Triangle Descriptor as LiDAR-IMU odometry and the loop detection method, respectively, LTA-OM is implemented to be functionally complete, including loop detection and correction, false positive loop closure rejection, long-term association mapping, and multi-session localization and mapping. One novelty of this paper is the real-time long-term association (LTA) mapping, which exploits the direct scan-to-map registration of FAST-LIO2 and employs the corrected history map to constrain the mapping process globally. LTA leads to more globally consistent map construction and drift-less odometry at revisit places. We exhaustively benchmark LTA-OM and other state-of-the-art LiDAR systems with 18 data sequences. The results show that LTA-OM steadily outperforms other systems regarding trajectory accuracy, map consistency, and time consumption. The robustness of LTA-OM is validated in a challenging scene - a multi-level building having similar structures at different levels. Besides, a multi-session mode is designed to allow the user to store current session’s results, including the corrected map points, optimized odometry, and descriptor database for future sessions. The benefits of this mode are additional accuracy improvement and consistent map stitching, which is helpful for life-long mapping. Furthermore, LTA-OM has valuable features for robot control and path planning, including high-frequency and real-time odometry, drift-less odometry at revisit places, and fast loop closing convergence. Moreover, LTA-OM is versatile as it is applicable to both multi-line spinning and solid-state LiDARs, mobile robots and handheld platforms.

The soil contact model (SCM) is widely used in practice for off-road wheeled vehicle mobility studies when simulation speed is important and highly accurate results are not a main concern. In practice, the SCM parameters are obtained via a bevameter test, which requires a complex apparatus and experimental procedure. Here, we advance the idea of running a virtual bevameter test using a high-fidelity terramechanics simulation. The latter employs the “continuous representation model” (CRM), which regards the deformable terrain as an elasto-plastic continuum that is spatially discretized using the smoothed particle hydrodynamics (SPH) method. The approach embraced is as follows: a virtual bevameter test is run in simulation using CRM terrain to generate “ground truth” data; in a Bayesian framework, this data is subsequently used to calibrate the SCM terrain. We show that (i) the resulting SCM terrain, while leading to fast terramechanics simulations, serves as a good proxy for the more complex CRM terrain; and (ii) the SCM-over-CRM simulation speedup is roughly one order of magnitude. These conclusions are reached in conjunction with two tests: a single wheel test, and a full rover simulation. The SCM and CRM simulations are run in an open-source software called Chrono. The calibration is performed using PyMC, which is a Python package that interactively communicates with Chrono to calibrate SCM. The models and scripts used in this contribution are available as open source for unfettered use and distribution in a public repository.

When there are obstacles around the target point, the mobile robot cannot reach the target using traditional Artificial Potential Field (APF). Besides, the traditional APF is prone to local oscillation in complex terrain such as three-point collinear or semi-closed obstacles. Aiming at solving the defects of traditional APF, a novel improved APF algorithm named back virtual obstacle setting strategy-APF (BVO-APF) has been proposed in this paper. There are two main advantages of the proposed method. Firstly, by redefining the gravitational function as logarithmic function, the proposed method can make the mobile robot reach the target point when there are obstacles around the target. Secondly, the proposed method can avoid falling into local oscillation for both three-point collinear and semi-closed obstacles. Compare with APF and other improved APF, the feasibility of the algorithm is proved through software simulation and practical application.