The recent era of Industry 4.0 extensively utilizes manipulators for various critical automation tasks. Handling a manipulator robot is a complex task. This research employs a 3-R (revolute) manipulator robot to achieve optimal joint trajectories for drawing different alphabets and shapes. The solution to inverse kinematics (IK) is essential; therefore, an artificial neural network (ANN) based model is used. The forward and backward reaching inverse kinematics (FABRIK) technique provides stability, though it is computationally intensive and time-consuming. To design a low-cost hybrid model, ANN combined with FABRIK is proposed to compute IK. Using the proposed model, coordinates for different alphabets and shapes within the confined workspace were calculated. We have developed a 3R (revolute) manipulator that utilizes deep learning (DL) and IK. The ANN automatically obtains specific end-effector coordinates. Due to the non-linear characteristics of IK’s mathematical model, inverse kinematics is a time-consuming procedure, making it difficult to provide a mathematical solution. Our primary goal is to reduce processing costs by using ANN and FABRIK with the IK model to solve IK problems and obtain coordinates for any form or alphabet within a given workspace. To ensure a safe region is selected, this model combines the control barrier function (CBF) with the Lyapunov function. The accuracy of the model exceeds 99.5 percent. We have calculated the mean square error (MSE) as 1.66, the root mean square error (RMSE) as 1.25, and the mean absolute error (MAE) as 0.96 for our model. The error between the model’s predicted and actual coordinates also demonstrates letter coordinates and shapes drawn using a physical 3R manipulator model. As a result, this approach can be used to accurately estimate the angles in complex 3DoF inverse kinematics models.

Plant diseases cause an annual global crop loss of 20-40%, leading to estimated economic losses of 30-50 billion dollars. Tomatoes are susceptible to more than 200 diseases. Breeding disease-resistant cultivars is more cost-effective and environmentally sustainable than the frequent use of pesticides. Traditional breeding methods for disease resistance, relying on direct visual observation to measure disease-related traits, are time-consuming, inaccurate, expensive, and require specific knowledge of tomato diseases. High-throughput disease phenotyping is essential to reduce labor costs, improve measurement accuracy, and expedite the release of new varieties, thereby more effectively identifying disease-resistant crops. Precision agriculture efforts have primarily focused on detecting diseases on individual tomato leaves under controlled laboratory conditions, neglecting the assessment of disease severity of the entire plant in the field. To address this, we created a synthetic dataset using existing field and individual leaf datasets, leveraging a game engine to minimize additional data labeling. Consequently, we developed a customized unsupervised domain-adaptive tomato disease segmentation algorithm that monitors the entire tomato plant and determines disease severity based on the proportion of affected leaf areas. The system-derived disease percentages show a high correlation with manually labeled data, evidenced by a correlation coefficient of 0.91. Our research demonstrates the feasibility of using ground robots equipped with deep-learning algorithms to monitor tomato disease severity under field conditions, potentially accelerating the automation and standardization of whole-plant disease severity monitoring in tomatoes. This high-throughput disease phenotyping system can also be adapted to analyze diseases in other crops with similar foliar diseases, such as maize, soybeans, and cotton.

The use of UAV for 3D surface inspection has become an important tool in the field of large-scale structure maintenance. However, the commonly used UAV inspection path planning algorithms for 3D surface suffer from problems such as path quality dependent model accuracy, path inspection efficiency, and low inspection quality. To address these issues, this paper proposes a UAV 3D surface inspection path planning algorithm based on normal vector filtering and integrated viewpoint Evaluation. Generate a safe and effective set of viewpoints through uniform sampling and normal vector viewpoint filtering, and then use a integrated viewpoint evaluation method combined with Monte Carlo tree search to select viewpoints, thereby generating a safe, efficient, and complete UAV surface inspection path. The results of simulation and physical experiments show that the proposed method can effectively solve the path planning problem when using UAV for surface inspection of large three-dimensional structures. Meanwhile, while ensuring the quality of inspection, this method can reduce path redundancy and improve the surface inspection efficiency of UAV.

Reforestation is pivotal in mitigating climate change, preserving biodiversity, and safeguarding ecosystems. Precision reforestation implies efficiently using materials and human resources to conduct this intensive task. In this sense, using unmanned aerial vehicles (UAVs), also known as drones, improves the accuracy of dispensed seeds while increasing coverage and reducing labor. However, drone-based reforestation still presents technological challenges that need to be addressed. An important challenge is the presence of disturbances during flights due to environmental conditions, primarily unexpected wind, and the unavoidable loss of mass presented by the vehicle caused by the dispensing task. This paper evaluates the use of an observer-based controller to reject the particular disturbances caused by the sowing activity. Through meticulous experimentation and analysis, the study demonstrates the observer’s adeptness in mitigating external disturbances, thereby enhancing the precision and stability of UAV operations. This technological advancement holds promise for diverse practical applications and has implications for environmental conservation efforts, particularly reforestation. The obtained experimental results confirm the viability of the proposed controller and observer framework, highlighting its potential to improve the robustness of environmental monitoring, conservation, and sustainable resource management practices.

The integration of precise landing capabilities into UAVs is crucial for enabling autonomous operations, particularly in challenging environments such as the offshore scenarios. This work proposes a heterogeneous perception system that incorporates a multimodal fiducial marker, designed to improve the accuracy and robustness of autonomous landing of UAVs in both daytime and nighttime operations. This work presents ViTAL-TAPE, a visual transformer-based model, that enhance the detection reliability of the landing target and overcomes the changes in the illumination conditions and viewpoint positions, where traditional methods fail. VITAL-TAPE is an end-to-end model that combines multimodal perceptual information, including photometric and radiometric data, to detect landing targets defined by a fiducial marker with 6 degrees-of-freedom. Extensive experiments have proved the ability of VITAL-TAPE to detect fiducial markers with an error of 0.01 m. Moreover, experiments using the RAVEN UAV, designed to endure the challenging weather conditions of offshore scenarios, demonstrated that the autonomous landing technology proposed in this work achieved an accuracy up to 0.1 m. This research also presents the first successful autonomous operation of a UAV in a commercial offshore wind farm with floating foundations installed in the Atlantic Ocean. These experiments showcased the system's accuracy, resilience and robustness, resulting in a precise landing technology that extends mission capabilities of UAVs, enabling autonomous and Beyond Visual Line of Sight offshore operations.

The regular inspection of the power transmission lines is essential for the uninterrupted transmission of electrical energy to demand points. This quickly requires actions with economically, efficiently, and safely. Therefore, the transmission line inspection robots are inevitable solution as an alternative to existing line inspection methods. This study present design and control of a transmission line inspection robot (I-Robot). Since the I-Robot exhibits nonlinear behaviour and has multiple inputs and multiple outputs, a model reference-based neural controller is determined to achieve nonlinear control. The robot design process consists of 4 stages which are kinematic modelling, dynamic modelling, actuator modelling and controller design. To meet inspection requirements, the conceptual design of the I-Robot is performed, and the kinematic model are calculated in terms of the transformation matrices. According to the design requirements and system constraints, the dynamic model of the I-Robot is created. To provide desired motions and trajectory tracking, the actuator models are determined. Then, the I-Robot is prototyped. According to the dynamics of joint, robot and constraints, the system identification is performed to create reference model. During the system identification, the logged data are used the train the reference model. Finally, the desired trajectory for the driving cycles is created by manual excitation of the I-Robot. During the manual excitation, the logged data are used to train the neural network-based controller. Eventually, the I-Robot is assessed under the test scenarios in term of the trajectory tracking performance as regression value and mean squared errors.

Multi-arm harvesting robots offer a promising solution to the labor shortage in fruit harvesting, due to their ability to improve harvesting efficiency. However, multi-arm harvesters necessitate additional visual sensors to acquire distribution information of fruits within larger working spaces. Greater demands are consequently imposed on graphics computation, leading to increased costs in computing hardware of robot system. To balance the graphics computing cost and reduce energy consumption, distributed graphics computation frameworks for multi-arm robot vision system are proposed in this study. First, a host-edge framework is proposed to assign the tasks of image inference and depth alignment to host computer and edge computing modules through a decentralized mode of local connection. Moreover, to increase the endurance time of robot in application, the edge computing modules are reduced and the fifth generation mobile communication is integrated into robot graphics computing system to transfer on-board image processing to a remote computing server with MQTT protocol. To verify the effectiveness of the proposed framework, comprehensive experiments were performed, demonstrating that, compared with traditional computing framework, the proposed local distributed framework reduced 35.6% average time consumption, and over 20 FPS average processing speed can be achieve. The remote distributed framework has reduced the computational power consumption of the on-board system by approximately 23.1% while ensuring the performance is not lower than the local distributed framework. Finally, by discussing the two frameworks in terms of stability and cost, we present the commercial viability for the application of multi-arm harvesting robot.

To deal with the issue of poor visibility caused by water turbidity during the operation of underwater robotics, we propose an attenuation prior formation model-guided enhancement algorithm for turbid underwater images. Specifically, we establish an imaging model suitable for turbid water by studying the influence of water turbidity on light attenuation and transmission. For this model, we first propose a scoring formula that takes into account multiple prior knowledge to estimate the global background light with the help of hierarchical searching technique. Then, we make full use of the advantages of different scale neighborhoods in image restoration, and propose an adaptive multi-scale weighted fusion transmission estimation method to balanc e brightness and contrast. In addition, to correct the color of the images with a natural appearance, a variation of white balance is introduced as post-processing. Extensive experiments on two image datasets show that our algorithm achieves better results than state-of-the-art methods.

In shallow water environments, traditional technologies face challenges in retrieving unexploded ordnance (UXO). We present an innovative and safe alternative robotic platform to replace traditional divers in this dangerous task. A waterproof eighteen-DOF legged robot MR. Crab has been developed specifically for the purpose of locating and recovering UXO. Utilizing walking legs for both locomotion and manipulation, the robot demonstrates the capability to pick up inert UXO weighing at 2.78kg (half robot’s mass) with one pair of walking legs and 10.5kg (184% of robot’s mass) when utilizing all three pairs of legs. To enhance our understanding of the gripping capacity, static analysis was conducted to predict the maximum weight of graspable UXO with respect to UXO radii. For easier control, a user-friendly GUI had been built for remote teleoperation. Additionally, an automatic gripping algorithm was developed to efficiently place the robot in the optimal gripping position and then grab the inert UXO.

This paper studies the coverage path planning and path following problems for an underwater biomimetic robotic fish to finish the deep-sea net cage water quality monitoring. Firstly, with a focus on minimizing total path length, repetition rate, and turning occurrences to enhance path coverage efficiency and reduce energy consumption, we propose a novel coverage path planning strategy. This strategy incorporates DQN, a reward function, and a rewrite strategy inspired by RRT*. Secondly, a high-performance path-following method, which takes into account robot performance, is designed to cope with adverse conditions in net cages. Finally, the simulation and field experiments demonstrate significant improvements over existing methods and the effectiveness in practical applications, showcasing its applicability in aquaculture management. The proposed algorithm offers valuable insights into optimizing coverage path planning for underwater robots in practical scenarios.

This article presents the first ever fully autonomous UAV (Unmanned Aerial Vehicle) mission to perform gas measurements after a real blast in an underground mine. The demonstration mission was deployed around 40 minutes after the blast took place, and as such realistic gas levels were measured. We also present multiple field robotics experiments in different mines detailing the development process. The presented novel autonomy stack, denoted as the Routine Inspection Autonomy (RIA) framework, combines a risk-aware 3D path planning D + ∗ , with 3D LiDAR-based global relocalization on a known map, and it is integrated on a custom hardware and a sensing stack with an onboard gas sensing device. In the presented framework, the autonomous UAV can be deployed in incredibly harsh conditions (dust, significant deformations of the map) shortly after blasting to perform inspections of lingering gases that present a significant safety risk to workers. We also present a change detection framework that can extract and visualize the areas that were changed in the blasting procedure, a critical parameter for planning the extraction of materials, and for updating existing mine maps. As will be demonstrated, the RIA stack can enable robust autonomy in harsh conditions, and provides reliable and safe navigation behavior for autonomous Routine Inspection missions.

Over the last few decades, the agricultural industry has witnessed significant advancements in autonomous systems, primarily aimed at improving efficiency while reducing environmental impact. The critical role of complete coverage path planning cannot be overstated in this context. It involves determining an optimal path for tasks such as harvesting, mowing, and spraying, taking into account various factors like land topography, operational requirements, and robot characteristics. Our previous approach introduced a tree-based exploration method to generate potential solutions, coupled with an optimization process to select the best ones, considering field complexities and robot characteristics. Yet, despite its strengths, it had certain limitations, notably in computational time and number of examined driving directions. In this paper, we present a novel hybrid method that combines the comprehensive coverage benefits of our original approach with the computational efficiency of the Fields2Cover algorithm. Besides combining our previous approach and Fieds2Cover strengths for optimizing coverage area, overlaps, non-working path length and overall travel time, it significantly improves the computation process, enhances the flexibility of trajectory generation. It also takes into account the working trajectory inclinations for more advanced optimization to address soil erosion and energy consumption. In an effort to support this innovative approach, we have also created and made available a public dataset. This dataset includes both 2D and 3D representations of thirty agricultural fields located in France. This resource not only illustrates the effectiveness of our approach but also provides an invaluable data for future research in complete coverage path planning within the context of modern agriculture.

The continuous and close combination of artificial intelligence technology and agriculture promotes the rapid development of smart agriculture, among which the agricultural robot navigation line recognition algorithm based on deep learning has achieved great success in detection accuracy and detection speed. However, there are still many problems, such as the large size of the algorithm is difficult to deploy in hardware equipment, and the accuracy and speed of crop row detection in real farmland environment are low. In order to solve the above problems, this paper proposed a navigation line extraction algorithm for corn spraying robot based on Light-YOLOv8s network. Firstly, the Convolution (Conv) module and C2f module of YOLOv8s network are replaced with Depthwise Convolution (DWConv) module and PP-LCNet module respectively to reduce the parameters (Params) and giga floating-point operations per second (GFLOPs) of the network, so as to achieve the purpose of network lightweight. Secondly, in order to reduce the precision loss caused by network lightweight, the spatial pyramid pooling fast (SPPF) module in the backbone network is changed to atrous spatial pyramid pooling faster (ASPPF) module to improve the accuracy of network feature extraction. Meanwhile, normalization-based attention module (NAM) is introduced into the network to improve the network’s attention to corn plants. Then the corn plant was located by using the midpoint of the corn plant detection box. Finally, the least square method is used to extract the maize crop row line, and the middle line of the maize crop row line is the navigation line of the maize spraying robot. According to the experimental results, the Params of Light-YOLOv8s network decreased by 29.24%, 86.64% and 55.38%, respectively, compared with YOLOv5s network, YOLOv7 network and YOLOv8s network. GFLOPs dropped 26.79%, 88.77%, and 58.74%, respectively, while accuracy lost only 1%, 0.6%, and 2.2%. It shows that the Light-YOLOv8s network proposed in this paper greatly reduces the size of the model, solves the problems such as the difficulty of deployment caused by the large size of the existing algorithm, and also greatly reduces the accuracy loss of the model, and solves the problems such as the reduced accuracy of the algorithm caused by the lightweight network. When the corn spraying robot works in the real farmland environment, the navigation line extraction algorithm proposed in this paper not only ensures the real-time navigation of the corn spraying robot, but also ensures the accuracy of the navigation, and makes a contribution to the development of agricultural robot visual navigation technology.

Plastering is dominated manually, exhibiting low levels of automation and inconsistent finished quality. A comprehensive review of literature indicates that extant plastering robots demonstrate a subpar performance when tasked with rectifying defects in transition area. The limitations encompass a lack of capacity to independently evaluate the quality of work or perform remedial plastering procedures. To address this issue, this research describes the system design of the Puttybot and a paradigm of plastering to solve the stated problems. The Puttybot consists of a mobile chassis, lift platform, and a macro/micro manipulator. The force-controlled scraper parameters have been calibrated to dynamically modify their rigidity in response to the applied putty. This strategy utilizes Convolutional Neural Networks to identify plastering defects and executes the plastering operation with force feedback. This paradigm’s effectiveness was validated during an autonomous plastering trial wherein a large-scale wall was processed without human involvement.

Agricultural robots offer a viable solution to the critical challenges of productivity and sustainability of modern agriculture. The widespread deployment of agricultural robotic fleets, however, is still hindered by the overall system’s complexity, requiring the integration of several non-trivial components for the operation of each robot but also the orchestration of robots working with each other and human workers. This paper proposes a topological map as the unifying representation and computational model to facilitate the smooth deployment of robotic fleets in agriculture. This topological abstraction of the system state results in an efficient representation of large-scale environments, but also offers the scalable and efficient operation of the entire fleet and allows for ex-situ modelling and analysis of operations. The practical use of the proposed framework is demonstrated in a horticultural use case with a fleet of robots supporting the work of human fruit pickers. The critical components of the system are analysed and evaluated in deployment in both realistic digital twin and real-life soft fruit farms of different scales, demonstrating the scalability and effectiveness of the proposed framework. The presented framework is general and should be easy to adopt in other multi-robot/multi-human scenarios such as warehouse logistics, cleaning and maintenance of public spaces.

A document by Simone Fontana. Click on the document to view its contents.

Projections for future air mobility envisage intensely utilised airspace that does not simply scale up from existing systems with centralised air traffic control. This paper considers the implementation and test of a software and hardware framework for decentralised control of aerial vehicles within intensely used airspace. Up to 10 rotary wing vehicles of maximum all up mass of 1 kg are flown in an outdoor volume with length scale of 100 m with GPS and WiFi connectivity. Flight control is implemented using a Pixhawk 4 flight controller running the PX4 firmware with guidance algorithms run on a separate onboard companion computer. Deconfliction is implemented using a simple elastic repulsion model with a guidance update rate of 10 Hz. Traffic structures are constructed from a path of directed waypoints and associated cross sectional geometry. Junctions are implemented when two paths converge into one or when one path diverges into two. Agents engage with structures through execution of flow, merge and swirl velocity rules. Calibration experiments showed that the worst case latency in agents sharing position information was of the order of 0 .5 s made up from delays due to finite guidance update rate, WiFi processing and centralised message processing. A choice of vehicle cruise speed of 2 m/s and conflict radius of 2 .5 m provided an acceptable compromise between experiment time efficiency (speed) and spatial efficiency (resolution) within the test volume. Results from recirculating junction experiments show that peak deconfliction activity occurs at the junction node, however biased distribution of agents within a corridor means the peak intensity is pushed ahead of the node. Use of meshed helical junction structures significantly reduces the intensity of conflict at the expense of reduced junction time efficiency.

This paper fuses ideas from Reinforcement Learning (RL), Learning from Demonstration (LfD), and Ensemble Learning into a single paradigm. Knowledge from a mixture of control algorithms (experts) are used to constrain the action space of the agent, enabling faster RL refining of a control policy, by avoiding unnecessary explorative actions. Domain-specific knowledge of each expert is exploited. However, the resulting policy is robust against errors of individual experts, since it is refined by a RL reward function without copying any particular demonstration. Our method has the potential to supplement existing RLfD methods when multiple algorithmic approaches are available to function as experts. We illustrate our method in the context of a Visual Servoing (VS) task, in which a 7-dof robot arm is controlled to maintain a desired pose relative to a target object. We explore four methods for bounding the actions of the RL agent during training. These methods include using a hypercube and convex hull with modified loss functions, ignoring actions outside the convex hull, and projecting actions onto the convex hull. We compare the training progress of each method with and without using the expert demonstrators. Our experiments show that using the convex hull with a modified loss function significantly improves training progress. Furthermore, we demonstrate faster VS error convergence while maintaining higher manipulability of the arm, compared to classical image-based VS, position-based VS, and hybrid-decoupled VS.