Granular grippers can manipulate a wide variety of objects, but need to be pressed on the object to conform to it. If the object is placed on unstable ground, e.g., on sand or water, this step might cause the object to sink or move away from the gripper, hindering proper operation. We introduce a granular gripper with an integrated suction cup, where suction and jamming are controlled independently. We demonstrate the system's robust and enhanced gripping capabilities by comparing its grasping performance with a typical granular gripper design. We show that the proposed device can grip objects that are challenging for typical granular grippers, including those placed on unstable ground, as the suction cup stabilizes the object, allowing the gripper to conform.

Performing complex tasks in open environments remains challenging for robots, even when using large language models (LLMs) as the core planner. Many LLM-based planners are inefficient due to their large number of parameters and prone to inaccuracies because they operate in open-loop systems. We think the reason is that only applying LLMs as planners is insufficient. In this work, we propose DaDu-E, a robust closed-loop planning framework for embodied AI robots. Specifically, DaDu-E is equipped with a relatively lightweight LLM, a set of encapsulated robot skill instructions, a robust feedback system, and memory augmentation. Together, these components enable DaDu-E to (i) actively perceive and adapt to dynamic environments, (ii) optimize computational costs while maintaining high performance, and (iii) recover from execution failures using its memory and feedback mechanisms. Extensive experiments on real-world and simulated tasks show that DaDu-E achieves task success rates comparable to embodied AI robots with larger models as planners like COME-Robot, while reducing computational requirements by 6 .6×. Users are encouraged to explore our system at: https://rlc-lab.github.io/dadu-e/.

Transparent objects are difficult to perceive due to their unique optical properties, and the dynamic interaction between the hand and object further complicates pose estimation. To address this problem, we propose HTPE-Net, a monocular instance-level 6D pose estimation method for hand-held transparent objects, addressing the significant challenges posed by the texture-less, non-Lambertian surfaces, and hand-object occlusions. HTPE-Net integrates hand and object features through a dual-stream feature extraction backbone and a hand-to-object feature enhancement module, generating geometric features and hand attention maps to improve robustness to background changes and occlusions. The network is trained on a modified version of the TransHand-14K dataset and demonstrates superior performance compared to state-of-the-art methods. Additionally, a sim-to-real experiment validates the practical applicability of HTPE-Net in real-world robot perception tasks. The proposed approach significantly advances the accuracy and robustness of 6D pose estimation for hand-held transparent objects, with potential applications in robotics, human-machine interaction, and augmented reality.

Transparent liquid volume estimation is crucial for robot manipulation tasks, such as pouring. However, estimating the volume of transparent liquids is a challenging problem. Most existing methods primarily focus on data collection in the real world, and the sensors are fixed to the robot body for liquid volume estimation. These approaches limit both the timeliness of the research process and the flexibility of perception. In this paper, we present SimLiquid20k, a high-fidelity synthetic dataset for liquid volume estimation, and propose a YOLO-based multi-modal network trained on fully synthetic data for estimating the volume of transparent liquids. Extensive experiments demonstrate that our method can effectively transfer from simulation to the real world. In scenarios involving changes in background, viewpoint, and container variations, our approach achieves an average error of 5% in real-world volume estimation. In addition, our work conducts two application experiments integrate with ChatGPT, showcasing the potential of our method in service robotics. The accompanying video and supplementary materials are available at https://simliquid.github.io/.

With the aging population and labor shortage, the proportion of labor costs in the production costs of tomato harvesting is increasing, making the research and development of tomato harvesting robots urgent. This paper develops a cherry tomato harvesting robot, consisting of a mobile chassis, lifting platform, robotic arm, end effector, depth camera, and control system, to achieve automated walking navigation, harvesting, and basket loading. This paper outlines the system design and harvesting scenario of the tomato harvesting robot, details the design and implementation of the robot’s chassis and walking route planning, end effector design, tomato recognition and positioning, robotic arm motion planning, and control system. Furthermore, the architecture and control flow of the software system are elaborated. Finally, field tests validate the robot’s performance and harvesting success rate. Tests show that the overall accuracy and recall rates of the visual recognition model for tomato bunches are 85.04% and 88.71%, respectively, and for the stalks are 82.72% and 81.51%, respectively, with a harvesting success rate of 70.77%. These efforts provide beneficial exploration for the future commercial application of cherry tomato harvesting robots.

： The secure gripping of soft grippers remains a difficult challenge to address. To solve this challenge, the Single-axis deformation (SD) soft grippers were designed that were inspired by the blooming behavior of petunias. Four SD finger structures were designed by analyzing the anatomical structure of petunia petals. The stress concentration resulting from the deformation of the fingers with different structures was analyzed using numerical methods under the same deformation load conditions. In order to prove the SD soft grippers have a improving ability of secure gripping, The dexterity, gripping capacity, and radial load-bearing capacity of the SD soft grippers were tested through experiments. The experimental results shown that the opening and closing range of the SD soft gripper is 18mm-144mm, the maximum gripping load is 39.4N, and the radial bending angle is only 2.76 when the radial load is 24N.

Recently, there has been a surge of international interest in extraterrestrial exploration targeting the Moon, Mars, the moons of Mars, and various asteroids. This contribution discusses how current state-of-the-art Earth-based testing for designing rovers and landers for these missions currently leads to overly optimistic conclusions about the behavior of these devices upon deployment on the targeted celestial bodies. The key misconception is that gravitational offset is necessary during the terramechanics testing of rover and lander prototypes on Earth. The body of evidence supporting our argument is tied to a small number of studies conducted during parabolic flights and insights derived from newly revised scaling laws. We argue that what has prevented the community from fully diagnosing the problem at hand is the absence of effective physics-based models capable of simulating terramechanics under low gravity conditions. We developed such a physics-based simulator and utilized it to gauge the mobility of early prototypes of the Volatiles Investigating Polar Exploration Rover (VIPER). This contribution discusses the results generated by this simulator, how they correlate with physical test results from the NASA-Glenn SLOPE lab, and the fallacy of the gravitational offset in rover and lander testing. The simulator, which is open-sourced and publicly available, supports trafficability analysis and facilitates principled studies into in-situ resource utilization activities like digging, bulldozing, and berming in low gravity environments.

In this paper, we discuss the development and deployment of a robust autonomous system capable of performing various tasks in the maritime domain under unknown dynamic conditions. We investigate a data-driven approach based on modular design for ease of transfer of autonomy across different maritime surface vessel platforms. The data-driven approach alleviates issues related to a priori identification of system models that may become deficient under evolving system behaviors and/or shifting, unanticipated, environmental influences. Our proposed learning-based platform comprises a deep Koopman system model and a change point detector that provides guidance on domain shifts prompting relearning under severe exogenous and endogenous perturbations. Motion control of the autonomous system is achieved via an optimal controller design. The Koopman linearized model naturally lends itself to a linear–quadratic regulator (LQR) control design. We propose the C3D control architecture “Cascade Control with Change Point Detection and Deep Koopman Learning”. The framework is verified in station keeping tasks on an ASV in both simulation and real experiments. The approach demonstrated a consistent improvement in mean distance error across all test cases compared to the methods that do not consider system changes.

The nonlinear control problem of quadrotor UAVs which perform cooperative transportation of payloads is treated with the use of nonlinear optimal and multi-loop flatness-based control methods. The load is suspended with a link from a cart which is turn in connected through cables with two quadrotors. The aim is to compute the flight path and the control inputs of the quadrotors that will allow to lift the load and move it to any desirable final position. First, the dynamic model of the cable-suspended load is obtained through Euler-Lagrange analysis. Despite underactuation the associated nonlinear optimal control problem is solved, thus allowing to compute the lift forces of the cables that enable the load to move on the vertical plane until it reaches the targeted position. These forces are also applied with opposite sign to the quadrotors’ side through joints at the other end of the cables. Thus, the dynamic model of the quadrotors is updated by including in it additional drag forces which are due to the tension of the cables. The flight paths for the two quadrotors that enables to bring the suspended load to its final position are also computed. Next, for each quadrotor the nonlinear control and path following problem is solved, taking into account the cable-induced drag forces effects. To this end, a flatness-based control approach which is implemented in successive loops is applied to each quadrotor. The state-space model of each quadrotor UAV is separated into subsystems, which are connected between them in cascading loops. Each one of these subsystems can be viewed independently as a differentially flat system and control about it can be performed with inversion of its dynamics as in the case of input-output linearized flat systems. The state variables of the second subsystem become virtual control inputs for the first subsystem. In turn, exogenous control inputs are applied to the second subsystem. The whole control method is implemented in two successive loops and its global stability properties are also proven through Lyapunov stability analysis. The whole procedure is repeated at each sampling instance, that is (i) solution of the nonlinear optimal control problem for the transportation of the payload (ii) computation of the drag forces which are exerted on the UAVs due to lifting the load, (iii) solution of the multi-loop flatness-based control problem for the individual UAVs. This control method allows each quadrotor to follow precisely the defined flight path and finally achieves to bring the load to the targeted position.

In this paper, we present a method for the manipulation of objects in maritime environments, in particular for the transportation of objects between two docked ships using a 6-DoF robotic arm. The presented method uses RGBD camera mounted on the robot arm end effector to localize the object with respect to the robot arm base. The design of a gripper, and a control method based on force and torque measurements provide a robust method for picking up object, that can compensate the influence of relative ship motion due to waves and localization error. The components of the proposed method, as well as the entire manipulation procedure have been tested both in a laboratory environment and in a real-world maritime environments scenario and have shown that the solution works reliably.

There is great interest in alternative pollination strategies for crop production in the face of climate change and perennial threats to the traditional pollination mechanisms. This review explores the potential for robotic pollination in response to these challenges to crop fertilization and global food production. Herein we describe the viability of novel robotic systems equipped with artificial intelligence and machine vision, as alternatives to traditional insect pollination. We examine the technological progress and challenges for both aerial and ground-based robotic artificial pollination systems and emphasize the need for continued research and development in this area to ensure sustainable agricultural productivity. This paper highlights the importance of robotic pollination as a practical and environmentally sustainable approach in modern agriculture, amidst burgeoning ecological threats and a dwindling agricultural workforce.

Miniature personal aerial vehicles (PAVs) with vertical take-off and landing (VTOL) capabilities offer numerous advantages over existing vehicles, particularly in terms of high maneuverability, manned flight capability, and load-carrying capacity required during rescue missions. However, the detailed research work was reported infrequently. In this paper, a miniature PAV capable of piloted operation in both standing and sitting postures is introduced. This PAV, weighing 45kg and measuring 45cm*87cm*154cm, is designed for easy transport and minimal spatial footprint. It incorporates five vertically arranged micro turbojet engines that enable VTOL capabilities and can carry loads exceeding 100kg while maintaining high maneuverability. Notably, a two-degree-of-freedom nozzle mechanism attached to the engines allows for precise thrust direction adjustments. Building upon this propulsion system and the physical model of the PAV, a cascade proportional-integral-derivative (PID) controller is specifically designed for the PAV to regulate its position and attitude. Additionally, a feed-forward-based proportional-derivative (PD) controller is implemented to enhance the engine’s thrust response. The PAV prototype underwent rigorous testing in various outdoor conditions, ranging from temperatures of -7℃ to 42℃ and wind speeds of 0 to 7.2m/s. The test results substantiate the feasibility of the proposed PAV’s working principles, demonstrating its adaptability to environmental fluctuations. While the focus of this paper lies on the miniature PAV system, its implications extend to broader applications within advanced air mobility research.

2D LiDAR simultaneous localization and mapping(SLAM) often encounters issues such as registration failures, cumulative errors, and low scan repeatability, which prevent the construction of consistent and accurate maps. To address these challenges and reduce the need for remapping, this study proposes an interactive solution for 2D LiDAR SLAM, commonly used in AGVs, by manually adding or modifying constraints. However, for large-scale environments, manually adding and correcting constraints can be highly labor-intensive and difficult. To overcome this, an automatic loop closure detection and edge enhancement strategy was developed. A multi-scale loop closure detection algorithm enables fast loop closure detection, while the edge enhancement strategy revisits the pose graph to reduce overall error. Additionally, a GUI application was developed to visualize the pose graph, edges, and node information, allowing for interactive batch optimization. Finally, several experiments were conducted to demonstrate the performance improvements achieved by the proposed method.

Pipelines, valves, and pressure vessels (PVs) are vital components in nuclear power, thermoelectric, and chemical systems, operating under high-temperature and high-pressure conditions. Ensuring their safe operation requires regular inspections, which current robotic systems cannot fully address due to the diverse and challenging environments. To address this need, we propose a robotic system featuring a narrow-waist spring torso, airbag foot supports, and propeller negative pressure adsorption. The robot exhibits three key features: First, the narrow-waist spring torso enables extensive telescopic and multi-directional bending deformation, allowing it to navigate sharp turns and avoid edges of necks within stop valves. Second, the airbags offer exceptional passive compliance and large deformation capacity, adapting to significant size variations and shape changes within valve cavities for reliable anchoring. Third the propeller negative pressure adsorption allows the robot to traverse PV inner walls with poor surface conditions, such as corrosion and scaling. The robot employs a worm-like creeping motion to adapt to complex internal channels and varying curvatures of pipelines and PVs. Experimental results demonstrate the robot’s ability to smoothly traverse a DN125 stop valve (84 mm-125 mm internal diameter) and move between pipelines and PVs. This confirms its capability to operate across multiple scenes, adapting to wall curvature radii from 55.5 mm to infinity and handling bosses, shape changes and size variations. This robotic system provides a valuable reference for designing non-disassembly internal inspection robots in gas and liquid transport systems, enhancing safety and reliability in high-temperature and high-pressure environments.

In this study, a hydraulic arm was developed for the robotic cotton picker and integrated with the e-vehicle. The hydraulic arm had three degrees of freedom (DOFs), that moved in vertical, rotational, and radial motion. A roller-type end-effector was installed at the front of the arm that came close to the cotton boll and separated it from the plant. A vacuum unit was generating a suction pressure in the hose pipe, allowing it to suck up the picked cotton bolls and transport them into the storage tank. The hydraulic arm with e-vehicle was operated in the field for three different plant-to-plant spacing 250, 350, and 450 mm at suction pressures 25, 50, and 75 mmHg to assess its performance. The maximum picking efficiency and picking capacity were 92.79 %, and 567 bolls picked/h, respectively observed for plant-to-plant spacing of 450 mm at a suction pressure of 75 mmHg. The minimum damage of green bolls and losses of cotton bolls were 126 bolls/ha, and 3.79 %, respectively observed for plant-to-plant spacing 450 mm at a suction pressure of 75 mmHg. The minimum trash content was 20.50 g observed for plant-to-plant spacing 450 mm at a suction pressure of 25 mmHg. The most suitable combination for achieving the acceptable performance of a hydraulic arm integrated with an e-vehicle in a cotton crop was the plant-to-plant spacing of 450 mm and suction pressure of 75 mmHg.

Despite the increasing prevalence of robots in daily life, their navigation capabilities are still limited to environments with prior knowledge, such as a global map. To fully unlock the potential of robots, it is crucial to enable them to navigate in large-scale unknown and changing unstructured scenarios. This requires the robot to construct an accurate static map in real-time as it explores, while filtering out moving objects to ensure mapping accuracy and, if possible, achieving high-quality pedestrian tracking and collision avoidance. While existing methods can achieve individual goals of spatial mapping or dynamic object detection and tracking, there has been limited research on effectively integrating these two tasks, which are actually coupled and reciprocal. In this work, we propose a solution called S 2MAT (Simultaneous and Self-Reinforced Mapping and Tracking) that integrates a front-end dynamic object detection and tracking module with a back-end static mapping module. S 2MAT leverages the close and reciprocal interplay between these two modules to efficiently and effectively solve the open problem of simultaneous tracking and mapping in highly dynamic scenarios. The proposed method is primarily designed for use with 3D LiDAR and offers a solution for real-time navigation in large-scale, unknown dynamic scenarios with a low computational cost, making it feasible for deployment on onboard computers equipped with only a single CPU. We conducted long-range experiments in real-world urban scenarios spanning over 7km, which included challenging obstacles like pedestrians and other traffic agents. The successful navigation provides a comprehensive test of S 2MAT’s robustness, scalability, efficiency, quality, and its ability to benefit autonomous robots in wild scenarios without pre-built maps.

not-yet-known not-yet-known not-yet-known unknown Public datasets play a crucial role in advancing autonomous robotics research. The rapid evolution of sensors and applications continually drives the need for new datasets. For instance, the shift from mechanical LiDAR (Light Detection and Ranging) sensors on autonomous vehicles to (hybrid) solid-state LiDAR technologies like MEMS (Micro-electromechanical systems) LiDAR has brought about enhanced durability and reduced costs. However, datasets supporting research on these sensors are scarce. This paper presents the multi-modular HK-MEMS dataset, incorporating data from LiDARs, a camera, GNSS, and Inertial Navigation Systems. Notably, it is the first dataset to offer automotive-grade MEMS LiDAR data for research in Simultaneous Localization and Mapping (SLAM). Compared with existing datasets, our data emphasize extreme environments like degenerate urban tunnels and dynamic scenarios, aiming to enhance the robustness of SLAM systems. The data are collected on various platforms including a handheld device, a mobile robot, and notably, buses with real driving behaviors. We collect 187 minutes and 75.4 kilometers of data. State-of-the-art SLAM methods are evaluated on this benchmark. The result highlights the challenges in extreme environments and underscores the ongoing need to enhance the robustness of SLAM systems. This dataset serves as a valuable platform for exploring the potential and limitations of MEMS LiDAR, and a challenge to enhance the robustness of SLAM in urban navigation scenarios. The data is available at https://github.com/RuanJY/HK_MEMS_Dataset.

In complex and dynamic environments, the decision-making sequence of individual robots significantly influences the effectiveness of collaboration and cooperation among multi-robot systems in completing tasks. This paper focuses on the division of labor in autonomous multi-robot systems, aiming to find optimal strategies for resource allocation among robots operating in complex scenarios. Each robot makes independent yet interacting decisions in relatively isolated dynamic environments. We propose a model that applies game theory from economics, classifying the robots into resource-providing robots and resource-consuming robots. Resource providers acquire resources and compete to determine the optimal strategy, while resource consumers purchase resources, making decisions based on the pricing set by providers.The problem is formulated as a two-stage game. In the first stage, resource providers engage in resource games, abstracted into Cournot or Stackelberg models, where optimal decisions are made based on available resources and estimated strategies of other participants. The second stage involves price games between providers and consumers, analogous to market supply and demand relationships. Price adjustments and demand changes lead to the discovery of Nash equilibria in the price game. Simulations are conducted to compare the system-wide benefits when providers adopt different strategies in the first stage. Results indicate that using the Stackelberg model yields higher overall benefits, further demonstrating the practicality and effectiveness of the proposed strategies. This highlights the importance of strategic model selection in optimizing the performance and resource efficiency of multi-robot systems operating in dynamic environments.