open-source

General-purpose clothes-folding robots do not yet exist owing to the deformable nature of textiles, making it hard to engineer manipulation pipelines or learn this task. In order to accelerate research for the learning of the robotic clothes folding task, we introduce a video dataset of human folding demonstrations. In total, we provide 8.5 hours of demonstrations from multiple perspectives leading to 1,000 folding samples of different types of textiles. The demonstrations are recorded in multiple public places, in different conditions with a diverse set of people. Our dataset consists of anonymized RGB images, depth frames, skeleton keypoint trajectories, and object labels. In this article, we describe our recording setup, the data format, and utility scripts, which can be accessed at https://adverley.github.io/folding-demonstrations.

We present a diverse dataset of industrial metal objects with unique characteristics such as symmetry, texturelessness, and high reflectiveness. These features introduce challenging conditions that are not captured in existing datasets. Our dataset comprises both real-world and synthetic multi-view RGB images with 6D object pose labels. Real-world data were obtained by recording multi-view images of scenes with varying object shapes, materials, carriers, compositions, and lighting conditions. This resulted in over 30,000 real-world images. We introduce a new public tool that enables the quick annotation of 6D object pose labels in multi-view images. This tool was used to provide 6D object pose labels for all real-world images. Synthetic data were generated by carefully simulating real-world conditions and varying them in a controlled and realistic way. This resulted in over 500,000 synthetic images. The close correspondence between synthetic and real-world data and controlled variations will facilitate sim-to-real research. Our focus on industrial conditions and objects will facilitate research on computer vision tasks, such as 6D object pose estimation, which are relevant for many industrial applications, such as machine tending. The dataset and accompanying resources are available on the project website.

Assistive robots should be able to wash, fold or iron clothes. However, due to the variety, deformability and self-occlusions of clothes, creating robot systems for cloth manipulation is challenging. Synthetic data is a promising direction to improve generalization, but the sim-to-real gap limits its effectiveness. To advance the use of synthetic data for cloth manipulation tasks such as robotic folding, we present a synthetic data pipeline to train keypoint detectors for almost-flattened cloth items. To evaluate its performance, we have also collected a real-world dataset. We train detectors for both T-shirts, towels and shorts and obtain an average precision of 64% and an average keypoint distance of 18 pixels. Fine-tuning on real-world data improves performance to 74% mAP and an average distance of only 9 pixels. Furthermore, we describe failure modes of the keypoint detectors and compare different approaches to obtain cloth meshes and materials. We also quantify the remaining sim-to-real gap and argue that further improvements to the fidelity of cloth assets will be required to further reduce this gap. The code, dataset and trained models are available here.

Unfolding cloth in the air is a fundamental task in robotic cloth manipulation, essential for various subsequent tasks like folding, hanging, or even assisted dressing. It has been extensively studied, yet, until now a standardized performance comparison across many different labs and methods has never been done. The ICRA 2024 Cloth Competition marks a significant milestone by offering a head-to-head evaluation of 11 diverse teams on a shared real-world robotic platform. This unprecedented event not only sets a new benchmark for the field but also fosters collaboration and innovation, while providing a comprehensive dataset to drive future research towards more robust and generalizable solutions in robotic cloth manipulation.

The study of the dynamic responses of plants to short-term environmental changes is becoming increasingly important in basic plant science, phenotyping, breeding, crop management, and modelling. These short-term variations are crucial in plant adaptation to new environments and, consequently, in plant fitness and productivity. Scalable, versatile, accurate, and low-cost data-logging solutions are necessary to advance these fields and complement existing sensing platforms such as high-throughput phenotyping. However, current data logging and sensing platforms do not meet the requirements to monitor these responses. Therefore, a new modular data logging platform was designed, named Gloxinia. Different sensor boards are interconnected depending upon the needs, with the potential to scale to hundreds of sensors in a distributed sensor system. To demonstrate the architecture, two sensor boards were designed—one for single-ended measurements and one for lock-in amplifier based measurements, named Sylvatica and Planalta, respectively. To evaluate the performance of the system in small setups, a small-scale trial was conducted in a growth chamber. Expected plant dynamics were successfully captured, indicating proper operation of the system. Though a large scale trial was not performed, we expect the system to scale very well to larger setups. Additionally, the platform is open-source, enabling other users to easily build upon our work and perform application-specific optimisations.

Monitoring climate change, and its impacts on ecological, agricultural, and other societal systems, is often based on temperature data derived from official weather stations. Yet, these data do not capture most microclimates, influenced by soil, vegetation and topography, operating at spatial scales relevant to the majority of organisms on Earth. Detecting and attributing climate change impacts with confidence and certainty will only be possible by a better quantification of temperature changes in forests, croplands, mountains, shrublands, and other remote habitats. There is an urgent need for a novel, miniature and simple device filling the gap between low-cost devices with manual data download (no instantaneous data) and high-end, expensive weather stations with real-time data access. Here, we develop an integrative real-time monitoring system for microclimate measurements: MIRRA (Microclimate Instrument for Real-time Remote Applications) to tackle this problem. The goal of this platform is the design of a miniature and simple instrument for near instantaneous, long-term and remote measurements of microclimates. To that end, we optimised power consumption and transfer data using a cellular uplink. MIRRA is modular, enabling the use of different sensors (e.g., air and soil temperature, soil moisture and radiation) depending upon the application, and uses an innovative node system highly suitable for remote locations. Data from separate sensor modules are wirelessly sent to a gateway, thus avoiding the drawbacks of cables. With this sensor technology for the long-term, low-cost, real-time and remote sensing of microclimates, we lay the foundation and open a wide range of possibilities to map microclimates in different ecosystems, feeding a next generation of models. MIRRA is, however, not limited to microclimate monitoring thanks to its modular and wireless design. Within limits, it is suitable or any application requiring real-time data logging of power-efficient sensors over long periods of time. We compare the performance of this system to a reference system in real-world conditions in the field, indicating excellent correlation with data collected by established data loggers. This proof-of-concept forms an important foundation to creating the next version of MIRRA, fit for large scale deployment and possible commercialisation. In conclusion, we developed a novel wireless cost-effective sensor system for microclimates.

Smart textiles have found numerous applications ranging from health monitoring to smart homes. Their main allure is their flexibility, which allows for seamless integration of sensing in everyday objects like clothing. The application domain also includes robotics; smart textiles have been used to improve human-robot interaction, to solve the problem of state estimation of soft robots, and for state estimation to enable learning of robotic manipulation of textiles. The latter application provides an alternative to computationally expensive vision-based pipelines and we believe it is the key to accelerate robotic learning of textile manipulation. Current smart textiles, however, maintain wired connections to external units, which impedes robotic manipulation, and lack modularity to facilitate state estimation of large cloths. In this work, we propose an open-source, fully wireless, highly flexible, light, and modular version of a piezoresistive smart textile. Its output stability was experimentally quantified and determined to be sufficient for classification tasks. Its functionality as a state sensor for larger cloths was also verified in a classification task where two of the smart textiles were sewn onto a piece of clothing of which three states are defined. The modular smart textile system was able to recognize these states with average per-class F1-scores ranging from 85.7 to 94.6% with a basic linear classifier.

The development of tactile sensing is expected to enhance robotic systems in handling complex objects like deformables or reflective materials. However, readily available industrial grippers generally lack tactile feedback, which has led researchers to develop their own tactile sensors, resulting in a wide range of sensor hardware. Reading data from these sensors poses an integration challenge: either external wires must be routed along the robotic arm, or a wireless processing unit has to be fixed to the robot, increasing its size. We have developed a microcontroller-based sensor readout solution that seamlessly integrates with Robotiq grippers. Our Arduino compatible design takes away a major part of the integration complexity of tactile sensors and can serve as a valuable accelerator of research in the field. Design files and installation instructions can be found at https://github.com/RemkoPr/airo-halberd.