publications

2022

1. Editorial : robots for learning

   Johal, Wafa, Belpaeme, Tony, and Chetouani, Mohamed

   2022

   Biblio
2. Distance-based delays in echo state networks

   Iacob, Stefan-Teodor, Freiberger, Matthias, and Dambre, Joni

   In Intelligent Data Engineering and Automated Learning (IDEAL 2022) : 23rd International Conference, IDEAL 2022, Proceedings 2022

   Abs Biblio

   Physical reservoir computing, a paradigm bearing the promise of energy-efficient high-performance computing, has raised much attention in recent years. We argue though, that the effect of signal propagation delay on reservoir task performance, one of the most central aspects of physical reservoirs, is still insufficiently understood in a more general learning context. Such physically imposed delay has been found to play a crucial role in some specific physical realizations, such as integrated photonic reservoirs. While delays at the readout layer and input of Echo State Networks (ESNs) have been successfully exploited before to improve performance, to our knowledge this feature has not been studied in a more general setting. We introduce inter-node delays, based on physical distances, into ESNs as model systems for physical reservoir computing. We propose a novel ESN design that includes variable signal delays along the connections between neurons, comparable to varying axon lengths in biological neural networks or varying length delay lines in physical systems. We study the impact of the resulting variable inter-node delays in this setup in comparison with conventional ESNs and find that incorporating variable delays significantly improves reservoir performance on the NARMA-10 benchmark task.

3. Hardware-Aware Mobile Building Block Evaluation for Computer Vision

   Bonnaerens, Maxim, Freiberger, Matthias, Verhelst, Marian, and Dambre, Joni

   Applied Sciences 2022

   Abs Biblio

   In this paper, we propose a methodology to accurately evaluate and compare the performance of efficient neural network building blocks for computer vision in a hardware-aware manner. Our comparison uses pareto fronts based on randomly sampled networks from a design space to capture the underlying accuracy/complexity trade-offs. We show that our approach enables matching of information obtained by previous comparison paradigms, but provides more insights into the relationship between hardware cost and accuracy. We use our methodology to analyze different building blocks and evaluate their performance on a range of embedded hardware platforms. This highlights the importance of benchmarking building blocks as a preselection step in the design process of a neural network. We show that choosing the right building block can speed up inference by up to a factor of two on specific hardware ML accelerators.

4. Visual conversation starters for human-robot interaction

   Janssens, Ruben, Demeester, Thomas, and Belpaeme, Tony

   2022

   Biblio
5. A comparative analysis on genome pleiotropy for evolved soft robots

   Marzougui, Dries, Biondina, Matthijs, and wyffels, Francis

   In Faculty of Engineering and Architecture Research Symposium 2022 (FEARS 2022), Abstracts 2022

   Abs Biblio

   Biological evolution shapes the body and brain of living creatures together over time. By contrast, in evolutionary robotics, brain-body co-optimization remains challenging. Conflicting mutations cause dissociation between morphology and control, which leads to premature convergence. Recent works have proposed algorithmic modifications to mitigate the impact of conflicting mutations. However, the importance of genetic design remains underexposed. Current approaches are divided between a single, pleiotropic genetic encoding and two isolated encodings representing morphology and control. This design choice is commonly made ad hoc, causing a lack of consistency for practitioners. To standardize this design, we performed a comparative analysis between these two configurations and two previously unexplored alternatives on a soft robot locomotion task.

6. Learning keypoints from synthetic data for robotic cloth folding

   Lips, Thomas, De Gusseme, Victor-Louis, and wyffels, Francis

   In Faculty of Engineering and Architecture Research Symposium 2022 (FEARS 2022), Abstracts 2022

   Abs Biblio

   Robotic cloth manipulation is challenging due to its deformability, which makes determining its full state infeasible. However, for cloth folding, it suffices to know the position of a few semantic keypoints. Convolutional neural networks (CNN) can be used to detect these keypoints, but require large amounts of annotated data, which is expensive to collect. To overcome this, we propose to learn these keypoint detectors purely from synthetic data, enabling low-cost data collection. In this paper, we procedurally generate images of towels and use them to train a CNN. We evaluate the performance of this detector for folding towels on a unimanual robot setup and find that the grasp and fold success rates are 77% and 53%, respectively. We conclude that learning keypoint detectors from synthetic data for cloth folding and related tasks is a promising research direction, discuss some failures and relate them to future work. A video of the system, as well as the codebase, more details on the CNN architecture and the training setup can be found at https://github.com/tlpss/workshop-icra-2022-cloth-keypoints

7. Photonic reservoir computing for nonlinear equalization of 64-QAM signals with a Kramers-Kronig receiver

   Masaad, Sarah, Gooskens, Emmanuel, Sackesyn, Stijn, Dambre, Joni, and Bienstman, Peter

   In European Conference on Optical Communication (ECOC 2022) 2022

   Biblio
8. BeCoS Corpus: Belgian Covid-19 Sign Language Corpus. A Corpus for Training Sign Language Recognition and Translation

   Vandeghinste, Vincent, Van Dyck, Bob, De Coster, Mathieu, Goddefroy, Maud, and Dambre, Joni

   Computational Linguistics in the Netherlands Journal 2022

   Biblio
9. Human-robot interaction

   Belpaeme, Tony

   2022

   Biblio
10. A social robot activity for novice programmers

    Staey, Zimcke, Gesquière, Natacha, and wyffels, Francis

    In RIE 2021, 12th International Conference on Robotics in Education, Proceedings 2022

    Abs Biblio

    We have developed an activity on social robots for learners of ages 11-14, as part of a broader AI and CT curriculum for secondary education. We thereby address some of the difficulties that teachers experience when teaching physical computing and challenges of CS education such as gender equality. During the project, learners will use a graphical programming language and simulator to design and implement their robot. Afterward, they can transfer their design to a physical robot.

11. The effectiveness of dynamically processed incremental descriptions in human robot interaction

    Wallbridge, Christopher D., Smith, Alex, Giuliani, Manuel, Melhuish, Chris, Belpaeme, Tony, and Lemaignan, Séverin

    ACM TRANSACTIONS ON HUMAN-ROBOT INTERACTION 2022

    Abs Biblio

    We explore the effectiveness of a dynamically processed incremental referring description system using under-specified ambiguous descriptions that are then built upon using linguistic repair statements, which we refer to as a dynamic system. We build a dynamically processed incremental referring description generation system that is able to provide contextual navigational statements to describe an object in a potential real-world situation of nuclear waste sorting and maintenance. In a study of 31 participants, we test the dynamic system in a case where a user is remote operating a robot to sort nuclear waste, with the robot assisting them in identifying the correct barrels to be removed. We compare these against a static non-ambiguous description given in the same scenario. As well as looking at efficiency with time and distance measurements, we also look at user preference. Results show that our dynamic system was a much more efficient method—taking only 62% of the time on average—for finding the correct barrel. Participants also favoured our dynamic system.

12. Multi-modal open world user identification

    Irfan, Bahar, Ortiz, Michael Garcia, Lyubova, Natalia, and Belpaeme, Tony

    ACM TRANSACTIONS ON HUMAN-ROBOT INTERACTION 2022

    Abs Biblio

    User identification is an essential step in creating a personalised long-term interaction with robots. This requires learning the users continuously and incrementally, possibly starting from a state without any known user. In this article, we describe a multi-modal incremental Bayesian network with online learning, which is the first method that can be applied in such scenarios. Face recognition is used as the primary biometric, and it is combined with ancillary information, such as gender, age, height, and time of interaction to improve the recognition. The Multi-modal Long-term User Recognition Dataset is generated to simulate various human-robot interaction (HRI) scenarios and evaluate our approach in comparison to face recognition, soft biometrics, and a state-of-the-art open world recognition method (Extreme Value Machine). The results show that the proposed methods significantly outperform the baselines, with an increase in the identification rate up to 47.9% in open-set and closed-set scenarios, and a significant decrease in long-term recognition performance loss. The proposed models generalise well to new users, provide stability, improve over time, and decrease the bias of face recognition. The models were applied in HRI studies for user recognition, personalised rehabilitation, and customer-oriented service, which showed that they are suitable for long-term HRI in the real world.

13. Modular piezoresistive smart textile for state estimation of cloths

    Proesmans, Remko, Verleysen, Andreas, Vleugels, Robbe, Veske, Paula, De Gusseme, Victor-Louis, and wyffels, Francis

    SENSORS 2022

    Abs Biblio

    Smart textiles have found numerous applications ranging from health monitoring to smart homes. Their main allure is their flexibility, which allows for seamless integration of sensing in everyday objects like clothing. The application domain also includes robotics; smart textiles have been used to improve human-robot interaction, to solve the problem of state estimation of soft robots, and for state estimation to enable learning of robotic manipulation of textiles. The latter application provides an alternative to computationally expensive vision-based pipelines and we believe it is the key to accelerate robotic learning of textile manipulation. Current smart textiles, however, maintain wired connections to external units, which impedes robotic manipulation, and lack modularity to facilitate state estimation of large cloths. In this work, we propose an open-source, fully wireless, highly flexible, light, and modular version of a piezoresistive smart textile. Its output stability was experimentally quantified and determined to be sufficient for classification tasks. Its functionality as a state sensor for larger cloths was also verified in a classification task where two of the smart textiles were sewn onto a piece of clothing of which three states are defined. The modular smart textile system was able to recognize these states with average per-class F1-scores ranging from 85.7 to 94.6% with a basic linear classifier.

14. On the pivotal role of water potential to model plant physiological processes

    De Swaef, Tom, Pieters, Olivier, Appeltans, Simon, Borra-Serrano, Irene, Coudron, Willem, Couvreur, Valentin, Garré, Sarah, Lootens, Peter, Nicolaï, Bart, Pols, Leroi, Saint Cast, Clément, Šalagovič, Jakub, Van Haeverbeke, Maxime, Stock, Michiel, and wyffels, Francis

    IN SILICO PLANTS 2022

    Abs Biblio

    Water potential explains water transport in the Soil-Plant-Atmosphere Continuum (SPAC), and is gaining interest as connecting variable between ‘pedo-, bio- and atmosphere’. It is primarily used to simulate hydraulics in the SPAC, and is thus essential for studying drought effects. Recent implementations of hydraulics in large-scale Terrestrial Biosphere Models (TBMs) improved their performance under water-limited conditions, while hydraulic features of recent detailed Functional-Structural Plant Models (FSPMs) open new possibilities for dissecting complex traits for drought tolerance. These developments in models across scales deserve a critical appraisal to evaluate its potential for wider use in FSPMs, but also in crop systems models (CSMs), where hydraulics are currently still absent. After refreshing the physical basis, we first address models where water potential is primarily used for describing water transport along the transpiration pathway from the soil to the leaves, through the roots, the xylem and the leaf mesophyll. Then, we highlight models for three ecophysiological processes, which have well-recognised links to water potential: phloem transport, stomatal conductance and organ growth. We identify water potential as the bridge between soil, root and shoot models, as the physiological variable integrating below- and above-ground abiotic drivers, but also as the link between water status and growth. Models making these connections enable identifying crucial traits for ecosystem resilience to drought and for breeding towards improved drought tolerance in crops. Including hydraulics often increases model complexity, and thus requires experimental data on soil and plant hydraulics. Nevertheless, modelling hydraulics is insightful at different scales (FSPMs, CSMs and TBMs).

15. Reservoir computing with plants

    Pieters, Olivier

    2022

    Abs Biblio

    Planten zijn complexe organismen, onderhevig aan een groot aantal omgevingsvariabelen, dewelke op hun beurt de fysiologie en het fenotype van de plant beïnvloeden. In ons werk wordt dit complex ingangsgedreven systeem beschouwd als een reservoir in physical reservoir computing (PRC). PRC is een paradigma uit de computerwetenschappen dat een fysisch substraat gebruikt als rekenmedium. In een eerste luik van het doctoraat werd de toepasbaarheid van hyperspectrale camera’s onderzocht. Dergelijke camera’s blijken niet geschikt te zijn om subtiele spectrale veranderingen, nodig voor PRC, te detecteren. Daarop werd een nieuw sensor platform ontwikkeld, specifiek ontworpen om contactsensoren uit te lezen. Aan de hand van bladdikte sensoren toonden we aan dat planten inderdaad als reservoir gebruikt kunnen worden voor PRC. Planten waren performanter dan een controle substraat in het oplossen van eco-fysiologische- en omgevingstaken. We onderzochten ook controle taken zoals NARMA en een vertragingslijn. Deze resultaten geven aan dat planten niet geschikt zijn om niet- plantspecifieke taken op te lossen, maar wel goed presteren in plant- gerelateerde taken. Deze eerste experimentele demonstratie van PRC met planten is een belangrijke mijlpaal naar een meer holistische kijk op fenotypering en informatieverwerking door planten.

16. Have I got the power? Analysing and reporting statistical power in HRI

    Bartlett, Madeleine E., Edmunds, C. E. R., Belpaeme, Tony, and Thill, Serge

    ACM TRANSACTIONS ON HUMAN-ROBOT INTERACTION 2022

    Abs Biblio

    This article presents a discussion of the importance of power analyses, providing an overview of when power analyses should be run in the context of the field of Human-Robot Interaction, as well as some examples of how to perform a power analysis. This work was motivated by the observation that the majority of papers published in the proceedings of recent HRI conferences did not report conducting a power analysis; an observation that has concerning implications for many conclusions drawn by these studies. This work is intended to raise awareness and encourage researchers to conduct power analyses when designing research studies using human participants.

17. A review of evaluation practices of gesture generation in embodied conversational agents

    Wolfert, Pieter, Robinson, Nicole, and Belpaeme, Tony

    IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS 2022

    Abs Biblio

    Embodied conversational agents (ECAs) are often designed to produce nonverbal behavior to complement or enhance their verbal communication. One such form of the nonverbal behavior is co-speech gesturing, which involves movements that the agent makes with its arms and hands that are paired with verbal communication. Co-speech gestures for ECAs can be created using different generation methods, divided into rule-based and data-driven processes, with the latter, gaining traction because of the increasing interest from the applied machine learning community. However, reports on gesture generation methods use a variety of evaluation measures, which hinders comparison. To address this, we present a systematic review on co-speech gesture generation methods for iconic, metaphoric, deictic, and beat gestures, including reported evaluation methods. We review 22 studies that have an ECA with a human-like upper body that uses co-speech gesturing in social human-agent interaction. This includes studies that use human participants to evaluate performance. We found most studies use a within-subject design and rely on a form of subjective evaluation, but without a systematic approach. We argue that the field requires more rigorous and uniform tools for co-speech gesture evaluation, and formulate recommendations for empirical evaluation, including standardized phrases and example scenarios to help systematically test generative models across studies. Furthermore, we also propose a checklist that can be used to report relevant information for the evaluation of generative models, as well as to evaluate co-speech gesture use.

18. ’Cool glasses, where did you get them?’ : generating visually grounded conversation starters for human-robot dialogue

    Janssens, Ruben, Wolfert, Pieter, Demeester, Thomas, and Belpaeme, Tony

    In PROCEEDINGS OF THE 2022 17TH ACM/IEEE INTERNATIONAL CONFERENCE ON HUMAN-ROBOT INTERACTION (HRI ’22) 2022

    Abs Biblio

    Visually situated language interaction is an important challenge in multi-modal Human-Robot Interaction (HRI). In this context we present a data-driven method to generate situated conversation starters based on visual context. We take visual data about the interactants and generate appropriate greetings for conversational agents in the context of HRI. For this, we constructed a novel open-source data set consisting of 4000 HRI-oriented images of people facing the camera, each augmented by three conversation-starting questions. We compared a baseline retrieval-based model and a generative model. Human evaluation of the models using crowdsourcing shows that the generative model scores best, specifically at correctly referencing visual features. We also investigated how automated metrics can be used as a proxy for human evaluation and found that common automated metrics are a poor substitute for human judgement. Finally, we provide a proof-of-concept demonstrator through an interaction with a Furhat social robot.

19. Assessment of code, which aspects do teachers consider and how are they valued?

    Neutens, Tom, Coolsaet, Kris, and wyffels, Francis

    ACM TRANSACTIONS ON COMPUTING EDUCATION 2022

    Abs Biblio

    In many countries, computer programming is becoming an integral part of the secondary school curriculum. However, many teachers, especially in the first years of Flemish secondary school, have limited experience with teaching programming. To improve their knowledge about programming, many different types of professional development programs have been proposed. Nevertheless, these programs mostly focus on technical skills and less on pedagogical skills. One aspect that is often overlooked in these programs is how teachers can assess code. To get insight into what teachers currently value when assessing code, we designed an experiment that analyzes the different aspects teachers consider during the assessment of code. During the experiment, the teachers (N=13) assess a set of programs from five different fictional learners. After the assessment, they participated in a structured interview giving us insight into the assessment process. We evaluated the transcripts of the interviews using deductive thematic analysis using a coding schema defining the different aspects of code that can be assessed. Additionally, we linked the assessment strategies of teachers to their teaching experience. Our results indicate that many teachers are unaware of the different concepts that can be part of the assessment of code which might lead to inaccurate or invalid feedback. Moreover, although our experimental group was too small to draw hard conclusions about the inter case results, our results indicate that the number of concepts considered by teachers seems to increase with experience. These results provide an initial insight into the code assessment practices of teachers and reveals interesting pathways for future research into the assessment of code.

20. Leveraging frozen pretrained written language models for neural sign language translation

    De Coster, Mathieu, and Dambre, Joni

    INFORMATION 2022

    Abs Biblio

    We consider neural sign language translation: machine translation from signed to written languages using encoder–decoder neural networks. Translating sign language videos to written language text is especially complex because of the difference in modality between source and target language and, consequently, the required video processing. At the same time, sign languages are low-resource languages, their datasets dwarfed by those available for written languages. Recent advances in written language processing and success stories of transfer learning raise the question of how pretrained written language models can be leveraged to improve sign language translation. We apply the Frozen Pretrained Transformer (FPT) technique to initialize the encoder, decoder, or both, of a sign language translation model with parts of a pretrained written language model. We observe that the attention patterns transfer in zero-shot to the different modality and, in some experiments, we obtain higher scores (from 18.85 to 21.39 BLEU-4). Especially when gloss annotations are unavailable, FPTs can increase performance on unseen data. However, current models appear to be limited primarily by data quality and only then by data quantity, limiting potential gains with FPTs. Therefore, in further research, we will focus on improving the representations used as inputs to translation models.

21. Learning self-supervised task progression metrics : a case of cloth folding

    Verleysen, Andreas, Biondina, Matthijs, and wyffels, Francis

    APPLIED INTELLIGENCE 2022

    Abs Biblio

    An important challenge for smart manufacturing systems is finding relevant metrics that capture task quality and progression for process monitoring to ensure process reliability and safety. Data-driven process metrics construct features and labels from abundant raw process data, which incurs costs and inaccuracies due to the labelling process. In this work, we circumvent expensive process data labelling by distilling the task intent from video demonstrations. We present a method to express the task intent in the form of a scalar value by aligning a self-supervised learned embedding to a small set of high-quality task demonstrations. We evaluate our method on the challenging case of monitoring the progress of people folding clothing. We demonstrate that our approach effectively learns to represent task progression without manually labelling sub-steps or progress in the videos. Using case-based experiments, we find that our method learns task-relevant features and useful invariances, making it robust to noise, distractors and variations in the task and shirts. The experimental results show that the proposed method can monitor processes in domains where state representation is inherently challenging.

22. Learning robotic cloth manipulation

    Verleysen, Andreas

    2022

    Abs Biblio

    Endowing robots with dexterous manipulation skills could spur economic welfare, create leisure time in society, reduce harmful manual labour, and provide care for an ageing population. However, while robots are producing our cars, we are still left to our own devices for doing the laundry at home. This shortcoming is due to the major difficulties in perceiving and handling the variability the real world presents. Robots in modern manufacturing require engineers to produce a safe and predictable environment: objects arrive at the same location, in the same orientation, and the robot is preprogrammed to perform a specific manipulation. Unfortunately, the need for a predictable environment is undesirable in dynamic environments that handle a wide range of objects, often in the presence of human activity. For example, should a human worker first unfold all clothes such that a robot can easily find the corner points and perform folding? Indeed, the high variability in modern production environments and households requires robots to handle objects that can take arbitrary shapes, weights, and configurations. This diversity renders traditional robotic control algorithms and grippers unsuitable for deployment in dynamic environments. To find methods that can handle the ever-changing nature of human environments, we study the perception and manipulation of objects that provide an infinite amount of variations: deformable objects. A deformable object changes shape on force interaction. Deformable objects are omnipresent in industry and society: food, paper, clothes, fruit, cables and sutures, among others. In particular, we study the task of automating the folding of clothes. Folding clothes is a common household task that will potentially be performed by service robots in the future. Handling cloth is also relevant in manufacturing, where technical textile is processed, and in the fashion industry. Dealing with the deformable nature of textiles requires fundamental improvements in both hardware and software. Mechanical engineering needs to incorporate actuators, links, joints and sensors into the limited space of a hand while using soft materials similar to the human skin. In addition to engineering more capable hands, control algorithms need to loosen assumptions about the environment in which robots operate. It is unattainable to expect highly deformable objects like cloth to always be in the same configuration before manipulating them with a robot. A solution for dealing with real-world variability can be found in the machine learning field. In particular, deep RLcombines the function approximation capabilities of deep neural networks with the learn by trial-and-error formalism provided by RL. Deep RL has shown to be capable of driving cars, flying helicopters and manipulating rigid objects. However, the data requirements for training highly parameterized functions, like neural networks, are considerable. This data-hungry property causes an incongruity between the representational learning features of deep neural networks and the high cost of generating real robotic trials. Our research focuses on reducing the required learning data for systems that perceive and manipulate clothing items. We implement a cloth simulation method to generate synthetic data, utilize smart textiles for state estimation of cloth, crowdsource a dataset of people folding clothing, and propose a method to estimate how well people are folding clothing without providing labels. Actuating a physical robot is slow, expensive and potentially dangerous. For this reason, roboticists resort to physics simulators that simulate the robot’s and environment’s dynamics. However, there exists no integrated robot and cloth simulator for use in learning experiments. Cloth simulators are built for offline render farms in the film industry, or for the game industry that sacrifices fidelity for real-time rendering. Unfortunately, cloth simulation for robotic learning requires performance characteristics similar to online rendering and accuracy aspects found in offline rendering. For this reason, we implement a custom cloth dynamics simulation on GPU and integrate it in the robotic simulation functionality of the Unity game engine. We found that we can utilize deep RL to train an agent in our simulation to fold a rectangular piece of cloth twice within 24 hours of wall time on standard computational hardware. The developed cloth simulation assumes full accessibility to the state of the cloth. However, state estimation of cloth in the real world relies on complex vision-based pipelines or high-cost sensing technology. We avoid this complexity and cost by integrating inexpensive tactile sensing technology into a cloth. The cloth becomes an active smart cloth by training a classifier that uses the tactile sensing data to estimate its state. We use this smart cloth to train a low-cost robotic platform to fold the cloth using RL. Our results demonstrate that it is possible to develop a smart cloth with off-the-shelf components and use it effectively for training on a real robotic platform. Our smart cloth bridges the gap between our cloth simulation on GPU and state estimation in the real world. However, it is still required to distil a scalar value that indicates task progress in order to acquire manipulation skills using RL. We believe that learning the reward function from demonstrations may overcome human bias in reward engineering. Unfortunately, when starting our research there existed no large dataset with people folding clothing. We fill this gap by crowdsourcing a dataset of people folding clothes. Our dataset consists of roughly 300000 multiperspective RGB-D frames, annotated with pose trajectories, quality labels and timestamps indicating substeps. This dataset can be used to benchmark research in action recognition and bootstrap learning by using example demonstrations. Learning from demonstrations is a prevalent domain in the robotics learning community. However, using our cloth folding dataset requires mapping the movements of demonstrators to the embodiment of a robot. Additionally, behavioural cloning is prone to blindly imitating trajectories instead of understanding how actions relate to solving the task. For this reason, estimating how well a process is being executed is preferred to learning the policy from demonstrations. Unfortunately, existing methods couple the learning of rewards with policy learning, thereby inheriting all problems associated with RL. To decouple reward and policy learning, we propose a method to learn the task progression from multiperspective videos of example demonstrations. We avoid incorporating human bias in the labelling process by using time as a self-supervised signal for learning. We demonstrate the first results on expressing task progression of people folding clothing, without labelling any data. General-purpose robots are not yet among us. Robots that are capable of working in dynamic environments will require a holistic view of software and hardware. We demonstrated the benefits of this approach by outsourcing the intelligence for state estimation to the cloth instead of the robot. By developing a smart cloth, we trained a robot to fold cloth in-vivo within a day. Extrapolating an integrated approach on hardware and software leads to embodied intelligence in which morphology closes the loop with control: co-optimizing the body and brain will allow evolving manipulators tailored to the tasks, and use them to build a representation of how the world works. Robots can use that feedback to understand how actions influence the environment and learn to solve tasks by using human examples, instrumented objects and their own experiences. This holistic process will enable future robots to understand human intent and solve a large repertoire of manipulation tasks.

23. A corpus for training sign language recognition and translation : the Belgian federal COVID-19 video corpus

    Vandeghinste, Vincent, Van Dyck, Bob, De Coster, Mathieu, and Goddefroy, Maud

    In Computational Linguistics in the Netherlands (CLIN 32), Abstracts 2022

    Biblio
24. Progress, challenges and innovations of the SignON project

    Vandeghinste, Vincent, De Sisto, Mirella, Shterionov, Dimitar, Brady, Aoife, De Coster, Mathieu, Leeson, Lorraine, Blat, Josep, Picron, Frankie, Scipioni, Marcello, Parikh, Aditya, Bosch, Louis, O’Flaherty, John, Dambre, Joni, and Rijckaert, Jorn

    In Computational Linguistics in the Netherlands (CLIN 32), Abstracts 2022

    Biblio
25. Sign language translation : ongoing development, challenges and innovations in the SignON project

    Shterionov, Dimitar, De Sisto, Mirella, Vandeghinste, Vincent, Brady, Aoife, De Coster, Mathieu, Leeson, Lorraine, Blat, Josep, Picron, Frankie, Scipioni, Marcello, Parikh, Aditya, Bosch, Louis, O’Flaherty, John, Dambre, Joni, and Rijckaert, Jorn

    In Proceedings of the 23rd Annual Conference of the European Association for Machine Translation 2022

    Abs Biblio

    The SignON project (www.signon-project.eu) focuses on the research and development of a Sign Language (SL) translation mobile application and an open communications framework. SignON rectifies the lack of technology and services for the automatic translation between signed and spoken languages, through an inclusive, humancentric solution which facilitates communication between deaf, hard of hearing (DHH) and hearing individuals. We present an overview of the current status of the project, describing the milestones reached to date and the approaches that are being developed to address the challenges and peculiarities of Sign Language Machine Translation (SLMT).

26. Attention analysis of a sign language recognition task on the AUTSL dataset

    Coppin, Jeanne, De Coster, Mathieu, and Dambre, Joni

    In Computational Linguistics in the Netherlands (CLIN 32), Abstracts 2022

    Biblio
27. Learning keypoints from synthetic data for robotic cloth folding

    Lips, Thomas, De Gusseme, Victor-Louis, and wyffels, Francis

    In ICRA 2022 Workshop on Representing and Manipulating Deformable Objects 2022

    Abs Biblio

    Robotic cloth manipulation is challenging due to its deformability, which makes determining its full state infeasible. However, for cloth folding, it suffices to know the position of a few semantic keypoints. Convolutional neural networks (CNN) can be used to detect these keypoints, but require large amounts of annotated data, which is expensive to collect. To overcome this, we propose to learn these keypoint detectors purely from synthetic data, enabling low-cost data collection. In this paper, we procedurally generate images of towels and use them to train a CNN. We evaluate the performance of this detector for folding towels on a unimanual robot setup and find that the grasp and fold success rates are 77% and 53%, respectively. We conclude that learning keypoint detectors from synthetic data for cloth folding and related tasks is a promising research direction, discuss some failures and relate them to future work. A video of the system, as well as the codebase, more details on the CNN architecture and the training setup can be found at https://github.com/tlpss/workshop-icra-2022-cloth-keypoints.git.

28. Challenges with sign language datasets for sign language recognition and translation

    De Sisto, Mirella, Vandeghinste, Vincent, Gomez, Santiago Egea, De Coster, Mathieu, Shterionov, Dimitar, and Saggion, Horacio

    In PROCEEDINGS OF THE 13TH INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION (LREC 2022) 2022

    Abs Biblio

    Sign Languages (SLs) are the primary means of communication for at least half a million people in Europe alone. However, the development of SL recognition and translation tools is slowed down by a series of obstacles concerning resource scarcity and standardization issues in the available data. The former challenge relates to the volume of data available for machine learning as well as the time required to collect and process new data. The latter obstacle is linked to the variety of the data, i.e., annotation formats are not unified and vary amongst different resources. The available data formats are often not suitable for machine learning, obstructing the provision of automatic tools based on neural models. In the present paper, we give an overview of these challenges by comparing various SL corpora and SL machine learning datasets. Furthermore, we propose a framework to address the lack of standardization at format level, unify the available resources and facilitate SL research for different languages. Our framework takes ELAN files as inputs and returns textual and visual data ready to train SL recognition and translation models. We present a proof of concept, training neural translation models on the data produced by the proposed framework.

29. Information processing capacity of a single-node reservoir computer : an experimental evaluation

    Vettelschoss, Benedikt, Rohm, Andre, and Soriano, Miguel C.

    IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2022

    Abs Biblio

    Physical dynamical systems are able to process information in a nontrivial manner. The machine learning paradigm of reservoir computing (RC) provides a suitable framework for information processing in (analog) dynamical systems. The potential of dynamical systems for RC can be quantitatively characterized by the information processing capacity (IPC) measure. Here, we evaluate the IPC measure of a reservoir computer based on a single-analog nonlinear node coupled with delay. We link the extracted IPC measures to the dynamical regime of the reservoir, reporting an experimentally measured nonlinear memory of up to seventh order. In addition, we find a nonhomogeneous distribution of the linear and nonlinear contributions to the IPC as a function of the system operating conditions. Finally, we unveil the role of noise in the IPC of the analog implementation by performing ad hoc numerical simulations. In this manner, we identify the so-called edge of stability as being the most promising operating condition of the experimental implementation for RC purposes in terms of computational power and noise robustness. Similarly, a strong input drive is shown to have beneficial properties, albeit with a reduced memory depth.

30. Modeling electronic response properties with an explicit-electron machine learning potential

    Cools-Ceuppens, Maarten, Dambre, Joni, and Verstraelen, Toon

    JOURNAL OF CHEMICAL THEORY AND COMPUTATION 2022

    Abs Biblio

    Explicit-electron force fields introduce electrons or electron pairs as semiclassical particles in force fields or empirical potentials, which are suitable for molecular dynamics simulations. Even though semiclassical electrons are a drastic simplification compared to a quantum-mechanical electronic wave function, they still retain a relatively detailed electronic model compared to conventional polarizable and reactive force fields. The ability of explicit-electron models to describe chemical reactions and electronic response properties has already been demonstrated, yet the description of short-range interactions for a broad range of chemical systems remains challenging. In this work, we present the electron machine learning potential (eMLP), a new explicit electron force field in which the short-range interactions are modeled with machine learning. The electron pair particles will be located at well-defined positions, derived from localized molecular orbitals or Wannier centers, naturally imposing the correct dielectric and piezoelectric behavior of the system. The eMLP is benchmarked on two newly constructed data sets: eQM7, an extension of the QM7 data set for small molecules, and a data set for the crystalline beta-glycine. It is shown that the eMLP can predict dipole moments, polarizabilities, and IR-spectra of unseen molecules with high precision. Furthermore, a variety of response properties, for example, stiffness or piezoelectric constants, can be accurately reproduced.

31. Wavelength dimension in waveguide-based photonic reservoir computing

    Gooskens, Emmanuel, Laporte, Floris, Ma, Chonghuai, Sackesyn, Stijn, Dambre, Joni, and Bienstman, Peter

    OPTICS EXPRESS 2022

    Abs Biblio

    Existing work on coherent photonic reservoir computing (PRC) mostly concentrates on single-wavelength solutions. In this paper, we discuss the opportunities and challenges related to exploiting the wavelength dimension in integrated photonic reservoir computing systems. Different strategies are presented to be able to process several wavelengths in parallel using the same readout. Additionally, we present multiwavelength training techniques that allow to increase the stable operating wavelength range by at least a factor of two. It is shown that a single-readout photonic reservoir system can perform with approximate to 0% BER on several WDM channels in parallel for bit-level tasks and nonlinear signal equalization. This even when taking manufacturing deviations and laser wavelength drift into account. (C) 2022 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement

32. Computational thinking in Flanders’ compulsory education

    Gesquière, Natacha, and wyffels, Francis

    In Proceedings of Sixth APSCE International Conference on Computational Thinking and STEM Education 2022 (CTE-STEM) 2022

    Abs Biblio

    To modernise education, the Flemish government defined new learning goals that take account of 21st-century competences, in particular on ‘digital competence and media literacy’, of which ‘computational thinking and acting’ is one of the building blocks. Since September 2019, ‘computational thinking and acting’ has been compulsory in secondary schools in Flanders. The basic concepts decomposition, abstraction, pattern recognition and generalisation, and algorithm have been pushed forward. A closer look at the newly defined learning goals clarified that ‘acting’ is about basic knowledge in computer science and computational thinking practices. The learning objectives show that ‘computational thinking and acting’ is best addressed interdisciplinary in a socially relevant context. Based on the abundant scientific literature on the subject, we found these goals to fit into an international perspective. To support teachers, we are adjusting the teaching materials we already developed on physical computing, programming, and AI

33. Anchor pruning for object detection

    Bonnaerens, Maxim, Freiberger, Matthias, and Dambre, Joni

    COMPUTER VISION AND IMAGE UNDERSTANDING 2022

    Abs Biblio

    This paper proposes anchor pruning for object detection in one-stage anchor-based detectors. While pruning techniques are widely used to reduce the computational cost of convolutional neural networks, they tend to focus on optimizing the backbone networks where often most computations are. In this work we demonstrate an additional pruning technique, specifically for object detection: anchor pruning. With more efficient backbone networks and a growing trend of deploying object detectors on embedded systems where post-processing steps such as non-maximum suppression can be a bottleneck, the impact of the anchors used in the detection head is becoming increasingly more important. In this work, we show that many anchors in the object detection head can be removed without any loss in accuracy. With additional retraining, anchor pruning can even lead to improved accuracy. Extensive experiments on SSD and MS COCO show that the detection head can be made up to 44% more efficient while simultaneously increasing accuracy. Further experiments on RetinaNet and PASCAL VOC show the general effectiveness of our approach. We also introduce ’overanchorized’ models that can be used together with anchor pruning to eliminate hyperparameters related to the initial shape of anchors. Code and models are available at https://github.com/Mxbonn/anchor_pruning.

34. A comparative analysis on genome pleiotropy for evolved soft robots

    Marzougui, Dries, Biondina, Matthijs, and wyffels, Francis

    In GECCO ’22 : Proceedings of the Genetic and Evolutionary Computation Conference Companion 2022

    Abs Biblio

    Biological evolution shapes the body and brain of living creatures together over time. By contrast, in evolutionary robotics, the co-optimization of these subsystems remains challenging. Conflicting mutations cause dissociation between morphology and control, which leads to premature convergence. Recent works have proposed algorithmic modifications to mitigate the impact of conflicting mutations. However, the importance of genetic design remains underexposed. Current approaches are divided between a single, pleiotropic genetic encoding and two isolated encodings representing morphology and control. This design choice is commonly made ad hoc, causing a lack of consistency for practitioners. To standardize this design, we performed a comparative analysis between these two configurations on a soft robot locomotion task. Additionally, we incorporated two currently unexplored alternatives that drive these configurations to their logical extremes. Our results demonstrate that pleiotropic representations yield superior performance in fitness and robustness towards premature convergence. Moreover, we showcase the importance of shared structure in the pleiotropic representation of robot morphology and control to achieve this performance gain. These findings provide valuable insights into genetic encoding design, which supply practitioners with a theoretical foundation to pursue efficient brain-body co-optimization.

35. Leveraging plant physiological dynamics using physical reservoir computing

    Pieters, Olivier, De Swaef, Tom, Stock, Michiel, and wyffels, Francis

    SCIENTIFIC REPORTS 2022

    Abs Biblio

    Plants are complex organisms subject to variable environmental conditions, which influence their physiology and phenotype dynamically. We propose to interpret plants as reservoirs in physical reservoir computing. The physical reservoir computing paradigm originates from computer science; instead of relying on Boolean circuits to perform computations, any substrate that exhibits complex non-linear and temporal dynamics can serve as a computing element. Here, we present the first application of physical reservoir computing with plants. In addition to investigating classical benchmark tasks, we show that Fragaria x ananassa (strawberry) plants can solve environmental and eco-physiological tasks using only eight leaf thickness sensors. Although the results indicate that plants are not suitable for general-purpose computation but are well-suited for eco-physiological tasks such as photosynthetic rate and transpiration rate. Having the means to investigate the information processing by plants improves quantification and understanding of integrative plant responses to dynamic changes in their environment. This first demonstration of physical reservoir computing with plants is key for transitioning towards a holistic view of phenotyping and early stress detection in precision agriculture applications since physical reservoir computing enables us to analyse plant responses in a general way: environmental changes are processed by plants to optimise their phenotype.

36. Personalised socially assistive robot for cardiac rehabilitation : critical reflections on long-term interactions in the real world

    Irfan, Bahar, Cespedes, Nathalia, Casas, Jonathan, Senft, Emmanuel, Gutierrez, Luisa F., Rincon-Roncancio, Monica, Cifuentes, Carlos A., Belpaeme, Tony, and Munera, Marcela

    USER MODELING AND USER-ADAPTED INTERACTION 2022

    Abs Biblio

    Lack of motivation and low adherence rates are critical concerns of long-term rehabilitation programmes, such as cardiac rehabilitation. Socially assistive robots are known to be effective in improving motivation in therapy. However, over longer durations, generic and repetitive behaviours by the robot often result in a decrease in motivation and engagement, which can be overcome by personalising the interaction, such as recognising users, addressing them with their name, and providing feedback on their progress and adherence. We carried out a real-world clinical study, lasting 2.5 years with 43 patients to evaluate the effects of using a robot and personalisation in cardiac rehabilitation. Due to dropouts and other factors, 26 patients completed the programme. The results derived from these patients suggest that robots facilitate motivation and adherence, enable prompt detection of critical conditions by clinicians, and improve the cardiovascular functioning of the patients. Personalisation is further beneficial when providing high-intensity training, eliciting and maintaining engagement (as measured through gaze and social interactions) and motivation throughout the programme. However, relying on full autonomy for personalisation in a real-world environment resulted in sensor and user recognition failures, which caused negative user perceptions and lowered the perceived utility of the robot. Nonetheless, personalisation was positively perceived, suggesting that potential drawbacks need to be weighed against various benefits of the personalised interaction.

37. Effective cloth folding trajectories in simulation with only two parameters

    De Gusseme, Victor-Louis, and wyffels, Francis

    FRONTIERS IN NEUROROBOTICS 2022

    Abs Biblio

    Robotic cloth folding remains challenging for robots due to its highly deformable nature. In order to deal with these deformations, several strategies with varying amounts of adaptability have been proposed. We study robotic cloth folding by simulating and evaluating a trajectory search space with only two parameters: one parameter for the trajectory’s height and one parameter to tilt it. We extensively analyzed folding a long-sleeved shirt in a high-fidelity simulator. To demonstrate that the trajectory is sufficiently adaptable and robust, we test several cloth shapes, cloth materials, an entire folding sequence and different folding velocities. We can deal with every folding scenario by tuning the two parameters correctly. The trajectories’ simplicity and their robustness in simulation make them ideal candidates for future transfer to real-world robotic setups.

38. Physical computing in primary and secondary education : techniques and tools for teaching and assessment

    Neutens, Tom

    2022

    Abs Biblio

    Computer science is becoming an increasingly important part of primary and secondary education. This change is backed by a societal shift towards a digital economy. The European Commis- sion wants 100% of key public services to be online and 90% of small- and medium-sized businesses to reach at least a basic level of digital intensity. It is clear that the integration of computer science in education is important in order to reach these goals. Nevertheless, this is not a trivial task. There are many unknowns about how to do this effectively, moreover, these unknowns are distributed across multiple areas of policy and practice. In this dissertation, we identify some of these unknowns and formulate some possible solutions. We mainly focus on supporting teach- ing practice but also formulate insights which can be helpful for making policy decisions. Since the field of computing education is very broad, we focus on the integration of programming in a physical computing context for learners ages 10 to 14. To conduct research in this field, we first created a visual programming environment for physical systems. This environment is based on existing literature and is used as a tool throughout this work. It allows learners to build and program robots both in the real world and in simulation. This strong link between the physical and virtual word allows the environment to be used in many different learning contexts, allowing us to conduct a variety of in classroom experiments. Using the environment we created, we have shown that not all ways of integrating programming into a physical computing class are equally effective. Specifically, our results show that having learners create a program from scratch is preferable to having them find errors in existing programs. Our metrics indicate that finding errors in an existing program has more risk of causing ex- traneous cognitive load in learners. Cognitive load theory states that the human mind has a limited working memory. In order for effective learning to take place, the capacity of the working memory should not be exceeded. Even though fixing a program only required learners to change one element in the code, the act of understanding what the code does, applying error detection strategies, and deducing what they can do to fix the issue, causes more cognitive load than constructing a fully functional program from scratch. As a result, learners in the fix group get stuck more often, lose motivation more quickly, tend to use trial-and-error solution strategies more often, and score lower on programming tests. Since effective teaching not only depends on the teaching method but also on the teacher, we have developed methods for support- ing teachers in their efforts to integrate programming into their teaching. To identify the most pressing needs teachers have, we set up a qualitative experiment during one of the professional de- velopment programs we created. By observing and interviewing teachers, we identified two main challenges teachers encounter. 1) The ability and willingness to create, reuse and share edu- cational materials. 2) How to assess the coding ability of their learners especially in an open learning context. We aimed to address these issues by designing and developing different tools and techniques for supporting teachers. Our ef- forts include the design and implementation of a learning object repository, a tool for sharing learning content on the web, as well as a framework for the assessment of code. The learning object repository defines a unified way of creating and sharing educational content. It solves the issues of granularity, heterogene- ity, reuse, personalization, and sharing. We argue that having a uniform system for defining learning content is important to facilitate personalized learning. The assessment framework we developed can be used by teach- ers to get insight into the aspects of code that can be assessed, giving them a grip on how to give quality feedback. It focusses on feedback of visual programs to learners ages 12 to 14. The teachers for these age groups need the most support since they usually have to teach multiple subjects and are less of an expert in each of the topics they teach. To develop this framework, we identified the gap between the existing literature about the assessment of code and how teachers perform the assessment in practice. Consequently, its strength is the link between theory and practice. The framework consists of eight main categories: 1) functionality 2) readability 3) programming concepts 4) algo- rithmic thinking 5) testing and debugging 6) mathematical con- cepts 7) creativity 8) personal aspects. Each of these categories is then split up into concepts relevant in the context of visual programming. Comparing this framework to what teachers as- sess in practice has shown that teachers are mostly unaware of these categories. This often results in incomplete, inaccurate, or invalid feedback. Additionally, the range of concepts being considered by teachers seems to grow with teaching experience. Nevertheless, even more experienced teachers are often not fully aware of which exact concepts they are assessing. Finally, we describe a method to assist teachers with their assess- ment of code by visualizing the solution paths of learners. This requires techniques to extract educationally relevant informa- tion from code. Consequently, we developed different machine learning based techniques for extracting didactical information from code and for visualizing solution paths in our programming environment. Our system extracts structural and functional in- formation from programs and embeds it into a vector space. The structural information is extracted using a Gaussian vertex edge histogram kernel. This kernel defines the structural similarity between two abstract syntax trees which can be used to cluster similar programs together. To extract functional information, we executed the programs in a simulated environment and captured the internal program state at each time step. By transforming these state variations in time into both frequency and amplitude components, we are able to create a vector representing the func- tion of a program. These vectors can be used as input for several machine learning algorithms. This allows us to cluster similar programs together and create visualizations based on the similarity of solution paths. This proof of concept application lays the groundwork for automated feedback systems.

2021

1. Analysis of attention in child-robot interaction among children diagnosed with cognitive impairment

   Ismail, Luthffi, Hanapiah, Fazah Akhtar, Belpaeme, Tony, Dambre, Joni, and wyffels, Francis

   INTERNATIONAL JOURNAL OF SOCIAL ROBOTICS 2021

   Abs Biblio

   Interacting with social robots has been reported as potentially beneficial for children with social communication difficulties, with one of the promising applications being the practising of social skills, such as joint attention. We present the analysis of attention skills in children with cognitive impairments over a series of child-robot interaction sessions. Here, an interaction consists of five different modules. The first module introduces the child to the robot. The next three modules are the task modules during which children are expected to improve their attention skills during the completion of a series of social tasks. The final module is a free style interaction, where the duration of interaction between the child and robot was used as a proxy to indicate the attention of the child towards a robot. Our analysis showed that the majority of the children reduced their task completion time in modules two to four, indicating an improvement in attention. Moreover, most of the children showed positive engagement towards the robot and spent an average of 120 s during the free style interaction in module five. The positive response suggests that the robot, via child-robot interaction could be a useful and engaging tool to improve attention skills of the children with cognitive impairment.

2. Collaborative and inclusive process with the autism community : a case study in Colombia about social robot design

   Ramirez-Duque, Andres A., Aycardi, Luis F., Villa, Adriana, Munera, Marcela, Bastos, Teodiano, Belpaeme, Tony, Frizera-Neto, Anselmo, and Cifuentes, Carlos A.

   INTERNATIONAL JOURNAL OF SOCIAL ROBOTICS 2021

   Abs Biblio

   One of the most promising areas in which social assistive robotics has been introduced is therapeutic intervention for children with autism spectrum disorders (CwASD). Even though there are promising results in therapeutic contexts, there is a lack of guidelines on how to select the appropriate robot and how to design and implement the child-robot interaction. The use of participatory design (PD) methods in the design of technology-based processes for CwASD is a recognition of the stakeholders as "experts" in their fields. This work explores the benefits brought by the use of PD methods in the design of a social robot, with a specific focus on their use in autism spectrum disorders therapies on the Colombian autism community. Based on what proved to be effective in our previous research, we implemented participatory methods for both the CwASD and the stakeholders. The process leverages the active role of participants using a focus group approach with parents and specialists, and scene cards, narrative and handmade generative methods with the children. To overcome some challenges of traditional PD processes, where not all community actors are considered, we included a Colombian community consisting of therapists, nurses, caregivers and parents. The proposed PD process provides an opportunity to learn from several community actors (and thus different cultural and social aspects of developing countries), improving traditional robot design methods. In this way, the findings are summarized through a set of guidelines regarding the design of a social robot-device suitable to be implemented for robot-assisted therapy for CwASD.

3. Social assistive robots : assessing the impact of a training assistant robot in cardiac rehabilitation

   Casas, Jonathan, Senft, Emmanuel, Gutierrez, Luisa F., Rincon-Rocancio, Monica, Munera, Marcela, Belpaeme, Tony, and Cifuentes, Carlos A.

   INTERNATIONAL JOURNAL OF SOCIAL ROBOTICS 2021

   Abs Biblio

   According to the world health organization, cardiovascular diseases are a major cause of death worldwide. Cardiac rehabilitation programmes are dedicated to approach this problem and reduce mortality rates due to the presence of a second event. However, the adherence and motivation of patients to assist to these programmes is not the expected. Therefore, this paper presents the incorporation of a SAR system into a cardiac rehabilitation scenario, where a social robot had the role of a training assistant during the therapy, aiming to increase motivation and encourage people to continue with the therapy. This study carried out a longitudinal experimental setup with a total of 209 sessions observed for a group of 6 patients in a period between 3 and 6 months. Results show that patients felt more encouraged to perform physical activity and continue with the rehabilitation when they perceived that monitored and supervised by the system, demonstrating that it can be implemented as a reliable tool that would potentially leverage tasks carried out by health professionals.

4. Analyzing coding behaviour of novice programmers in different instructional settings : creating vs. debugging

   Neutens, Tom, and wyffels, Francis

   In 2020 International Conference on Computational Science and Computational Intelligence (CSCI) 2021

   Biblio
5. Unsupervised functional analysis of graphical programs for physical computing

   Neutens, Tom, and wyffels, Francis

   In 2020 International Conference on Computational Science and Computational Intelligence (CSCI) 2021

   Biblio
6. The robot made me do it : human-robot interaction and risk-taking behavior

   Hanoch, Yaniv, Arvizzigno, Francesco, Hernandez García, Daniel, Denham, Sue, Belpaeme, Tony, and Gummerum, Michaela

   CYBERPSYCHOLOGY BEHAVIOR AND SOCIAL NETWORKING 2021

   Abs Biblio

   Empirical evidence has shown that peer pressure can impact human risk-taking behavior. With robots becoming ever more present in a range of human settings, it is crucial to examine whether robots can have a similar impact. Using the balloon analogue risk task (BART), participants’ risk-taking behavior was measured when alone, in the presence of a silent robot, or in the presence of a robot that actively encouraged risk-taking behavior. In the BART, shown to be a proxy for real risk-taking behavior, participants must weigh risk against potential payout. Our results reveal that participants who were encouraged by the robot did take more risks, while the mere presence of the robot in the robot control condition did not entice participants to show more risk-taking behavior. Our results point to both possible benefits and perils that robots might pose to human decision-making. Although increasing risk-taking behavior in some cases has obvious advantages, it could also have detrimental consequences that are only now starting to emerge.

7. Robots in education : an introduction to high-tech social agents, intelligent tutors, and curricular tools

   Alnajjar, Fady, Bartneck, Christoph, Baxter, Paul, Belpaeme, Tony, Cappuccio, Massimiliano, Di Dio, Cinzia, Eyssel, Friederike, Handke, Jürgen, Mubin, Omar, Obaid, Mohammad, and Reich-Stiebert, Natalia

   2021

   Biblio
8. A proposal of accessibility guidelines for human-robot interaction

   Qbilat, Malak, Iglesias, Ana, and Belpaeme, Tony

   ELECTRONICS 2021

   Abs Biblio

   We will increasingly become dependent on automation to support our manufacturing and daily living, and robots are likely to take an important place in this. Unfortunately, currently not all the robots are accessible for all users. This is due to the different characteristics of users, as users with visual, hearing, motor or cognitive disabilities were not considered during the design, implementation or interaction phase, causing accessibility barriers to users who have limitations. This research presents a proposal for accessibility guidelines for human-robot interaction (HRI). The guidelines have been evaluated by seventeen HRI designers and/or developers. A questionnaire of nine five-point Likert Scale questions and 6 open-ended questions was developed to evaluate the proposed guidelines for developers and designers, in terms of four main factors: usability, social acceptance, user experience and social impact. The questions act as indicators for each factor. The majority (15 of 17 participants) agreed that the guidelines are helpful for them to design and implement accessible robot interfaces and applications. Some of them had considered some ad hoc guidelines in their design practice, but none of them showed awareness of or had applied all the proposed guidelines in their design practice, 72% of the proposed guidelines have been applied by less than or equal to 8 participants for each guideline. Moreover, 16 of 17 participants would use the proposed guidelines in their future robot designs or evaluation. The participants recommended the importance of aligning the proposed guidelines with safety requirements, environment of interaction (indoor or outdoor), cost and users’ expectations.

9. Effect of compliance on morphological control of dynamic locomotion with HyQ

   Urbain, Gabriel, Barasuol, Victor, Semini, Claudio, Dambre, Joni, and wyffels, Francis

   AUTONOMOUS ROBOTS 2021

   Abs Biblio

   Classic control theory applied to compliant and soft robots generally involves an increment of computation that has no equivalent in biology. To tackle this, morphological computation describes a theoretical framework that takes advantage of the computational capabilities of physical bodies. However, concrete applications in robotic locomotion control are still rare. Also, the trade-off between compliance and the capacity of a physical body to facilitate its own control has not been thoroughly studied in a real locomotion task. In this paper, we address these two problems on the state-of-the-art hydraulic robot HyQ. An end-to-end neural network is trained to control HyQ’s joints positions and velocities using only Ground Reaction Forces (GRF). Our simulations and experiments demonstrate better controllability using less memory and computational resources when increasing compliance. However, we show empirically that this effect cannot be attributed to the ability of the body to perform intrinsic computation. It invites to give an increased emphasis on compliance and co-design of the controller and the robot to facilitate attempts in machine learning locomotion.

10. A large, crowdsourced evaluation of gesture generation systems on common data : the GENEA Challenge 2020

    Kucherenko, T., Jonell, P., Yoon, Y., Wolfert, Pieter, and Henter, G. E.

    In IUI ’21: 26th International Conference on Intelligent User Interfaces 2021

    Abs Biblio

    Co-speech gestures, gestures that accompany speech, play an important role in human communication. Automatic co-speech gesture generation is thus a key enabling technology for embodied conversational agents (ECAs), since humans expect ECAs to be capable of multi-modal communication. Research into gesture generation is rapidly gravitating towards data-driven methods. Unfortunately, individual research efforts in the field are difficult to compare: there are no established benchmarks, and each study tends to use its own dataset, motion visualisation, and evaluation methodology. To address this situation, we launched the GENEA Challenge, a gesture-generation challenge wherein participating teams built automatic gesture-generation systems on a common dataset, and the resulting systems were evaluated in parallel in a large, crowdsourced user study using the same motion-rendering pipeline. Since differences in evaluation outcomes between systems now are solely attributable to differences between the motion-generation methods, this enables benchmarking recent approaches against one another in order to get a better impression of the state of the art in the field. This paper reports on the purpose, design, results, and implications of our challenge.

11. Communicative function of eye blinks of virtual avatars may not translate onto physical platforms

    Oetringer, Djamari, Wolfert, Pieter, Deschuyteneer, Jorre, Thill, Serge, and Belpaeme, Tony

    In HRI ’21 Companion: Companion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction 2021

    Abs Biblio

    Eye behaviour is one of the main modalities used to regulate face-to-face conversation. Gaze aversion and mutual gaze, for example, serve to signal cognitive load, interest or turns during a conversation. While eye blinking is mainly thought to have a physiological function, the rate of blinking is known to increase during conversation suggesting a communicative function for the eye. Recently, it has been shown that a virtual avatar, acting as the receiver in a conversation, could use blinking as a kind of conversational marker influencing the speaker’s communicative behaviour. In particular, it has been demonstrated that long eye blinks resulted in shorter answers by the speaker compared with short ones. Here, we set out to investigate this effect when using a humanoid robot as interaction partner, given that robots have both a physical and social presence. Interestingly, however, we could not replicate the result: short or long blinks did not modulate the length of the responses by the human interactant.

12. A socially assistive robot for long-term cardiac rehabilitation in the real world

    Cespedes, Nathalia, Irfan, Bahar, Senft, Emmanuel, Cifuentes, Carlos A., Gutierrez, Luisa F., Rincon-Roncancio, Monica, Belpaeme, Tony, and Munera, Marcela

    FRONTIERS IN NEUROROBOTICS 2021

    Abs Biblio

    What are the benefits of using a socially assistive robot for long-term cardiac rehabilitation? To answer this question we designed and conducted a real-world long-term study, in collaboration with medical specialists, at the Fundacion Cardioinfantil-Instituto de Cardiologia clinic (Bogota, Colombia) lasting 2.5 years. The study took place within the context of the outpatient phase of patients’ cardiac rehabilitation programme and aimed to compare the patients’ progress and adherence in the conventional cardiac rehabilitation programme (control condition) against rehabilitation supported by a fully autonomous socially assistive robot which continuously monitored the patients during exercise to provide immediate feedback and motivation based on sensory measures (robot condition). The explicit aim of the social robot is to improve patient motivation and increase adherence to the programme to ensure a complete recovery. We recruited 15 patients per condition. The cardiac rehabilitation programme was designed to last 36 sessions (18 weeks) per patient. The findings suggest that robot increases adherence (by 13.3%) and leads to faster completion of the programme. In addition, the patients assisted by the robot had more rapid improvement in their recovery heart rate, better physical activity performance and a higher improvement in cardiovascular functioning, which indicate a successful cardiac rehabilitation programme performance. Moreover, the medical staff and the patients acknowledged that the robot improved the patient motivation and adherence to the programme, supporting its potential in addressing the major challenges in rehabilitation programmes.

13. Roboter in der Bildung : wie Robotik das Lernen im digitalen Zeitalter bereichern kann

    Alnajjar, Fady, Bartneck, Christoph, Baxter, Paul, Belpaeme, Tony, Cappuccio, Massimiliano L., Dio, Cinzia, Eyssel, Friederike, Handke, Jürgen, Mubin, Omar, Obaid, Mohammad, and Reich-Stiebert, Natalia

    2021

    Biblio
14. Comparing learning ecologies of primary graphical programming : create or fix?

    Neutens, Tom, Barbion, Evelien, Coolsaet, Kris, and wyffels, Francis

    JOURNAL OF COMPUTER ASSISTED LEARNING 2021

    Abs Biblio

    In the last few years, programming, computational thinking, and robotics are more frequently integrated into elementary education. This integration can be done in many different ways. However, it is still unclear which teaching methods work in which situations. To provide some clarity in this area, we compared two methods of integrating programming into a primary robotics workshop for learners aged ten to twelve. In one method, students create programs from scratch; in the other, they start with a faulty program they have to fix. These teaching methods were evaluated using the framework of learning ecology, which provides a holistic framework for assessing complex learning environments. We identified different indicators of learning ecology and assessed our workshops using a mixed-methods approach. Our results showed no difference between the groups on the intrinsic dimension of a learning ecology. However, on the experiential dimension, the learners in the create group scored better on all tests. Our results show the value of a multidimensional assessment of learning ecology to understand different teaching techniques. Additionally, the results provide us with important insights on how to integrate programming into a primary robotics curriculum enabling teachers to select better methods for teaching computing in their classroom.

15. cREAtIve : reconfigurable embedded artificial intelligence

    Bahrebar, Poona, Denis, Leon, Bonnaerens, Maxim, Coddens, Kristof, Dambre, Joni, Favoreel, Wouter, Khvastunov, Illia, Munteanu, Adrian, Nguyen-Duc, Hung, Schulte, Stefan, Stroobandt, Dirk, Valvekens, Ramses, Broeck, Nick, and Verbruggen, Geert

    In CF ’21, Proceedings of the 18th ACM International Conference on Computing Frontiers 2021

    Abs Biblio

    cREAtIve targets the development of novel highly-adaptable embedded deep learning solutions for automotive and traffic monitoring applications, including position sensor processing, scene interpretation based on LiDAR, and object detection and classification in thermal images for traffic camera systems. These applications share the need for deep learning solutions tailored for deployment on embedded devices with limited resources and featuring high adaptability and robustness to changing environmental conditions. cREAtIve develops knowledge, tools and methods that enable hardware-efficient, adaptable, and robust deep learning.

16. Addressing limited weight resolution in a fully optical neuromorphic reservoir computing readout

    Ma, Chonghuai, Laporte, Floris, Dambre, Joni, and Bienstman, Peter

    SCIENTIFIC REPORTS 2021

    Abs Biblio

    Using optical hardware for neuromorphic computing has become more and more popular recently, due to its efficient high-speed data processing capabilities and low power consumption. However, there are still some remaining obstacles to realizing the vision of a completely optical neuromorphic computer. One of them is that, depending on the technology used, optical weighting elements may not share the same resolution as in the electrical domain. Moreover, noise of the weighting elements are important considerations as well. In this article, we investigate a new method for improving the performance of optical weighting components, even in the presence of noise and in the case of very low resolution. Our method utilizes an iterative training procedure and is able to select weight connections that are more robust to quantization and noise. As a result, even with only 8 to 32 levels of resolution, in noisy weighting environments, the method can outperform both nearest rounding low-resolution weighting and random rounding weighting by up to several orders of magnitude in terms of bit error rate and can deliver performance very close to full-resolution weighting elements.

17. MIRRA : a modular and cost-effective microclimate monitoring system for real-time remote applications

    Pieters, Olivier, Deprost, Emiel, Van Der Donckt, Jonas, Brosens, Lore, Sanczuk, Pieter, Vangansbeke, Pieter, De Swaef, Tom, De Frenne, Pieter, and wyffels, Francis

    SENSORS 2021

    Abs Biblio

    Monitoring climate change, and its impacts on ecological, agricultural, and other societal systems, is often based on temperature data derived from official weather stations. Yet, these data do not capture most microclimates, influenced by soil, vegetation and topography, operating at spatial scales relevant to the majority of organisms on Earth. Detecting and attributing climate change impacts with confidence and certainty will only be possible by a better quantification of temperature changes in forests, croplands, mountains, shrublands, and other remote habitats. There is an urgent need for a novel, miniature and simple device filling the gap between low-cost devices with manual data download (no instantaneous data) and high-end, expensive weather stations with real-time data access. Here, we develop an integrative real-time monitoring system for microclimate measurements: MIRRA (Microclimate Instrument for Real-time Remote Applications) to tackle this problem. The goal of this platform is the design of a miniature and simple instrument for near instantaneous, long-term and remote measurements of microclimates. To that end, we optimised power consumption and transfer data using a cellular uplink. MIRRA is modular, enabling the use of different sensors (e.g., air and soil temperature, soil moisture and radiation) depending upon the application, and uses an innovative node system highly suitable for remote locations. Data from separate sensor modules are wirelessly sent to a gateway, thus avoiding the drawbacks of cables. With this sensor technology for the long-term, low-cost, real-time and remote sensing of microclimates, we lay the foundation and open a wide range of possibilities to map microclimates in different ecosystems, feeding a next generation of models. MIRRA is, however, not limited to microclimate monitoring thanks to its modular and wireless design. Within limits, it is suitable or any application requiring real-time data logging of power-efficient sensors over long periods of time. We compare the performance of this system to a reference system in real-world conditions in the field, indicating excellent correlation with data collected by established data loggers. This proof-of-concept forms an important foundation to creating the next version of MIRRA, fit for large scale deployment and possible commercialisation. In conclusion, we developed a novel wireless cost-effective sensor system for microclimates.

18. Isolated sign recognition from RGB video using pose flow and self-attention

    De Coster, Mathieu, Van Herreweghe, Mieke, and Dambre, Joni

    In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) 2021

    Abs Biblio

    Automatic sign language recognition lies at the intersection of natural language processing (NLP) and computer vision. The highly successful transformer architectures, based on multi-head attention, originate from the field of NLP. The Video Transformer Network (VTN) is an adaptation of this concept for tasks that require video understanding, e.g., action recognition. However, due to the limited amount of labeled data that is commonly available for training automatic sign (language) recognition, the VTN cannot reach its full potential in this domain. In this work, we reduce the impact of this data limitation by automatically pre-extracting useful information from the sign language videos. In our approach, different types of information are offered to a VTN in a multi-modal setup. It includes per-frame human pose keypoints (extracted by OpenPose) to capture the body movement and hand crops to capture the (evolution of) hand shapes. We evaluate our method on the recently released AUTSL dataset for isolated sign recognition and obtain 92.92% accuracy on the test set using only RGB data. For comparison: the VTN architecture without hand crops and pose flow achieved 82% accuracy. A qualitative inspection of our model hints at further potential of multi-modal multi-head attention in a sign language recognition context.

19. Frozen pretrained transformers for neural sign language translation

    De Coster, Mathieu, D’Oosterlinck, Karel, Pizurica, Marija, Rabaey, Paloma, Verlinden, Severine, Van Herreweghe, Mieke, and Dambre, Joni

    In Proceedings of the 1st International Workshop on Automatic Translation for Signed and Spoken Languages (AT4SSL) 2021

    Abs Biblio

    One of the major challenges in sign language translation from a sign language to a spoken language is the lack of parallel corpora. Recent works have achieved promising results on the RWTH-PHOENIX-Weather 2014T dataset, which consists of over eight thousand parallel sentences between German sign language and German. However, from the perspective of neural machine translation, this is still a tiny dataset. To improve the performance of models trained on small datasets, transfer learning can be used. While this has been previously applied in sign language translation for feature extraction, to the best of our knowledge, pretrained language models have not yet been investigated. We use pretrained BERT-base and mBART-50 models to initialize our sign language video to spoken language text translation model. To mitigate overfitting, we apply the frozen pretrained transformer technique: we freeze the majority of parameters during training. Using a pretrained BERT model, we outperform a baseline trained from scratch by 1 to 2 BLEU-4. Our results show that pretrained language models can be used to improve sign language translation performance and that the self-attention patterns in BERT transfer in zero-shot to the encoder and decoder of sign language translation models.

20. Chatbot : leerkrachtenhandleiding

    Gesquière, Natacha, Hoste, Veronique, Van Hee, Cynthia, and wyffels, Francis

    2021

    Abs Biblio

    Waar taal en technologie samenkomen, ontstaat het domein van Natural Language Processing (NLP). Met technieken uit machine learning, zoals het trainen van modellen om tekst, afbeeldingen of geluidsfrequenties te herkennen, kunnen we computers op een intelligente manier laten werken met taal. Bekende NLP-toepassingen van machine learning zijn bijvoorbeeld vertaalprogramma’s, spraakbesturingssystemen of virtual assistants zoals Siri, en chatrobots. We buigen ons over vragen als "Hoe kunnen we automatisch teksten begrijpen, vertalen en zelfs schrijven?", "Kan een computer emoties of ironie herkennen in een tekst?", en vooral "Hoe belangrijk zijn data?”. Leerlingen van de tweede en de derde graad secundair onderwijs krijgen inzicht in het domein NLP en zetten hun eerste stappen in de wereld van het programmeren met Python.

21. Biologically inspired locomotion of compliant robots

    Urbain, Gabriel

    2021

    Abs Biblio

    A better understanding of locomotion, and the processes that it involves, has potential benefits in our society. On one hand, it could help to design efficient legged robots, with better accessibil- ity to the different environments on the globe, whereas wheeled platforms remain generally limited on even terrains without ob- stacles. This could be used to build social robots or to deal with exploration and rescue operations in hazardous environments. On the other hand, improving the understanding of the locomotion mechanisms can also contribute to biological sciences and, in par- ticular, neurosciences. In this regard, studies on locomotion are a typical illustration of an embodied problem, which is often cited as a key concept to bridge the different scales in brain research, from the chemical and physical processes to the behavioral and psychological aspects. To improve the agility and adaptability of robotic locomotion plat- forms, an appealing path is to use compliant structures and actua- tors rather than stiff elements. However, compliant and soft robots are not well suited for control with traditional computational ar- chitectures, generally directed towards centralized commands and exactness. New models, driven by data, or inspired by the broad locomotion abilities encountered within biological systems, create an opportunity to improve the state of the art in robotics. But they also raise questions on several fundamental aspects, limiting their maturity and their diffusion in the society. This dissertation tries to better investigate three of these research questions: how can we transfer knowledge from simulation to real robots, how does the mechanical compliance correlate with the locomotion performance and the controller complexity, and, thirdly, how can reflex-based control on compliant structures benefit from a stance correction mechanism taking its inspiration in the biological Cerebellum. In the introduction and the state-of-the-art chapters, I identify these questions more precisely and provide an overview of the existing literature on the subject. The next chapter gives more de- tails on the control methodology used in this dissertation. Chapter 4 presents three different robot platforms to target the research questions: a simulated network of masses and springs (MSD struc- ture), a cheap passive compliant quadruped robot (Tigrillo), and a state-of-the-art quadruped robot with active compliant actuators (HyQ). The next four chapters expose the approach and discuss the results of the experimental trials conducted on these platforms, to formulate contributions in the domain. Finally, a conclusion is provided in the last chapter. The contribution of this dissertation is manifold and five main topics are investigated throughout the manuscript. First, the re- cent progress in machine learning has led to impressive results for the locomotion of simulated creatures. However, mechanical compliance is not often considered in the work from this field and the difficulty to transfer trained parametric models from simula- tion to the real world has been highlighted in different research tracks. In this dissertation, I suggest an optimization procedure of the physics simulation model to reduce the difference between observations in simulation and the real world. Secondly, an analysis of the state of the art in the field of robotics shows that compliance is not a straightforward defined concept, although it is generally linked to two physical parameters, damping and stiffness. I further analyze this relation through empirical analysis of non-linear robotic systems. They seem to indicate that the concept of resonance, only strictly defined in systems with second-order ordinary differential equations, could serve as a first approximation of the link between stiffness, damping, and optimal locomotion frequency. An investigation of this dependence is conducted on the MSD structures and the HyQ robot. Thirdly, this dissertation presents a training method in two steps: a parameter optimization of open-loop biologically inspired models, followed by a supervised training of a feed-forward neural network, linking robot’s sensors and actuators, to reproduce the targets obtained in the first step in closed-loop. This architecture demon- strates the ability to learn a reflex-based locomotion model without the need for centralized control. The properties of this closed-loop dynamical system are investigated on the three different robotic platforms and the robustness against external disturbance is also discussed. Fourthly, to connect this reflex-based model with biological obser- vations, I also compare two architectural hypotheses in a simple stability controller for a quadruped robot: one using an internal spatiotemporal representation of the body system, and the other based on afferent sensor signals from the lower limbs. I show that the first model performs better in maintaining a target locomotion frequency and resisting external disturbances, which corroborates biological observations conducted on the Cerebellum’s functioning. All these items put together establish an ideal framework to fur- ther investigate the potential exchange of computation capacity between physical body and controller during locomotion, as ex- pected from the theory of morphological computation. A reflection on the matter constitutes the fifth contribution of this disserta- tion. The experiments on HyQ and the MSD networks promote another formulation of this phenomenon: although it is not pos- sible to confirm a transfer of computation per se in my trials, we observe that increased structural complexity and larger mechan- ical compliance contribute to the simplification of computational requirements in the controller and promote a more stable locomo- tion process against external disturbance. In conclusion, this work contributes to better understanding the impact of mechanical compliance on the design and the tuning of a locomotion controller. The experimental results advocate in favor of the use of compliance in robotics, not only to improve performance but also to simplify the control, in association with generic and data-driven controllers. Furthermore, anchoring the architecture of these controllers into biological observations proves to be a source of inspiration for enhanced robots but also a way to test hypotheses and better understand the phenomena in action during human and animal locomotion.

22. To rate or not to rate : investigating evaluation methods for generated co-speech gestures

    Wolfert, Pieter, Girard, Jeffrey M., Kucherenko, Taras, and Belpaeme, Tony

    In ICMI ’21 : Proceedings of the 2021 International Conference on Multimodal Interaction 2021

    Abs Biblio

    While automatic performance metrics are crucial for machine learning of artificial human-like behaviour, the gold standard for evaluation remains human judgement. The subjective evaluation of artificial human-like behaviour in embodied conversational agents is however expensive and little is known about the quality of the data it returns. Two approaches to subjective evaluation can be largely distinguished, one relying on ratings, the other on pairwise comparisons. In this study we use co-speech gestures to compare the two against each other and answer questions about their appropriateness for evaluation of artificial behaviour. We consider their ability to rate quality, but also aspects pertaining to the effort of use and the time required to collect subjective data. We use crowd sourcing to rate the quality of co-speech gestures in avatars, assessing which method picks up more detail in subjective assessments. We compared gestures generated by three different machine learning models with various level of behavioural quality. We found that both approaches were able to rank the videos according to quality and that the ranking significantly correlated, showing that in terms of quality there is no preference of one method over the other. We also found that pairwise comparisons were slightly faster and came with improved inter-rater reliability, suggesting that for small-scale studies pairwise comparisons are to be favoured over ratings.

23. HEMVIP : human evaluation of multiple videos in parallel

    Jonell, Patrik, Yoon, Youngwoo, Wolfert, Pieter, Kucherenko, Taras, and Henter, Gustav Eje

    In ICMI ’21 : Proceedings of the 2021 International Conference on Multimodal Interaction 2021

    Abs Biblio

    In many research areas, for example motion and gesture generation, objective measures alone do not provide an accurate impression of key stimulus traits such as perceived quality or appropriateness. The gold standard is instead to evaluate these aspects through user studies, especially subjective evaluations of video stimuli. Common evaluation paradigms either present individual stimuli to be scored on Likert-type scales, or ask users to compare and rate videos in a pairwise fashion. However, the time and resources required for such evaluations scale poorly as the number of conditions to be compared increases. Building on standards used for evaluating the quality of multimedia codecs, this paper instead introduces a framework for granular rating of multiple comparable videos in parallel. This methodology essentially analyses all condition pairs at once. Our contributions are 1) a proposed framework, called HEMVIP, for parallel and granular evaluation of multiple video stimuli and 2) a validation study confirming that results obtained using the tool are in close agreement with results of prior studies using conventional multiple pairwise comparisons.

24. Coffee with a hint of data : towards using data-driven approaches in personalised long-term interactions

    Irfan, Bahar, Hellou, Mehdi, and Belpaeme, Tony

    FRONTIERS IN ROBOTICS AND AI 2021

    Abs Biblio

    While earlier research in human-robot interaction pre-dominantly uses rule-based architectures for natural language interaction, these approaches are not flexible enough for long-term interactions in the real world due to the large variation in user utterances. In contrast, data-driven approaches map the user input to the agent output directly, hence, provide more flexibility with these variations without requiring any set of rules. However, data-driven approaches are generally applied to single dialogue exchanges with a user and do not build up a memory over long-term conversation with different users, whereas long-term interactions require remembering users and their preferences incrementally and continuously and recalling previous interactions with users to adapt and personalise the interactions, known as the lifelong learning problem. In addition, it is desirable to learn user preferences from a few samples of interactions (i.e., few-shot learning). These are known to be challenging problems in machine learning, while they are trivial for rule-based approaches, creating a trade-off between flexibility and robustness. Correspondingly, in this work, we present the text-based Barista Datasets generated to evaluate the potential of data-driven approaches in generic and personalised long-term human-robot interactions with simulated real-world problems, such as recognition errors, incorrect recalls and changes to the user preferences. Based on these datasets, we explore the performance and the underlying inaccuracies of the state-of-the-art data-driven dialogue models that are strong baselines in other domains of personalisation in single interactions, namely Supervised Embeddings, Sequence-to-Sequence, End-to-End Memory Network, Key-Value Memory Network, and Generative Profile Memory Network. The experiments show that while data-driven approaches are suitable for generic task-oriented dialogue and real-time interactions, no model performs sufficiently well to be deployed in personalised long-term interactions in the real world, because of their inability to learn and use new identities, and their poor performance in recalling user-related data.

    </p>
25. Experimental realization of integrated photonic reservoir computing for nonlinear fiber distortion compensation

    Sackesyn, Stijn, Ma, Chonghuai, Dambre, Joni, and Bienstman, Peter

    OPTICS EXPRESS 2021

    Abs Biblio

    Nonlinearity mitigation in optical fiber networks is typically handled by electronic Digital Signal Processing (DSP) chips. Such DSP chips are costly, power-hungry and can introduce high latencies. Therefore, optical techniques are investigated which are more efficient in both power consumption and processing cost. One such a machine learning technique is optical reservoir computing, in which a photonic chip can be trained on certain tasks, with the potential advantages of higher speed, reduced power consumption and lower latency compared to its electronic counterparts. In this paper, experimental results are presented where nonlinear distortions in a 32 GBPS OOK signal are mitigated to below the 0.2 x 10(-3) FEC limit using a photonic reservoir. Furthermore, the results of the reservoir chip are compared to a tapped delay line filter to clearly show that the system performs nonlinear equalisation. (C) 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

26. CASIE : computing affect and social intelligence for healthcare in an ethical and trustworthy manner

    Vasiliu, Laurentiu, Cortis, Keith, McDermott, Ross, Kerr, Aphra, Peters, Arne, Hesse, Marc, Hagemeyer, Jens, Belpaeme, Tony, McDonald, John, Villing, Rudi, Mileo, Alessandra, Capulto, Annalina, Scriney, Michael, Griffiths, Sascha, Koumpis, Adamantios, and Davis, Brian

    PALADYN 2021

    Abs Biblio

    This article explores the rapidly advancing innovation to endow robots with social intelligence capabilities in the form of multilingual and multimodal emotion recognition, and emotion-aware decision-making capabilities, for contextually appropriate robot behaviours and cooperative social human–robot interaction for the healthcare domain. The objective is to enable robots to become trustworthy and versatile social robots capable of having human-friendly and human assistive interactions, utilised to better assist human users’ needs by enabling the robot to sense, adapt, and respond appropriately to their requirements while taking into consideration their wider affective, motivational states, and behaviour. We propose an innovative approach to the difficult research challenge of endowing robots with social intelligence capabilities for human assistive interactions, going beyond the conventional robotic sense-think-act loop. We propose an architecture that addresses a wide range of social cooperation skills and features required for real human–robot social interaction, which includes language and vision analysis, dynamic emotional analysis (long-term affect and mood), semantic mapping to improve the robot’s knowledge of the local context, situational knowledge representation, and emotion-aware decision-making. Fundamental to this architecture is a normative ethical and social framework adapted to the specific challenges of robots engaging with caregivers and care-receivers.

27. Silicon photonics for brain-inspired neuromorphic information processing

    Bienstman, Peter, Dambre, Joni, Lugnan, Alessio, Sackesyn, Stijn, Ma, Chonghuai, Gooskens, Emmanuel, Gouda, Muhammed Gouda Ahmed, and Masaad, Sarah

    In 1st workshop on neuromorphic photonics, Abstracts 2021

    Biblio
28. Photonic neuromorphic computing using silicon chips

    Bienstman, Peter, Dambre, Joni, Lugnan, Alessio, Sackesyn, Stijn, Ma, Chonghuai, Gooskens, Emmanuel, Gouda, Muhammed Gouda Ahmed, and Masaad, Sarah

    In Huawei STW, Abstracts 2021

    Biblio
29. Reservoir computing for high-speed photonic information processing (invited)

    Bienstman, Peter, Dambre, Joni, Lugnan, Alessio, Sackesyn, Stijn, Ma, Chonghuai, Gooskens, Emmanuel, Gouda, Muhammed Gouda Ahmed, and Masaad, Sarah

    In Photonics in Switching and Computing, Abstracts 2021

    Biblio
30. Experimental demonstration of nonlinear fibre distortion compensation with integrated photonic reservoir computing

    Sackesyn, Stijn, Ma, Chonghuai, Dambre, Joni, and Bienstman, Peter

    In 2021 European Conference on Optical Communication (ECOC) 2021

    Abs Biblio

    Optical reservoir computing is a machine learning technique in which a photonic chip can be trained on classification tasks of time signals. This paper presents experimental results where linear and nonlinear fibre distortions are mitigated to below the 0.2×10 −3 FEC limit using a photonic reservoir.

31. GENEA Workshop 2021 : the 2nd Workshop on Generation and Evaluation of Non-verbal Behaviour for Embodied Agents

    Kucherenko, Taras, Jonell, Patrik, Yoon, Youngwoo, Wolfert, Pieter, Yumak, Zerrin, and Henter, Gustav

    In ICMI ’21 : proceedings of the 2021 International Conference on Multimodal Interaction 2021

    Abs Biblio

    Embodied agents benefit from using non-verbal behavior when communicating with humans. Despite several decades of non-verbal behavior-generation research, there is currently no well-developed benchmarking culture in the field. For example, most researchers do not compare their outcomes with previous work, and if they do, they often do so in their own way which frequently is incompatible with others. With the GENEA Workshop 2021, we aim to bring the community together to discuss key challenges and solutions, and find the most appropriate ways to move the field forward.

32. Photonic reservoir computing for high-speed neuromorphic computing applications

    Lugnan, Alessio, Sackesyn, Stijn, Ma, Chonghuai, Gooskens, Emmanuel, Gouda, Muhammed Gouda Ahmed, Masaad, Sarah, Dambre, Joni, and Bienstman, Peter

    In 2021 Summer Topicals Meetings Series, Abstracts 2021

    Biblio
33. Comparing different nonlinearities in readout systems for optical neuromorphic computing networks

    Ma, Chonghuai, Lambrecht, Joris, Laporte, Floris, Yin, Xin, Dambre, Joni, and Bienstman, Peter

    SCIENTIFIC REPORTS 2021

    Abs Biblio

    Nonlinear activation is a crucial building block of most machine-learning systems. However, unlike in the digital electrical domain, applying a saturating nonlinear function in a neural network in the analog optical domain is not as easy, especially in integrated systems. In this paper, we first investigate in detail the photodetector nonlinearity in two main readout schemes: electrical readout and optical readout. On a 3-bit-delayed XOR task, we show that optical readout trained with backpropagation gives the best performance. Furthermore, we propose an additional saturating nonlinearity coming from a deliberately non-ideal voltage amplifier after the detector. Compared to an all-optical nonlinearity, these two kinds of nonlinearities are extremely easy to obtain at no additional cost, since photodiodes and voltage amplifiers are present in any system. Moreover, not having to design ideal linear amplifiers could relax their design requirements. We show through simulation that for long-distance nonlinear fiber distortion compensation, using only the photodiode nonlinearity in an optical readout delivers BER improvements over three orders of magnitude. Combined with the amplifier saturation nonlinearity, we obtain another three orders of magnitude improvement of the BER.

34. Simulating self-learning in photorefractive optical reservoir computers

    Laporte, Floris, Dambre, Joni, and Bienstman, Peter

    SCIENTIFIC REPORTS 2021

    Abs Biblio

    Photorefractive materials exhibit an interesting plasticity under the influence of an optical field. By extending the finite-difference time-domain method to include the photorefractive effect, we explore how this property can be exploited in the context of neuromorphic computing for telecom applications. By first priming the photorefractive material with a random bit stream, the material reorganizes itself to better recognize simple patterns in the stream. We demonstrate this by simulating a typical reservoir computing setup, which gets a significant performance boost on performing the XOR on two consecutive bits in the stream after this initial priming step.

35. Computing with integrated photonic reservoirs

    Dambre, Joni, Katumba, Andrew, Ma, Chonghuai, Sackesyn, Stijn, Laporte, Floris, Freiberger, Matthias, and Bienstman, Peter

    2021

    Abs Biblio

    The idea of using photonic systems as reservoirs to perform general-purpose computing was first introduced in 2008. Since then, a wide range of systems using either discrete or integrated optical components has been explored. In this chapter, we summarise a decade of research into integrated coherent photonic reservoirs. In these systems, information is carried by the intensity and the phase of light waves. Computations emerge from the way the light propagates inside the system, and the ways in which light that travels along different paths is mixed and transformed. We discuss the computational capabilities of these reservoirs and the trade-offs that can be made to optimise them. We also discuss the technological constraints that are encountered in building such systems and the ways these reflect on their design and training. Finally, we give an overview of recent approaches to combining multiple such reservoirs into larger and computationally more powerful systems.

36. The SignON project : a sign language translation framework

    Shterionov, Dimitar, Vandeghinste, Vincent, Saggion, Horacio, Blat, Josep, De Coster, Mathieu, Dambre, Joni, Heuvel, Henk, Murtagh, Irene, Leeson, Lorraine, and Schuurman, Ineke

    In Computational Linguistics in the Netherlands (CLIN 31), Abstracts 2021

    Biblio
37. Photonic neuromorphic computing using silicon chips

    Bienstman, Peter, Dambre, Joni, Lugnan, Alessio, Sackesyn, Stijn, Ma, Chonghuai, Gooskens, Emmanuel, Gouda, Muhammed Gouda Ahmed, and Masaad, Sarah

    In bePOM - Belgian Photonics Online Meetup, Abstracts 2021

    Biblio

2020

1. User-driven design of robot costume for child-robot interactions among children with cognitive impairment

   Ismail, Luthffi, Hanapiah, Fazah Akhtar, and wyffels, Francis

   In ROBOTICS IN EDUCATION: CURRENT RESEARCH AND INNOVATIONS 2020

   Abs Biblio

   The involvement of arts and psychology elements in robotics research for children with cognitive impairment is still limited. However, the combination of robots, arts, psychology and education in the development of robots could significantly contribute to the improvement of social interaction skills among children with cognitive impairment. In this article, we would like to share our work on building and innovating the costume of LUCA’s robot, which incorporating the positive psychological perspectives and arts values for children with cognitive impairment. Our goals are (1) to educate arts students in secondary arts school on the importance of social robot appearance for children with cognitive impairment, and (2) to select the best costume for future child-robot interaction study with children with cognitive impairments.

2. Curriculum deep reinforcement learning with different exploration strategies : a feasibility study on cardiac landmark detection

   Astudillo, Patricio, Mortier, Peter, De Beule, Matthieu, and wyffels, Francis

   In PROCEEDINGS OF THE 13TH INTERNATIONAL JOINT CONFERENCE ON BIOMEDICAL ENGINEERING SYSTEMS AND TECHNOLOGIES, VOL 2: BIOIMAGING 2020

   Abs Biblio

   Transcatheter aortic valve implantation (TAVI) is associated with conduction abnormalities and the mechanical interaction between the prosthesis and the atrioventricular (AV) conduction path cause these life-threatening arrhythmias. Pre-operative assessment of the location of the AV conduction path can help to understand the risk of post-TAVI conduction abnormalities. As the AV conduction path is not visible on cardiac CT, the inferior border of the membranous septum can be used as an anatomical landmark. Detecting this border automatically, accurately and efficiently would save operator time and thus benefit pre-operative planning. This preliminary study was performed to identify the feasibility of 3D landmark detection in cardiac CT images with curriculum deep Q-learning. In this study, curriculum learning was used to gradually teach an artificial agent to detect this anatomical landmark from cardiac CT. This agent was equipped with a small field of view and burdened with a large ac tion-space. Moreover, we introduced two novel action-selection strategies: α-decay and action-dropout. We compared these two strategies to the already established ε-decay strategy and observed that α-decay yielded the most accurate results. Limited computational resources were used to ensure reproducibility. In order to maximize the amount of patient data, the method was cross-validated with k-folding for all three action-selection strategies. An inter-operator variability study was conducted to assess the accuracy of the method

3. Improving time series recognition and prediction with networks and ensembles of passive photonic reservoirs

   Freiberger, Matthias, Sackesyn, Stijn, Ma, Chonghuai, Katumba, Andrew, Bienstman, Peter, and Dambre, Joni

   IEEE JOURNAL OF SELECTED TOPICS IN QUANTUM ELECTRONICS 2020

   Abs Biblio

   As the performance increase of traditional Von-Neumann computing attenuates, new approaches to computing need to be found. A promising approach for low-power computing at high bitrates is integrated photonic reservoir computing. In the past though, the feasible reservoir size and computational power of integrated photonic reservoirs have been limited by hardware constraints. An alternative solution to building larger reservoirs is the combination of several small reservoirs to match or exceed the performance of a single bigger one. This paper summarizes our efforts to increase the available computational power by combining multiple reservoirs into a single computing architecture. We investigate several possible combination techniques and evaluate their performance using the classic XOR and header recognition tasks as well as the well-known Santa Fe chaotic laser prediction task. Our findings suggest that a new paradigm of feeding a reservoir’s output into the readout structure of the next one shows consistently good results for various tasks as well as for both electrical and optical readouts and coupling schemes.

4. Pyglmnet : Python implementation of elastic-net regularized generalized linear models

   Jas, Mainak, Achakulvisut, Titipat, Idrizović, Aid, Acuna, Daniel, Antalek, Matthew, Marques, Vinicius, Odland, Tommy, Garg, Ravi Prakash, Agrawal, Mayank, Umegaki, Yu, Foley, Peter, Fernandes, Hugo, Harris, Drew, Li, Beibin, Pieters, Olivier, Otterson, Scott, De Toni, Giovanni, Rodgers, Chris, Dyer, Eva, Hamalainen, Matti, Kording, Konrad, and Ramkumar, Pavan

   JOURNAL OF OPEN SOURCE SOFTWARE 2020

   Biblio
5. Photonic neuromorphic information processing and reservoir computing

   Lugnan, Alessio, Katumba, Andrew, Laporte, Floris, Freiberger, Matthias, Sackesyn, Stijn, Ma, C., Gooskens, Emmanuel, Dambre, Joni, and Bienstman, Peter

   APL PHOTONICS 2020

   Abs Biblio

   Photonic neuromorphic computing is attracting tremendous research interest now, catalyzed in no small part by the rise of deep learning in many applications. In this paper, we will review some of the exciting work that has been going in this area and then focus on one particular technology, namely, photonic reservoir computing.

6. Security risks of social robots used to persuade and manipulate : a proof of concept study

   Wolfert, Pieter, Deschuyteneer, Jorre, Oetringer, Djamari, Robinson, Nicole, and Belpaeme, Tony

   In HRI’20: COMPANION OF THE 2020 ACM/IEEE INTERNATIONAL CONFERENCE ON HUMAN-ROBOT INTERACTION 2020

   Abs Biblio

   Earlier research has shown that robots can provoke social responses in people, and that robots often elicit compliance. In this paper we discuss three proof of concept studies in which we explore the possibility of robots being hacked and taken over by others with the explicit purpose of using the robot’s social capabilities. Three scenarios are explored: gaining access to secured areas, extracting sensitive and personal information, and convincing people to take unsafe action. We find that people are willing to do these tasks, and that social robots tend to be trusted, even in situations that would normally cause suspicion.

7. Conditional BRUNO : a neural process for exchangeable labelled data

   Korshunova, Iryna, Gal, Yarin, Gretton, Arthur, and Dambre, Joni

   NEUROCOMPUTING 2020

   Biblio
8. Challenges of a real-world HRI study with non-native English speakers : can personalisation save the day?

   Irfan, Bahar, Hellou, Mehdi, Mazel, Alexandre, and Belpaeme, Tony

   In HRI’20: COMPANION OF THE 2020 ACM/IEEE INTERNATIONAL CONFERENCE ON HUMAN-ROBOT INTERACTION 2020

   Abs Biblio

   Real-world studies allow for testing the limits of HRI systems and observing how people react to failures. We developed a fully autonomous personalised barista robot and deployed the robot on an international student campus for five days. We experienced several challenges, the most important one being speech recognition failures due to foreign accents. Nonetheless, these failures showed a different perspective on HRI, and we demonstrate how personalisation can overcome a negative user experience.

9. HRI’20 : Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction

   2020

   Biblio
10. Sign language recognition with transformer networks

    De Coster, Mathieu, Van Herreweghe, Mieke, and Dambre, Joni

    In PROCEEDINGS OF THE 12TH INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION (LREC 2020) 2020

    Abs Biblio

    Sign languages are complex languages. Research into them is ongoing, supported by large video corpora of which only small parts are annotated. Sign language recognition can be used to speed up the annotation process of these corpora, in order to aid research into sign languages and sign language recognition. Previous research has approached sign language recognition in various ways, using feature extraction techniques or end-to-end deep learning. In this work, we apply a combination of feature extraction using OpenPose for human keypoint estimation and end-to-end feature learning with Convolutional Neural Networks. The proven multi-head attention mechanism used in transformers is applied to recognize isolated signs in the Flemish Sign Language corpus. Our proposed method significantly outperforms the previous state of the art of sign language recognition on the Flemish Sign Language corpus: we obtain an accuracy of 74.7% on a vocabulary of 100 classes. Our results will be implemented as a suggestion system for sign language corpus annotation.

11. Century‐long apparent decrease in iWUE with no evidence of progressive nutrient limitation in African tropical forests

    Bauters, Marijn, Meeus, Sofie, Barthel, Matti, Stoffelen, Piet, De Deurwaerder, Hannes, Meunier, Félicien, Drake, Travis W., Ponette, Quentin, Ebuy, Jerôme, Vermeir, Pieter, Beeckman, Hans, wyffels, Francis, Bodé, Samuel, Verbeeck, Hans, Vandelook, Filip, and Boeckx, Pascal

    GLOBAL CHANGE BIOLOGY 2020

    Abs Biblio

    Forests exhibit leaf- and ecosystem-level responses to environmental changes. Specifically, rising carbon dioxide (CO2) levels over the past century are expected to have increased the intrinsic water-use efficiency (iWUE) of tropical trees while the ecosystem is gradually pushed into progressive nutrient limitation. Due to the long-term character of these changes, however, observational datasets to validate both paradigms are limited in space and time. In this study, we used a unique herbarium record to go back nearly a century and show that despite the rise in CO2 concentrations, iWUE has decreased in central African tropical trees in the Congo Basin. Although we find evidence that points to leaf-level adaptation to increasing CO2-that is, increasing photosynthesis-related nutrients and decreasing maximum stomatal conductance, a decrease in leaf delta C-13 clearly indicates a decreasing iWUE over time. Additionally, the stoichiometric carbon to nitrogen and nitrogen to phosphorus ratios in the leaves show no sign of progressive nutrient limitation as they have remained constant since 1938, which suggests that nutrients have not increasingly limited productivity in this biome. Altogether, the data suggest that other environmental factors, such as increasing temperature, might have negatively affected net photosynthesis and consequently downregulated the iWUE. Results from this study reveal that the second largest tropical forest on Earth has responded differently to recent environmental changes than expected, highlighting the need for further on-ground monitoring in the Congo Basin.

12. Gloxinia—an open-source sensing platform to monitor the dynamic responses of plants

    Pieters, Olivier, De Swaef, Tom, Lootens, Peter, Stock, Michiel, Roldán-Ruiz, Isabel, and wyffels, Francis

    SENSORS 2020

    Abs Biblio

    The study of the dynamic responses of plants to short-term environmental changes is becoming increasingly important in basic plant science, phenotyping, breeding, crop management, and modelling. These short-term variations are crucial in plant adaptation to new environments and, consequently, in plant fitness and productivity. Scalable, versatile, accurate, and low-cost data-logging solutions are necessary to advance these fields and complement existing sensing platforms such as high-throughput phenotyping. However, current data logging and sensing platforms do not meet the requirements to monitor these responses. Therefore, a new modular data logging platform was designed, named Gloxinia. Different sensor boards are interconnected depending upon the needs, with the potential to scale to hundreds of sensors in a distributed sensor system. To demonstrate the architecture, two sensor boards were designed—one for single-ended measurements and one for lock-in amplifier based measurements, named Sylvatica and Planalta, respectively. To evaluate the performance of the system in small setups, a small-scale trial was conducted in a growth chamber. Expected plant dynamics were successfully captured, indicating proper operation of the system. Though a large scale trial was not performed, we expect the system to scale very well to larger setups. Additionally, the platform is open-source, enabling other users to easily build upon our work and perform application-specific optimisations.

13. Automatic detection of the aortic annular plane and coronary ostia from multidetector computed tomography

    Astudillo, Patricio, Mortier, Peter, Bosmans, Johan, De Backer, Ole, Jaegere, Peter, Iannaccone, Francesco, De Beule, Matthieu, and Dambre, Joni

    JOURNAL OF INTERVENTIONAL CARDIOLOGY 2020

    Abs Biblio

    Anatomic landmark detection is crucial during preoperative planning of transcatheter aortic valve implantation (TAVI) to select the proper device size and assess the risk of complications. The detection is currently a time-consuming manual process influenced by the image quality and subject to operator variability. In this work, we propose a novel automatic method to detect the relevant aortic landmarks from MDCT images using deep learning techniques. We trained three convolutional neural networks (CNNs) with 344 multidetector computed tomography (MDCT) acquisitions to detect five anatomical landmarks relevant for TAVI planning: the three basal attachment points of the aortic valve leaflets and the left and right coronary ostia. The detection strategy used these three CNN models to analyse a single MDCT image and yield three segmentation volumes as output. These segmentation volumes were averaged into one final segmentation volume, and the final predicted landmarks were obtained during a postprocessing step. Finally, we constructed the aortic annular plane, defined by the three predicted hinge points, and measured the distances from this plane to the predicted coronary ostia (i.e., coronary height). The methodology was validated on 100 patients. The automatic landmark detection was able to detect all the landmarks and showed high accuracy as the median distance between the ground truth and predictions is lower than the interobserver variations (1.5 mm [1.1-2.1], 2.0 mm [1.3-2.8] with a paired difference -0.5 +/- 1.3 mm and p value <0.001). Furthermore, a high correlation is observed between predicted and manually measured coronary heights (for both R-2 = 0.8). The image analysis time per patient was below one second. The proposed method is accurate, fast, and reproducible. Embedding this tool based on deep learning in the preoperative planning routine may have an impact in the TAVI environments by reducing the time and cost and improving accuracy.

14. Simpler learning of robotic manipulation of clothing by utilizing DIY smart textile technology

    Verleysen, Andreas, Holvoet, Thomas, Proesmans, Remko, Den Haese, Cedric, and wyffels, Francis

    APPLIED SCIENCES-BASEL 2020

    Abs Biblio

    Deformable objects such as ropes, wires, and clothing are omnipresent in society and industry but are little researched in robotics research. This is due to the infinite amount of possible state configurations caused by the deformations of the deformable object. Engineered approaches try to cope with this by implementing highly complex operations in order to estimate the state of the deformable object. This complexity can be circumvented by utilizing learning-based approaches, such as reinforcement learning, which can deal with the intrinsic high-dimensional state space of deformable objects. However, the reward function in reinforcement learning needs to measure the state configuration of the highly deformable object. Vision-based reward functions are difficult to implement, given the high dimensionality of the state and complex dynamic behavior. In this work, we propose the consideration of concepts beyond vision and incorporate other modalities which can be extracted from deformable objects. By integrating tactile sensor cells into a textile piece, proprioceptive capabilities are gained that are valuable as they provide a reward function to a reinforcement learning agent. We demonstrate on a low-cost dual robotic arm setup that a physical agent can learn on a single CPU core to fold a rectangular patch of textile in the real world based on a learned reward function from tactile information.

15. From leaf to label : a robust automated workflow for stomata detection

    Meeus, Sofie, Bulcke, Jan, and wyffels, Francis

    ECOLOGY AND EVOLUTION 2020

    Abs Biblio

    1. Plant leaf stomata are the gatekeepers of the atmosphere-plant interface and are essential building blocks of land surface models as they control transpiration and photosynthesis. Although more stomatal trait data is needed to significantly reduce the error in these model predictions, recording these traits is time-consuming and no standardized protocol is currently available. Some attempts were made to automate stomatal detection from photomicrographs, however, these approaches have the disadvantage of using classic image processing or targeting a narrow taxonomic entity which makes these technologies less robust and generalizable to other plant species. We propose an easy-to-use and adaptable workflow from leaf to label. A methodology for automatic stomata detection was developed using deep neural networks according to the state-of-the-art and its applicability demonstrated across the phylogeny of the angiosperms. 2. We used a patch-based approach for training/tuning three different deep learning architectures. For training, we used 431 micrographs taken from leaf prints made according the nail polish method from herbarium specimens of 19 species. The best performing architecture was tested on 595 images of 16 additional species spread across the angiosperm phylogeny. 3. The nail polish method was successfully applied in 78% of the species sampled here. The VGG19 architecture slightly outperformed the basic shallow and deep architectures, with a confidence threshold equal to 0.7 resulting in an optimal trade-off between precision and recall. Applying this threshold the VGG19 architecture obtained an average F-score of 0.87, 0.89 and 0.67 on the training, validation and unseen test set, respectively. The average accuracy was very high (94%) for computed stomatal counts on unseen images of species used for training. 4. The leaf-to-label pipeline is an easy-to-use workflow for researchers of different areas of expertise interested in detecting stomata more efficiently. The described methodology was based on multiple species and well-established methods so that it can serve as a reference for future work.

16. Human-robot interaction : evaluation methods and their standardization

    2020

    Biblio
17. Scaling up integrated photonic reservoirs towards low-power high-bandwidth computing

    Freiberger, Matthias

    2020

    Biblio
18. Human-robot interaction : an introduction

    Bartneck, Christoph, Belpaeme, Tony, Eyssel, Friederike, Kanda, Takayuki, Keijsers, Merel, and Šabanović, Selma

    2020

    Abs Biblio

    The role of robots in society keeps expanding and diversifying, bringing with it a host of issues surrounding the relationship between robots and humans. This introduction to human-robot interaction (HRI), written by leading researchers in this developing field, is the first to provide a broad overview of the multidisciplinary topics central to modern HRI research. Students and researchers from robotics, artificial intelligence, psychology, sociology, and design will find it a concise and accessible guide to the current state of the field. Written for students from diverse backgrounds, it presents relevant background concepts, describing how robots work, how to design them, and how to evaluate their performance. Self-contained chapters discuss a wide range of topics, including the different communication modalities such as speech and language, non-verbal communication and the processing of emotions, as well as ethical issues around the application of robots today and in the context of our future society.

19. Video dataset of human demonstrations of folding clothing for robotic folding

    Verleysen, Andreas, Biondina, Matthijs, and wyffels, Francis

    INTERNATIONAL JOURNAL OF ROBOTICS RESEARCH 2020

    Abs Biblio

    General-purpose clothes-folding robots do not yet exist owing to the deformable nature of textiles, making it hard to engineer manipulation pipelines or learn this task. In order to accelerate research for the learning of the robotic clothes folding task, we introduce a video dataset of human folding demonstrations. In total, we provide 8.5 hours of demonstrations from multiple perspectives leading to 1,000 folding samples of different types of textiles. The demonstrations are recorded in multiple public places, in different conditions with a diverse set of people. Our dataset consists of anonymized RGB images, depth frames, skeleton keypoint trajectories, and object labels. In this article, we describe our recording setup, the data format, and utility scripts, which can be accessed at https://adverley.github.io/folding-demonstrations.

20. Advice to new human-robot interaction researchers

    Belpaeme, Tony

    2020

    Biblio
21. Improvement of child-robot interaction for children diagnosed with cognitive impairments

    Ismail, Luthffi

    2020

    Biblio
22. The DREAM dataset : supporting a data-driven study of autism spectrum disorder and robot enhanced therapy

    Billing, Erik, Belpaeme, Tony, Cai, Haibin, Cao, Hoang-Long, Ciocan, Anamaria, Costescu, Cristina, David, Daniel, Homewood, Robert, Garcia, Daniel Hernandez, Esteban, Pablo Gomez, Liu, Honghai, Nair, Vipul, Matu, Silviu, Mazel, Alexandre, Selescu, Mihaela, Senft, Emmanuel, Thill, Serge, Vanderborght, Bram, Vernon, David, and Ziemke, Tom

    PLOS ONE 2020

    Abs Biblio

    We present a dataset of behavioral data recorded from 61 children diagnosed with Autism Spectrum Disorder (ASD). The data was collected during a large-scale evaluation of Robot Enhanced Therapy (RET). The dataset covers over 3000 therapy sessions and more than 300 hours of therapy. Half of the children interacted with the social robot NAO supervised by a therapist. The other half, constituting a control group, interacted directly with a therapist. Both groups followed the Applied Behavior Analysis (ABA) protocol. Each session was recorded with three RGB cameras and two RGBD (Kinect) cameras, providing detailed information of children’s behavior during therapy. This public release of the dataset comprises body motion, head position and orientation, and eye gaze variables, all specified as 3D data in a joint frame of reference. In addition, metadata including participant age, gender, and autism diagnosis (ADOS) variables are included. We release this data with the hope of supporting further data-driven studies towards improved therapy methods as well as a better understanding of ASD in general.

23. Photonic reservoir computing

    Laporte, Floris, Katumba, Andrew, Freiberger, Matthias, Ma, Chonghuai, Sackesyn, Stijn, Gooskens, Emmanuel, Lugnan, Alessio, Dambre, Joni, and Bienstman, Peter

    In Photonic Integration Week, Abstracts 2020

    Biblio
24. Development of a quantitative comparison tool for plant models

    Pieters, Olivier, De Swaef, Tom, and wyffels, Francis

    In FSPM2020: Towards Computable Plants; 9th International Conference on Functional-structural Plant Models 2020

    Biblio
25. Self-organized dynamic attractors in recurrent neural networks

    Vettelschoss, Benedikt, Freiberger, Matthias, and Dambre, Joni

    In ESANN 2020, European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, Proceediings 2020

    Biblio
26. Close range hyperspectral camera dataset with high temporal resolution of strawberry with eco-physiological data of one leaf

    Pieters, Olivier, De Swaef, Tom, Lootens, Peter, Stock, Michiel, Roldán-Ruiz, Isabel, and wyffels, Francis

    2020

    Abs Biblio

    This high temporal resolution dataset of a strawberry plant was captured in two experiments, each lasting 100h. On leaf was inserted into a leaf chamber of the LI-6400XT gas exchange system, capturing information on transpiration, photosynthesis and stomatal conductance. Environmental characteristics are also captured at canopy height. These experiments were conducted in a growth chamber at ILVO (Melle, Belgium) and only covered conditions that did not result in stress in the plant. As such, this dataset attempts to capture subtle dynamic variation in the plant.

27. High-resolution time series of leaf length and leaf thickness of strawberry and tomato measured in a growth chamber

    Pieters, Olivier, De Swaef, Tom, Lootens, Peter, Stock, Michiel, Roldàn-Ruiz, Isabel, and wyffels, Francis

    2020

    Abs Biblio

    This dataset captures leaf thickness and leaf length of two strawberry plants and one tomato seeding. Leaf thickness measurements are only available of the strawberry plants. Additional data included light intensity data, soil water content (of one plant), relative humidity and temperature data. All sensors were read every second over a period of two weeks. Original data and calibrated data is provided, along with the necessary scripts to process the data.

28. Limitations of snapshot hyperspectral cameras to monitor plant response dynamics in stress-free conditions

    Pieters, Olivier, De Swaef, Tom, Lootens, Peter, Stock, Michiel, Roldán-Ruiz, Isabel, and wyffels, Francis

    COMPUTERS AND ELECTRONICS IN AGRICULTURE 2020

    Abs Biblio

    Plants’ dynamic eco-physiological responses are vital to their productivity in continuously fluctuating conditions, such as those in agricultural fields. However, it is currently still very difficult to capture these responses at the field scale for phenotyping purposes. Advanced hyperspectral imaging tools are increasingly used in phenotyping, and have been applied to detect changes in plants in response to a specific treatment, phenological state or monitor its growth and development. Phenotyping has to evolve towards capturing dynamic behaviour under more subtle fluctuations in environmental conditions, without the presence of clear treatments or stresses. Therefore, we investigated the potential of hyperspectral imaging to capture dynamic behaviour of plants in stress-free conditions at a temporal resolution of seconds. Two growth chamber experiments were set up, in which strawberry plants and four different background materials, serving as controls, were monitored by a snapshot hyperspectral camera in variable conditions of light, temperature and relative humidity. The sampling period was set to three seconds, triggering image acquisition and gas exchange measurements. Different background materials were used to assess the influence of the environment and the camera in both experiments. To separate the plant and background data, static masks were determined. Two datasets were created, which encompass both experiments. One dataset was constructed after averaging over the entire mask to acquire one value per spectral band. These values were then used to calculate a set of vegetation indices. The other dataset used spatial subsampling to retain spatial information. From both datasets, linear models were constructed using ridge regression, which estimated the measured eco-physiological and environmental data. Leaf temperature and vapour pressure deficit based on leaf temperature are the two main eco-physiological characteristics that could be predicted successfully. Stomatal conductance, photosynthesis and transpiration rate show less promising results. We suspect that limited variation, and low spectral resolution and range are the main causes of the inability of the models to extract meaningful predictions. Furthermore, the models that were only trained on background data also showed good predictive performance. This is probably because the main drivers for good performing eco-physiological variables are temperature and incident light intensity. Environmental characteristics that have good performance are photosynthetically active radiation and air temperature. Current hyperspectral sensing technologies are not yet able to uncover most plant dynamic eco-physiological responses when plants are cultivated in stress-free conditions.

29. An open-source social robot based on CompliAnt SofT Robotics for therapy with children with ASD

    Casas-Bocanegra, Diego, Gomez-Vargas, Daniel, Pinto Bernal, Maria Jose, Maldonado, Juan, Munera, Marcela, Villa-Moreno, Adriana, Stoelen, Martin F., Belpaeme, Tony, and Cifuentes, Carlos A.

    ACTUATORS 2020

    Abs Biblio

    Therapy with robotic tools is a promising way to help improve verbal and nonverbal communication in children. The robotic tools are able to increase aspects such as eye contact and the ability to follow instructions and to empathize with others. This work presents the design methodology, development, and experimental validation of a novel social robot based on CompliAnt SofT Robotics called the CASTOR robot, which intends to be used as an open-source platform for the long-term therapy of children with autism spectrum disorder (CwASD). CASTOR integrates the concepts of soft actuators and compliant mechanisms to create a replicable robotic platform aimed at real therapy scenarios involving physical interaction between the children and the robot. The validation shows promising results in terms of robustness and the safety of the user and robot. Likewise, mechanical tests assess the robot’s response to blocking conditions for two critical modules (i.e., neck and arm) in interaction scenarios. Future works should focus on the validation of the robot’s effectiveness in the therapy of CwASD.

30. Kunstmatige intelligentie, klimaatverandering, stomata : KIKS : lerarenhandleiding

    Gesquière, Natacha, Meeus, Sofie, Bulcke, Jan, and wyffels, Francis

    2020

    Biblio
31. Unsupervised orientation learning using autoencoders

    Daems, Rembert, and wyffels, Francis

    In NeurIPS 2020 DiffGeo4DL Workshop 2020

    Abs Biblio

    We present a method to learn the orientation of symmetric objects in real-world images in an unsupervised way. Our method explicitly maps in-plane relative rotations to the latent space of an autoencoder, by rotating both in the image domain and latent domain. This is achieved by adding a proposed \textitcrossing loss to a standard autoencoder training framework which enforces consistency between the image domain and latent domain rotations. This relative representation of rotation is made absolute, by using the symmetry of the observed object, resulting in an unsupervised method to learn the orientation. Furthermore, orientation is disentangled in latent space from other descriptive factors. We apply this method on two real-world datasets: aerial images of planes in the DOTA dataset and images of densely packed honeybees. We empirically show this method can learn orientation using no annotations with high accuracy compared to the same models trained with annotations.

32. Using a personalised socially assistive robot for cardiac rehabilitation : a long-term case study

    Irfan, Bahar, Gomez, Nathalia Cespedes, Casas, Jonathan, Senft, Emmanuel, Gutierrez, Luisa F., Rincon-Roncancio, Monica, Munera, Marcela, Belpaeme, Tony, and Cifuentes, Carlos A.

    In 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN) 2020

    Abs Biblio

    This paper presents a longitudinal case study of Robot Assisted Therapy for cardiac rehabilitation. The patient, who is a 60-year old male that suffered a myocardial infarction and received angioplasty surgery, successfully recovered after 35 sessions of rehabilitation with a social robot, lasting 18 weeks. The sessions took place directly at the clinic and relied on an exercise regime which was designed by the clinicians and delivered with the support of a social robot and a sensor suite. The robot monitored the patient’s progress, and provided personalised encouragement and feedback. We discuss the recovery of the patient and illustrate how the use of a social robot, its sensory systems and its personalised interaction was instrumental to maintain engagement with the programme and to the patient’s recovery. Of note is a critical event that was promptly detected by the robot, which allowed fast intervention measures to be taken by the medical staff for the referral of the patient for further surgery.

33. PyTorch-Hebbian : facilitating local learning in a deep learning framework

    Talloen, Jules, Vandesompele, Alexander, and Dambre, Joni

    In NeurIPS 2020 Beyond backpropagation Workshop 2020

    Biblio
34. A training algorithm for networks of high-variability reservoirs

    Freiberger, Matthias, Bienstman, Peter, and Dambre, Joni

    SCIENTIFIC REPORTS 2020

    Abs Biblio

    Physical reservoir computing approaches have gained increased attention in recent years due to their potential for low-energy high-performance computing. Despite recent successes, there are bounds to what one can achieve simply by making physical reservoirs larger. Therefore, we argue that a switch from single-reservoir computing to multi-reservoir and even deep physical reservoir computing is desirable. Given that error backpropagation cannot be used directly to train a large class of multi-reservoir systems, we propose an alternative framework that combines the power of backpropagation with the speed and simplicity of classic training algorithms. In this work we report our findings on a conducted experiment to evaluate the general feasibility of our approach. We train a network of 3 Echo State Networks to perform the well-known NARMA-10 task, where we use intermediate targets derived through backpropagation. Our results indicate that our proposed method is well-suited to train multi-reservoir systems in an efficient way.

35. Machine learning issues and opportunities in ultrafast particle classification for label-free microflow cytometry

    Lugnan, Alessio, Gooskens, Emmanuel, Vatin, Jeremy, Dambre, Joni, and Bienstman, Peter

    SCIENTIFIC REPORTS 2020

    Abs Biblio

    Machine learning offers promising solutions for high-throughput single-particle analysis in label-free imaging microflow cytomtery. However, the throughput of online operations such as cell sorting is often limited by the large computational cost of the image analysis while offline operations may require the storage of an exceedingly large amount of data. Moreover, the training of machine learning systems can be easily biased by slight drifts of the measurement conditions, giving rise to a significant but difficult to detect degradation of the learned operations. We propose a simple and versatile machine learning approach to perform microparticle classification at an extremely low computational cost, showing good generalization over large variations in particle position. We present proof-of-principle classification of interference patterns projected by flowing transparent PMMA microbeads with diameters of 15.2 mu m and 18.6 mu m. To this end, a simple, cheap and compact label-free microflow cytometer is employed. We also discuss in detail the detection and prevention of machine learning bias in training and testing due to slight drifts of the measurement conditions. Moreover, we investigate the implications of modifying the projected particle pattern by means of a diffraction grating, in the context of optical extreme learning machine implementations.

36. Stance control inspired by cerebellum stabilizes reflex-based locomotion on HyQ robot

    Urbain, Gabriel, Barasuol, Victor, Semini, Claudio, Dambre, Joni, and wyffels, Francis

    In 2020 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA) 2020

    Abs Biblio

    Advances in legged robotics are strongly rooted in animal observations. A clear illustration of this claim is the generalization of Central Pattern Generators (CPG), first identified in the cat spinal cord, to generate cyclic motion in robotic locomotion. Despite a global endorsement of this model, physiological and functional experiments in mammals have also indicated the presence of descending signals from the cerebellum, and reflex feedback from the lower limb sensory cells, that closely interact with CPGs. To this day, these interactions are not fully understood. In some studies, it was demonstrated that pure reflex-based locomotion in the absence of oscillatory signals could be achieved in realistic musculoskeletal simulation models or small compliant quadruped robots. At the same time, biological evidence has attested the functional role of the cerebellum for predictive control of balance and stance within mammals. In this paper, we promote both approaches and successfully apply reflex-based dynamic locomotion, coupled with a balance and gravity compensation mechanism, on the state-of-art HyQ robot. We discuss the importance of this stability module to ensure a correct foot lift-off and maintain a reliable gait. The robotic platform is further used to test two different architectural hypotheses inspired by the cerebellum. An analysis of experimental results demonstrates that the most biologically plausible alternative also leads to better results for robust locomotion.

37. Van motorvoertuig tot autonome robot

    wyffels, Francis

    2020

    Biblio
38. Dendritic computation in a point neuron model

    Vandesompele, Alexander, wyffels, Francis, and Dambre, Joni

    In ARTIFICIAL NEURAL NETWORKS AND MACHINE LEARNING, ICANN 2020, PT II 2020

    Abs Biblio

    Biological neurons possess elaborate dendrites that perform elaborate computations. They are however ignored in the widely used point neuron models. Here, we present a simple addition to the commonly used leaky integrate-and-fire model that introduces the concept of a dendrite. All synapses on the dendrite have a mutual relationship. The result is a form of short term plasticity in which synapse strengths are influenced by recent activity in other synapses. This improves the ability of the neuron to recognize temporal sequences.

39. The GENEA Challenge 2020 : benchmarking gesture-generation systems on common data

    Kucherenko, Taras, Jonell, Patrik, Yoon, Youngwoo, Wolfert, Pieter, and Henter, Gustav Eje

    In GENEA workshop 2020, part of IVA’20, Proceedings 2020

    Abs Biblio

    Automatic gesture generation is a field of growing interest, and a key technology for enabling embodied conversational agents. Research into gesture generation is rapidly gravitating towards data-driven methods. Unfortunately, individual research efforts in the field are difficult to compare: there are no established benchmarks, and each study tends to use its own dataset, motion visualisation, and evaluation methodology. To address this situation, we launched the GENEA gesture-generation challenge, wherein participating teams built automatic gesture-generation systems on a common dataset, and the resulting systems were evaluated in parallel in a large, crowdsourced user study. Since differences in evaluation outcomes between systems now are solely attributable to differences between the motion-generation methods, this enables benchmarking recent approaches against one another and investigating the state of the art in the field. This paper provides a first report on the purpose, design, and results of our challenge, with each individual team’s entry described in a separate paper also presented at the GENEA Workshop. Additional information about the workshop can be found at genea-workshop.github.io/202

2019

1. A personalized and platform-independent behavior control system for social robots in therapy : development and applications

   Cao, Hoang-Long, Perre, Greet, Kennedy, James, Senft, Emmanuel, Esteban, Pablo Gomez, De Beir, Albert, Simut, Ramona, Belpaeme, Tony, Lefeber, Dirk, and Vanderborght, Bram

   IEEE TRANSACTIONS ON COGNITIVE AND DEVELOPMENTAL SYSTEMS 2019

   Abs Biblio

   Social robots have been proven beneficial in different types of healthcare interventions. An ongoing trend is to develop (semi-)autonomous socially assistive robotic systems in healthcare context to improve the level of autonomy and reduce human workload. This paper presents a behavior control system for social robots in therapies with a focus on personalization and platform-independence. This system architecture provides the robot an ability to behave as a personable character, which behaviors are adapted to user profiles and responses during the human-robot interaction. Robot behaviors are designed at abstract levels and can be transferred to different social robot platforms. We adopt the component-based software engineering approach to implement our proposed architecture to allow for the replaceability and reusability of the developed components. We introduce three different experimental scenarios to validate the usability of our system. Results show that the system is potentially applicable to different therapies and social robots. With the component-based approach, the system can serve as a basic framework for researchers to customize and expand the system for their targeted healthcare applications.

2. Leveraging robotics research for children with autism : a review

   Ismail, Luthffi, Verhoeven, Thibault, Dambre, Joni, and wyffels, Francis

   INTERNATIONAL JOURNAL OF SOCIAL ROBOTICS 2019

   Biblio
3. A differentiable physics engine for deep learning in robotics

   Degrave, Jonas, Hermans, Michiel, Dambre, Joni, and wyffels, Francis

   FRONTIERS IN NEUROROBOTICS 2019

   Abs Biblio

   An important field in robotics is the optimization of controllers. Currently, robots are often treated as a black box in this optimization process, which is the reason why derivative-free optimization methods such as evolutionary algorithms or reinforcement learning are omnipresent. When gradient-based methods are used, models are kept small or rely on finite difference approximations for the Jacobian. This method quickly grows expensive with increasing numbers of parameters, such as found in deep learning. We propose an implementation of a modern physics engine, which can differentiate control parameters. This engine is implemented for both CPU and GPU. Firstly, this paper shows how such an engine speeds up the optimization process, even for small problems. Furthermore, it explains why this is an alternative approach to deep Q-learning, for using deep learning in robotics. Finally, we argue that this is a big step for deep learning in robotics, as it opens up new possibilities to optimize robots, both in hardware and software.

4. Leveraging deep learning for eye contact analysis in child-robot interaction

   Ismail, Luthffi, Hanapiah, Fazah Akhtar, Dambre, Joni, Belpaeme, Tony, and wyffels, Francis

   2019

   Biblio
5. Conditional BRUNO : a neural process for exchangeable labelled data

   Korshunova, Iryna, Gal, Yarin, Gretton, Arthur, and Dambre, Joni

   In Proceedings of the 27th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN 2019) 2019

   Abs Biblio

   We present a neural process that models exchangeable sequences of high-dimensional complex observations conditionally on a set of labels or tags. Our model combines the expressiveness of deep neural networks with the data-efficiency of Gaussian processes, resulting in a probabilistic model for which the posterior distribution is easy to evaluate and sample from, and the computational complexity scales linearly with the number of observations. The advantages of the proposed architecture are demonstrated on a challenging few-shot view reconstruction task which requires generalisation from short sequences of viewpoints.

6. Response pattern of child-robot interaction among children with cognitive impairment

   Ismail, Luthffi, Hanapiah, Fazah Akhtar, Belpaeme, Tony, and wyffels, Francis

   In Proceedings of HUSO 2019, the fifth international conference on human and social analytics 2019

   Abs Biblio

   Children with cognitive impairments reported having an engaging initial response in child-robot interaction. However, the interaction modules for child-robot interaction vary and depend on the aim of the studies in the literature. In this study, we designed child-robot interaction modules in order to help children with cognitive impairment in improving their social interaction skills. Hence, we would like to report the overall pattern of their response for each module in 3 sessions of child-robot interaction. Their response pattern for each module is important in future behaviour analysis, especially with regards to their attention skills analysis and their eye contact engagement analysis during child-robot interaction.

7. Body randomization reduces the sim-to-real gap for compliant quadruped locomotion

   Vandesompele, Alexander, Urbain, Gabriel, Mahmud, Hossain, wyffels, Francis, and Dambre, Joni

   FRONTIERS IN NEUROROBOTICS 2019

   Abs Biblio

   Designing controllers for compliant, underactuated robots is challenging and usually requires a learning procedure. Learning robotic control in simulated environments can speed up the process whilst lowering risk of physical damage. Since perfect simulations are unfeasible, several techniques are used to improve transfer to the real world. Here, we investigate the impact of randomizing body parameters during learning of CPG controllers in simulation. The controllers are evaluated on our physical quadruped robot. We find that body randomization in simulation increases chances of finding gaits that function well on the real robot.

8. Design of a bio-inspired compliant quadruped robot for research on closed-loop locomotion control

   Urbain, Gabriel, Vandesompele, Alexander, wyffels, Francis, and Dambre, Joni

   In 2nd HBP student conference : transdisciplinary research linking neuroscience, brain medicine and computer science : book of abstracts 2019

   Biblio
9. Second language tutoring using social robots : a large-scale study

   Vogt, Paul, Berghe, Rianne, Haas, Mirjam, Hoffman, Laura, Kanero, Junko, Mamus, Ezgi, Montanier, Jean-Marc, Oranc, Cansu, Oudgenoeg-Paz, Ora, Garcia, Daniel Hernandez, Papadopoulos, Fotios, Schodde, Thorsten, Verhagen, Josje, Wallbridge, Christopher D., Willemsen, Bram, Wit, Jan, Belpaeme, Tony, Goksun, Tilbe, Kopp, Stefan, Krahmer, Emiel, Kuntay, Aylin C., Leseman, Paul, and Pandey, Amit Kumar

   In HRI ’19: 2019 14TH ACM/IEEE INTERNATIONAL CONFERENCE ON HUMAN-ROBOT INTERACTION 2019

   Abs Biblio

   We present a large-scale study of a series of seven lessons designed to help young children learn English vocabulary as a foreign language using a social robot. The experiment was designed to investigate 1) the effectiveness of a social robot teaching children new words over the course of multiple interactions (supported by a tablet), 2) the added benefit of a robot’s iconic gestures on word learning and retention, and 3) the effect of learning from a robot tutor accompanied by a tablet versus learning from a tablet application alone. For reasons of transparency, the research questions, hypotheses and methods were preregistered. With a sample size of 194 children, our study was statistically well-powered. Our findings demonstrate that children are able to acquire and retain English vocabulary words taught by a robot tutor to a similar extent as when they are taught by a tablet application. In addition, we found no beneficial effect of a robot’s iconic gestures on learning gains.

10. Second language tutoring using social robots : L2TOR-the movie

    Vogt, Paul, Berghe, Rianne, Haas, Mirjam, Hoffman, Laura, Kanero, Junko, Mamus, Ezgi, Montanier, Jean-Marc, Oranc, Cansu, Oudgenoeg-Paz, Ora, Garcia, Daniel Hernandez, Papadopoulos, Fotios, Schodde, Thorsten, Verhagen, Josje, Wallbridge, Christopher D., Willemsen, Bram, Wit, Jan, Belpaeme, Tony, Goksun, Tilbe, Kopp, Stefan, Krahmer, Emiel, Kuntay, Aylin C., Leseman, Paul, and Pandey, Amit Kumar

    In HRI ’19: 2019 14TH ACM/IEEE INTERNATIONAL CONFERENCE ON HUMAN-ROBOT INTERACTION 2019

    Biblio
11. Towards generating spatial referring expressions in a social robot : dynamic vs non-ambiguous

    Wallbridge, Christopher D., Lemaignan, Severin, Senft, Emmanuel, and Belpaeme, Tony

    In HRI ’19: 2019 14TH ACM/IEEE INTERNATIONAL CONFERENCE ON HUMAN-ROBOT INTERACTION 2019

    Abs Biblio

    We present in this paper our work towards a new dynamic method of generating spatial referring expressions. While people are generally ambiguous in their description of locations, previous methods of artificial generation mostly considered non-ambiguous descriptions. However, to increase the naturalness of interaction and share workload in the communication, robots should be able to generate language in a more dynamic way. Our method initially produces ambiguous spatial referring expressions followed by dynamically generating repair statements. We built a classifier using data from 18 participants as they described locations to each other. We perform a preliminary analysis on this method using two further pilot studies.

12. A CycleGAN for style transfer between drum and bass subgenres

    Vande Veire, Len, De Bie, Tijl, and Dambre, Joni

    In ML4MD at ICML2019, Proceedings 2019

    Abs Biblio

    In this work, we apply the CycleGAN image-to-image translation framework to Mel-scaled log-amplitude spectrograms, successfully realizing audio texture transfer between excerpts from two musically related genres. Such automatic musical transfer could provide music producers and DJs with new creative tools, e.g. to quickly prototype a remix of an existing song in another genre, or to use as an advanced effect during a live performance. We show that meaningful style transfer can be realized using only a limited amount of data and computational resources. A high-quality audio reconstruction is obtained from the generated amplitude spectrogram by simply using the phase of the original audio as an approximation for the phase of the generated spectrogram. This results in a significant quality improvement over traditional phase reconstruction methods.

13. From leaf to label : a robust automated workflow for stomata detection

    Meeus, Sofie, wyffels, Francis, and Bulcke, Jan

    In BIODIVERSITY INFORMATION SCIENCE AND STANDARDS 2019

    Biblio
14. Robot-enhanced therapy developing and validating a supervised autonomous robotic system for autism spectrum disorders therapy

    Cao, Hoang-Long, Esteban, Pablo G., Bartlett, Madeleine, Baxter, Paul, Belpaeme, Tony, Billing, Erik, Cai, Haibin, Coeckelbergh, Mark, Costescu, Cristina, David, Daniel, De Beir, Albert, Hernandez, Daniel, Kennedy, James, Liu, Honghai, Matu, Silviu, Mazel, Alexandre, Pandey, Amit, Richardson, Kathleen, Senft, Emmanuel, Thill, Serge, Perre, Greet, Vanderborght, Bram, Vernon, David, Wakunuma, Kutoma, Yu, Hui, Zhou, Xiaolong, and Ziemke, Tom

    IEEE ROBOTICS & AUTOMATION MAGAZINE 2019

    Biblio
15. Should beat gestures be learned or designed? A benchmarking user study

    Wolfert, Pieter, Kucherenko, T., Kjelström, H., and Belpaeme, Tony

    In ICDL-EPIROB 2019 : workshop on naturalistic non-verbal and affective human-robot interactions, Proceedings 2019

    Biblio
16. What can you see? Identifying cues on internal states from the movements of natural social interactions

    Bartlett, Madeleine E., Edmunds, Charlotte E. R., Belpaeme, Tony, Thill, Serge, and Lemaignan, Severin

    FRONTIERS IN ROBOTICS AND AI 2019

    Abs Biblio

    In recent years, the field of Human-Robot Interaction (HRI) has seen an increasing demand for technologies that can recognize and adapt to human behaviors and internal states (e.g., emotions and intentions). Psychological research suggests that human movements are important for inferring internal states. There is, however, a need to better understand what kind of information can be extracted from movement data, particularly in unconstrained, natural interactions. The present study examines which internal states and social constructs humans identify from movement in naturalistic social interactions. Participants either viewed clips of the full scene or processed versions of it displaying 2D positional data. Then, they were asked to fill out questionnaires assessing their social perception of the viewed material. We analyzed whether the full scene clips were more informative than the 2D positional data clips. First, we calculated the inter-rater agreement between participants in both conditions. Then, we employed machine learning classifiers to predict the internal states of the individuals in the videos based on the ratings obtained. Although we found a higher inter-rater agreement for full scenes compared to positional data, the level of agreement in the latter case was still above chance, thus demonstrating that the internal states and social constructs under study were identifiable in both conditions. A factor analysis run on participants’ responses showed that participants identified the constructs interaction imbalance, interaction valence and engagement regardless of video condition. The machine learning classifiers achieved a similar performance in both conditions, again supporting the idea that movement alone carries relevant information. Overall, our results suggest it is reasonable to expect a machine learning algorithm, and consequently a robot, to successfully decode and classify a range of internal states and social constructs using low-dimensional data (such as the movements and poses of observed individuals) as input.

17. Information processing and computation with photonic reservoir systems

    Dambre, Joni

    2019

    Biblio
18. Closed loop control of a compliant quadruped with spiking neural networks

    Vandesompele, Alexander, Urbain, Gabriel, wyffels, Francis, and Dambre, Joni

    In Biologically inspired cognitive architectures 2019 2019

    Biblio
19. Populations of spiking neurons for reservoir computing : closed loop control of a compliant quadruped

    Vandesompele, Alexander, Urbain, Gabriel, wyffels, Francis, and Dambre, Joni

    COGNITIVE SYSTEMS RESEARCH 2019

    Abs Biblio

    Compliant robots can be more versatile than traditional robots, but their control is more complex. The dynamics of compliant bodies can however be turned into an advantage using the physical reservoir computing frame- work. By feeding sensor signals to the reservoir and extracting motor signals from the reservoir, closed loop robot control is possible. Here, we present a novel framework for implementing central pattern generators with spik- ing neural networks to obtain closed loop robot control. Using the FORCE learning paradigm, we train a reservoir of spiking neuron populations to act as a central pattern generator. We demonstrate the learning of predefined gait patterns, speed control and gait transition on a simulated model of a compliant quadrupedal robot.

20. Highly parallel simulation and optimization of photonic circuits in time and frequency domain based on the deep-learning framework PyTorch

    Laporte, Floris, Dambre, Joni, and Bienstman, Peter

    SCIENTIFIC REPORTS 2019

    Abs Biblio

    We propose a new method for performing photonic circuit simulations based on the scatter matrix formalism. We leverage the popular deep-learning framework PyTorch to reimagine photonic circuits as sparsely connected complex-valued neural networks. This allows for highly parallel simulation of large photonic circuits on graphical processing units in time and frequency domain while all parameters of each individual component can easily be optimized with well-established machine learning algorithms such as backpropagation.

21. A neuromorphic silicon photonics nonlinear equalizer for optical communications with intensity modulation and direct detection

    Katumba, Andrew, Yin, Xin, Dambre, Joni, and Bienstman, Peter

    JOURNAL OF LIGHTWAVE TECHNOLOGY 2019

    Abs Biblio

    We present the design and numerical study of a nonlinear equalizer for optical communications based on silicon photonics and reservoir computing. The proposed equalizer leverages the optical information processing capabilities of integrated photonic reservoirs to combat distortions both in metro links of a few hundred kilometers and in high-speed short-reach intensity-modulation-direct-detection links. We show nonlinear compensation in unrepeated metro links of up to 200 km that outperform electrical feedforward equalizers based equalizers, and ultimately any linear compensation device. For a high-speed short-reach 40Gb/s link based on a distributed feedback laser and an electroabsorptive modulator, and considering a hard decision forward error correction limit of 0.2 x 10(-2), we can increase the reach by almost 10 km. Our equalizer is compact (only 16 nodes) and operates in the optical domain without the need for complex electronic DSP, meaning its performance is not bandwidth constrained. The approach is, therefore, a viable candidate even for equalization techniques far beyond 100G optical communication links.

22. Advanced X-ray CT scanning can boost tree-ring research for earth-system sciences

    Bulcke, Jan, Boone, Marijn A, Dhaene, Jelle, Van Loo, Denis, Van Hoorebeke, Luc, Boone, Matthieu, wyffels, Francis, Beeckman, Hans, Van Acker, Joris, and De Mil, Tom

    ANNALS OF BOTANY 2019

    Abs Biblio

    Background and Aims: Tree rings, as archives of the past and biosensors of the present, offer unique opportunities to study influences of the fluctuating environment over decades to centuries. As such, tree-ring-based wood traits are capital input for global vegetation models. To contribute to earth system sciences, however, sufficient spatial coverage is required of detailed individual-based measurements, necessitating large amounts of data. X-ray computed tomography (CT) scanning is one of the few techniques that can deliver such data sets. Methods: Increment cores of four different temperate tree species were scanned with a state-of-the-art X-ray CT system at resolutions ranging from 60 mu m down to 4.5 mu m, with an additional scan at a resolution of 0.8 mu m of a splinter-sized sample using a second X-ray CT system to highlight the potential of cell-level scanning. Calibration-free densitometry, based on full scanner simulation of a third X-ray CT system, is illustrated on increment cores of a tropical tree species. Key Results: We show how multiscale scanning offers unprecedented potential for mapping tree rings and wood traits without sample manipulation and with limited operator intervention. Custom-designed sample holders enable simultaneous scanning of multiple increment cores at resolutions sufficient for tree ring analysis and densitometry as well as single core scanning enabling quantitative wood anatomy, thereby approaching the conventional thin section approach. Standardized X-ray CT volumes are, furthermore, ideal input imagery for automated pipelines with neural-based learning for tree ring detection and measurements of wood traits. Conclusions: Advanced X-ray CT scanning for high-throughput processing of increment cores is within reach, generating pith-to-bark ring width series, density profiles and wood trait data. This would allow contribution to large-scale monitoring and modelling efforts with sufficient global coverage.

23. Training passive photonic reservoirs with integrated optical readout

    Freiberger, Matthias, Katumba, Andrew, Bienstman, Peter, and Dambre, Joni

    IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2019

    Abs Biblio

    As Moore’s law comes to an end, neuromorphic approaches to computing are on the rise. One of these, passive photonic reservoir computing, is a strong candidate for computing at high bitrates (>10 Gb/s) and with low energy consumption. Currently though, both benefits are limited by the necessity to perform training and readout operations in the electrical domain. Thus, efforts are currently underway in the photonic community to design an integrated optical readout, which allows to perform all operations in the optical domain. In addition to the technological challenge of designing such a readout, new algorithms have to be designed in order to train it. Foremost, suitable algorithms need to be able to deal with the fact that the actual on-chip reservoir states are not directly observable. In this paper, we investigate several options for such a training algorithm and propose a solution in which the complex states of the reservoir can be observed by appropriately setting the readout weights, while iterating over a predefined input sequence. We perform numerical simulations in order to compare our method with an ideal baseline requiring full observability as well as with an established black-box optimization approach (CMA-ES).

24. Generating spatial referring expressions in a social robot : dynamic vs. non-ambiguous

    Wallbridge, Christopher D., Lemaignan, Severin, Senft, Emmanuel, and Belpaeme, Tony

    FRONTIERS IN ROBOTICS AND AI 2019

    Abs Biblio

    Generating spatial referring expressions is key to allowing robots to communicate with people in an environment. The focus of most algorithms for generation is to create a non-ambiguous description, and how best to deal with the combination explosion this can create in a complex environment. However, this is not how people naturally communicate. Humans tend to give an under-specified description and then rely on a strategy of repair to reduce the number of possible locations or objects until the correct one is identified, what we refer to here as a dynamic description. We present here a method for generating these dynamic descriptions for Human Robot Interaction, using machine learning to generate repair statements. We also present a study with 61 participants in a task on object placement. This task was presented in a 2D environment that favored a non-ambiguous description. In this study we demonstrate that our dynamic method of communication can be more efficient for people to identify a location compared to one that is non-ambiguous.

25. Combining evolutionary and adaptive control strategies for quadruped robotic locomotion

    Massi, Elisa, Vannucci, Lorenzo, Albanese, Ugo, Capolei, Marie Claire, Vandesompele, Alexander, Urbain, Gabriel, Sabatini, Angelo Maria, Dambre, Joni, Laschi, Cecilia, Tolu, Silvia, and Falotico, Egidio

    FRONTIERS IN NEUROROBOTICS 2019

    Abs Biblio

    In traditional robotics, model-based controllers are usually needed in order to bring a robotic plant to the next desired state, but they present critical issues when the dimensionality of the control problem increases and disturbances from the external environment affect the system behavior, in particular during locomotion tasks. It is generally accepted that the motion control of quadruped animals is performed by neural circuits located in the spinal cord that act as a Central Pattern Generator and can generate appropriate locomotion patterns. This is thought to be the result of evolutionary processes that have optimized this network. On top of this, fine motor control is learned during the lifetime of the animal thanks to the plastic connections of the cerebellum that provide descending corrective inputs. This research aims at understanding and identifying the possible advantages of using learning during an evolution-inspired optimization for finding the best locomotion patterns in a robotic locomotion task. Accordingly, we propose a comparative study between two bio-inspired control architectures for quadruped legged robots where learning takes place either during the evolutionary search or only after that. The evolutionary process is carried out in a simulated environment, on a quadruped legged robot. To verify the possibility of overcoming the reality gap, the performance of both systems has been analyzed by changing the robot dynamics and its interaction with the external environment. Results show better performance metrics for the robotic agent whose locomotion method has been discovered by applying the adaptive module during the evolutionary exploration for the locomotion trajectories. Even when the motion dynamics and the interaction with the environment is altered, the locomotion patterns found on the learning robotic system are more stable, both in the joint and in the task space.

26. Towards safe and efficient pre-operative planning of transcatheter mitral valve interventions

    Astudillo, Patricio, De Beule, Matthieu, Dambre, Joni, and Mortier, Peter

    MORPHOLOGIE 2019

    Biblio
27. Enabling automated device size selection for transcatheter aortic valve implantation

    Astudillo, Patricio, Mortier, Peter, Bosmans, Johan, De Backer, Ole, De Jaegere, Peter, De Beule, Matthieu, and Dambre, Joni

    JOURNAL OF INTERVENTIONAL CARDIOLOGY 2019

    Abs Biblio

    The number of transcatheter aortic valve implantation (TAVI) procedures is expected to increase significantly in the coming years. Improving efficiency will become essential for experienced operators performing large TAVI volumes, while new operators will require training and may benefit from accurate support. In this work, we present a fast deep learning method that can predict aortic annulus perimeter and area automatically from aortic annular plane images. We propose a method combining two deep convolutional neural networks followed by a postprocessing step. The models were trained with 355 patients using modern deep learning techniques, and the method was evaluated on another 118 patients. The method was validated against an interoperator variability study of the same 118 patients. The differences between the manually obtained aortic annulus measurements and the automatic predictions were similar to the differences between two independent observers (paired diff. of 3.3 +/- 16.8 mm(2) vs. 1.3 +/- 21.1 mm(2) for the area and a paired diff. of 0.6 +/- 1.7 mm vs. 0.2 +/- 2.5 mm for the perimeter). The area and perimeter were used to retrieve the suggested prosthesis sizes for the Edwards Sapien 3 and the Medtronic Evolut device retrospectively. The automatically obtained device size selections accorded well with the device sizes selected by operator 1. The total analysis time from aortic annular plane to prosthesis size was below one second. This study showed that automated TAVI device size selection using the proposed method is fast, accurate, and reproducible. Comparison with the interobserver variability has shown the reliability of the strategy, and embedding this tool based on deep learning in the preoperative planning routine has the potential to increase the efficiency while ensuring accuracy.

28. Classification of aortic valve morphology using a three-dimensional convolutional neural network

    Dowling, Cameron, Astudillo, Patricio, Brecker, Stephen, De Beule, Matthieu, Mortier, Peter, and Dambre, Joni

    In PCR London Valves, Abstracts 2019

    Biblio
29. Development of a smart cycling support device for people with an intellectual disability

    Deprez, Kenneth, Holvoet, Emiel, Reynders, Esther, Vandepoele, Michelle, Verstringe, Maud, wyffels, Francis, Verstockt, Steven, and Saldien, Jelle

    In Project and design literacy as cornerstones of smart education : proceedings of the 4th International Conference on Smart Learning Ecosystems and Regional Development 2019

    Abs Biblio

    While cycling in traffic, people encounter many different situations with external triggers, that need special attention, such as vehicles, other bicycles, pedestrians, road surface and intersections. Furthermore, people need to remember the route to their destination and take into account the traffic regulations. For people with an intellectual disability, this can be a challenging task.

30. Teaching robots social autonomy from in situ human guidance

    Senft, Emmanuel, Lemaignan, Séverin, Baxter, Paul E., Bartlett, Madeleine, and Belpaeme, Tony

    SCIENCE ROBOTICS 2019

    Abs Biblio

    Striking the right balance between robot autonomy and human control is a core challenge in social robotics, in both technical and ethical terms. On the one hand, extended robot autonomy offers the potential for increased human productivity and for the off-loading of physical and cognitive tasks. On the other hand, making the most of human technical and social expertise, as well as maintaining accountability, is highly desirable. This is particularly relevant in domains such as medical therapy and education, where social robots hold substantial promise, but where there is a high cost to poorly performing autonomous systems, compounded by ethical concerns. We present a field study in which we evaluate SPARC (supervised progressively autonomous robot competencies), an innovative approach addressing this challenge whereby a robot progressively learns appropriate autonomous behavior from in situ human demonstrations and guidance. Using online machine learning techniques, we demonstrate that the robot could effectively acquire legible and congruent social policies in a high-dimensional child-tutoring situation needing only a limited number of demonstrations while preserving human supervision whenever desirable. By exploiting human expertise, our technique enables rapid learning of autonomous social and domain-specific policies in complex and nondeterministic environments. Last, we underline the generic properties of SPARC and discuss how this paradigm is relevant to a broad range of difficult human-robot interaction scenarios.

31. Towards automatic sign language corpus annotation using deep learning

    De Coster, Mathieu, Van Herreweghe, Mieke, and Dambre, Joni

    In 6th Workshop on Sign Language Translation and Avatar Technology (SLTAT) 2019

    Abs Biblio

    Sign classification in sign language corpora is a challenging problem that requires large datasets. Unfortunately, only a small portion of those corpora is labeled. To expedite the annotation process, we propose a gloss suggestion system based on deep learning. We improve upon previous research in three ways. Firstly, we use a proven feature extraction method called OpenPose, rather than learning end-to-end. Secondly, we propose a more suitable and powerful network architecture, based on GRU layers. Finally, we exploit domain and task knowledge to further increase the accuracy. We show that we greatly outperform the previous state of the art on the used dataset. Our method can be used for suggesting a top 5 of annotations given a video fragment that is selected by the corpus annotator. We expect that it will expedite the annotation process to the benefit of sign language translation research.

32. A power-efficient architecture for on-chip reservoir computing

    Sackesyn, Stijn, Ma, Chonghuai, Katumba, Andrew, Dambre, Joni, and Bienstman, Peter

    In Artificial neural networks and machine learning – ICANN 2019 : workshop and special sessions 2019

    Abs Biblio

    Reservoir computing is a neuromorphic computing paradigm which is well suited for hardware implementations. In this work, an enhanced reservoir architecture is introduced as to lower the losses and improve mixing behaviour in silicon photonic reservoir computing designs.

33. Photontorch : simulation and optimization of large photonic circuits using the deep learning framework PyTorch

    Laporte, Floris, Dambre, Joni, and Bienstman, Peter

    In 2019 IEEE Photonics Society Summer Topical Meeting Series (SUM) 2019

    Abs Biblio

    We propose a new framework for simulating and optimizing large photonic circuits. The framework utilizes gpu-acceleration to efficiently simulate the circuits in a highly parallel manner, while optimization is achieved through backpropagation by essentially viewing the network as a sparsely connected recurrent neural network.

34. Neuromorphic information processing using silicon photonics

    Bienstman, Peter, Dambre, Joni, Katumba, Andrew, Freiberger, Matthias, Laporte, Floris, Lugnan, Alessio, Sackesyn, Stijn, and Ma, Chonghuai

    In Active Photonic Platforms XI 2019

    Biblio
35. All-optical readout for integrated photonic reservoir computing

    Ma, Chonghuai, Sackesyn, Stijn, Dambre, Joni, and Bienstman, Peter

    In 2019 21st International Conference on Transparent Optical Networks (ICTON) 2019

    Abs Biblio

    Photonic neuromorphic computing has gained a lot of attention for its strong potential to deliver machine learning computation capability at high bitrates (> 32 Gbps) with very low energy consumption. Reservoir computing is one of the strong candidates that delivers a huge advantage on a real hardware implementation. However, one of the challenges is that current readout systems are the bottleneck of the high-speed link involving heavy power consumption from opto-electrical conversions. In this paper, we present our design of an integrated all-optical readout system that overcomes the challenges with optical weighting elements, which works at 32 Gbps and can deliver computation capability on-chip with one final readout signal channel. We especially compare the memory capability difference between the optical readout scheme with conventional electrical readout and show that optical readout is superior. Furthermore, this paper discusses the problem of some non-volatile optical weighting elements having limited weighting resolution. As a result, our readout system can still perform very well at an low resolution (4 bit) compared to using full resolution weighting elements.

36. Non-linear signal equalisation using silicon photonic reservoir computing

    Bienstman, Peter, Dambre, Joni, Katumba, Andrew, Freiberger, Matthias, Laporte, Floris, Lugnan, Alessio, Sackesyn, Stijn, Ma, Chonghuai, and Gooskens, Emmanuel

    In ECOC Machine Learning Workshop 2019, Abstracts 2019

    Biblio
37. Silicon photonics reservoir computing at 32 Gbit/s

    Bienstman, Peter, Dambre, Joni, Katumba, Andrew, Freiberger, Matthias, Laporte, Floris, Lugnan, Alessio, Sackesyn, Stijn, Ma, Chonghuai, and Gooskens, Emmanuel

    In 2019 Workshop on Dynamical Systems and Brain-Inspired Information Processing, Abstracts 2019

    Biblio
38. Reinforcement learning and insight in the artificial pigeon

    Colin, Thomas R., and Belpaeme, Tony

    In Creativity + cognition + computation : 41st Annual Meeting of the Cognitive Science Society (CogSci 2019) 2019

    Abs Biblio

    The phenomenon of insight (also called "Aha!" or "Eureka!" moments) is considered a core component of creative cognition. It is also a puzzle and a challenge for statistics-based approaches to behavior such as associative learning and reinforcement learning. We simulate a classic experiment on insight in pigeons using deep Reinforcement Learning. We show that prior experience may produce large and rapid performance improvements reminiscent of insights, and we suggest theoretical connections between concepts from machine learning (such as the value function or overfitting) and concepts from psychology (such as feelings-of-warmth and the einstellung effect). However, the simulated pigeons were slower than the real pigeons at solving the test problem, requiring a greater amount of trial and error: their "insightful" behavior was sudden by comparison with learning from scratch, but slow by comparison with real pigeons. This leaves open the question of whether incremental improvements to reinforcement learning algorithms will be sufficient to produce insightful behavior.

2018

1. Beyond temporal pooling : recurrence and temporal convolutions for gesture recognition in video

   Pigou, Lionel, Oord, Aäron, Dieleman, Sander, Van Herreweghe, Mieke, and Dambre, Joni

   INTERNATIONAL JOURNAL OF COMPUTER VISION 2018

   Abs Biblio

   Recent studies have demonstrated the power of recurrent neural networks for machine translation, image captioning and speech recognition. For the task of capturing temporal structure in video, however, there still remain numerous open research questions. Current research suggests using a simple temporal feature pooling strategy to take into account the temporal aspect of video. We demonstrate that this method is not sufficient for gesture recognition, where temporal information is more discriminative compared to general video classification tasks. We explore deep architectures for gesture recognition in video and propose a new end-to-end trainable neural network architecture incorporating temporal convolutions and bidirectional recurrence. Our main contributions are twofold; first, we show that recurrence is crucial for this task; second, we show that adding temporal convolutions leads to significant improvements. We evaluate the different approaches on the Montalbano gesture recognition dataset, where we achieve state-of-the-art results.

2. Towards improved design and evaluation of epileptic seizure predictors

   Korshunova, Iryna, Kindermans, Pieter-Jan, Degrave, Jonas, Verhoeven, Thibault, Brinkmann, Benjamin, and Dambre, Joni

   IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING 2018

   Biblio
3. Automated diagnosis of temporal lobe epilepsy in the absence of interictal spikes

   Verhoeven, Thibault, Coito, Ana, Plomp, Gijs, Thomschewski, Aljoscha, Pittau, Francesca, Trinka, Eugen, Wiest, Roland, Schaller, Karl, Michel, Christoph, Seeck, Margitta, Dambre, Joni, Vulliemoz, Serge, and Mierlo, Pieter

   NEUROIMAGE-CLINICAL 2018

   Abs Biblio

   Objective: To diagnose and lateralise temporal lobe epilepsy (TLE) by building a classification system that uses directed functional connectivity patterns estimated during EEG periods without visible pathological activity. Methods: Resting-state high-density EEG recording data from 20 left TLE patients, 20 right TLE patients and 35 healthy controls was used. Epochs without interictal spikes were selected. The cortical source activity was obtained for 82 regions of interest and whole-brain directed functional connectivity was estimated in the theta, alpha and beta frequency bands. These connectivity values were then used to build a classification system based on two two-class Random Forests classifiers: TLE vs healthy controls and left vs right TLE. Feature selection and classifier training were done in a leave-one-out procedure to compute the mean classification accuracy. Results: The diagnosis and lateralization classifiers achieved a high accuracy (90.7% and 90.0% respectively), sensitivity (95.0% and 90.0% respectively) and specificity (85.7% and 90.0% respectively). The most important features for diagnosis were the outflows from left and right medial temporal lobe, and for lateralization the right anterior cingulate cortex. The interaction between features was important to achieve correct classification.

4. The multi-modal interface of Robot-Era multi-robot services tailored for the elderly

   Di Nuovo, Alessandro, Broz, Frank, Wang, Ning, Belpaeme, Tony, Cangelosi, Angelo, Jones, Ray, Esposito, Raffaele, Cavallo, Filippo, and Dario, Paolo

   INTELLIGENT SERVICE ROBOTICS 2018

   Abs Biblio

   Socially assistive robotic platforms are now a realistic option for the long-term care of ageing populations. Elderly users may benefit from many services provided by robots operating in different environments, such as providing assistance inside apartments, serving in shared facilities of buildings or guiding people outdoors. In this paper, we present the experience gained within the EU FP7 ROBOT-ERA project towards the objective of implementing easy-to-use and acceptable service robotic system for the elderly. In particular, we detail the user-centred design and the experimental evaluation in realistic environments of a web-based multi-modal user interface tailored for elderly users of near future multi-robot services. Experimental results demonstrate positive evaluation of usability and willingness to use by elderly users, especially those less experienced with technological devices who could benefit more from the adoption of robotic services. Further analyses showed how multi-modal modes of interaction support more flexible and natural elderly-robot interaction, make clear the benefits for the users and, therefore, increase its acceptability. Finally, we provide insights and lessons learned from the extensive experimentation, which, to the best of our knowledge, is one of the largest experimentation of a multi-robot multi-service system so far.

5. Bringing computer science education to secondary school : a teacher first approach

   Neutens, Tom, and wyffels, Francis

   In SIGCSE’18: PROCEEDINGS OF THE 49TH ACM TECHNICAL SYMPOSIUM ON COMPUTER SCIENCE EDUCATION 2018

   Abs Biblio

   The Progra-MEER professional development workshop is a one year program organized collaboratively by the computer science departments of three Flemish universities. It aims to improve the computer science knowledge of in service teachers in a physical computing context. Since Flemish schools are starting to implement STEM in their schools, the program links computer science to STEM and project based learning. This paper gives a description of the design and implementation of the program while providing an analysis of its strengths and weaknesses. We show that the program leads to the successful implementation of different physical computing projects. However, it needs to further support the practical project implementations while spending more attention on assessment and context definition. Additionally, the program has to invest more effort in creating a sustainable community of practice so knowledge and experiences can still be shared even after the program has finished.

6. Robot enhanced therapy for children with autism (DREAM) : a social model of autism

   Richardson, Kathleen, Coeckelbergh, Mark, Wakunuma, Kutoma, Billing, Erik, Ziemke, Tom, Gomez, Pablo, Vanderborght, Bram, and Belpaeme, Tony

   IEEE TECHNOLOGY AND SOCIETY MAGAZINE 2018

   Biblio
7. Social assistive robot for cardiac rehabilitation

   Casas, Jonathan, Irfan, Bahar, Senft, Emmanuel, Gutiérrez, Luisa, Rincon-Roncancio, Monica, Munera, Marcela, Belpaeme, Tony, and Cifuentes, Carlos A.

   In HRI ’18 Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction 2018

   Biblio
8. Guidelines for designing social robots as second language tutors

   Belpaeme, Tony, Vogt, Paul, Berghe, Rianne, Bergmann, Kirsten, Göksun, Tilbe, Haas, Mirjam, Kanero, Junko, Kennedy, James, Küntay, Aylin C., Oudgenoeg-Paz, Ora, Papadopoulos, Fotios, Schodde, Thorsten, Verhagen, Josje, Wallbridge, Christopher D., Willemsen, Bram, Wit, Jan, Geçkin, Vasfiye, Hoffmann, Laura, Kopp, Stefan, Krahmer, Emiel, Mamus, Ezgi, Montanier, Jean-Marc, Oranç, Cansu, and Pandey, Amit Kumar

   INTERNATIONAL JOURNAL OF SOCIAL ROBOTICS 2018

   Biblio
9. Social robots for education : a review

   Belpaeme, Tony, Kennedy, James, Ramachandran, Aditi, Scassellati, Brian, and Tanaka, Fumihide

   SCIENCE ROBOTICS 2018

   Abs Biblio

   Social robots can be used in education as tutors or peer learners. They have been shown to be effective at increasing cognitive and affective outcomes and have achieved outcomes similar to those of human tutoring on restricted tasks. This is largely because of their physical presence, which traditional learning technologies lack. We review the potential of social robots in education, discuss the technical challenges, and consider how the robot’s appearance and behavior affect learning outcomes.

10. Calibration method to improve transfer from simulation to quadruped robots

    Urbain, Gabriel, Vandesompele, Alexander, wyffels, Francis, and Dambre, Joni

    In FROM ANIMALS TO ANIMATS 15 2018

    Abs Biblio

    Using passive compliance in robotic locomotion has been seen as a cheap and straightforward way of increasing the performance in energy consumption and robustness. However, the control for such systems remains quite challenging when using traditional robotic techniques. The progress in machine learning opens a horizon of new possibilities in this direction but the training methods are generally too long and laborious to be conducted on a real robot platform. On the other hand, learning a control policy in simulation also raises a lot of complication in the transfer. In this paper, we designed a cheap quadruped robot and detail a calibration method to optimize a simulation model in order to facilitate the transfer of parametric motor primitives. We present results validating the transfer of Central Pattern Generators (CPG) learned in simulation to the robot which already give positive insights on the validity of this method.

11. Low-loss photonic reservoir computing with multimode photonic integrated circuits

    Katumba, Andrew, Heyvaert, Jelle, Schneider, Bendix, Uvin, Sarah, Dambre, Joni, and Bienstman, Peter

    SCIENTIFIC REPORTS 2018

    Biblio
12. Silicon photonics for neuromorphic information processing

    Bienstman, Peter, Dambre, Joni, Katumba, Andrew, Freiberger, Matthias, Laporte, Floris, and Lugnan, Alessio

    In OPTICAL DATA SCIENCE: TRENDS SHAPING THE FUTURE OF PHOTONICS 2018

    Abs Biblio

    We present our latest results on silicon photonics neuromorphic information processing based a.o. on techniques like reservoir computing. We will discuss aspects like scalability, novel architectures for enhanced power efficiency, as well as all-optical readout. Additionally, we will touch upon new machine learning techniques to operate these integrated readouts. Finally, we will show how these systems can be used for high-speed low-power information processing for applications like recognition of biological cells.

13. Photonic reservoir computing: a brain-inspired approach for information processing (invited paper)

    Bienstman, Peter, Dambre, Joni, Katumba, Andrew, Freiberger, Matthias, Laporte, Floris, and Lugnan, Alessio

    In 2018 OPTICAL FIBER COMMUNICATIONS CONFERENCE AND EXPOSITION (OFC) 2018

    Abs Biblio

    We present our latest results on silicon photonics neuromorphic information processing based on techniques like reservoir computing. We will discuss aspects like scalability, novel architectures for enhanced power efficiency, as well as all-optical readout. Additionally, we will touch upon new machine learning techniques to operate these integrated readouts.

14. Numerical demonstration of neuromorphic computing with photonic crystal cavities

    Laporte, Floris, Katumba, Andrew, Dambre, Joni, and Bienstman, Peter

    OPTICS EXPRESS 2018

    Biblio
15. Neuromorphic computing based on silicon photonics and reservoir computing

    Katumba, Andrew, Freiberger, Matthias, Laporte, Floris, Lugnan, Alessio, Sackesyn, Stijn, Ma, Chonghuai, Dambre, Joni, and Bienstman, Peter

    IEEE JOURNAL OF SELECTED TOPICS IN QUANTUM ELECTRONICS 2018

    Biblio
16. All-optical reservoir computing on a photonic chip using silicon-based ring resonators

    Le Courier, Florian Denis, Sciamanna, Marc, Katumba, Andrew, Freiberger, Matthias, Dambre, Joni, Bienstman, Peter, and Rontani, Damien

    IEEE JOURNAL OF SELECTED TOPICS IN QUANTUM ELECTRONICS 2018

    Abs Biblio

    We present in our work numerical results on the performance of a 4 x 4 swirl-topology photonic reservoir integrated on a silicon chip. Nonlinearmicroring resonators are used as nodes. We analyze the performance of such a reservoir on a classical non-linear Boolean task (the delayed XOR task) for: various designs of the reservoir in terms of lengths of the waveguides between consecutive nodes, and various injection parameters (injected power and optical detuning). From this analysis, we find that this kind of reservoir can perform-for a large variety of parameters-the delayed XOR task at 20 Gb/s with bit error rates lower than 10(-3) and an averaged injection power lower than 2.5 mW.

17. Toward neuro-inspired computing using a small network of micro-ring resonators on an integrated photonic chip

    Denis-le Coarer, Florian, Freiberger, Matthias, Dambre, Joni, Bienstman, Peter, Rontani, Damien, Katumba, Andrew, and Sciamanna, Marc

    In NEURO-INSPIRED PHOTONIC COMPUTING 2018

    Abs Biblio

    We present in this work numerical simulations of the performance of an on-chip photonic reservoir computer using nonlinear microring resonator as neurons. We present dynamical properties of the nonlinear node and the reservoir computer, and we analyse the performance of the reservoir on a typical nonlinear Boolean task : the delayed XOR task. We study the performance for various designs (number of nodes, and length of the synapses in the reservoir), and with respect to the properties of the optical injection of the data (optical detuning and power). From this work, we find that such a reservoir has state-of-the art level of performance on this particular task - that is a bit error rate of 2.5 10(-4) - at 20 Gb/s, with very good power efficiency (total injected power lower than 1.0 mW).

18. Silicon photonics neuromorphic computing and its application to telecommunications

    Katumba, Andrew, Yin, Xin, Dambre, Joni, and Bienstman, Peter

    In 44th European Conference on Optical Communication 2018

    Biblio
19. Solis : a smart interactive system for houseplants caring

    Penders, Arno, Octavia, Johanna Renny, Caron, Michiel, Haan, Fay, Devoogdt, Thomas, Nop, Samnang, McAtear, Anthony, Pieters, Olivier, wyffels, Francis, Verstockt, Steven, and Saldien, Jelle

    In 2018 INTERNATIONAL CONFERENCE ON ORANGE TECHNOLOGIES (ICOT) 2018

    Biblio
20. The PInSoRo dataset : supporting the data- driven study of child-child and child-robot social dynamics

    Lemaignan, Severin, Edmunds, Charlotte E. R., Senft, Emmanuel, and Belpaeme, Tony

    PLOS ONE 2018

    Abs Biblio

    The study of the fine-grained social dynamics between children is a methodological challenge, yet a good understanding of how social interaction between children unfolds is important not only to Developmental and Social Psychology, but recently has become relevant to the neighbouring field of Human-Robot Interaction (HRI). Indeed, child-robot interactions are increasingly being explored in domains which require longer-term interactions, such as healthcare and education. For a robot to behave in an appropriate manner over longer time scales, its behaviours have to be contingent and meaningful to the unfolding relationship. Recognising, interpreting and generating sustained and engaging social behaviours is as such an important-and essentially, open-research question. We believe that the recent progress of machine learning opens new opportunities in terms of both analysis and synthesis of complex social dynamics. To support these approaches, we introduce in this article a novel, open dataset of child social interactions, designed with data-driven research methodologies in mind. Our data acquisition methodology relies on an engaging, methodologically sound, but purposefully underspecified free-play interaction. By doi ng so, we capture a rich set of behavioural patterns occurring in natural social interactions between children. The resulting dataset, called the PInSoRo dataset, comprises 45+ hours of hand-coded recordings of social interactions between 45 child-child pairs and 30 child-robot pairs. In addition to annotations of social constructs, the dataset includes fully calibrated video recordings, 3D recordings of the faces, skeletal informations, full audio recordings, as well as game interactions.

21. Teaching computing in primary school : create or fix?

    Neutens, Tom, and wyffels, Francis

    In WIPSCE’18: PROCEEDINGS OF THE 13TH WORKSHOP IN PRIMARY AND SECONDARY COMPUTING EDUCATION 2018

    Biblio
22. Dual Rectified Linear Units (DReLUs) : a replacement for tanh activation functions in quasi-recurrent neural networks

    Godin, Fréderic, Degrave, Jonas, Dambre, Joni, and De Neve, Wesley

    PATTERN RECOGNITION LETTERS 2018

    Abs Biblio

    In this paper, we introduce a novel type of Rectified Linear Unit (ReLU), called a Dual Rectified Linear Unit (DReLU). A DReLU, which comes with an unbounded positive and negative image, can be used as a dropin replacement for a tanh activation function in the recurrent step of Quasi-Recurrent Neural Networks (QRNNs) [1]. Similar to ReLUs, DReLUs are less prone to the vanishing gradient problem, they are noise robust, and they induce sparse activations. We independently reproduce the QRNN experiments of Bradbury et al. [1] and compare our DReLUbased QRNNs with the original tanh-based QRNNs and Long Short-Term Memory networks (LSTMs) on sentiment classification and word-level language modeling. Additionally, we evaluate on character-level language modeling, showing that we are able to stack up to eight QRNN layers with DReLUs, thus making it possible to improve the current state-of-the-art in character-level language modeling over shallow architectures based on LSTMs. (C) 2018 Elsevier B.V. All rights reserved.

23. Explaining character-aware neural networks for word-level prediction : do they discover linguistic rules?

    Godin, Fréderic, Demuynck, Kris, Dambre, Joni, De Neve, Wesley, and Demeester, Thomas

    2018

    Biblio
24. BRUNO : a deep recurrent model for exchangeable data

    Korshunova, Iryna, Degrave, Jonas, Huszar, Ferenc, Gal, Yarin, Gretton, Arthur, and Dambre, Joni

    In ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 31 (NIPS 2018) 2018

    Abs Biblio

    We present a novel model architecture which leverages deep learning tools to perform exact Bayesian inference on sets of high dimensional, complex observations. Our model is provably exchangeable, meaning that the joint distribution over observations is invariant under permutation: this property lies at the heart of Bayesian inference. The model does not require variational approximations to train, and new samples can be generated conditional on previous samples, with cost linear in the size of the conditioning set. The advantages of our architecture are demonstrated on learning tasks that require generalisation from short observed sequences while modelling sequence variability, such as conditional image generation, few-shot learning, and anomaly detection.

25. Integrated dielectric scatterers for fast optical classification of biological cells

    Lugnan, Alessio, Dambre, Joni, and Bienstman, Peter

    In NEURO-INSPIRED PHOTONIC COMPUTING 2018

    Abs Biblio

    The development of label-free, high-speed, automated and integrated cell sorting solutions is of particular interest for several biomedical applications. The employment of digital holographic microscopy in microfluidic flow cytometry gives access to a large amount of information regarding the 3D refractive index structure of a cell. In the presented work a passive, linear, integrated photonic stage is proposed as an effective nonlinear mixing interface between the hologram projection and the image sensor, allowing for a fast, compact and power-efficient extreme learning machine (ELM) implementation. The required nonlinearity comes from the sinusoid-based transfer function between the phase-shift accumulated by the light through the cell and the field intensity measured by the detector. 2D FDTD simulations with 2 classes of randomized cell models (normal and cancer cells differing in their average nucleus size) have been employed to train and test a readout linear classifier. A collection of silicon nitride pillar scatterers embedded in a silica cladding are interposed between the cell and the intensity monitor, in order to increase the complexity of the acquired interference pattern and to assist the readout linear classifier. The results show that, employing green light, the presence of scatterer layers decreases the classification error rate up to similar to 50% with respect to the case without scatterers. Such improvement can be further increased to a factor similar to 5 when a properly designed integrated optical cavity containing the cell is considered. An intuitive argumentation that explains these results is provided.

26. Integrated dielectric scatterers for speeding up classification of cell diffraction patterns

    Lugnan, Alessio, Dambre, Joni, and Bienstman, Peter

    In 2018 20TH ANNIVERSARY INTERNATIONAL CONFERENCE ON TRANSPARENT OPTICAL NETWORKS (ICTON) 2018

    Biblio
27. Kraken: a Modifiable Compliant Robot

    Doms, Natan, Dambre, Joni, and wyffels, Francis

    2018

    Biblio
28. Improving time series prediction with ensembles of integrated photonic reservoirs

    Freiberger, Matthias, Bienstman, Peter, and Dambre, Joni

    In Cognitive Computing 2018 : meeting abstracts 2018

    Biblio
29. Towards a full spectrum diagnosis of autistic behaviours using human robot interactions

    Bartlett, Madeleine, Belpaeme, Tony, and Thill, Serge

    2018

    Abs Biblio

    Autism Spectrum Disorder (ASD) is conceptualised by the Diag-nostic and Statistical Manual of Mental Disorders (DSM-V) [1] asa spectrum, and diagnosis involves scoring behaviours in termsof a severity scale. Whilst the application of automated systemsand socially interactive robots to ASD diagnosis would increase ob-jectivity and standardisation, most of the existing systems classifybehaviours in a binary fashion (ASD vs. non-ASD). To be useful ininterventions, and to overcome ethical concerns regarding overlysimpli€ed diagnostic measures, a robot therefore needs to be ableto classify target behaviours along a continuum, rather than indiscrete groups. Here we discuss an approach toward this goalwhich has the potential to identify the full spectrum of observableASD traits.

30. Using a robot peer to encourage the production of spatial concepts in a second language

    Wallbridge, Christopher D, Berghe, Rianne, Hernández Garcia, Daniel, Kanero, Junko, Lemaignan, Séverin, Edmunds, Charlotte, and Belpaeme, Tony

    In HAI’18: PROCEEDINGS OF THE 6TH INTERNATIONAL CONFERENCE ON HUMAN-AGENT INTERACTION 2018

    Biblio
31. Architecture for a social assistive robot in cardiac rehabilitation

    Casas, Jonathan, Gomez, Nathalia Cespedes, Senft, Emmanuel, Irfan, Bahar, Gutierrez, Luisa F., Rincon, Monica, Munera, Marcela, Belpaeme, Tony, and Cifuentes, Carlos A.

    In 2018 IEEE 2ND COLOMBIAN CONFERENCE ON ROBOTICS AND AUTOMATION (CCRA) 2018

    Abs Biblio

    Social robots are demonstrating to have potential in several healthcare applications, especially in rehabilitation areas. This paper presents an architecture for a socially assistive robot system for cardiac rehabilitation, based on a model-controller structure through a finite-state machine and a behaviour module. The platform has been designed to provide social support and assistance during the therapy, aiming to improve the quality of the provided service, as well as the engagement and performance of the patients. This architecture has been tested under clinical conditions with a patient during a typical therapy session. The results show that the proposed architecture is able to adapt to the various situations present during a session, providing a robust framework to further develop the robot’s behaviour towards a more natural and intuitive interaction with the patients.

32. UNDERWORLDS : cascading situation assessment for robots

    Lemaignan, Severin, Sallami, Yoan, Wallhridge, Christopher, Clodic, Aurelic, Belpaeme, Tony, and Alami, Rachid

    In 2018 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS) 2018

    Abs Biblio

    We introduce UNDERWORLDS, a novel lightweight framework for cascading spatio-temporal situation assessment in robotics. UNDERWORLDS allows programmers to represent the robot’s environment as real-time distributed data structures, containing both scene graphs (for representation of 3D geometries) and timelines (for representation of temporal events). UNDERWORLDS supports cascading representations: the environment is viewed as a set of worlds that can each have different spatial and temporal granularities, and may inherit from each other. UNDERWORLDS also provides a set of high-level client libraries and tools to introspect and manipulate the environment models. This article presents the design and architecture of this open-source tool, and explores some applications, along with examples of use.

33. Experiences of a motivational interview delivered by a robot : qualitative study

    Silva, Joana, Kavanagh, David J, Belpaeme, Tony, Taylor, Lloyd, Beeson, Konna, and Andrade, Jackie

    JOURNAL OF MEDICAL INTERNET RESEARCH 2018

    Abs Biblio

    Background: Motivational interviewing is an effective intervention for supporting behavior change but traditionally depends on face-to-face dialogue with a human counselor. This study addressed a key challenge for the goal of developing social robotic motivational interviewers: creating an interview protocol, within the constraints of current artificial intelligence, which participants will find engaging and helpful. Objective: The aim of this study was to explore participants’ qualitative experiences of a motivational interview delivered by a social robot, including their evaluation of usability of the robot during the interaction and its impact on their motivation. Methods: NAO robots are humanoid, child-sized social robots. We programmed a NAO robot with Choregraphe software to deliver a scripted motivational interview focused on increasing physical activity. The interview was designed to be comprehensible even without an empathetic response from the robot. Robot breathing and face-tracking functions were used to give an impression of attentiveness. A total of 20 participants took part in the robot-delivered motivational interview and evaluated it after 1 week by responding to a series of written open-ended questions. Each participant was left alone to speak aloud with the robot, advancing through a series of questions by tapping the robot’s head sensor. Evaluations were content-analyzed utilizing Boyatzis’ steps: (1) sampling and design, (2) developing themes and codes, and (3) validating and applying the codes. Results: Themes focused on interaction with the robot, motivation, change in physical activity, and overall evaluation of the intervention. Participants found the instructions clear and the navigation easy to use. Most enjoyed the interaction but also found it was restricted by the lack of individualized response from the robot. Many positively appraised the nonjudgmental aspect of the interview and how it gave space to articulate their motivation for change. Some participants felt that the intervention increased their physical activity levels. Conclusions: Social robots can achieve a fundamental objective of motivational interviewing, encouraging participants to articulate their goals and dilemmas aloud. Because they are perceived as nonjudgmental, robots may have advantages over more humanoid avatars for delivering virtual support for behavioral change.

34. Robots in the classroom : learning to be a good tutor

    Senft, Emmanuel, Lemaignan, Séverin, Bartlett, Madeleine, Baxter, Paul, and Belpaeme, Tony

    In 4th Workshop on Robots for Learning - Inclusive Learning, HRI 2018, Proceedings 2018

    Biblio
35. Children conform, adults resist : a robot group induced peer pressure on normative social conformity

    Vollmer, Anna-Lisa, Read, Robin, Trippas, Dries, and Belpaeme, Tony

    SCIENCE ROBOTICS 2018

    Abs Biblio

    People are known to change their behavior and decisions to conform to others, even for obviously incorrect facts. Because of recent developments in artificial intelligence and robotics, robots are increasingly found in human environments, and there, they form a novel social presence. It is as yet unclear whether and to what extent these social robots are able to exert pressure similar to human peers. This study used the Asch paradigm, which shows how participants conform to others while performing a visual judgment task. We first replicated the finding that adults are influenced by their peers but showed that they resist social pressure from a group of small humanoid robots. Next, we repeated the study with 7- to 9-year-old children and showed that children conform to the robots. This raises opportunities as well as concerns for the use of social robots with young and vulnerable cross-sections of society; although conforming can be beneficial, the potential for misuse and the potential impact of erroneous performance cannot be ignored.

36. An enhanced architecture for silicon photonic reservoir computing

    Sackesyn, Stijn, Ma, Chonghuai, Dambre, Joni, and Bienstman, Peter

    In Cognitive Computing 2018 - Merging Concepts with Hardware 2018

    Biblio
37. Neuromorphic computing with signal-mixing cavities

    Laporte, Floris, Dambre, Joni, and Bienstman, Peter

    In 2018 IEEE INTERNATIONAL CONFERENCE ON REBOOTING COMPUTING (ICRC) 2018

    Biblio
38. Towards automated TAVI device size selection using artificial intelligence

    Astudillo, Patricio, Mortier, Peter, De Beule, Matthieu, Bosmans, Johan, De Backer, Ole, Jaegere, Peter, and Dambre, Joni

    In 5th European Congress on eCardiology and eHealth, Abstracts 2018

    Biblio
39. Detection of the aortic annular plane and coronary ostia from MDCT using deep learning techniques.

    Astudillo, Patricio, Iannaccone, Francesco, Jaegere, Peter, Bosmans, Johan, De Backer, Ole, De Beule, Matthieu, Mortier, Peter, and Dambre, Joni

    In Advanced Techniques and Therapies in Cardiovascular Care, ATTICC 2018, abstracts 2018

    Biblio
40. Photonic reservoir computing using silicon chips

    Vandoorne, Kristof, Mechet, Pauline, Fiers, Martin, Van Vaerenbergh, Thomas, Schneider, Bendix, Katumba, Andrew, Laporte, Floris, Verstraeten, David, Schrauwen, benjamin, Dambre, Joni, and Bienstman, Peter

    In European Nanoelectronics Applications, Design & Technology Conference (ADTC 2018), Abstracts 2018

    Biblio
41. Accurate eye center localization via hierarchical adaptive convolution

    Cai, Haibin, Liu, Bangli, Ju, Zhaojie, Thill, Serge, Belpaeme, Tony, Vanderborght, Bram, and Liu, Honghai

    In Proceedings of the 29th British Machine Vision Conference 2018

    Biblio

2017

1. Morphological properties of mass-spring networks for optimal locomotion learning

   Urbain, Gabriel, Degrave, Jonas, Carette, Benonie, Dambre, Joni, and wyffels, Francis

   FRONTIERS IN NEUROROBOTICS 2017

   Abs Biblio

   Robots have proven very useful in automating industrial processes. Their rigid components and powerful actuators, however, render them unsafe or unfit to work in normal human environments such as schools or hospitals. Robots made of compliant, softer materials may offer a valid alternative. Yet, the dynamics of these compliant robots are much more complicated compared to normal rigid robots of which all components can be accurately controlled. It is often claimed that, by using the concept of morphological computation, the dynamical complexity can become a strength. On the one hand, the use of flexible materials can lead to higher power efficiency and more fluent and robust motions. On the other hand, using embodiment in a closed-loop controller, part of the control task itself can be outsourced to the body dynamics. This can significantly simplify the additional resources required for locomotion control. To this goal, a first step consists in an exploration of the trade-offs between morphology, efficiency of locomotion, and the ability of a mechanical body to serve as a computational resource. In this work, we use a detailed dynamical model of a Mass–Spring–Damper (MSD) network to study these trade-offs. We first investigate the influence of the network size and compliance on locomotion quality and energy efficiency by optimizing an external open-loop controller using evolutionary algorithms. We find that larger networks can lead to more stable gaits and that the system’s optimal compliance to maximize the traveled distance is directly linked to the desired frequency of locomotion. In the last set of experiments, the suitability of MSD bodies for being used in a closed loop is also investigated. Since maximally efficient actuator signals are clearly related to the natural body dynamics, in a sense, the body is tailored for the task of contributing to its own control. Using the same simulation platform, we therefore study how the network states can be successfully used to create a feedback signal and how its accuracy is linked to the body size.

2. How to build a supervised autonomous system for robot-enhanced therapy for children with autism spectrum disorder

   Esteban, Pablo G, Baxter, Paul, Belpaeme, Tony, Billing, Erik, Cai, Haibin, Cao, Hoang-Long, Coeckelbergh, Mark, Costescu, Cristina, David, Daniel, De Beir, Albert, Fang, Yinfeng, Ju, Zhaojie, Kennedy, James, Liu, Honghai, Mazel, Alexandre, Pandey, Amit, Richardson, Kathleen, Senft, Emmanue, Thill, Serge, Perre, Greet, Vanderborght, Bram, Vernon, David, Yu, Hui, and Ziemke, Tom

   PALADYN : JOURNAL OF BEHAVIORAL ROBOTICS 2017

   Biblio
3. Robot education peers in a situated primary school study : personalisation promotes child learning

   Baxter, Paul, Ashurst, Emily, Read, Robin, Kennedy, James, and Belpaeme, Tony

   PLOS ONE 2017

   Biblio
4. Nonverbal immediacy as a characterisation of social behaviour for human–robot interaction

   Kennedy, James, Baxter, Paul, and Belpaeme, Tony

   INTERNATIONAL JOURNAL OF SOCIAL ROBOTICS 2017

   Biblio
5. Supervised autonomy for online learning in human-robot interaction

   Senft, Emmanuel, Baxter, Paul, Kennedy, James, Lemaignan, Séverin, and Belpaeme, Tony

   PATTERN RECOGNITION LETTERS 2017

   Biblio
6. The impact of robot tutor nonverbal social behavior on child learning

   Kennedy, James, Baxter, Paul, and Belpaeme, Tony

   FRONTIERS IN ICT 2017

   Biblio
7. Child speech recognition in human-robot interaction : evaluations and recommendations

   Kennedy, James, Lemaignan, Séverin, Montassier, Caroline, Lavalade, Pauline, Irfan, Bahar, Papadopoulos, Fotios, Senft, Emmanuel, and Belpaeme, Tony

   In Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction 2017

   Abs Biblio

   An increasing number of human-robot interaction (HRI) studies are now taking place in applied settings with children. These interactions often hinge on verbal interaction to effectively achieve their goals. Great advances have been made in adult speech recognition and it is often assumed that these advances will carry over to the HRI domain and to interactions with children. In this paper, we evaluate a number of automatic speech recognition (ASR) engines under a variety of conditions, inspired by real-world social HRI conditions. Using the data collected we demonstrate that there is still much work to be done in ASR for child speech, with interactions relying solely on this modality still out of reach. However, we also make recommendations for child-robot interaction design in order to maximise the capability that does currently exist.

8. Leveraging Human Inputs in Interactive Machine Learning for Human Robot Interaction

   Senft, Emmanuel, Lemaignan, Séverin, Baxter, Paul E., and Belpaeme, Tony

   2017

   Biblio
9. Workshop on Robots for Learning

   Johal, Wafa, Dillenbourg, Pierre, Vogt, Paul, Kennedy, James, Haas, Mirjam, Paiva, Ana, Castellano, Ginevra, Okita, Sandra, Tanaka, Fumihide, and Belpaeme, Tony

   2017

   Biblio
10. New methods for the design of smart products : project based learning with industry

    Neutens, Tom, Saldien, Jelle, Verstockt, Steven, and wyffels, Francis

    2017

    Biblio
11. GECO : the development of a wall-climbing robot

    Cardinaels, Rob, Maes, Julie, Deseure, Jordi, Saldien, Jelle, Aelterman, Kevin, Verstockt, Steven, and wyffels, Francis

    In Proceedings of CLAWAR 2017 : 20th International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines 2017

    Biblio
12. OTO : A DIY platform for mobile social robots in education

    Vervisch, Thomas, Doms, Natan, Descamps, Sander, Vandevelde, Cesar, wyffels, Francis, Verstockt, Steven, and Saldien, Jelle

    2017

    Abs Biblio

    This paper describes the design of OTO, a do-it-yourself expansion kit for OPSORO (Open Platform for Social Robots), that enables and facilitates the creation of mobile social robots. The expansion kit consists of modular, adaptable building blocks combined with a software toolkit, and is aimed at applications within the maker community, STEM education, and the market for creative inventor kits. Keeping reproducibility and adaptability in mind, the expansion kit can be produced entirely using digital manufacturing technology and low-cost, off-the-shelf components. Using the building blocks offered by this system, users can easily design, build and customize mobile social robots. The software is designed to address a wide range of users by offering different programming options depending on the user’s skill and experience. Inexperienced users are offered a graphical programming environment based on Blockly, whereas more advanced users can program their robot using Lua or Python. The OTO toolkit offers a fun and playful context in which a wide range of STEM-related skills are addressed.

13. Quadruped Robots Benefit from Compliant Leg Designs

    Willems, Brecht, Degrave, Jonas, Dambre, Joni, and wyffels, Francis

    2017

    Abs Biblio

    A major bottleneck in building autonomous robots is the lack of suitable power supplies. Consequently, researchers aim for more energy-efficient locomotion. One way to achieve this is by using compliant materials in the robot bodies. In this work, we introduce Tigrillo, a small, low-cost quadruped robot platform with modular legs. The robot’s legs consist of a simple two-segmented construction which can be configured stiff or passive compliant. Each leg is actuated at the hip with one motor. By thorough optimization of the open loop control signals we demonstrate that compliance not only results in gaits that are more insensitive to parameter variations, but also leads to more energy-efficient gaits.

14. Fast face-swap using convolutional neural networks

    Korshunova, Iryna, Shi, Wenzhe, Dambre, Joni, and Theis, Lucas

    In 2017 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV) 2017

    Biblio
15. Implementation and evaluation of a simulator and debugger for physical computing environments

    Neutens, Tom, Staes, Juta, and wyffels, Francis

    2017

    Biblio
16. Do-it-yourself design for social robots

    Vandevelde, Cesar, wyffels, Francis, Vanderborght, Bram, and Saldien, Jelle

    IEEE ROBOTICS & AUTOMATION MAGAZINE 2017

    Biblio
17. Improving language modeling using densely connected recurrent neural networks

    Godin, Fréderic, Dambre, Joni, and De Neve, Wesley

    2017

    Biblio
18. On-chip passive photonic reservoir computing with integrated optical readout

    Freiberger, Matthias, Katumba, Andrew, Bienstman, Peter, and Dambre, Joni

    In 2017 IEEE INTERNATIONAL CONFERENCE ON REBOOTING COMPUTING (ICRC) 2017

    Abs Biblio

    Photonic reservoir computing is a recent bio-inspired paradigm for signal processing. Despite first successes, the paradigm still faces challenges. We address some of these challenges and introduce our approaches to solve them. In detail, we discuss how integrated reservoirs can be scaled up by injecting multiple copies of the input. Further we introduce a new hardware-friendly training method for integrated optical readouts.

19. Gesture and sign language recognition with temporal residual networks

    Pigou, Lionel, Van Herreweghe, Mieke, and Dambre, Joni

    In 2017 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS (ICCVW 2017) 2017

    Biblio
20. Integrated pillar scatterers for speeding up classification of cell holograms

    Lugnan, Alessio, Dambre, Joni, and Bienstman, Peter

    OPTICS EXPRESS 2017

    Biblio
21. A multiple-input strategy to efficient integrated photonic reservoir computing

    Katumba, Andrew, Freiberger, Matthias, Bienstman, Peter, and Dambre, Joni

    COGNITIVE COMPUTATION 2017

    Abs Biblio

    Photonic reservoir computing has evolved into a viable contender for the next generation of analog computing platforms as industry looks beyond standard transistor-based computing architectures. Integrated photonic reservoir computing, particularly on the silicon-on-insulator platform, presents a CMOS-compatible, wide bandwidth, parallel platform for implementation of optical reservoirs. A number of demonstrations of the applicability of this platform for processing optical telecommunication signals have been made in the recent past. In this work, we take it a stage further by performing an architectural search for designs that yield the best performance while maintaining power efficiency. We present numerical simulations for an optical circuit model of a 16-node integrated photonic reservoir with the input signal injected in combinations of 2, 4, and 8 nodes, or into all 16 nodes. The reservoir is composed of a network of passive photonic integrated circuit components with the required nonlinearity introduced at the readout point with a photodetector. The resulting error performance on the temporal XOR task for these multiple input cases is compared with that of the typical case of input to a single node. We additionally introduce for the first time in our simulations a realistic model of a photodetector. Based on this, we carry out a full power-level exploration for each of the above input strategies. Multiple-input reservoirs achieve better performance and power efficiency than single-input reservoirs. For the same input power level, multiple-input reservoirs yield lower error rates. The best multiple-input reservoir designs can achieve the error rates of single-input ones with at least two orders of magnitude less total input power. These results can be generally attributed to the increase in richness of the reservoir dynamics and the fact that signals stay longer within the reservoir. If we account for all loss and noise contributions, the minimum input power for error-free performance for the optimal design is found to be in the approximate to 1 mW range.

22. Building scalable integrated silicon photonics reservoirs for signal processing

    Katumba, Andrew, Freiberger, Matthias, Heyvaert, Jelle, Dambre, Joni, and Bienstman, Peter

    In Frontiers in Optics 2017 2017

    Biblio
23. Novel photonic reservoir computing architectures

    Laporte, Floris, Lugnan, Alessio, Dambre, Joni, and Bienstman, Peter

    In Workshop on Dynamical systems and Brain-Inspired Information Processing 2017

    Biblio
24. Photonic reservoir computing using a small network of micro-ring resonators

    Rontani, D., Denis, F., Katumba, Andrew, Freiberger, Matthias, Dambre, Joni, Bienstman, Peter, and Sciamanna, M.

    In Workshop on Dynamical systems and Brain-Inspired Information Processing 2017

    Biblio
25. Integrated-photonics implementation of reservoir computing neural networks

    Katumba, Andrew, Laporte, Floris, Lugnan, Alessio, Dambre, Joni, and Bienstman, Peter

    In Workshop Machine Learning @ ECOC 2017 2017

    Biblio
26. Reservoir computing on an active silicon photonics chip using nonlinear microring resonators

    Denis-le Coarer, F., Rontani, D., Katumba, Andrew, Freiberger, Matthias, Dambre, Joni, Bienstman, Peter, and Sciamanna, M.

    In XXXVII Dynamic Days Europe 2017

    Biblio
27. Passive integrated photonics reservoir computing with on-chip readout for telecommunication applications

    Katumba, Andrew, Freiberger, Matthias, Dambre, Joni, and Bienstman, Peter

    In Dynamical Systems and Brain-Inspired Computing Conference 2017

    Biblio
28. Integrated photonic pillar scatterers for speeding up classification of cell holograms

    Lugnan, Alessio, Dambre, Joni, and Bienstman, Peter

    In Dynamical Systems and Brain-Inspired Computing Conference 2017

    Biblio
29. Continuous multi-modal interaction causes human-robot alignment

    Wallkötter, Sebastian, Joannou, Michael, Westlake, Samuel, and Belpaeme, Tony

    In Proceedings of the 5th International Conference on Human Agent Interaction (HAI17) 2017

    Biblio
30. Header recognition with signal-mixing cavities

    Laporte, Floris, Dambre, Joni, and Bienstman, Peter

    In Workshop on Dynamical Systems and Brain Inspired Computing 2017

    Biblio
31. Reservoir computing with signal-mixing cavities

    Laporte, Floris, Dambre, Joni, and Bienstman, Peter

    In 2017 19TH INTERNATIONAL CONFERENCE ON TRANSPARENT OPTICAL NETWORKS (ICTON) 2017

    Biblio
32. Human-robot sensor interface for cardiac rehabilitation

    Lara, Juan S., Casas, Jonathan, Aguirre, Andres, Munera, Marcela, Rincon-Roncancio, Monica, Irfan, Bahar, Senft, Emmanuel, Belpaeme, Tony, and Cifuentes, Carlos A.

    In 2017 INTERNATIONAL CONFERENCE ON REHABILITATION ROBOTICS (ICORR) 2017

    Abs Biblio

    Cardiovascular disease is the leading cause of death in the world. A program of cardiac rehabilitation (CR) is related to physical activities or exercises to regain the optimal quality of life. CR relies on the necessity to evaluate, control and supervise a patient’s status and progress. This work has two objectives: on the one hand, provide a tool for clinicians to assess the patient’s status during CR. On the other hand, there is evidence that robots can motivate patients during therapeutic procedures. Our sensor interface explores the possibility to integrate a robotic agent into cardiac therapy. This work presents an exploratory experiment for on-line assessment of typical CR routines.

33. Reservoir computing with nonlinear micro

    Rontani, Damien, Denis-Le Coarer, Florian, Katumba, Andrew, Freiberger, Matthias, Dambre, Joni, Bienstman, Peter, and Sciamanna, Marc

    In 2017 International Symposium on Nonlinear Theory and Its Applications, NOLTA2017, Proceedings 2017

    Biblio
34. Reservoir computing with nonlinear micro-resonators on a silicon photonics chip

    Rontani, Damien, Denis-Le Coarer, Florian, Katumba, Andrew, Freiberger, Matthias, Dambre, Joni, Bienstman, Peter, and Sciamanna, Marc

    In 2017 International Symposium on Nonlinear Theory and Its Applications, NOLTA2017, Proceedings 2017

    Biblio

2016

1. Design and evaluation of a DIY construction system for educational robot kits

   Vandevelde, Cesar, wyffels, Francis, Ciocci, Maria-Cristina, Vanderborght, Bram, and Saldien, Jelle

   INTERNATIONAL JOURNAL OF TECHNOLOGY AND DESIGN EDUCATION 2016

   Abs Biblio

   Building a robot from scratch in an educational context can be a challenging prospect. While a multitude of projects exist that simplify the electronics and software aspects of a robot, the same cannot be said for construction systems for robotics. In this paper, we present our efforts to create a low-cost do-it-yourself construction system for small robots. We have created three different construction systems (laser-cut screw connectors, printed friction-fit connectors, and printed hybrid connectors) using small aluminium T-slot extrusions, based on prior work done by Industrial Design college students. Eighty-six secondary school students and 35 teachers tested these three systems during a five-day robotics contest where they had to build firefighting robots. Follow-up questionnaires and an expert evaluation were used to measure the usability, affective appraisal and functionality of the three systems in order to determine which system should serve as a basis for further design iterations. Overall, a clear preference was shown for the hybrid system, which relies on its interlocking shape as well as on a screw connection to create robot frames that are both quick to construct and very rigid once assembled. We believe our work represents a solid first step toward an inexpensive, “hackable” construction system for educational robotics.

2. Spatial chirp-Z transformer networks

   Degrave, Jonas, Dieleman, Sander, Dambre, Joni, and wyffels, Francis

   In European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN 2016), Proceedings 2016

   Abs Biblio

   Convolutional Neural Networks are often used for computer vision solutions, because of their inherent modeling of the translation invariance in images. In this paper, we propose a new module to model rotation and scaling invariances in images. To do this, we rely on the chirp-Z transform to perform the desired translation, rotation and scaling in the frequency domain. This approach has the benefit that it scales well and that it is differentiable because of the computationally cheap sincinterpolation.

3. Fast particle characterization using digital holography and neural networks

   Schneider, Bendix, Dambre, Joni, and Bienstman, Peter

   APPLIED OPTICS 2016

   Biblio
4. Deep dynamic neural networks for multimodal gesture segmentation and recognition

   Wu, Di, Pigou, Lionel, Kindermans, Pieter-Jan, LE, Nam, Shao, Ling, Dambre, Joni, and Odobez, Jean-Marc

   IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2016

   Abs Biblio

   This paper describes a novel method called Deep Dynamic Neural Networks (DDNN) for multimodal gesture recognition. A semi-supervised hierarchical dynamic framework based on a Hidden Markov Model (HMM) is proposed for simultaneous gesture segmentation and recognition where skeleton joint information, depth and RGB images, are the multimodal input observations. Unlike most traditional approaches that rely on the construction of complex handcrafted features, our approach learns high-level spatio-temporal representations using deep neural networks suited to the input modality: a Gaussian-Bernouilli Deep Belief Network (DBN) to handle skeletal dynamics, and a 3D Convolutional Neural Network (3DCNN) to manage and fuse batches of depth and RGB images. This is achieved through the modeling and learning of the emission probabilities of the HMM required to infer the gesture sequence. This purely data driven approach achieves a Jaccard index score of 0.81 in the ChaLearn LAP gesture spotting challenge. The performance is on par with a variety of state-of-the-art hand-tuned feature-based approaches and other learning-based methods, therefore opening the door to the use of deep learning techniques in order to further explore multimodal time series data.

5. Sign classification in sign language Corpora with deep neural networks

   Pigou, Lionel, Van Herreweghe, Mieke, and Dambre, Joni

   In International Conference on Language Resources and Evaluation (LREC), Workshop, Proceedings 2016

   Abs Biblio

   Automatic and unconstrained sign language recognition (SLR) in image sequences remains a challenging problem. The variety of signers, backgrounds, sign executions and signer positions makes the development of SLR systems very challenging. Current methods try to alleviate this complexity by extracting engineered features to detect hand shapes, hand trajectories and facial expressions as an intermediate step for SLR. Our goal is to approach SLR based on feature learning rather than feature engineering. We tackle SLR using the recent advances in the domain of deep learning with deep neural networks. The problem is approached by classifying isolated signs from the Corpus VGT (Flemish Sign Language Corpus) and the Corpus NGT (Dutch Sign Language Corpus). Furthermore, we investigate cross-domain feature learning to boost the performance to cope with the fewer Corpus VGT annotations.

6. Identifying and removing micro-drift in ground-based electromagnetic induction data

   De Smedt, Philippe, Delefortrie, Samuël, and wyffels, Francis

   JOURNAL OF APPLIED GEOPHYSICS 2016

   Biblio
7. Automatic architectural style detection using one-class support vector machines and graph kernels

   Strobbe, Tiemen, wyffels, Francis, Verstraeten, Ruben, De Meyer, Ronald, and Van Campenhout, Jan

   AUTOMATION IN CONSTRUCTION 2016

   Biblio
8. Building ArtBots to attract students into STEM learning

   wyffels, Francis, Van De Steene, Willem, Roets, Jelle, Ciocci, Maria-Cristina, and Carbajal, Juan Pablo

   2016

   Abs Biblio

   There is an increasing worldwide demand for people educated into science and technology. Unfortunately, girls and underprivileged students are often underrepresented in Science, Technology, Engineering and Mathematics (STEM) education programs. We believe that by inclusion of art in these programs, educational activities might become more attractive to a broader audience. In this work we present an example of such an educational activity: an international robotics and art week for secondary school students. This educational activity builds up on the project-based and inquiry learning framework. This article is intended as a brief manual to help others organise such an activity. It also gives insights in how we led a highly heterogeneous group of students into learning STEM and becoming science and technology ambassadors for their peers.

9. Reducing BCI calibration time with transfer learning: a shrinkage approach

   Verhoeven, Thibault, Kindermans, Pieter-Jan, Vandenberghe, Stefaan, and Dambre, Joni

   In Proceedings of the Sixth International Brain-Computer Interface Meeting: BCI Past, Present, and Future 2016

   Abs Biblio

   Introduction: A brain-computer interface system (BCI) allows subjects to make use of neural control signals to drive a computer application. Therefor a BCI is generally equipped with a decoder to differentiate between types of responses recorded in the brain. For example, an application giving feedback to the user can benefit from recognizing the presence or absence of a so-called error potential (Errp), elicited in the brain of the user when this feedback is perceived as being ‘wrong’, a mistake of the system. Due to the high inter- and intra- subject variability in these response signals, calibration data needs to be recorded to train the decoder. This calibration session is exhausting and demotivating for the subject. Transfer Learning is a general name for techniques in which data from previous subjects is used as additional information to train a decoder for a new subject, thereby reducing the amount of subject specific data that needs to be recorded during calibration. In this work we apply transfer learning to an Errp detection task by applying single-target shrinkage to Linear Discriminant Analysis (LDA), a method originally proposed by Höhne et. al. to improve accuracy by compensating for inter-stimuli differences in an ERP-speller [1]. Material, Methods and Results: For our study we used the error potential dataset recorded by Perrin et al. in [2]. For 26 subjects each, 340 Errp/nonErrp responses were recorded with a #Errp to #nonErrp ratio of 0.41 to 0.94. 272 responses were available for training the decoder and the remaining 68 responses were left out for testing. For every subject separately we built three different decoders. First, a subject specific LDA decoder was built solely making use of the subject’s own train data. Second, we added the train data of the other 25 subjects to train a global LDA decoder, naively ignoring the difference between subjects. Finally, the single-target-shrinkage method (STS) [1] is used to regularize the parameters of the subject specific decoder towards those of the global decoder. Making use of cross validation this method assigns an optimal weight to the subject specific data and data from previous subjects to be used for training. Figure 1 shows the performance of the three decoders on the test data in terms of AUC as a function of the amount of subject specific calibration data used. Discussion: The subject specific decoder in Figure 1 shows how sensitive the decoding performance is to the amount of calibration data provided. Using data from previously recorded subjects the amount of calibration data, and as such the calibration time, can be reduced as shown by the global decoder. A certain amount of quality is however sacrificed. Making an optimal compromise between the subject specific and global decoder, the single-target-shrinkage decoder allows the calibration time to be reduced by 20% without any change in decoder quality (confirmed by a paired sample t-test giving p=0.72). Significance: This work serves as a first proof of concept in the use of shrinkage LDA as a transfer learning method. More specific, the error potential decoder built with reduced calibration time boosts the opportunity for error correcting methods in BCI.

10. Automatic diagnosis of temporal lobe epilepsy and its lateralization using EEG-based directed functional connectivity

    Verhoeven, Thibault, Coito, Ana, Mierlo, Pieter, Seeck, Margitta, Michel, Christoph, Plomp, Gijs, Dambre, Joni, and Vulliemoz, Serge

    In Organization for Human Brain Mapping 2016 2016

    Abs Biblio

    Introduction: Temporal lobe epilepsy (TLE) is the most common type of pharmaco-resistant epilepsy in adults. During pre-surgical evaluation, EEG is recorded to identify pathological activity such as interictal spikes that can help to identify the epileptogenic zone. However, sometimes no spikes are recorded in the scalp EEG. The characterization of brain network dysfunction in the absence of visible scalp epileptic activity could be very important to improve diagnostic accuracy and treat these patients. Here, we aimed to build a classification system that uses EEG-based directed functional connectivity patterns to assign a patient to one of three classes: left TLE (LTLE), right TLE (RTLE) or healthy control. Methods: Sixty subjects underwent a resting-state high-density EEG recording: 20 LTLE, 20 RTLE and 20 healthy controls. For each subject sixty 1-sec epochs free of artifacts or interictal spikes were selected. The cortical source activity was obtained for 82 regions of interest (ROIs) using an individual head model and distributed linear inverse solution. The whole-brain directed functional connectivity was estimated in the theta, alpha and beta frequency bands using Granger-causal modeling (weighted Partial Directed Coherence[1-3]) applied to the source signal of all ROIs. The summed outflow from each ROI was computed for each frequency band (total of 246 connectivity values per subject). Details of this analysis can be found in [4]. Here, we used logistic regression classifiers which take a linear combination of these connectivity values and compare it with a threshold in order to assign the subject to one of the three given classes. A leave-one-out procedure was used to compute the mean classification accuracy on subjects that are not used as example to build the classifier system. Due to the low amount of recorded subjects compared to the number of connectivity values, a subset was selected for classification purpose. We selected a maximum of six connectivity values making use of a greedy forward selection algorithm: gradually adding connectivity values to improve the classification accuracy. Finally, three classifiers were built: ‘Control vs. LTLE’, ‘Control vs. RTLE’ and ‘LTLE vs. RTLE’. In the final classification system, a new subject is assigned to the class that was most voted for by these three individual classifiers. Subjects for which every class is voted once were classified as undefined. Results: The selected brain regions and corresponding frequency bands for the three classifiers are depicted in Figure 1. The ‘Control vs. RTLE’ classifier achieved an accuracy of 92.5% (sensitivity: 95.0%, specificity 90.0%), the ‘Control vs. LTLE’ classifier an accuracy of 87.5% (sensitivity 90.0%, specificity 85.0%) and the ‘LTLE vs. RTLE’ classifier an accuracy of 80.0% (sensitivity 85.0%, specificity 75.0%). Combining these individual classifiers in one system resulted in the confusion matrix shown in Figure 2. Conclusion: We had previously found significant differences in the summed outflow from regions included in the Default-Mode Network and also known to be important in TLE in patients vs healthy controls using these EEG periods without interictal spikes [4]. Here, the high accuracy achieved in classifying a subject as LTLE, RTLE or healthy control in periods of EEG without visible interictal pathological events further demonstrates the potential of resting-state EEG-based directed functional connectivity for the diagnosis and lateralization of TLE. This could thus constitute a new important clinical biomarker in surgical candidates or even earlier in the course of the disease, especially to lateralize the epilepsy.

11. Innovative design of a hexapod scorpion through digital production techniques

    Flamand, Stephan, Saldien, Jelle, Deconinck, Pieterjan, wyffels, Francis, Verstockt, Steven, and Terryn, Robbe

    In 19th International Conference on Climbing and Walking Robots and Support Technologies for Mobile Machines 2016

    Biblio
12. Teacher professional development through a physical computing workshop

    Neutens, Tom, and wyffels, Francis

    2016

    Abs Biblio

    In recent years there has been a push towards more CS and STEM education in Flanders. These two domains require a set of skills with which teachers are currently often unfamiliar. To enable teachers to acquire these skills, professional development programs should be implemented. In this paper we first present a way of identifying the properties of such a program to allow comparison with other programs. Next, we describe a professional development program in the form of a physical computing workshop.

13. Embodied language learning and cognitive bootstrapping: methods and design principles

    Lyon, C, Nehaniv, CL, Saunders, J, Belpaeme, Tony, Bisio, A, Fischer, K, Forster, F, Lehmann, H, Metta, G, Mohan, V, Morse, A, Nolfi, S, Nori, F, Rohlfing, K, Sciutti, A, Tani, J, Tuci, E, Wrede, B, Zeschel, A, and Cangelosi, A

    INTERNATIONAL JOURNAL OF ADVANCED ROBOTIC SYSTEMS 2016

    Abs Biblio

    Co-development of action, conceptualization and social interaction mutually scaffold and support each other within a virtuous feedback cycle in the development of human language in children. Within this framework, the purpose of this article is to bring together diverse but complementary accounts of research methods that jointly contribute to our understanding of cognitive development and in particular, language acquisition in robots. Thus, we include research pertaining to developmental robotics, cognitive science, psychology, linguistics and neuroscience, as well as practical computer science and engineering. The different studies are not at this stage all connected into a cohesive whole; rather, they are presented to illuminate the need for multiple different approaches that complement each other in the pursuit of understanding cognitive development in robots. Extensive experiments involving the humanoid robot iCub are reported, while human learning relevant to developmental robotics has also contributed useful results. Disparate approaches are brought together via common underlying design principles. Without claiming to model human language acquisition directly, we are nonetheless inspired by analogous development in humans and consequently, our investigations include the parallel co-development of action, conceptualization and social interaction. Though these different approaches need to ultimately be integrated into a coherent, unified body of knowledge, progress is currently also being made by pursuing individual methods.

14. Experimental evaluation of a multi-modal user interface for a robotic service

    Di Nuovo, Alessandro, Wang, Ning, Broz, Frank, Belpaeme, Tony, Jones, Ray, and Cangelosi, Angelo

    In TOWARDS AUTONOMOUS ROBOTIC SYSTEMS, TAROS 2016 2016

    Abs Biblio

    This paper reports the experimental evaluation of a Multi-Modal User Interface (MMUI) designed to enhance the user experience in terms of service usability and to increase acceptability of assistive robot systems by elderly users. The MMUI system offers users two main modalities to send commands: they are a GUI, usually running on the tablet attached to the robot, and a SUI, with a wearable microphone on the user. The study involved fifteen participants, aged between 70 and 89 years old, who were invited to interact with a robotic platform customized for providing every-day care and services to the elderly. The experimental task for the participants was to order a meal from three different menus using any interaction modality they liked. Quantitative and qualitative data analyses demonstrate a positive evaluation by users and show that the multi-modal means of interaction can help to make elderly-robot interaction more flexible and natural.

15. From characterising three years of HRI to methodology and reporting recommendations

    Baxter, P, Kennedy, J, Senft, E, Lemaignan, S, and Belpaeme, Tony

    In ACMIEEE International Conference on Human-Robot Interaction 2016

    Abs Biblio

    Human-Robot Interaction (HRI) research requires the integration and cooperation of multiple disciplines, technical and social, in order to make progress. In many cases using different motivations, each of these disciplines bring with them different assumptions and methodologies. We assess recent trends in the field of HRI by examining publications in the HRI conference over the past three years (over 100 full papers), and characterise them according to 14 categories. We focus primarily on aspects of methodology. From this, a series of practical recommendations based on rigorous guidelines from other research fields that have not yet become common practice in HRI are proposed. Furthermore, we explore the primary implications of the observed recent trends for the field more generally, in terms of both methodology and research directions. We propose that the interdisciplinary nature of HRI must be maintained, but that a common methodological approach provides a much needed frame of reference to facilitate rigorous future progress.

16. Heart vs hard drive : children learn more from a human tutor than a social robot

    Kennedy, J, Baxter, P, Senft, E, and Belpaeme, Tony

    In ACMIEEE International Conference on Human-Robot Interaction 2016

    Abs Biblio

    The field of Human-Robot Interaction (HRI) is increasingly exploring the use of social robots for educating children. Commonly, non-academic audiences will ask how robots compare to humans in terms of learning outcomes. This question is also interesting for social roboticists as humans are often assumed to be an upper benchmark for social behaviour, which influences learning. This paper presents a study in which learning gains of children are compared when taught the same mathematics material by a robot tutor and a non-expert human tutor. Significant learning occurs in both conditions, but the children improve more with the human tutor. This difference is not statistically significant, but the effect sizes fall in line with findings from other literature showing that humans outperform technology for tutoring. We discuss these findings in the context of applying social robots in child education.

17. Providing a robot with learning abilities improves its perception by users

    Senft, E, Baxter, P, Kennedy, J, Lemaignan, S, and Belpaeme, Tony

    In ACMIEEE International Conference on Human-Robot Interaction 2016

    Abs Biblio

    Subjective appreciation and performance evaluation of a robot by users are two important dimensions for Human Robot Interaction, especially as increasing numbers of people become involved with robots. As roboticists we have to carefully design robots to make the interaction as smooth and enjoyable as possible for the users, while maintaining good performance in the task assigned to the robot. In this paper, we examine the impact of providing a robot with learning capabilities on how users report the quality of the interaction in relation to objective performance. We show that humans tend to prefer interacting with a learning robot and will rate its capabilities higher even if the actual performance in the task was lower. We suggest that adding learning to a robot could reduce the apparent load felt by a user for a new task and improve the user’s evaluation of the system, thus facilitating the integration of such robots into existing work flows.

18. Socially contingent humanoid robot head behaviour results in increased charity donations

    Wills, P, Baxter, P, Kennedy, J, Senft, E, and Belpaeme, Tony

    In ACMIEEE International Conference on Human-Robot Interaction 2016

    Abs Biblio

    The role of robot social behaviour in changing people’s behaviour is an interesting and yet still open question, with the general assumption that social behaviour is beneficial. In this study, we examine the effect of socially contingent robot behaviours on a charity collection task. Manipulating only behavioural cues (maintaining the same verbal content), we show that when the robot exhibits contingent behaviours consistent with those observable in humans, this results in a 32% increase in money collected over a non-reactive robot. These results suggest that apparent social agency on the part of the robot, even when subtle behavioural cues are used, can result in behavioural change on the part of the interacting human.

19. Review of semantic-free utterances in social human-robot interaction (vol 32, pg 63, 2016)

    Yilmazyildiz, S, Read, R, Belpaeme, Tony, and Verhelst, W

    2016

    Biblio
20. People interpret robotic non-linguistic utterances categorically

    Read, R, and Belpaeme, Tony

    INTERNATIONAL JOURNAL OF SOCIAL ROBOTICS 2016

    Abs Biblio

    We present results of an experiment probing whether adults exhibit categorical perception when affectively rating robot-like sounds (Non-linguistic Utterances). The experimental design followed the traditional methodology from the psychology domain for measuring categorical perception: stimulus continua for robot sounds were presented to subjects, who were asked to complete a discrimination and an identification task. In the former subjects were asked to rate whether stimulus pairs were affectively different, while in the latter they were asked to rate single stimuli affectively. The experiment confirms that Non-linguistic Utterances can convey affect and that they are drawn towards prototypical emotions, confirming that people show categorical perception at a level of inferred affective meaning when hearing robot-like sounds. We speculate on how these insights can be used to automatically design and generate affect-laden robot-like utterances.

21. An integrated photonics reservoir computing approach to signal equalization for telecommunications

    Katumba, Andrew, Schneider, Bendix, Dambre, Joni, and Bienstman, Peter

    In 2016 18TH INTERNATIONAL CONFERENCE ON TRANSPARENT OPTICAL NETWORKS (ICTON) 2016

    Biblio
22. Photorefractive crystals as brain-inspired photonic reservoir computing systems

    Laporte, Floris, Dambre, Joni, and Bienstman, Peter

    2016

    Biblio
23. Using Random Forest for Diagnosis and Lateralization of Temporal Lobe Epilepsy from EEG-based Directed Functional Connectivity

    Verhoeven, Thibault, Coito, Ana, Mierlo, Pieter, Seeck, Margitta, Michel, Christoph, Plomp, Gijs, Dambre, Joni, and Vulliemoz, Serge

    In 12th European Congress on Epileptology, Prague, Czech Republic11th-15thSeptember, 2016 2016

    Abs Biblio

    Purpose: We aimed to build a classification system that uses resting-state (no visible scalp epileptic activity) EEG-based directed functional connectivity values to assign a patient to one of three classes: left TLE (LTLE), right TLE (RTLE) or healthy control. Methods: Twenty LTLE, 20 RTLE and 35 healthy controls underwent resting-state high-density EEG. For each subject, sixty 1-sec epochs free of artifacts or interictal spikes were selected. The source activity was obtained for 82 regions of interest using an individual head model and distributed linear inverse solution. The summed outflow and whole-brain directed functional connectivity were estimated in the theta, alpha and beta frequency bands using Granger-causal modeling. A Random Forest classifier (an ensemble of decision tree classifiers) was then used to assign the subject to one of three classes. The mean classification accuracy was computed with a leave-one-out procedure. We selected a maximum of six connectivity values for classification, using a greedy forward selection algorithm. Finally, three classifiers were built: ‘Control vs. LTLE’, ‘Control vs. RTLE’ and ‘LTLE vs. RTLE’. In the final classification system, a new subject is assigned to the class that was most voted by these three classifiers. Results: The ‘Control vs. RTLE’ classifier achieved an accuracy of 78.2% (sensitivity: 80.0%, specificity 77.2%), ‘Control vs. LTLE’ an accuracy of 83.6% (sensitivity 85.0%, specificity 82.9%) and ‘LTLE vs. RTLE’ an accuracy of 85.0% (sensitivity 85.0%, specificity 85.0%). Combining these classifiers into one system yielded that 16, 15 and 27 subjects were correctly classified as being, respectively, RTLE, LTLE and control. Conclusion: The high accuracy achieved demonstrates the potential of resting-state EEG-based directed functional connectivity for the diagnosis and lateralization of TLE. This could constitute a new clinical biomarker for surgical candidates and earlier in the course of the disease.

24. Using digital masks to enhance the bandwidth tolerance and improve the performance of on-chip reservoir computing systems

    Schneider, Bendix, Dambre, Joni, and Bienstman, Peter

    IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2016

    Abs Biblio

    Reservoir computing (RC) is a computing scheme related to recurrent neural network theory. As a model for neural activity in the brain, it attracts a lot of attention, especially because of its very simple training method. However, building a functional, on-chip, photonic implementation of RC remains a challenge. Scaling delay lines down from optical fiber scale to chip scale results in RC systems that compute faster, but at the same time requires that the input signals be scaled up in speed, which might be impractical or expensive. In this brief, we show that this problem can be alleviated by a masked RC system in which the amplitude of the input signal is modulated by a binary-valued mask. For a speech recognition task, we demonstrate that the necessary input sample rate can be a factor of 40 smaller than in a conventional RC system. In addition, we also show that linear discriminant analysis and input matrix optimization is a well-performing alternative to linear regression for reservoir training.

25. Using deep learning to estimate systolic and diastolic volumes from MRI-images

    Degrave, Jonas, Burms, Jeroen, Korshunova, Iryna, and Dambre, Joni

    In BENELEARN 2016 : proceedings of the 25th Belgian-Dutch conference on machine learning 2016

    Biblio
26. Hierarchical reinforcement learning as creative problem solving

    Colin, Thomas R, Belpaeme, Tony, Cangelosi, Angelo, and Hemion, Nikolas

    ROBOTICS AND AUTONOMOUS SYSTEMS 2016

    Biblio
27. Towards long-term social child-robot interaction : using multi-activity switching to engage young users

    Coninx, Alexandre, Baxter, Paul, Oleari, Elettra, Bellini, Sara, Bierman, Bert, Blanson Henkemans, Olivier, Cañamero, Lola, Cosi, Piero, Enescu, Valentin, Ros Espinoza, Raquel, Hiolle, Antoine, Humbert, Rémi, Kiefer, Bernd, Kruijff-Korbayovà, Ivana, Looije, Rosemarijn, Mosconi, Marco, Neerincx, Mark, Paci, Giulio, Patsis, Georgios, Pozzi, Clara, Sacchitelli, Francesca, Sahli, Hichem, Sanna, Alberto, Sommavilla, Giacomo, Tesser, Fabio, Demiris, Yiannis, and Belpaeme, Tony

    JOURNAL OF HUMAN-ROBOT INTERACTION 2016

    Biblio
28. Embodied Language Learning and Cognitive Bootstrapping : Methods and Design Principles

    Lyon, Caroline, Nehaniv, Chrystopher L., Saunders, Joe, Belpaeme, Tony, Bisio, Ambra, Fischer, Kerstin, Förster, Frank, Lehmann, Hagen, Metta, Giorgio, Mohan, Vishwanathan, Morse, Anthony, Nolfi, Stefano, Nori, Francesco, Rohlfing, Katharina, Sciutti, Alessandra, Tani, Jun, Tuci, Elio, Wrede, Britta, Zeschel, Arne, and Cangelosi, Angelo

    INTERNATIONAL JOURNAL OF ADVANCED ROBOTIC SYSTEMS 2016

    Biblio
29. Social Development of Artificial Cognition

    Belpaeme, Tony, Adams, Samantha, Greeff, Joachim, Nuovo, Alessandro, Morse, Anthony, and Cangelosi, Angelo

    TOWARD ROBOTIC SOCIALLY BELIEVABLE BEHAVING SYSTEMS-VOLUME I 2016

    Biblio
30. The Effect of Repeating Tasks on Performance Levels in Mediated Child-Robot Interactions

    Baxter, Paul, Kennedy, James, Ashurst, Emily, and Belpaeme, Tony

    2016

    Biblio
31. The Cautious Attitude of Teachers Towards Social Robots in Schools

    Kennedy, James, Lemaignan, Severin, and Belpaeme, Tony

    2016

    Biblio
32. Towards “Machine-Learnable” Child-Robot Interactions: the PInSoRo Dataset

    Lemaignan, Severin, Kennedy, James, Baxter, Paul, and Belpaeme, Tony

    2016

    Biblio
33. Social robot tutoring for child second language learning

    Kennedy, James, Baxter, Paul, Senft, Emmanuel, and Belpaeme, Tony

    2016

    Biblio
34. Heart vs hard drive: Children learn more from a human tutor than a social robot

    Kennedy, James, Baxter, Paul, Senft, Emmanuel, and Belpaeme, Tony

    2016

    Biblio