Computer Science

Faculty of Engineering, LTH

Denna sida på svenska This page in English

PhD project

AI for HRI: Mixed-initiative interaction in multi-agent settings


This page might be updated more or less any time I think of more information to consider, however, if you have questions do not hesitate to contact me!

The Context

The group for Robotics and Semantic Systems (RSS) at the Computer Science Department, Faculty of Engineering at Lund University can offer one PhD project, funded for four years through the ELLIIT initiative and to be supervised by me, Elin A. Topp. Some more details on how a PhD studentship looks like in Sweden, and specifically in the RSS-group, can be found below.

The Project

Human users can provide robots (and other systems) with an immense amount of knowledge, e.g. through demonstrations, immediate (classic) programming, instructions, dialogues, or combinations thereof. However, due to robots being different from humans regarding their physical and perceptual capabilities, it is not always obvious to a human user what - and sometimes even how - to communicate to make the robot or underlying system understand. This is to some extent due to the fact that the human user is not necessarily familiar with the actual capabilities and models the system can make use of.

Hence, we must assume, that when autonomous systems, or robots, interact with humans in an open-ended world (the real world), there will be situations where the system works according to its technical specifications, but its behaviour is incomprehensible for the user. This can be because user and system do not quite share the same understanding of the given context and situation.

This PhD project will thus investigate how a better situation understanding can be provided for systems that interact with humans and how to support this interaction in a mixed-initiative setting. Example problems include how such an interactive system can identify patterns in the user's behaviour that can be used to detect ambiguities in the situation, how it can in general assess unexpected and ambiguous situations, and how suitable responses to such detections can be computed. Such responses can mean to act autonomously, or to ask the human for help to resolve an ambiguous situation. It is also possible to consider direct support for the user in an instruction or programming setting, where the prediction of intentions and goals for a sequence of instructions or activities can be used to generate suggestions or reminders for the user to prevent problematic situations from occurring. In particular settings in a command central / control room, e.g. on a marine vessel, or in a multi-robot search-and-rescue mission, where the overall situation might have to be assessed to adapt visualisations and suggestions given to an operator, can be of interest for the investigations carried out.

Machine learning and pattern classification techniques, including Bayesian methods, are assumed suitable for the identification of patterns in user behaviour, while the situation assessment and computation of appropriate actions and responses can be based on reasoning mechanisms, potentially including defeasible reasoning. Since there are various perspectives and a variety of interaction scenarios possible, the actual project can be adapted to the background and interests of the candidate to quite some extent. 

The project is part of an ELLIIT-collaboration project (Project "Collaborative Robotics", B09), in which there will be the challenge (and opportunity) to investigate the idea of mixed-initiative interaction in multi-agent settings.

Otherwise, testbeds and opportunities for the demonstration of results include different mobile platforms, classic industrial manipulators, and collaborative robots available in the LTH Robot Lab, as well as the WASP demonstrator arena for Public Safety (WARA-PS).

The Candidate

Apart from the formal requirements specified in the official call, the applicant should match the following criteria. The ideal candidate has a solid background in computer science or a related field (also including, e.g., cognitive science) with very good mathematical and programming skills. Experience in at least one of the following areas is beneficial: Machine Learning, AI Reasoning, Robotics, Human-Robot Interaction, User Studies, Cognitive Science, and Programming Languages suitable for Robotics Research and ROS. Besides technical and mathematical skills, the candidate is expected to be curious and ambitious, with strong motivation to conduct research, but also willing to implement for and experiment with actual robotic systems. The applicant should be used to a well-structured work style, and have the ability to work both individually and in teams. Very good communication skills in both oral and written English are required. A short research proposal showing the applicant's interests in relation to the project area should be included in the application material.

The Group

In addition to the obvious interest in mixed-initiative interaction with both mobile (service) and industrial robot systems and the underlying approaches for situation understanding, intention recognition and general interaction monitoring represented by myself, the prospective supervisor for the project, the research group for Robotics and Semantic Systems has a well established track record in knowledge representation and management as well as integrated end-to-end software systems for (mainly, but not exclusively) industrial robotic applications, and is or has been involved in a series of respective EU-funded projects (SIARAS, ROSETTA, PRACE, SMErobotics, SARAFun). Recently, several PhD projects have started to investigate various approaches to Machine Learning within and for Robotics, many of those being supervised by Prof. Volker Krueger, who joined the group in 2018, Assistant Prof. Luigi Nardi, who joined us in 2019, and myself. 

A second, very strong and well established line of research is pursued by Pierre Nugues in the area of natural language processing (NLP), specifically semantic parsing, role labelling, and entity recognition. Some rather recent efforts of integrating NLP into work on intuitive programming of industrial robots have led to the establishment of an end-to-end support chain from high-level (spoken) commands formulated in natural language through levels of task and skill representation and abstraction down to executable robot code for industrial, both native and external, robot controllers (see Maj Stenmark’s respective publications).

The group is collaborating closely with the robotics researchers within the Dept of Automatic Control, enabling this type of research on all system levels, from investigations of high-level programming tools to sensor-based force-controlled manipulation approaches.

In addition to the opportunities for collaboration through the ELLIIT-funded project, there are strong connections in the group to the Graduate School and various research projects within the Wallenberg AI, Autonomous Systems and Software Program (WASP). The prospective candidate has thus the opportunity to benefit from the different topics, perspectives, and core competencies offered within RSS and the RobotLab and beyond.

PhD Student

The situation of a PhD student in Sweden is two-fold: While being a student with a certain course work load (currently at the Department of Computer Science this corresponds to 90 credits of in total 240, i.e., 37.5% of the nominal load), the candidate will also be fully employed by the University to carry out both course work and research. The employment entails a salary usually sufficient to sustain a small household, which is raised during the studentship according to certain milestones (in terms of credits) being achieved. 

The studentship / employment in this particular case is fully funded for four years through ELLIIT, however, it is common for PhD students to also take on up to 20% of full working time of departmental duties, most often as teaching assistant. These commitments are then funded by the department, which means that the overall time for the PhD studentship is spread out over five years with on average 80% of a full time work load per year assigned to the PhD studies (course work and research), and 20% dedicated to teaching. About half way through the project, i.e. after approximately 2.5 years, it is common for PhD candidates to obtain a Licentiate degree. This is not compulsory, however, it can be a good opportunity to consolidate the achievements of the first part of the project, and helps to shape the direction of further research.