Welcome to Peng Kuang's licentiate thesis seminar on Thursday April 25th at 13:15-15:00 in E:A

Thesis title: Toward Gaze-enabled Programming Tool Assistance

Author: Peng Kuang, Department of Computer Science, Lund University

Faculty opponent: Professor Roman Bednarik, University of Eastern Finland

Examiner: Professor Björn Regnell, LTH

Supervisor: Docent Emma Söderberg, LTH

Co-supervisors: Professor Martin Höst, Malmö University & Dr. Diederick Niehorster, HumLab Lund University

Date and time: Thursday 25 April at 13:15

Location: E:A, E-huset, LTH, Ole Römers väg 3, Lund

Link to download thesis: https://portal.research.lu.se/en/publications/toward-gaze-enabled-programming-tool-assistance

Link to online zoom meeting: https://lu-se.zoom.us/j/63634033658

Abstract

Programming is a cognitively demanding exercise. In particular, today’s software development requires a collective effort of programmers and the orchestration of a complex programming infrastructure. As disruptive technologies emerge, e.g., AI and quantum computing, the programming practice is undergoing a change, facing an uncertain future that we may not be able to accurately predict but can envision and work toward.

With the maturity of eye-tracking and its integration into everyday consumer electronics such as Alienware’s laptops and Apple’s Vision Pro, we expect it will eventually make its way into everyday use just as touchpad, camera, and microphone. Therefore, we see an opportunity to design eye-tracking based assistance to support programmers. Given programmers spend a large amount of their time reading and understanding code, which heavily relies on eyes, we deem this to be a promising problem domain where eye-tracking can be of assistance.

To explore this inquiry, we undertook two mapping studies to establish the problem and solution constructs. We then surveyed professional developers to understand this representative cohort of our prospective users and gather concrete, situated problems from them. We conducted these studies under the guiding design science model for empirical software engineering which centers on a problem-solution pair.

From the first study, we found that eye-tracking so far is used mostly for education-oriented studies in the research community focused on software development. There is a need to bring it closer to practitioners. From the second study, we identify that the gaze data produced by eye trackers has been explored with a collection of machine learning techniques. However, these models were trained with small samples that might carry bias and insufficiency. Contemporary machine learning techniques may be able to compensate for that. From the survey, we learned that developers have already adopted AI assistance, and they are mostly positive about it despite room for greater accuracy and capability. As eye-tracking is relatively novel to them, most developers are unsure about how it can help them.

For future work, we plan to practice designing with programmers to develop and evaluate our proof of concept and explore gaze data with more tailored machine learning techniques, which aims to generate integration into our system.