This position is for School of Computing and Information Systems (SCIS).
Extend and maintain the lab's VR testbed (Unity front-end, Python back-end) integrating live speech recognition, eye tracking, hand tracking, and a scene-graph representation of the task environment.
Implement an incremental multimodal reference resolver that fuses speech, gaze, and gesture into a continuously-updated belief distribution over candidate interpretations, meeting real-time latency requirements (less than 200ms end to end).
Design and implement a library of in-situ visual overlays that render the agent's belief in VR, and integrate them with the communication policy developed by the Co-PI's team.
Design, pilot, and run controlled lab user studies (between- and within-subjects, N≈24-48), including experiment scripting, instrumentation and logging of behavioural measures, participant recruitment and scheduling, and IRB documentation.
Analyse quantitative (ANOVA, trust-calibration and error measures) and qualitative (interview, think-aloud) study data, and co-author submissions to CHI, UIST, DIS, CSCW, or HRI.
Collect, annotate, and release an expanded public corpus of natural deictic instructions, including running elicitation sessions and maintaining the annotation scheme and documentation.
Other duties as assigned.
Qualifications
Minimum Master's degree in Computer Science, Information Technology, Information Systems or closely related disciplines from a reputable institution of higher learning; with particular focus on Human-Robot Interaction.
Minimum 1 year of relevant research experience in Human-Robot Interaction as a graduate student, research assistant or research engineer.
Proficiency in programming software/languages such as C# and Python, and the Unity environment is required.
Hands-on experience developing for VR head-mounted displays (e.g., Meta Quest, OpenXR), including eye- and hand-tracking input.