Boi Mai Quach

This project will explore how multi-modal context-aware user models can be used for the purposes of enhancing productivity by allowing computer systems in real-time to intelligently adapt to a user’s state at that point in time and the context of their current mental state. Imagine sitting at your computer being bombarded with requests over instant messaging while trying to complete other tasks. This is something you might be able to manage on any other day without problem, but today you are having difficulty because you are tired as you didn’t sleep, and the three coffees you had earlier in the day are not having the desired effect. The systems and software you use daily cannot adapt appropriately to this because they are not aware that you woke too early and haven’t eaten, and that in that moment you were feeling overwhelmed with the task load. Multi-modal context-aware user-modelling would allow for such systems to be aware of your current mental state in that moment by leveraging signals (neural, physiological, lifelog, etc) captured up to that point in time. Furthermore, having such information would enable powerful personal information retrieval and summarization systems that would allow you to understand what areas of your work or day have been most impacted in terms of productivity, enabling you to change or identify behaviors (or how you schedule work tasks) in order to achieve optimal throughput. By using signals produced from the body such as EEG (Electroencephalography), EOG, movement, heart rate, GSR and breathing, in concert with the capture of signals from our environment such as those from a lifelog camera or computer interaction, this research aims explore the types of multi- modal context-aware user models that can be built based on machine learning that can be used for the purposes of enhancing productivity by allowing computer systems in real-time to intelligently adapt to a user’s state at that point in time and the context of their mental current state.