Pinar Sungu Isiacik

This research project investigates the training and application of foundation models for time series classification. Recent developments in machine learning and AI have put forward many deep learning architectures, especially transformers, for learning with time series data.
In the field of Time Series Classification, there are many applications where deep learning models (e.g., transformers, GPT) do not work well, they are not robust to noise in the data and are computationally very expensive. Also, deep learning models often suffer from overfitting when they are trained on small TSC datasets (generalisation problem). However, transformer-based architectures like BERT, GPT, and T5, which are pre-trained on large- scale datasets, can be fine-tuned for more specific downstream tasks. This project aims to investigate the use and development of foundation models for TSC, with a focus on the application area of human motion analysis.

Some of the main research questions are:
(1) What are the current foundation models for Time Series Classification (TSC), and what are their strengths and limitations? Are open source foundation models feasible to train with laptop-type computational resources?

(2) What type of techniques can help with improving the scalability and accuracy of foundation models for TSC? Can symbolic transformations of time series (e.g., SAX or SFA) be used to train effective foundation models for TSC?

(3) What application areas most benefit from foundation models for TSC?

(4) What are effective techniques for explaining the foundation models’ predictions?