This research project investigates the training and application of foundation models for time series classification. Recent developments in machine learning have introduced many deep learning architectures, especially transformers, for learning with time series data. In Time Series Classification (TSC), DL models often struggle; they are not robust to noise and are computationally expensive. Additionally, these models tend to overfit when trained on small TSC datasets, leading to generalization issues. However, transformer-based architectures like BERT, GPT, and T5, which are pre-trained on large-scale datasets, can be fine-tuned for more specific downstream tasks.