An article by Cohort 5 students Oluwadara Adedeji (UCD), Abigail Naa Amankwaa Abeo (DCU) and Thabang Isaka (TU Dublin)
It has been some weeks since my first ML Labs Summer School, and I continue to reminisce about the knowledge and insights I gained from this exciting and instructive program. One of my favourite experiences was getting advice on navigating the PhD from graduates and industry partners. I was also able to relearn the fundamental concepts in different areas of AI, especially Gen AI. My gratitude goes to the Student Organizing Committee and to SFI Centre for Research Training in Machine Learning for organising this.
I believe it would have been extra informative to know more about activities of the ML Labs before I began my PhD. Nevertheless, I would like to write this piece to give an insight into activities of the PhD program at ML Labs. Took awhile, but I finally found time to put this together.
The summer school was a 5-day programme with a lot of activities. In parts, I will be sharing key highlights, and I hope not to bore you. I ask for pardon from my fellow participants should I inadvertently omit any necessary detail. Additionally, I want to thank my colleagues Abigail and Thabang for their immense contributions. This article is born out of a collective effort and without them, it would not have come to light.
Day 1
The ML Labs Summer School, held in Dublin City University (DCU), was power-packed from beginning till the end with exciting activities and lightning talks from different speakers. It commenced on June 10th as student researchers from all three universities in ML Labs—DCU, Technological University Dublin (TUD), and University College Dublin (UCD)—trooped in, full of energy.
The program began with a welcome address by Dr. Georginia Ifrim, Associate Prof. at UCD. This really got me pumped up on what to expect in the week-long programme.
Dr. Georgianna Ifrim, ML-Labs Director delivering the welcome address
Right after her welcome address, we had an ice breaker activity; a crazy Bingo game! Look, this game is not popular back home in Nigeria and I have never played it before. I must admit that, the game was fun and created a good atmosphere to loosen everyone up and get ready for the day.
Our vibrant MCs: Thabang and Abigail
Thereafter, Dr. Tony O’Donnell, Director of Engineering at Meta, gave an interesting industry keynote. He provided a summary of his work on integrity, focusing in particular on how they guard vulnerable consumers and identify inappropriate behaviours. A significant insight from his presentation was that the emergence of generative AI makes it more challenging to distinguish between real and fake media. He also gave the PhD students an overview of what to anticipate if interested in joining Meta, and requirements such as programming, system design, and behavioural evaluation.
Dr. Tony O’Donnell giving his industry keynote
After the morning sessions, we were treated to a sumptuous lunch, which quickly became one of my favourite parts of the summer school (winks). And as you know, what’s a day without lunch? Thereafter, Dr. Susan Leavy gave a thought-provoking talk on AI Governance, where we delved into the EU AI Act and explored its lessons for our research. Building on the theme of AI’s broader implications, Dr. Michael Scheibenreif, a regional manager at UNICEF, wrapped up the day with the last lecture, focusing on AI and its social impact through the use of drone technology in Africa.
Dr. Susan Leavy’s lecture on AI Governance
Dr. Michael Scheibenreif, presenting on UNICEF’s Drone academy
Day 2
The lively ice breakers continued on Day 2. Indeed, the organisers had thought through various ways to keep us entertained. This time, we faced a ‘finish the lyrics’ challenge, where those unfamiliar with the song had to dance—and PhD students dancing is a spectacle worth saving for another discussion 😂! After the fun, the atmosphere shifted as Dr. Kathleen Curran took the stage to discuss Emerging Trends in AI in Medical Imaging and Diagnostics. As someone passionate about AI applications in healthcare, I found this topic particularly engaging. Dr. Curran’s insights into the challenges of data annotation and the critical role of selecting appropriate metrics in medical imaging were enlightening.
Dr. Kathleen Curran giving her talk
Thereafter, there was a fierce and captivating debate with the motion, “AI will have a positive impact on the future workforce”. Personally, if you had asked for my stance prior to the debate, I would say the impact is largely positive, being someone inclined to AI research. But it was interesting to see colleagues not being biased by their AI background, and give objective points as to why this impact could be positive or negative. The representatives were Min Wu (DCU), Ajay Kumar M (UCD), and Francesco Chiumento (DCU) for the motion, and Faithful C. Onwuegbuche (UCD), Joyce Mahon (UCD), and Cathal Mellon Whelan (UCD) against the motion. Some interesting points raised for the positive aspects were that AI will make jobs easier, by making mundane work easy and augmenting human creativity.
Despite the good points from the first group, the panel against the motion composed themselves and gave a rebuttal to argue on the negative aspects of AI. Some of the points raised were that AI currently takes more jobs than it replaces, and some of these jobs are even the so-called creative jobs. Furthermore, AI has the potential to make future labour cheaper, which could negatively impact people’s earning potential.
The debaters and the judges
There was also an industry talk on ML models as API and in platform architectures delivered by Kemal Araz from Fresco. Some key takeaways are on stateful and stateless applications. Stateful applications store all information from previous history while the stateless do not. In addition, stateless is easily scalable, programming language agnostic and also easy to monitor. Moreover, an introduction to model serving through APIs was discussed.
Industry Talk by Kemal Araz, Fresco
Finally, the last activity for the day was an alumni panel featuring Na Li, Carles Garcia, Cabrera, and Agatha de Mattos. Although I’m only in my first year, I was keen to learn from the experiences of the program’s alumni. Key lessons included being proactive in research and the importance of proper organisation. There was a strong emphasis on the value of quality publications over quantity, especially in today’s ‘publish or perish’ academic culture. It’s also crucial to engage with the research community beyond Ireland. Additionally, for those looking to transition into the industry, it’s beneficial to work on projects that can be showcased during interviews and to prepare for technical interviews. Another vital lesson was the importance of self-awareness regarding the value you bring, which boosts self-confidence. To identify post-PhD opportunities, it is important to read papers and note which labs are conducting research that interests you, with the aim of applying to these labs after completing your PhD.
Alumni panel discussion
Day 3
Day 3 of our summer school kicked off with a lively “Guess the Country Flag” activity. Although my otherwise stellar geography skills could only manage a top 5 finish, the morning was charged with enthusiasm. The first talk of the day was on Reproducibility and Transparency in AI delivered by Dr. Claudia Mazo. It was interesting to refresh our understanding of machine learning concepts such as different types of machine learning, the machine learning cycle, and why it is important to approach machine learning in a reproducible and transparent manner.
Academic Talk by Dr. Claudia Mazo
Following the morning activities, an industry panel composed of industrial professionals took the stage. A key takeaway from the panel discussion was the industry’s focus on impacts and business value. Additionally, the panel highlighted in-demand skills such as design pattern expertise, unit testing, and strong collaboration and communication abilities. It’s crucial to always start with the “why” for each project, before diving into the technical aspects that make achieving that goal possible.
Industry Panel Discussion. From left to right: Jonathan Costello, ML-Labs, Dr Brian McNamee, ML-Labs, Abigail Naa Amankwaa Abeo ML-Labs PhD candidate, Brian Mullan, Principal Group Data Science Manager at Microsoft, Dr Esraa Ali, Senior Data Scientist at Carelon Global, Alexey Tarasov, Senior Machine Learning Scientist at Intercom and Adi Botea, Principal Data Scientist at Eaton
Subsequently, the panel shifted focus to the importance of cultivating a habit of continuous learning. While engineers and other technical professionals often enhance their technical skills, they may neglect their communication skills. We were encouraged to develop these skills to become more effective communicators. For PhDs, leveraging opportunities such as research presentations and academic writing can significantly improve their communication abilities. Additionally, it’s crucial to exhibit these skills during interviews. For example, clearly articulating your thought process during a coding interview is essential.
I also learned the value of a well-crafted CV and the benefits of gaining experience through internships. Another key lesson from the panel was the importance of having faith in oneself and seeking environments where one is valued and that offer a good work-life balance. The panelists also stressed the importance of networking at conferences and technical events, which can help build a supportive network and facilitate career growth.
Following the panel discussion, there was an academic keynote on Knowledge Graphs for Social Good delivered by Dr. Bojan Bozic. It was fascinating to learn about knowledge graphs, particularly the insight that their real information lies in their links or connections. Despite the vast amount of data on the web, the content’s meaning is not machine-understandable. Knowledge graphs aim to provide semantic meaning to web data using elements like entities (nodes), relationships (edges), properties (attributes), labels and tags, and ontologies. It was intriguing to discover that ontologies help set restrictions and define how data is organised in knowledge graphs. Although I have experience with graphs, this was my first in-depth lesson on knowledge graphs, and I learned a lot.
Academic keynote by Dr. Bojan Bozic
The day wrapped up with a dynamic industry visit to Intercom—a delightful change from the usual academic environment, offering a refreshing change of pace. I extend my heartfelt gratitude to the organising committee for their thoughtful planning of each activity. Special thanks also go to the team at Intercom for their warm welcome.
During the visit, we were given a glimpse into the vibrant work culture at Intercom. As students enjoyed some light refreshments, we were treated to an enlightening presentation on transformers and language modelling. It is intriguing to note that although language models have recently gained prominence with the success of ChatGPT, they are not a new concept. The presentation covered the evolution of language models from basic N-gram models to the sophisticated large language models of today. The importance of solid fundamentals, adaptability, and robust scientific skills were highlighted throughout the talk. This industry visit proved to be both informative and educational, a perfect conclusion to an engaging day.
Day 4
This is the final day of the in-house activities. Disclaimer, the details I will share about this day are more technical, especially to a non-machine learning audience. On this day, there was a workshop on Generative Models and Diffusion Models presented by Dr. Xavier Giro-i-Nieto, an Applied Scientist at Amazon Science and academic at Universitat Politecnica de Catalunya (UPC) in Barcelona. During the workshop, the students were taken through the fundamentals of generative models such as Generative Adversarial Networks (GANs), Variational Autoenconders (VAE), Diffusion models, and Autoregressive models.
Workshop delivered by Dr. Xavier Giro-i-Nieto
The workshop began with an overview of discriminative models, which are designed to predict a response Y from a given input X, commonly used in classification and regression tasks. In contrast, generative models are capable of producing new samples by modelling the data clusters within super high-dimensional spaces. Essentially, generating samples from a high-dimensional probability distribution that mirrors the training data’s distribution. An example provided was the Gaussian Mixture Model. Additionally, students were introduced to advanced concepts like conditional generative models and sampling techniques using deterministic deep neural networks through interpolation. The facilitator also discussed the concept of disentanglement in these networks, which allows for the manipulation of various output aspects by adjusting the corresponding latent variables. A significant challenge with Generative AI, as pointed out during the session, is evaluating the output of these models. Evaluation metrics specific to computer vision, such as the Frechet Inception Distance (FID), Maximum Mean Discrepancy for fidelity, and CLIP score for alignment, were explained.
The session further covered Generative Adversarial Networks (GANs), detailing their development rooted in game theory. Auto-encoders were also highlighted, particularly their use in compressing data at the bottleneck to achieve compression. The discussion on diffusion models, which are developed by adding Gaussian noise to data and then reconstructing the original data, underscored the growing prominence of UNets and the burgeoning use of transformers in these models. The workshop concluded with an exploration of autoregressive models like pixelRNN and transformers, which form the basis of the popular GPT models, including ChatGPT. To ensure hands-on experience, students were provided with lab resources to practise these models. This workshop proved to be highly educational, offering both a deep dive into the theoretical underpinnings and practical applications of machine learning technologies.
Students with smiley faces after an exciting summer school
As the afternoon rolled in, it was time for what could best be described as the academic fashion show. I mean, the poster presentations! Each student arrived, posters in hand, colourful, flashy, and ready to dazzle. It was like watching peacocks fan their feathers, but instead of feathers, they had graphs and bullet points. It was great to see all the interesting research from the students in machine learning, including topics like fundamental machine learning, machine learning applications, and quantum machine learning. At the end of the session, awards were presented to the best posters as voted by the rest of the students. The awardees were Eanna Curran (1st), Mehran Alizadeh Pirbasti (2nd), Sidra Aleem (3rd), and Milad Dadgar (3rd).
Day 5
It’s the last day of summer school! We ended on a high note with a trip to the Irish National Stud & Japanese Gardens in County Kildare. Soaking up the tranquility of the Japanese Gardens, everyone shared beautiful moments together, fully embracing the stunning surroundings. They captured plenty of pictures, preserving the sweet memories of today to reminisce in the future. Although the lands around the Tully have been breeding horses since the 13th century, it wasn’t until the 19th century that Colonel William Hall-Walker, who later became Lord Wavertee, set up the stud farm.
Now to the really interesting parts of our tour that absolutely left me shocked, seeing horses getting the VIP treatment really blew me away! We’re talking about a red carpet class for these horses. The stud primarily houses stallions, these top-notch male horses bred to mate with mares. And these aren’t just any stallions; they’re high-quality breeds carrying genes meant for producing champions in racing. Each stallion at the stud not only has its own name but also a private paddock. They’re booked a whole year in advance for mating sessions and, trust me, it does not come cheap. Some stallions are even flown as far as Australia just to mate, transported in custom paddocks designed for maximum comfort. Honestly, it was like watching a ‘Keeping Up with the Kardashians’ episode, but instead, it was the ritzy lives of these Irish stud horses!
I also learned a lot about how horses live, like what they eat, how costly it is to feed them, their ability to sleep while standing, their gestation periods, the duration foals stay with their moms before they’re separated, their unbelievable body mass, and their lifespan. They seem to live a fulfilling and enjoyable life. Honestly, in anyone’s next life, coming back as a horse might actually not be a bad idea; I recommend it. Ahah.
Finally, the students participated in a simulated horse race. It was a great climax to our incredibly fun experience at the stud farm, and a fantastic way to wrap up the summer school.
What a sweet way to end the summer school!