By Niamh Belton, Jack Nicholls // Illustration by Kyle Hamilton
Contact detection is a method being used by organisations around the world in a bid to tackle the spread of Covid-19, but how exactly does it work and how can it be improved?
Using data produced by your phone, applications like the HSE Covid tracker app allow for the tracking and notification of potential contacts with people who have been tested positive for the virus. The HSE’s app uses Bluetooth data in order to calculate the distance and length of time people have been in contact with others. Testing of the app showed an accuracy of 72% in the detection of close contacts. Can machine learning methods improve the accuracy of close contact detection using other data produced by your phone?
NIST (National Institute of Standards and Technology of the USA) in collaboration with MIT’s research group PACT (Private Automated Contact Tracing) have created a challenge titled TC4TL, or Too Close For Too Long. The challenge saw universities around the world participate in an attempt to develop accurate and robust models to accurately calculate close contact detection. Positive results were produced using Bluetooth data, accelerometer and transmission power information. The results and methodology of the various groups can be viewed on the TC4TL challenge website (https://tc4tlchallenge.nist.gov/).
The Centre for Research Training in Machine Learning approached NIST after the initial competition and proposed that the new cohort of students take part in this challenge. The new cohort of thirty PhD students from across UCD, DCU, and TU Dublin were divided into six groups and tasked with tackling the “Too Close” or distance aspect of the TC4TL challenge. NIST created a direct line of communication with the cohort, allowing us to ask questions on the data, the methods of evaluation, and any other enquiries.
This challenge gave us the unique opportunity to use our machine learning skills to make an impact on a global issue. NIST provided large amounts of data for almost 25,000 events where each event had one label. This gave us a chance to immerse ourselves in a project that uses real-world data, where the data was noisy, weakly labelled and the data collection methods varied from one event to another. The wide scope of the problem allowed us to be creative and formulate the problem in a number of different ways – some choosing complex time series analysis and others fusing domain knowledge with machine learning techniques. An array of algorithms were used from deep learning techniques to tree-based ensembles. The effects of bringing a diverse range of expertise together and implementing a wide range of approaches resulted in teams performing on par and ahead of groups that are on the top of the leaderboard!
One of the key learning take-aways from the project was how to formulate a Machine Learning solution. We found that defining the objective of the project, establishing a clear baseline performance and effectively evaluating the solution were crucial components of developing a robust machine learning solution. We learned that evaluating our solution meant not only assessing it on the basis of a performance metric but to also consider other factors such as the complexity of the solution, the training time and the feasibility of productionising the solution. Moreover, the project highlighted the significance of being able to effectively communicate our solution to others. We practiced both verbal and written communication skills by composing a final report and regularly presenting updates to fellow ML-Labs students and supervisors. These are skills that we can utilise and continue to develop in our future projects!
Similarly with the majority of the global workforce we have had to collaborate and work together through online technology like Slack, and Zoom. As some students have not travelled to Ireland due to the pandemic, time scheduling and coordination of working schedules was the first hurdle for many. With any project, there are different skills and backgrounds of each team member. With such diverse backgrounds for each student of ML-Labs, the groups had different levels of experience in project management, software development, and model evaluation to learn from each other.