Chris Vorster

A clear trend in Internet of Things (IoT) research is to move intelligent decision-making away from centralised cloud servers and towards low-power, low-latency smart devices close to the data source and user, while supported by centralised oversight and configuration control. Challenges in moving Machine Learning (ML) from cloud to edge devices include the relatively constrained computing power and memory of edge devices, which in turn constrains the accuracy of ML models.In such limited settings, Video Object Detection is frequently accomplished by performing Image Object Detection on a per frame basis. Real-time video streams therefore need to be decoded into individual images, which is inefficient. This approach also neglects temporal information.The first avenue of this research is to investigate the feasibility and accuracy of Object Detection by applying deep learning models directly to key frames and using the non-key frames as a proxy for motion. Objects can be detected and recognised using the spatial information in the key frames, aided by the object cohesiveness which motion provides through the non-key frames. This research will build on the work done by Wang et al. in Fast Object Detection in Compressed Video (2019) and Real-time and accurate object detection in compressed video by long short-term feature aggregation (2021).Further research directions will include the creation of a dataset that combines compressed video and neuromorphic camera data along with the application of ML on such data. Neuromorphic cameras are event-based, low-power sensors that detect changes in brightness per pixel. The similarity between neuromorphic camera’s representation of reality and that of compressed video will be investigated.Possible research avenues include memory utilization, novel model architectures and training data transformations. Additionally, this research will develop new architecture designs, techniques and models for implementing ML-based applications on edge devices in order to efficiently perform ML on IoT devices.Use cases of this work will focus on computer vision applications, being data intensive processes requiring ML support, which is particularly challenging for edge devices. Output will be evaluated against current state-of-art deep learning platforms, including Google Cloud ML Engine.Applications of this work are wide-ranging, including automotive driver assist, predictive maintenance, intruder detection and wildlife monitoring. Any application with scene understanding from cameras could benefit from this work.