When UAVs autonomously navigate, it is important that the time it takes them to process sensor input and make a movement decision (i.e. the inference time) is as low as possible. Edge computing can help with reducing this inference time by allowing computation to take place on-UAV or at nearby edge servers. Deep reinforcement learning (DRL) is popular method of UAV autonomous navigation. A drawback of DRL is the large amount of time it can take to train a good model. We examine methods of reducing training and inference time of DRL based UAV autonomous navigation models at the edge.