This project explores the intersection of machine learning and numerical analysis, focusing on optimizing models governed by systems of ordinary differential equations (ODEs). Leveraging advanced techniques from machine learning and high-performance computing, the research aims to develop efficient methods for optimizing large parameter arrays in complex, non-convex problems with linear and non-linear constraints. Initially, classical interior point optimization methods will be implemented, followed by the deployment of machine learning architectures like RNNs and transformers to reduce dimensionality and optimize parameters. Additionally, the integration of neural networks into the optimization process will be investigated, capitalizing on their inherent parallelization capabilities.