The ability to learn a hierarchical and compact representation from data stands as a fundamental principle behind the rapid growth of Deep Learning, particularly evident in Computer Vision. Despite the significant progress on the perception tasks, these models still fall short in terms of reasoning and planning capabilities, and cannot generalise systematically despite being trained with extensive amount of data and compute resources. How to effectively scale up a representation learning system in terms of computation and data, and extend the capabilities of the visual representations toward high-level tasks is the central research topic of this project.