Multi-Modal Generative Models of Stylistic Tuning — Making Personality more Personal.
The world is progressively globalising. As a result, many people want to learn new languages to exploit travel, business, and cultural enrichment opportunities. Unfortunately, not everyone has easy access to language learning classes, and they can be costly for people who do. The amount of virtual content and services people are using is increasing year to year. The rising volume of virtual services and content paired with the finite amount of spare time that all users have makes the selection of services and content within services more critical. Many services can be provided or supported by automatic text generation. There are support chatbots, virtual assistants, data to text generators, joke generators, and language tutors. Many of these advancements in text generation had become more feasible once deep neural networks were popularised and applied to the area. Personalisation applied to these services can make them more relevant to individual users. One fruitful application domain of these areas is language learning. An intelligent virtual tutor can service people regardless of location a lot cheaper than a human tutor. On the other hand, some people may not relate to a text generator like a human tutor. However, an intelligent virtual tutor that can adapt to user personality in a continuous fashion can become more relatable to the user. Another approach that may promote relatability is to combine a virtual reality avatar that can perform simple gestures in line with the generated text. The personality of a virtual tutor can be considered a form of style. This style can be customised to suit a user better. Generative adversarial networks or GANs are a neural mechanism often used to tune the outputs of media. Text can be passed through a network and tuned to provide a custom output. A research challenge would be to make these outputs fluid, dynamic, and automatic. We could combine GANs with other generative models to simulate personality for the language tutor. User studies would likely be used to evaluate this research. This project links to the ML-Labs challenge area of machine learning in language, a fundamental area of machine learning.