Despite recent advancements in large language models (LLMs), current LLM-based chatbots often come across as unnatural, lacking the spontaneity typical of human conversations. Their responses also suffer from high latency and are frequently too conservative, leading to dull and unengaging interactions. The objectives of this project are to:

(1) Propose a diversity-enhanced fine-tuning approach for LLMs to encourage the generation of responses that are free from bias, sufficiently diverse, and creative.

(1) Develop a real-time speech-to-speech chatbot that does not rely on text as an intermediate representation. This approach simplifies the pipeline and reduces latency, enabling the chatbot to produce not only speech but also paralinguistic signals, such as laughter and back-channeling, to enhance conversational fluidity.