Leveraging Approximate Nearest Neighbors for Improved Text Generation in LLMs

Milad Rastian and Mikk Rätsep presented a comprehensive talk on enhancing text generation of Large Language Models (LLMs) using Elixir. They began by discussing their backgrounds and interest in AI and Elixir. The talk then delved into the limitations of LLMs, such as hallucination, and introduced the concept of Retrieval Augmented Generation (RAG) using Approximate Nearest Neighbors (ANN). They demonstrated the use of vector embeddings to semantically capture data and showcased a practical implementation involving a PostgreSQL extension called PG Vector. The speakers also covered how to address common issues such as optimal chunk size, context fragmentation, and scalability. They introduced techniques like multi-query expansion and cross-encoder reranking to handle ambiguous queries and improve retrieval accuracy. The talk concluded with resources for further learning and a live demo, making it a rich source of practical insights for leveraging ANN in LLMs using Elixir.