Release of TextChunker, a Semantic Text Chunking Library for Elixir

TextChunker is an Elixir library that provides developers with the ability to segment large text documents into semantically meaningful chunks. It was created by Revelry specifically to support their AI integration needs within an Elixir app they are developing. It aims to fill a gap in the Elixir ecosystem by offering intelligent text segmentation, which exists in Python and JavaScript through projects like LangChain. TextChunker offers features such as semantic chunking for various formats (plaintext, markdown, code, etc.), configurable chunking for variable chunk size and overlap, and metadata tracking to ensure the integrity of the original text. The library is particularly useful for Retrieval Augmented Generation applications since it provides the contextual relevance needed for large language models to work effectively. Information about how to get started with TextChunker, including installation and documentation, is provided. Revelry also invites contributions from the community on their GitHub repository.