Efficient Data Processing in a Monolithic System

In the session presented by Brian Meeker at ElixirConf US 2024, the focus is on the scaling issues faced by GetThru regarding their text messaging infrastructure. The existing system became complex due to various external requirements, leading to inaccuracies and difficulties in management. The presentation outlines a case study of how the team developed a new concurrent, multi-stage data processing pipeline that was designed to efficiently adapt to these external requirements while maintaining flexibility for future demands. Notably, the implementation was done within their existing monolith, thus avoiding the pitfalls commonly associated with microservices.