Benchee 1.3.0: Memory Optimizations and Enhancements

The article by PragTob delves into the latest version of Benchee (1.3.0), a benchmarking tool for Elixir. It focuses on a reduction in memory usage when running benchmarks, especially those with large inputs. A new function, `Benchee.report/1`, simplifies the process of working with saved benchmarks. The update also ensures that configured times are displayed in a more readable format. PragTob outlines the motivation behind the release, which was to fix ergonomic issues and high memory consumption identified while writing a large benchmark. The development process itself led to the realization of an actual issue that prompted the new release. Notably, Benchee 1.3 addressed a memory leak that reduced memory consumption from 6.5 GB to approximately 5.1 GB for an individual run, and file sizes of saved results plummeted from 226MB to 4MB or lower. The time taken to create reports also decreased significantly. There was a downside, however; the fix introduced a breaking change for formatter plugins, a decision PragTob regards as a necessary evil to rectify a substantial mistake. Additionally, an entire feature was implemented and subsequently removed after realizing that it didn't address the core problem and provided no tangible benefits. The article concludes on a light-hearted note, looking forward to sharing the actual benchmark and wishing readers happy holidays.