An Introduction and Demonstration of Erlfuzz: A Fuzzing Tool for Erlang

Robin Morisset's talk from Code BEAM Europe 2023 introduces Erlfuzz, a recently open-sourced fuzzing tool that generates random valid Erlang programs. This tool is used to test the Erlang compiler (erlc), the BEAM VM, and various other tools such as dialyzer, eqWAlizer, and erlfmt, among others. To date, Erlfuzz has discovered more than 80 bugs, including over 60 in erlc alone. The talk covers how Erlfuzz works, the techniques employed to uncover subtle VM bugs, and its application in testing any desired tool. It also touches on the unique and occasionally overlooked aspects of the Erlang language, including its distinctive scoping rules. The talk includes a live demonstration where Robin shows how Erlfuzz can be operated, including its various commands. A key highlight is the discussion on the generation of Erlang code, maintaining validity while ensuring maximal coverage of potential bugs. The intricacies of Erlang's variable scope rules, assignments, and how Erlfuzz handles these complexities are explained. Robin also describes various approaches to make the compiler and runtime environment more crash-prone intentionally to reveal hidden bugs, such as enabling assertions and using tools like ASan (AddressSanitizer). The session concludes with an overview of the bugs discovered by Erlfuzz, and some practical advice on how to integrate fuzzing into a testing regime, stressing the importance of running these tests continuously. Finally, the community's engagement and quick reaction to bug fixes are acknowledged, with a special mention of the OTP team's responsiveness.