Evaluating Cloud vs. On-Premise GPUs for Development and Deployment in Elixir

Thomas Millar writes a thought-provoking piece questioning the cost-effectiveness of renting cloud GPUs from services like Fly.io for AI development with Elixir. He emphasizes the high costs associated with using cloud GPUs and points out that ownership of physical GPUs, such as an RTX 4090 rig, can be more economical in the long term. Thomas proposes an alternative where developers can use Erlang distribution to connect a production environment to a powerful home PC, thus benefiting from scale-to-zero cost when there are no users, while still maintaining the flexibility of cloud services through the use of Fly.io's FLAME library as a fallback for resiliency and scaling. He juxtaposes the recurring costs of cloud GPUs against the one-time expense (which can also be a tax write-off) and subsequent ownership of a high-performance GPU that can serve multiple purposes, including gaming.