Optimizing Hyper-Q for High Throughput Database Virtualization

G. Mitra and M. Pope share their journey of diagnosing and solving bottlenecks in Hyper-Q, a database virtualization application. Hyper-Q allows legacy SQL databases to be emulated over new cloud databases, facilitating migrations without significant rewrites. The talk covers diagnosing performance issues through profiling and architectural changes to enhance throughput and memory usage. Key improvements included switching to row-level parallelism and asynchronous disk writes, which significantly boosted performance and prevented system crashes. The improved Hyper-Q helps transition legacy data warehouses to cloud solutions efficiently.