Doomfana: Engineering WebAssembly and Time Series Streaming for Real-Time Game Rendering in Grafana

The intersection of high-performance gaming and observability platforms might initially appear to be a realm of technological incongruity. However, the emergence of the "Doomfana" project—a specialized Grafana datasource developed during the March 2022 Grafana Labs hackathon—demonstrates a profound capability within the modern observability stack. This technical endeavor was not merely a demonstration of novelty but a rigorous stress test of the Grafana Live streaming architecture and the performance boundaries of WebAssembly (WASM) integration within a browser-based visualization environment. Born from the creative impulse to follow the precedent set by previous "Rickrolling" experiments using Prometheus and Grafana, the engineering team sought to answer a fundamental question regarding the limits of data streaming: Can the engine designed for monitoring microservices and infrastructure metrics be repurposed to render a fully functional, real-time instance of the classic game Doom?

The project, spearheaded by software engineers Bogdan Matei, Kostas Pelelis, and Domas Lapinskas, utilized the WebAssembly-compiled version of Doom to function as a custom Grafana datasource. By leveraging the Grafana Live feature, the implementation transformed the traditional time series panel—typically used for tracking CPU utilization or request latency—into a rendering canvas. In this configuration, the game state is translated into a high-frequency stream of data points, where each color in the game's resolution is represented as an individual time series. This architectural approach pushed the limits of the Grafana Live streaming feature, testing the platform's ability to handle massive influxes of concurrent, high-frequency data updates without compromising the stability of the browser's main thread or the accuracy of the observability dashboard.

The Architectural Framework of Doomfana

The development of Doomfana required a complex decision-making process regarding the distribution of computational load between the client and the server. The engineering team evaluated two distinct architectural patterns for executing the game engine within the Grafana ecosystem. The first possibility involved a server-side execution model where the game logic would reside on a backend server, transmitting processed frames to the client via WebSockets. While this would centralize the computational heavy lifting, it introduced significant overhead in terms of bandwidth and-server-side resource consumption. Specifically, this approach required extensive C code implementation to ensure that individual frames could be safely and efficiently transported through WebSocket protocols to the connected clients.

The second, and ultimately selected, architectural pattern utilized Doom compiled to WebAssembly (WASM) to run directly within the user's browser. By executing the game engine on the client side, the developers could leverage the local processing power of the user's hardware, significantly reducing the server-side burden of frame transmission. This method allowed the plugin to fetch data through the WASM interface, essentially treating the game engine as a highly dynamic, real-time data producer. This shift to client-side execution was pivotal in achieving the necessary frame rates required for a playable experience, even while the data was being funneled through the structured pipelines of a Grafana datasource.

The structural components of the project involve a specialized datasource that interfaces between the WASM-based game engine and the Grafana visualization layer. This relationship is defined by the following technical layers:

  • The WebAssembly Engine: A compiled version of the Doom engine capable of executing within a browser sandbox.
  • The Grafana Datasource: A custom plugin that acts as the bridge, translating game-state changes into a format recognizable by Graf and its time series panels.
  • The Grafana Live Layer: The streaming backbone that facilitates the real-time delivery of data points from the engine to the visualization panels.
  • The Visualization Panel: A configured time series panel that interprets incoming data as color-coded dots representing pixels.

Technical Specifications and Performance Metrics

The rendering of Doom within a Grafana dashboard is not a standard video playback but a sophisticated reconstruction of game frames using time series data. To maintain a semblance of playable performance, the developers utilized a "half resolution" mode. This reduction in resolution is a critical optimization that allows the system to manage the massive volume of data points without crashing the browser's rendering engine.

The following table outlines the specific data density and performance metrics observed during the implementation of the half-resolution mode:

Metric Value Impact on System Performance
Active Time Series 256 Each time series corresponds to a specific color in the rendering palette.
Data Point Volume 16,000 The total number of individual data points rendered per frame.
Target Frame Rate ~30 FPS The frequency of updates required to maintain visual fluidity.
Rendering Mode Half Resolution Reduces the pixel grid to 100x160 to manage computational load.

The consequence of managing 256 active time series, each receiving updates 30 times per second, is an extreme demand on the browser's JavaScript engine and the Grafana frontend. This high-density data environment serves as a perfect laboratory for identifying performance bottlenecks in real-time monitoring tools. If the data pipeline is not optimized, the accumulation of unrendered frames can lead to significant memory leaks and eventual browser freezing.

Implementation and Build Orchestration

Building and deploying the Doomfana plugin requires a precise sequence of dependency installations and compilation steps. Because the project relies on Emscripten to bridge C/C++ code to WebAssembly, the environment must be configured with specific toolchains. The build process involves compiling the original Doom source code, integrating the game assets (WAD files), and then packaging the result as a functional Grafana plugin.

The initial environment setup requires a package manager like Homebrew to install the necessary low-level dependencies:

bash brew install emscripten automake sdl2 sdl2_mixer sdl2_net pkg-config

Once the toolchain is established, the integration of the game assets must be performed manually. The core game data, contained within the doom1.wad file, must be placed within the specific source directory of the WASM implementation to ensure the engine can access the necessary textures, levels, and sprites.

```bash

Move the WAD file to the correct source directory

cp ./doom-wasm/src/doom1.wad ./doom-wasm/src/
```

After the assets are in place, the build pipeline follows a multi-stage process involving cleaning the previous build artifacts and re-compiling the C code into WASM. The following commands are essential for the compilation phase:

bash ./doom-wasm/scripts/clean.sh ./doom-wasm/scripts/build.sh

Following the compilation of the engine, the resulting WebAssembly binaries and their corresponding source maps must be moved into the Grafana plugin's distribution folder. This ensures that the Grafana frontend can load the executable logic:

```bash

Copying the compiled assets to the plugin source directory

cp ./doom-wasm/src/doom1.wad ./src/img/
cp ./doom-wasm/src/websockets-doom.wasm ./src/img/
cp ./doom-wasm/src/websockets-doom.wasm.map ./src/img/
```

The final stage of the build involves the JavaScript layer of the plugin. Using Yarn, the developer must install the Node.js dependencies and execute the production build command to generate the final, optimized plugin files ready for deployment in a Grafana instance:

bash yarn install yarn build

To complete the installation, the plugin must be made visible to the Grafana server. This is achieved by creating a symbolic link from the project's root directory to the official Grafana plugins folder:

bash ln -s /path/to/doomfana-project-root /path/to/grafana/data/plugins/doomfana

After the link is established, the user must navigate to the Grafana UI to create a new "Doom" datasource and take note of its unique ID, which is required for configuring the panels.

Optimization Strategies for High-Frequency Time Series

The development of Doomfana provided the Grafana Labs team with invaluable insights into optimizing the Grafana Live streaming feature. When dealing with the massive throughput required for 30 FPS rendering, standard data-handling practices are insufficient. The engineers identified several critical optimizations that can be applied to any high-frequency observability dashboard to prevent performance degradation.

One of the primary findings was the importance of field stability. In Grafana, mutating the configuration of fields within a data frame is an expensive operation. To optimize performance, the list of fields should be kept constant; instead of adding or removing fields, the system should only update the values within the existing field structure. This prevents the expensive reconfiguration cycles that occur when the UI attempts to re-map the data schema.

Furthermore, the method of data construction significantly impacts the speed of the rendering loop. When assembling data frames in JavaScript, the engineers discovered that initializing a fixed-length array and then assigning values via index is substantially faster than using the Array.push method. This is due to the reduced overhead in memory reallocation that occurs when an array grows dynamically.

The following list details the critical performance optimizations identified during the hackathon:

  • Maintain a constant list of fields to avoid expensive configuration updates in the UI.
  • Utilize fixed-length arrays and index-based assignment instead of Array.push when building data frames.
  • Configure explicit minimum and maximum Y-axis values in the panel settings to prevent Grafana from recalculating axis scales on every frame.
  • Synchronize the pushing of new frames with the window.requestAnimationFrame tick to ensure smooth rendering aligned with the browser's refresh rate.
  • For Chrome users, ensure that "Canvas 2d out of process rasterization" is enabled via the chrome://gpu interface to offload rendering tasks to the GPU.

These optimizations are not merely applicable to gaming; they are essential for any enterprise-scale implementation of real-time streaming, such as monitoring high-cardinality microservices or live IoT sensor networks where data volatility is high.

Future Trajectories in Observability Visualization

While the successful execution of Doom within Grafana is a landmark achievement for the hackathon, the project remains an ongoing endeavor. The engineering team has outlined several future technical goals intended to further bridge the gap between traditional observability and immersive data visualization.

One significant area of focus is the expansion of the data integration. The current implementation focuses on rendering pixels, but the team aims to bring all relevant Doom statistics—such as player health, ammunition levels, and enemy counts—into dedicated Grafana panels. This would allow for a dual-view experience: a visual rendering of the game state alongside structured, queryable metrics.

Additionally, the team is exploring the integration of the new Grafana Heatmap panel as a potential alternative to the time series panel. A heatmap could potentially offer a more efficient way to visualize high-density pixel data by aggregating color distributions, thereby reducing the number of active time series required. There is also interest in implementing a recording feature, which would allow users to capture high-fidelity streams of the gameplay for later playback and analysis, essentially creating a "DVR" for observability data.

The integration of technologies like Grafana Mimir also presents an opportunity. As Mimir provides robust, long-term storage and high-scale querying for Prometheus-compatible metrics, it could eventually be used to store historical "gameplay" metrics, allowing for the analysis of trends in the data stream over much longer durations than a standard live session.

Analysis of the Engineering Achievement

The creation of Doomfana represents a paradigm shift in how we perceive the utility of observability tools. By successfully repurposing a monitoring platform to host a real-time, compute-intensive application, the Grafana Labs team has demonstrated that the boundaries between "data visualization" and "graphics rendering" are increasingly porous. The technical achievement lies not just in the successful compilation of the Doom engine to WebAssembly, but in the sophisticated orchestration of the Grafana Live streaming architecture to handle the resulting data deluge.

The project serves as a rigorous benchmark for the scalability of the Grafana ecosystem. The ability to manage 256 active time series at 30 frames per second proves that the underlying streaming infrastructure is capable of handling extreme workloads, provided that the data-handling strategies—such as fixed-length arrays and constant field schemas—are strictly followed. Furthermore, the project highlights the importance of the WebAssembly ecosystem in modernizing web-based applications, allowing for the execution of legacy C/C++ logic with near-native performance within the browser.

Ultimately, Doomfana is a testament to the power of the "hackathon" culture in driving technical innovation. It took a tool designed for the sober, analytical task of infrastructure monitoring and pushed it into the realm of high-performance entertainment, uncovering critical optimization patterns that will undoubtedly benefit the wider community of engineers building real-time, high-cardinality observability pipelines.

Sources

  1. GitHub: grafana/doom-datasource
  2. Grafana Blog: Can Grafana run Doom?
  3. Grafana Events: Hackathon Showcase

Related Posts