From Web 3D Engine Trade-offs to Consulting Graphics Veteran Sebastian Aaltonen: Exploring the Feasibility of Bringing NoGraphicsAPI to WebGPU

Abstract: In my previous post, I recapped a personal weekend project exploring a high-performance hybrid WebGPU runtime prototype built on Google Dawn and WKWebView, leveraging an EmscriptenGLX-like C++ static linking mechanism to achieve a steady 72 FPS with a single IPC dispatch per frame. However, while experimenting with AI-driven game generation, I encountered fresh architectural trade-offs: LLMs strongly favor JavaScript-based Three.js, which cannot benefit from native static .a linking; meanwhile, Unity is mature and delivers stellar performance, but as a full-featured industrial engine, it is relatively heavy for lightweight instant-play scenarios, and automating its comprehensive editor and project workflows via AI agents is quite complex.

> Seeking a lighter, code-first, and modernized graphics foundation, I came across NoGraphicsAPI, shared at SIGGRAPH 2026 by legendary graphics architect Sebastian Aaltonen (co-founder & technical director at Ubisoft RedLynx, former Unity senior rendering research lead, co-founder of Second Order, and HypeHype rendering lead). It targets modern Vulkan and Metal, without WebGPU support.

> Intrigued by the possibilities, I reached out to Sebastian to ask whether WebGPU was on his roadmap, leading to an in-depth technical discussion on modern low-level hardware abstractions versus web sandboxing constraints. This article documents our exchange, dissects the architectural gaps of WebGPU, and explores a potential path forward using Google Dawn and Slang.

---

1. Context & Dilemma: Engine Choices in the Age of AI Game Generation

In the previous post, I introduced a client-side graphics runtime prototype explored as a personal side project using Google Dawn and WKWebView. The architecture implements a hybrid container model reminiscent of WeChat Mini Games, centered around an EmscriptenGLX-like mechanism:

flowchart LR
    subgraph WASM ["WebAssembly / C++ Container"]
        Logic["Game Logic"] -->|Direct Call| WireClient["Statically Linked Wire Client (.a)"]
    end

    subgraph NativeHost ["Native Host Process (Dawn Native / Metal)"]
        WireServer["Dawn Wire Server"] --> DawnMetal["Metal Hardware Pipeline"]
    end

    WireClient -->|Single IPC Binary Submit per Frame| WireServer

By statically linking the Dawn Wire client library (.a) directly into the WebAssembly binary, command serialization is completed entirely in C++, bypassing the browser's JavaScript WebGPU wrappers and garbage collection (GC) overhead. This yielded a solid 72 FPS with steady-state single-dispatch IPC per frame.

However, as I ventured deeper into autonomous AI-generated games (AI Game Generation), an unexpected architectural dilemma surfaced:

1. Three.js: AI's Favorite, but an Architectural Island

Yet Three.js is pure JavaScript. It cannot plug into our native .a static linking pipeline, having to fall back to the browser's standard navigator.gpu interface (or a passive hook layer). Furthermore, Three.js's heavy scene graph traversals, dirty-flag evaluations, and frequent JS garbage collection introduce single-thread CPU bottlenecks under demanding rendering workloads.

2. Unity: Production-Grade and Mature, but Complex for Lightweight AI Generation

• Full-featured, heavy footprint: Engineered for commercial AAA and mid-core development, exported WebAssembly packages and runtime heaps are relatively heavy for instant-loading, lightweight playables;
• Complex automation workflows for AI: Unity features a powerful visual editor and asset pipeline (Inspector, Prefabs, Asset Database metadata, component lifecycles). While highly productive for human engineers, orchestrating this multi-tiered editor and asset ecosystem end-to-end via AI agents is significantly more complex than outputting self-contained code.

This spurred me to look for a different kind of graphics foundation:

• Code-First: Scenes and rendering defined purely in code, free of monolithic editor asset databases, naturally tailored for AI generation;
• Minimal Ceremony: Bypassing legacy pipeline abstractions to drive modern GPU hardware with minimal CPU overhead;
• Cross-Platform & Web-Friendly: Capable of full-throttle native performance while retaining the architectural viability to run on WebGPU / WASM.

This search led me to NoGraphicsAPI, a new open-source project by Sebastian Aaltonen.

---

2. Discovering NoGraphicsAPI: Sebastian Aaltonen's Radical Philosophy

Sebastian Aaltonen is a celebrated pioneer in real-time computer graphics (co-founder & technical director at Ubisoft RedLynx, former senior rendering research lead at Unity, co-founder of Second Order, and rendering lead at HypeHype). At SIGGRAPH 2026, he unveiled the core philosophy behind NoGraphicsAPI.

The core premise is embedded in its name: "No Graphics API" — eradicating the cumbersome boilerplate and intermediate abstractions of modern graphics APIs.

flowchart TD
    subgraph Traditional ["Traditional Graphics APIs (Vulkan / WebGPU)"]
        direction TB
        T1["Vertex Buffer & Vertex Layout declarations"]
        T2["Descriptor Pool / Descriptor Set / BindGroup"]
        T3["PipelineLayout slot tracking & hashing"]
        T4["Fine-grained Barrier / Image Layout state machines"]
    end

    subgraph NoGraphics ["The Minimalist NoGraphicsAPI Paradigm"]
        direction TB
        N1["GPU is just a computer with memory"]
        N2["64-bit Virtual GPU Pointers (Buffer Device Address)"]
        N3["Global Texture Heap indexing (Fully Bindless)"]
        N4["Single Root Argument struct pointer per draw"]
        N5["Coarse-grained Global Memory Barriers"]
    end

    Traditional -->|Philosophical Shift & Paradigm Reset| NoGraphics

In Aaltonen's view, modern GPU architectures have converged to the point where micro-managing binding slots like a 20-year-old fixed-function pipeline is obsolete. NoGraphicsAPI rests upon four core pillars:

1. 64-bit GPU Shaders Pointers (Buffer Device Address): Shaders (written in Slang) use 64-bit pointers directly to read vertices, materials, skinning palettes, and bounding boxes, completely eliminating vertex layout declarations and explicit buffer bindings;
2. Fully Bindless: A single global texture heap and sampler heap; shaders sample textures directly via integer indices;
3. Single Root Arguments: Every draw or dispatch passes a single root struct (via push constants or root pointers);
4. Persistently Mapped CPU Writes: Memory is mapped once on allocation; the CPU writes via *root.cpu = ... and the GPU reads immediately upon submission, with no explicit copy queues or unmap state transitions.

This paradigm aligns remarkably well with what an "AI-native engine" would desire: no binding ceremony, just clean data structures, pointer math, and Slang shaders.

Looking at the repository, however, NoGraphicsAPI only targets Vulkan 1.4 and Metal 3/4. WebGPU was nowhere to be found.

Curious whether there was any plan to support WebGPU, I sent a private message to Sebastian Aaltonen to ask about his perspective.

---

3. Reaching Out: Sebastian Aaltonen Points Out the Three Crucial Blockers

Sebastian responded promptly and candidly, highlighting three fundamental architectural discrepancies where WebGPU diverges from modern hardware trends:

"NoGraphicsAPI is fully bindless (no texture bindings), uses 64-bit GPU pointers in shaders and allows CPU go write directly to persistently mappy GPU pointers. WebGPU has none of these 3 crucial features. WebGPU feature set is similar to 10 year old Vulkan 1.0. No bindless or descriptor indexing, precreated descriptor sets (bind groups) for bindings and driver managed CPU->GPU copy queue. The bind groups are immutable and copy queue doesn’t allow direct GPU read without a copy command. It is even more limited in those ways than Vulkan 1.0."

Aaltonen objectively articulated three fundamental barriers separating NoGraphicsAPI from WebGPU, observing that WebGPU's 1.0 specification remains notably conservative compared to modern native APIs:

Technical Deep Dive: Why is WebGPU So Conservative?

Developers arriving from native platforms (DirectX 12, Vulkan, Metal) often wonder why WebGPU omits features hardware has supported for years.

The divide stems from the tension between raw hardware efficiency and the browser security model:

1. Why does the Web prohibit raw GPU pointers (BDA)? In a web browser, the GPU is a shared physical device across all tabs, iframes, and the underlying OS. If shaders were permitted to dereference arbitrary 64-bit virtual addresses, a malicious page could construct an out-of-bounds pointer to snoop on memory from adjacent tabs (passwords, banking sessions, decrypted images) or trigger sandbox escapes. WebGPU mandates Robust Buffer Access (guaranteed out-of-bounds clamping or zero-fills), an invariant that cannot be enforced efficiently on arbitrary 64-bit pointers without software MMU emulation.
2. Why no persistent mapping without unmap? In modern browser architecture, web code runs inside a sandboxed Renderer Process, while the graphics driver resides in an isolated GPU Process. Exposing a persistently writable pointer during GPU command execution opens a critical TOCTOU (Time-of-Check to Time-of-Use) attack surface: an attacker could tamper with offsets or descriptors in shared memory immediately after validation passes, deceiving the GPU into out-of-bounds reads. WebGPU enforces strict mutual exclusion via unmap() to sever CPU access before GPU dispatch.

---

4. Digging Into Dawn: Deductions & Potential Workarounds

Does this mean the NoGraphicsAPI paradigm is fundamentally precluded from ever running on WebGPU?

To explore whether practical bridges exist, I investigated the source tree of Chromium's WebGPU implementation — Google Dawn — and found that inside Dawn's experimental branches and upstream proposals, the landscape is shifting.

1. The Bindless Horizon: Kangz's Resource Table

• ChromiumExperimentalSamplingResourceTable: Dawn introduces a concept akin to DirectX 12 descriptor heaps or Metal argument buffers — the Resource Table. In WGSL, by enabling the experimental feature:

    enable chromium_experimental_resource_table;
  
    @fragment
    fn fs_main() -> @location(0) vec4f {
        // Say goodbye to explicit @group(0) @binding(1) var my_texture declarations!
        let tex = getResource<texture_2d<f32>>(root.diffuse_texture_index);
        let smp = getResource<sampler>(root.sampler_index);
        return textureSample(tex, smp, vec2f(0.5, 0.5));
    }
    

  On the C++ host side, creating and binding the table once globally is all that is required:

    passEncoder.SetResourceTable(globalResourceTable);
    

  This directly overcomes the first major hurdle (Fully Bindless)!

2. Emulating 64-bit Pointers: Mega-Buffer + Programmable Vertex Pulling

• Zero Vertex Layout Declarations: The pipeline is created with an empty vertex.buffers array;
• Programmable Vertex Pulling: Inside the vertex shader, vertex attributes are retrieved manually from the Mega-Buffer using the built-in @builtin(vertex_index):

    fn load_vertex(byte_offset: u32) -> Vertex {
        // Convert byte offset to u32 array index (byte_offset / 4)
        let base = byte_offset >> 2u;
        return Vertex(
            vec3f(bitcast<f32>(mega_buffer[base + 0u]),
                  bitcast<f32>(mega_buffer[base + 1u]),
                  bitcast<f32>(mega_buffer[base + 2u]))
        );
    }
    

• Dawn ImmediateData (Push Constants): For each draw call, Dawn's pass.SetImmediates(0, &root, sizeof(root)) passes the Root struct (holding buffer offsets, texture indices, and transform matrices) in a single ultra-low-overhead call.

3. CPU Upload Strategy: Host Bump Allocator + Batched WriteBuffer

• Implement a lightweight linear allocator (Bump Allocator) in standard host memory on the CPU side;
• Gameplay and scene logic write directly to plain system RAM throughout the frame;
• Just prior to submitting the command pass, invoke device.GetQueue().WriteBuffer(...) or utilize a staging ring buffer to upload the frame's dynamic payloads to the GPU Mega-Buffer in a single batched operation.

Takeaway: While literal 64-bit physical pointers cannot breach web sandboxing, pairing Dawn's Resource Table + Programmable Vertex Pulling + ImmediateData + Batched CPU Staging yields a highly efficient rendering model that captures the core benefits of NoGraphicsAPI (no binding tables, no vertex layouts, zero BindGroup thrashing, and ultra-lean root argument dispatch).

---

5. Further Exchange: Aaltonen's Feedback and the HypeHype Legacy

I compiled these architectural deductions, Dawn prototype findings, and proposed workarounds, and followed up with Sebastian Aaltonen.

Sebastian reviewed the technical breakdown and provided insightful feedback, affirming the viability of the approach while shedding light on key compiler considerations and prior production experience:

"Depends whether Slang pointers translate to wgsl in a meaningful way. Definitely don’t want to wrap all memory accesses in macros. That would be bad user experience. Shaders are filled with memory accesses. Wgsl has pointers already, but not pointers to GPU memory ATM. Need to investigate futher what would the Slang path be. Currently you can write one Slang shader with pointers and global texture heap indexing and it works fine on both vulkan and metal, allowing shipping to win, mac, ios, android, linux at once.

> Yeah, if we can get pointers working in WGSL, then we can port with the new bindless extension. But data upload from CPU would need a completely new path. No direct GPU pointers mapped for CPU. That would be much closer to the API I wrote for HypeHype. It supported WebGPU. Unfortunately I can't open source it, since I don't own it."

flowchart TD
    subgraph AaltonenView ["Sebastian Aaltonen's Core Insights"]
        A1["1. Developer ergonomics are non-negotiable:\nReject wrapping memory accesses in ugly macros"]
        A2["2. Slang compiler lowering is key:\nPointers must lower cleanly into idiomatic WGSL"]
        A3["3. Unified multiplatform shader codebase:\nSingle Slang shader targets Win, Mac, iOS, Android, Linux"]
        A4["4. Proven by HypeHype's production architecture:\nCPU uploads require a dedicated staging path, already validated in practice"]
    end

1. Why "Macro Wrapping" is an Ergonomic Red Line

Aaltonen emphasized that this is a poor developer experience. Modern complex shaders (PBR materials, compute skinning, ray queries, post-processing) are dominated by memory operations. Turning code into:

// 💥 The macro hell experience:
vec3 pos = LOAD_FLOAT3(g_MegaBuffer, pVertex + OFFSET_OF_POS);
Material mat = UNPACK_MAT(g_MegaBuffer, pMat);

The linchpin is the Slang compiler itself: Slang needs to compile high-level shaders featuring native pointer semantics down into WGSL by automatically and cleanly lowering pointers into valid, performant storage buffer loads.

2. The HypeHype Architectural Precedent

He confirmed that for platforms lacking direct host-visible persistent pointers, CPU uploads must be restructured through a dedicated staging/copy path (similar to the host bump allocator + batched staging strategy discussed above). This proves that our deduction is not merely theoretical, but has been proven in commercial production.

---

6. Outlook: What Kind of Rendering Engine Does the AI Era Need?

Reflecting on this investigation across WebGPU specifications, experimental Dawn internals, and discussions with Sebastian Aaltonen, an intriguing trajectory emerges.

1. Why the NoGraphicsAPI Model is a Sweet Spot for AI Generation

• Three.js: Trivial for LLMs to generate rapidly, but faces CPU, GC, and memory ceilings in complex scenes;
• Full-featured Engines (such as Unity): Extremely mature and feature-complete for production games, but orchestrating their extensive visual editors and asset pipelines autonomously via AI agents remains complex;
• The NoGraphicsAPI Paradigm:
• Minimal State: Exactly 1 global pipeline layout, no BindGroup juggling, zero descriptor pool tracking;
• Plain Data as ABI: Standard C++ structs act as Root Arguments, dispatched with direct memcpy semantics;
• Pure Code Definition: Readily understood, generated, tested, and refactored by LLMs, significantly reducing hallucinations caused by disparate visual configuration files.

2. Hopes for an Experimental WebGPU Backend

I hope to see (and contribute toward) an experimental WebGPU backend for NoGraphicsAPI in the near future:

1. Unified Shader Assets: Authoring clean Slang shaders that compile to Vulkan and Metal for native builds, while seamlessly lowering to WebGPU for web deployment;
2. Pure Data-Driven Rendering: In WebAssembly environments, eliminating massive engine dependencies in favor of a lean, "zero-ceremony" runtime that brings modern rendering principles to the browser and hybrid application containers.

This journey is not only about returning to the fundamental simplicity of hardware, but also about laying the groundwork for lightweight, AI-native graphics programming.