What crossing the WASM boundary costs

In a plugin host built on the WebAssembly Component Model, every argument and every return value crosses a boundary between two memory spaces. Most of the time you can ignore that. With request bodies you cannot, and the numbers are sharp enough to change the design.

Bodies as buffers

The first version declared HTTP bodies the obvious way:

record http-request {
    method: string,
    uri: string,
    headers: list<http-header>,
    body: option<list<u8>>,
}

A list<u8> is a buffer. So an upload arrives in the host, is buffered there, copied across the boundary, and materialised again in the guest's linear memory — three copies of the same bytes before the plugin has looked at any of them. Worse, that cost is paid by every plugin, including the ones that never touch the body.

Switching both directions to stream<u8> changed two things:

A body the plugin ignores became nearly free. An 8 MiB request went from 2638 µs to 8 µs. Nothing is copied because nothing is read.

A body the plugin does read got slower. Roughly 4–5× on the marginal cost per MiB: 543 µs against 112 µs for the old bulk copy. Small JSON posts pay tens of microseconds they did not pay before.

That is a real trade, not a free win. It happens to be the right one here, because most plugin routes are status checks and small commands, and the occasional large upload was making everything else pay for its existence. If the workload were mostly large bodies that all get read, the buffer version would win.

Streaming changes what an error can be

Once the host returns status and headers as soon as the guest produces them — which is the point of streaming — a failure halfway through the body arrives after the status line is already on the wire. It cannot become a 500. The guest can trap, the fuel budget can run out mid-body, and the client still sees 200 followed by a truncated response.

So the design has to say what that means. Here, call_http keeps its fuel budget of ten million units, a large body can now exhaust it, and a trap mid-body cannot change an already-sent status. Nothing about that is exotic — it is the same constraint any streaming server has — but moving from buffers to streams quietly converts a category of "return an error" into a category of "the client got half a response".

The sandbox has no clock

The other lesson had nothing to do with performance. Plugins could not sleep. Not "sleeping was slow" — there was no way to express it. The guests target wasm32-wasip2, and the host exposed no timer, so anything time-based simply was not available.

Adding it was not a matter of importing a crate. WASI has to be linked by the host, and putting it into the same world as the plugin interface made the host's bindgen demand a Host implementation and collide with the p3 runtime. The fix was to split worlds: a plugin-guest world that adds wasi:clocks@0.3 on top of the plugin interface, with the clock definition copied from the version of wasmtime-wasi the host actually links, so the two agree.

The general point: a sandbox is defined by what the host hands over. It is easy to think of it as "normal code, but isolated". It is closer to the opposite — nothing exists until it is granted, including capabilities so ordinary you have never had to think of them as capabilities. Time is the one that surprised me. It will not be the last.

Was the Component Model worth it

For a plugin system, yes. Plugins compile once to .wasm and load into a host they were not built with, declare their routes in a spec the host merges, and run under fuel limits with no access to anything not explicitly linked. Getting that from a dynamic library would mean trusting the library.

The cost is that every interface decision is a data-representation decision, made in a WIT file, with performance consequences you only see under load. That is a fair trade for isolation. It is not a free one.