Skip to content

Run executors inside of separate tasks.

Cory Massarorequested to merge
apine-fix-some-rust into main

List of changes:

  • fix the Blubber variant to build Rust evaluator from locally-compiled binary (developer experience);
  • run run_function_call inside of a tokio::task, allowing for WASI setup to be done before function execution;
  • replace Arc/Mutex pattern in Executor with channels, simplifying the code and removing one source of async woes.

Motivation: It turns out that we are still incurring the WASI startup costs because Rust Futures don't start until they are awaited. To get the desired behavior, run_function_call has to be called as a separate tokio task.

With this, we do achieve some nice numbers. In Node, the entire fetch call from the orchestrator (i.e., including I/O latency) is about 10 to 20 ms slower IN OPTIMAL CONDITIONS:

node-latency.txt rust-latency.txt

These numbers reflect what happens when the evaluator gets hit with 100 concurrent requests. Note the massive 20-second-or-so delays when the Rust executors are reconstituting themselves. This is apparently due to some kind of blocking behavior that prevents any of the tasks from advancing (it seems that 15 concurrent tasks is the limit, based on the pattern in these numbers.

More experimentation is needed to determine where exactly the blocking is taking place (I think it's because we're using the wrong Mutex in src/executor.rs, so we might need to switch back to an RwLock), but this is a big improvement over the current situation, where every call incurs the WASI startup costs.

Bug: T380809

Bug: T380808

Edited by Cory Massaro

Merge request reports