fetchObject: Cap (per-pod) concurrent requests to external APIs

Jforresterrequested to merge
T282922 into main

As part of our threat modelling, we've been aware that a run-away call (e.g. a composition looping over ZID or Wikidata dereferences) could flood the shared production MediaWiki APIs (the Wikifunctions wiki and Wikidata) and threaten cluster stability. Every limit we have today (RateLimitCache, EvaluatorBudget, the wall-clock deadline) bounds a single request; none bounds a pod's aggregate outbound demand on a host. This adds that missing dimension. I was particularly wanting to implement this ahead of us potentially adding Commons as a third API target in the orchestrator some time soon.

This is done as a per-host concurrency bouncer, owned by the process- invariant OrchestratorConfig and hooked at the single outbound fetch choke point, with the fetch code changed to acquire a slot before each external request; if the host is at its cap, retry briefly within the request's remaining deadline, then reject. Rejection reuses Z570 (already HTTP 429) with the host in the args; 429 not 422 because saturation is aggregate load, not the refused request's own fault.

This is the smallest plausible change that delivers the limit: one choke point already carrying the host and deadline (so it covers both APIs, counts per HTTP call, and excludes the evaluator for free); no new lifecycle (rides the OrchestratorConfig from T426260); and per- pod only, with the acquire()/release interface as the seam a future shared-state backend would replace. Out of scope: cross-pod aggregation, and the per-request outbound budget (T282918).

Note that this provides two new ORCHESTRATOR_CONFIG keys to set:

  • maxConcurrentWikifunctionsRequests (default 256), and
  • maxConcurrentWikidataRequests (also default 256).

These are runaway tripwires, not traffic shapers — normal per-pod concurrency to a host is in the low tens, a runaway is thousands — so we enforce immediately and rely on that gap plus new per-host metrics to catch a mis-set value. We will likely override both per environment in deployment-charts. Note the cap cannot disable limiting (a host is unlimited only when its virtual host is unset) and 0 rejects every call; to effectively disable, set a very high number.

Bug: T282922
Assisted-by: Claude Opus 4.8

Merge request reports