Keep processing TK experiments when GrowthBook is unavailable

Clare Mingrequested to merge
T428222/preserve-tk-experiments-when-gb-n/a into main

What?

Adds a gbAvailable flag through poll >> process >> store so the pipeline degrades gracefully:

  • poll() catches fetchExperiments errors, passes { experiments: [] } and { gbAvailable: false } to processConfig, and increments growthbook_polls_total{outcome="partial"} instead of calling it a fail
    • The last success timestamp gauge only proceeds on full success
  • process() skips validateAllExperiments and adaptAllExperiments when gbAvailable is false, builds no per-experiment preservation set, and passes preserveAllGb: true to store()
  • store() adds .whereNot('source', 'growthbook') to its delete query when preserveAllGb is set so existing GB-source rows survive the outage while TK rows keep being inserted, updated, and deleted normally

Distinguishes three poll outcomes for alerting:

  1. outcome="success": full cycle ok - sets last-success timestamp
  2. outcome="test_kitchen_only": GB down, TK processed - does not set timestamp
  3. outcome="failure": TK load or store threw - does not set timestamp

The existing saved-groups preservation stays the same. The new preserveAllGb flag runs along with it (both clauses are chained via knex).

Why?

A failed GrowthBook fetch previously aborted the whole poll cycle, so TK didn't get processed either.

Bug: T428222

Edited by Clare Ming

Merge request reports