fuzzy/diacritic-tolerant category & Wikidata autocomplete (T426538)

Daanvrrequested to merge
feat/T426538-fuzzy-search into main

Summary

  • Categories: switch searchCategories from action=opensearch (prefix-only) to action=query&list=search&srnamespace=14 (CirrusSearch). CirrusSearch on Commons handles diacritic folding, hyphen tokenization and case-insensitivity server-side — so typing "self portrait" surfaces Category:Self-portraits, "cafe" surfaces Category:Café, etc.
  • Client-side filter: the synchronous matchVocab wrapper now folds both query and candidate through NFD + strip-combining-marks + hyphen/underscore/dash-to-space + lowercase before scoring. This is what unblocks Wikidata items too — wbsearchentities already returns the right hit for "zurich" → Zürich, but the previous text.includes(q) would discard Zürich because the substring isn't literal. Same fold is reused in isKnownCategory so chip-color stays consistent.

No new dependency, no extra per-keystroke API call, 250 ms debounce and the existing 5-min apiCache unchanged.

API research (why CirrusSearch over wbsearchentities-style for categories)

The task explicitly asks "look into the api documentation and the different apis". Candidates considered:

API Diacritic fold Hyphen/space Case Verdict
action=opensearch (current) No No Partial Prefix-only — the root cause
action=query&list=prefixsearch No No Partial Same prefix limitation
action=query&list=search (CirrusSearch) Yes Yes Yes Picked for categories
wbsearchentities (Wikidata) Yes (already in use) Yes Yes Kept as-is for items

Live API verification (Commons, before this MR) confirmed all three failure modes from the task:

  • srsearch=cafe&srnamespace=14 returns Category:Café (position 2)
  • srsearch=self-portraits&srnamespace=14 returns Category:Self-portraits (position 1)
  • srsearch=Lac Leman&srnamespace=14 returns Category:Lac Léman (position 2)
  • wbsearchentities&search=zurich returns Q72 Zurich + Q40313234 Schauspielhaus Zürich

Concrete failing inputs from the task description, now working:

  • "-" vs space: typing "self portrait" finds Category:Self-portraits (was: empty)
  • é vs e: typing "cafe" finds Category:Café and Q-id items containing "café" (was: empty)
  • Case mismatch: "SELF PORTRAIT" matches the same set as "self portrait" (CirrusSearch + client-side fold both case-insensitive)

Phabricator

  • T426538 — better search/suggestions

Test plan

  • Open the Categories editor (chip cell) and type "self portrait" — Category:Self-portraits should appear in the dropdown.
  • Type "cafe" in Categories — Category:Café should appear.
  • Open the Depicts editor (Wikidata items) and type "zurich" — Q72 Zurich should appear (and Schauspielhaus Zürich).
  • Type a known-good prefix like "Mountains of" or "Albert Ein" — ordering should still feel right (exact/startsWith bubble to the top).
  • Existing categories already on a row should keep their (green/red) chip color — isKnownCategory now folds, so previously-okay names still pass.
  • Confirm no console errors and no extra requests per keystroke (still gated by the 250 ms debounce + 5-min apiCache).

🤖 Generated with Claude Code

Merge request reports