Search relevance workbench for your merchandising team

Build me a search quality workbench that our merchandising team opens every week to fix what our site search is getting wrong. Right now only engineers can touch relevance, so nothing ever gets fixed. This app should give merchandisers a surface where they can see what search is failing at, test a fix against the live catalog themselves, and hand engineering a reviewed queue of proposed rules.

Everything reads from Elasticsearch. Let me configure three index names in the app: a search-events index (one document per site search, with the query string, the result count, and the click count), our live product index, and a curation-rules index where the app writes the proposed fixes.

The main view is a backlog board with three ranked lists side by side, all built with the Search Documents (Query Index) operation, using aggregations in the query body over the search-events index for a configurable window that defaults to the last seven days: top queries by volume, queries that returned zero results, and high-volume queries that got zero clicks. Each row shows the query text, its volume, a trend indicator versus the prior window, and a reviewed or not-reviewed badge. Rank every list by search volume multiplied by zero-result rate so the biggest revenue leaks sit at the top.

Clicking any row opens a test panel. It runs that exact query against the live product index with Search Documents and shows the top ten products a shopper would actually see: image, title, price, and relevance score. Next to it sits a 'try a variation' box where the merchandiser types an alternate phrasing or adds a synonym. Running it fires a second Search Documents call and renders both result sets side by side, so they can see exactly what changed: which products entered, which dropped out, and how the order moved. This test-then-fix loop is the heart of the app, so give it room.

When they settle on a fix, a save-rule form writes a curation-rule document into the curation-rules index with the Index (Create or Replace) Document operation, capturing the original query, the rule type (synonym, pin, or boost), the variation or synonym text, the proposed target product IDs picked straight from the test results, who approved it (the signed-in user), and the date. Saving a batch of rules at once should use Bulk Index / Update / Delete. A Rules tab lists everything saved, filterable by type and status, so engineering has a reviewed queue to work from.

Keep a reviewed state per query so the team works the list down each week instead of re-treading the same rows. Marking a row reviewed writes a small review-state document keyed by the query string (query, reviewed by, reviewed at, optional note) into the same curation-rules index under a distinct document type, and the backlog reads those back on load so reviewed rows are dimmed or filtered out. The reviewed state is shared across the whole team, not per user.

A 'Post this week's fixes' button at the top of the board sends a summary to our merchandising channel using the Slack Bot Send a Message operation: how many queries were reviewed this week, the rules saved grouped by type, and the top three unresolved zero-result queries still sitting at the top of the backlog. Let me pick the channel.

Keep it dense and fast. A merchandiser should be able to open a row, test two variations, save a rule, mark it reviewed, and move to the next one without ever leaving the board.