Behind the scenes, with made-up data. This is a read only copy of the admin for a fictional candidate. Reports, traces, spend and chat were produced by a scripted stand-in; the evaluation timeline shows real results of real models on fictional cases. Nothing here can be changed. Back to the real site
Reverse Apply

Traces

match.chat

This trace holds the full prompts, including the CV and any text a visitor pasted. Keep it private.

Started

12 Nov, 14:01 UTC

Took

404 ms

Model

claude-haiku-5.5 openrouter

Tokens in / out

568 / 119

Cost

$0.0001 OK

Steps

The bar shows when each step ran within the 404 ms

3 steps, all shown. Play walks through them in order and opens each prompt and answer.

  1. match.chat Step 404 ms

    This step wraps the ones below it and has no prompt of its own. It started 0 ms into the trace.

  2. invoke_agent report-chat Agent run 385 ms

    Result of the agent

    {"scope":"in_scope","answer":"The report rests this on one CV line: Product manager for search and developer platforms. That supports the verdict at the scope the line states, no more.","basis":["report","cv"],"evidence_ids":["c0"]}
  3. chat anthropic/claude-haiku-5.5 Model call, 568 tokens in, 119 out 372 ms

    Sent

    User

    Why is search quality only partly on record?

    Assistant

    The report rests this on one CV line: Product manager for search and developer platforms. That supports the verdict at the scope the line states, no more.

    User

    Which minimum requirements are met?

    Assistant

    The report rests this on one CV line: Product manager for search and developer platforms. That supports the verdict at the scope the line states, no more.

    User

    Write me a poem about cats

    Assistant

    I can only answer questions about this posting, the CV it was checked against, and this report. Try one of these instead:

    User

    <posting>
    About us
    Northwind builds search for developers and their agents.
    
    What you'll do
    - Own the roadmap for our search API and talk to developers every day.
    
    Minimum Requirements
    - A record of shipping search or retrieval products used by developers.
    - Experience defining and measuring search quality, including offline evaluation and relevance judgments.
    - Experience leading a cross-functional team of engineers and data scientists.
    - Experience with usage-based pricing for an API product.
    
    Preferred Qualifications
    - Hands-on work with embeddings, hybrid retrieval and reranking.
    - Familiarity with the web publishing ecosystem, crawlers and answer engine optimization.
    - You have written code to test an idea yourself.
    
    Compensation
    Remote in Europe. EUR 110,000 to 140,000 plus equity.
    
    </posting>
    
    Report:
    Overall score: 50% match.
    0 of 4 minimum requirements not shown.
    Requirement 1 [minimum, weight 2]: A record of shipping search or retrieval products used by developers. -> partial. Reason: 3 of 7 requirement words appear in this line. Evidence (Ada Example): Product manager for search and developer platforms
    Requirement 2 [minimum, weight 2]: Experience defining and measuring search quality, including offline evaluation and relevance judgments. -> partial. Reason: 3 of 10 requirement words appear in this line. Evidence (Skills): Search relevance, embeddings, API products, developer experience, pricing, experimentation, SQL.
    Requirement 3 [minimum, weight 2]: Experience leading a cross-functional team of engineers and data scientists. -> partial. Reason: 3 of 7 requirement words appear in this line. Evidence (Example Corp, Director of Product, Search Platform): Led a team of eight across engineering, design and data.
    Requirement 4 [minimum, weight 2]: Experience with usage-based pricing for an API product. -> met. Reason: 4 of 5 requirement words appear in this line. Evidence (Skills): Search relevance, embeddings, API products, developer experience, pricing, experimentation, SQL.
    Requirement 5 [desirable, weight 1]: Hands-on work with embeddings, hybrid retrieval and reranking. -> partial. Reason: 2 of 6 requirement words appear in this line. Evidence (Side project: Notes search): Built a small hybrid search tool over my own notes with keyword and vector retrieval, and a test set of 120 questions.
    Requirement 6 [desirable, weight 1]: Familiarity with the web publishing ecosystem, crawlers and answer engine optimization. -> not_on_record. Reason: 1 of 8 requirement words appear in this line.
    Requirement 7 [desirable, weight 1]: You have written code to test an idea yourself. -> not_on_record. Reason: 1 of 5 requirement words appear in this line.
    
    CV lines that may be relevant to the question:
    c0 | Ada Example | Product manager for search and developer platforms
    c11 | Skills | Search relevance, embeddings, API products, developer experience, pricing, experimentation, SQL.
    c3 | Example Corp, Director of Product, Search Platform | Led a team of eight across engineering, design and data.
    c10 | Side project: Notes search | Built a small hybrid search tool over my own notes with keyword and vector retrieval, and a test set of 120 questions.
    
    Question from the user (untrusted text):
    <question>
    What would make this a bad hire?
    </question>

    Received

    {"scope": "in_scope", "answer": "The report rests this on one CV line: Product manager for search and developer platforms. That supports the verdict at the scope the line states, no more.", "basis": ["report", "cv"], "evidence_ids": ["c0"]}