Dylan Matthews

Case study · Personally operated tool · synthetic public demo

Quarry

A job-search workflow with explainable matching and AI proposals that change nothing until approved.

The problem

A useful job search needs more than a stream of postings: it needs reasons to pursue an opportunity, visibility into hard requirements, and a way to keep decisions organized as preferences change. Quarry combines a tracking board with an explainable discovery inbox across public employer sources. A conversational assistant proposes search changes for review. The scope is a small personally operated workflow with a separate synthetic preview.

Contribution and AI assistance

Dylan supplied the September 15 brief, including MVP scope and an operating-cost ceiling, and authorized expanded discovery. The September 17 search retarget lives in his own configuration. He operates the private workspace and daily collection. These are documented product and operating actions; implementation assistance is disclosed separately.

How AI contributed

Claude Code implemented the backend and a review/fix pass; Codex built the interface, integration, browser tests and deployment files. Later coding-agent work improved the read model and qualification interpreter, with review corrections before release. Within the product, hosted AI proposes typed configuration changes or supplies bounded posting checks; it cannot apply its own proposal or fetch arbitrary URLs.

  • TypeScript
  • Next.js
  • PostgreSQL

Two consequential decisions

Decision 1

Proposals, not actions

The assistant returns a typed configuration patch. Quarry stores it on the server with its base version and an expiring one-use token. The user reviews the difference and chooses Apply or Cancel, and Quarry rejects stale or replayed proposals. Common commands work locally without a model. The cost is an extra step on every change. Hosted calls stay off until credentials and explicit per-token prices exist, and the worst-case cost of each request is reserved before it is sent.

Decision 2

Deterministic, explainable scoring

Scores come from qualification and keyword rules applied to a confirmed profile. They are not a trained model or a hiring probability. A mandatory experience or degree gap caps the score before salary or watchlist bonuses can raise it. A medical-coding audit job once reached the inbox. In response, the October 1 release added domain disqualifiers, a single quality gate and a score floor of 42 chosen from a labelled golden set. The tradeoff is that the rules need upkeep as postings change.

Verification

  • September 15, 2026 hosted release: 158 automated tests passed, all 10,000 migrated postings and descriptions matched the migration backup, and live checks covered login, wrong-password rejection and 401 responses to unauthenticated API calls.
  • September 24 discovery repair: a read-only audit of 99 reviewable postings found the scorer was missing stated experience years, geography and paraphrased customer-facing duties. After the fix, 684 tests passed with one skipped, and all 16 saved or dismissed decisions survived the rollout.
  • An inspected implementation limit: production collectors do not currently populate the content hash required by the AI final posting check. The preview fills this field separately. This limits that optional check, not the deterministic matcher or configuration-proposal workflow.
  • These are checks recorded in the repository, not a fresh audit for this page.

Outcome and progression

9.96 s → 226 ms

First Discovery visit on the hosted workspace

Same live browser workload on 10,000 postings, before and after the read-model release (September 15, 2026). First load right after a restart still took 5.9 s; full-description search took 1.47–1.71 s.

21 of 21

Postings labelled “yes” that the golden evaluation kept

Real postings exported October 1, 2026 and labelled yes, no or maybe. A small set that catches over-filtering of known cases; not a precision or hiring-outcome measure.

99 → 3 eligible, 6 review, 90 excluded

Offline rescore of the audited review set

September 24, 2026 repair. The named false positives were excluded. These are classification counts, not proof of suitability.

Milestones

  1. 2026-09-15MVP validated locally; hosted read-model release deployed.
  2. 2026-09-17Live search retargeted toward analyst-to-developer bridge roles.
  3. 2026-09-24Discovery repair deployed after a read-only audit of 99 reviewable postings.
  4. 2026-10-01Single quality gate, score floor and labelled golden evaluation set added.

What I would measure next

Weekly precision of “Ready to act”: the share of actionable postings that receive a thumbs-up versus a thumbs-down, using the feedback votes added on October 1. Not yet reported.

Try the workflow

  1. Open the synthetic public preview and select Discovery inbox.
  2. Select Requirements & match details on a card to read its score explanation.
  3. Select Pipeline assistant, enter ‘Set my salary floor to $80,000’, and press Send message.
  4. Inspect the Salary floor difference and cached-posting preview in Review pipeline changes, then choose Cancel.
  5. Open Pipeline studio to inspect configured sources and run history.

Evidence and further reading