Case Study
Evidence-based ATS
We built an applicant tracking system for small in-house recruiting teams, with three AI features that each have to prove their output before anyone sees it. A notetaker drafts the interview scorecard from raw notes, a question box answers plain questions about the hiring pipeline, and a screening summary sums up each candidate in sentences that cite their source. The model runs on the recruiter's own computer, so candidate data never leaves the machine and no call costs anything. Every quote is checked word for word against its source, every number comes from code rather than the model, and a person makes every hiring decision.
- of interview evidence filed correctly
- 82%
- pipeline questions read right
- 26/26
- unanswerable questions refused
- 6/6
- invented quotes on a scorecard
- 0
The Challenge
Hiring is where a confident wrong sentence does the most damage. A made-up quote on a scorecard is a made-up reason to reject a person, and rules on automated hiring tools keep spreading, from New York City's Local Law 144 to the EU AI Act and Colorado's AI Act. Most recruiting software adds AI as a box that answers in a sure voice and leaves the checking to the reader. The goal was AI that saves real time on interview notes, reporting and screening, without ever putting an unchecked claim in front of a hiring manager.
Our Approach
We started each feature from one question: what does the model have to prove before its output reaches a screen? Four rules hold across the product. The model proposes, and plain code decides what is shown. Anything the checks throw away stays on screen, struck through, with the reason in words. The person keeps every decision, so the model can suggest a rating but never select one, and a summary never recommends. And the model runs locally through Ollama, free per call and fully optional, with a fallback that works without it and says which path produced what is on screen. Instead of a vague AI label, the interface says "Drafted by qwen3.6, running on this computer", so readers know exactly what wrote the text and where their data went.
Scorecard Drafts
The interviewer pastes raw notes, and the model drafts up to three quotes per attribute, a suggested rating and a one-line reason. A schema limits the model to the attributes in the interview kit, so it cannot invent one. Every quote is then matched against the notes. The match forgives changes in case, spacing and quote marks, but not paraphrase, and a quote under three words fails because it proves nothing. Quotes that touch a trait protected by US employment law, such as age, family or health, are held back with the reason shown. A suggested rating only survives when a real quote supports it, and the rating buttons stay empty until the interviewer picks one.
Pipeline Questions
Recruiters can ask plain questions such as "which source turns into hires most often?" The model only reads the question. It picks a metric and filters from closed lists built from live data, and the same code that draws every chart computes the answer, so the model never states a number. Before any figure appears, small chips show how the question was read: the metric, the time window, the role and the source, so a misreading is caught at a glance. Questions no metric can answer, such as "which candidate is the best culture fit?" or "what salary should we offer?", are turned down with a reason instead of being matched to the nearest metric.
Cited Summaries
On request, the model writes up to five sentences about a candidate from their scorecards, recruiter notes and rejection reasons, with each record numbered and labeled. Every sentence must cite one record and pass four checks: the record exists, the quote is in it, every number appears in it, and nothing touches a protected trait. Sentences that pass open to their source with the quoted words marked. Sentences that fail are listed under "Left out after checking" with the reason. The page is also honest about its limits: the checks confirm that the words exist, not that a sentence reads them fairly, and it says so under every summary.
The Result
An eval script scores every feature against fixed test cases, both before and after the checks, so the value of the checks is measured rather than claimed. On the default local model, qwen3.6, the notetaker filed 82% of labeled evidence under the right attribute, against 30% for plain keyword matching, and no invented or protected-trait quote reached the form. It read all 26 pipeline questions fully right, including the 6 that should be refused. 98% of summary sentences passed every check, and the one that failed was held back. A smaller 2 GB model also runs, and the eval shows plainly where it falls short: it adds filters nobody asked for, which is exactly the mistake the chips that show how a question was read are there to catch.
Technology & Deployment
The ATS runs on Next.js 15, React 19 and TypeScript, with SQLite for storage and Tailwind CSS for styling. It ships as a web app or as a Mac desktop app built with Tauri. The models run through Ollama on the same machine, with qwen3.6 by default and llama3.2 as a lighter option, and a single setting turns the model off entirely. Funnel and pass-through numbers come from an event log of stage changes rather than from current state, so reports never shift after the fact. The interface was audited against WCAG 2.2 AA, and its weakest text pairing has a contrast ratio of 7.22 to 1.