Knowledge Base

Title-Guard C3: From String Validator to Claims Verifier

Design & brainstorm document — 2026-07-15. Status: ideation approved for write-up; no C3 code written yet. All API probes in this document were run live on 2026-07-15 from this VPS.

1. Where this came from

Title-guard C1–C2.7 answers one question well: is this string shaped like a real job title (or name)? It does so lexically and statistically — TF-IDF nearest-neighbour over 63,934 O*NET title variants, seniority grammar, profanity/gibberish/placeholder flags — at ~900 titles/s on this 4-core box, with 97.7% accuracy on the frozen eval and 100% on SEC officer titles.

The C2.7 holdout taught us the lesson this whole design is built on: memorized lists decay, statistical signals hold. Fictional-title detection fell from 98% (frozen set) to 12% (one-shot holdout), while gibberish and legit-title detection stayed at 100%. Any signal that amounts to "a list of things we've seen" will rot. Signals grounded in structure or in external reality won't.

The user's challenge: given that records usually carry (name, title, company), why not use the company? Check the company's careers page, about page, public filings — build production rules from public data. This document is the worked-out answer.

2. The reframe: three layers of validation

Layer Question Status
L1 — Intrinsic Is this string shaped like a real title? ✅ built (C1–C2.7)
L2 — Contextual Is this title plausible at this company? this design
L3 — Extrinsic Does public evidence support this exact person+title+company claim? this design

That progression turns title-guard from a string validator into a claims verifier — which is what KYC, lead-list hygiene (ZoomInfo/Apollo-style data vendors), and fraud teams actually pay for.

3. The core design principle: asymmetric evidence

Every external source is evidence with three possible polarities, and they are not symmetric:

A second structural insight: different sources see different slices of a company's true title universe.

They complement rather than duplicate; a good combiner treats them as independent witnesses.

4. Live probe results (2026-07-15)

Every source below was tested from this VPS before being written into the plan.

Source Probe Result
Greenhouse boards-api.greenhouse.io/v1/boards/{slug}/jobs stripe, airbnb, databricks ✅ structured JSON, no auth
Lever api.lever.co/v0/postings/{slug} spotify ✅, netflix ∅ (not a Lever shop) ✅ works; empty ≠ error
Ashby api.ashbyhq.com/posting-api/job-board/{slug} openai ✅ titles + departments + teams
SEC EDGAR data.sec.gov/submissions/CIK…json Apple ✅ (UA header required, ~8 req/s)
GLEIF api.gleif.org/api/v1/lei-records "Stripe" ✅ 55 legal entities, no key
NPPES npiregistry.cms.hhs.gov/api John Smith ✅ full provider records, no key
ProPublica Nonprofit Explorer v2 "red cross" ✅ 190 orgs, no key
Wikidata SPARQL (P169 chief executive officer) Apple ✅ returned "Tim Cook"
OpenAlex api.openalex.org/authors Yoshua Bengio ✅ 89 author records, no key
DOL H-1B LCA disclosure page HTTP check ✅ 200, quarterly bulk files
PatentsView search ❌ now requires (free) API key

5. Architecture

claim = (name?, title, company?, country?)        # any subset works

titleguard/
  evidence/
    base.py       # EvidenceAdapter: lookup(claim) -> Evidence(source, polarity, strength, raw, fetched_at)
    slugs.py      # company name -> domain -> ATS detection  (the one genuinely hard component)
    cache.py      # sqlite, per-source TTL (ATS 7d, EDGAR 90d, GLEIF 180d)
    adapters/     # one small file per source, all behind the same protocol
  offline/        # L2: built on disk, zero network at query time
    lca_index/    # H-1B employer->title index (sqlite FTS)
    sic_soc.py    # industry x occupation compatibility matrix
  combine.py      # log-odds evidence combiner; per-source weights calibrated on labeled eval
  cascade.py      # L1 lexical (900/s) -> L2 offline -> L3 network; escalate only the uncertain band

The cascade preserves the throughput story. L1 clears ~95% of rows at current speed. L2 is local sqlite lookups (thousands/s, no network). Only the uncertain residue pays network latency — behind an aggressive cache, a rate limiter, and a per-batch budget. Each tier is 10–100× more expensive and exponentially rarer.

The combiner is naive-Bayes-style log-odds aggregation — the same statistical ethos as C2, calibratable with the existing eval machinery. It natively expresses the asymmetry: per-source weights for match/absence/contradiction are learned, and absence weights will calibrate near zero for low-coverage sources automatically.

8GB-VPS discipline (per project rules): offline indexes are streamed builds into sqlite/FTS, no pandas, adapters are plain requests with generators; nothing holds a dataset in RAM.

6. Sources — development plan and honest success forecast

Tier 1 — build these

DOL H-1B LCA bulk files — the gem. Quarterly XLSX disclosures with EMPLOYER_NAME, JOB_TITLE, SOC_CODE, WAGE: millions of real employer→title→occupation rows, bulk download, no rate limits, fully offline — a perfect match for this box. Answers both "does this employer hire" and "is this title in their real vocabulary" locally, for incumbents rather than openings. Plan: downloader + streaming XLSX→sqlite ETL (~1 day); the real work is employer-name normalization (legal suffixes, DBA names — reuse existing fold/clean code). Forecast: strong coverage of mid-to-large US employers, skewed toward skilled roles; high precision; ~85% success for the US L2 layer. Blind spot: small non-sponsoring businesses.

SEC EDGAR officers. fetch_sec_titles.py already speaks EDGAR; extend to the submissions API + Form 3/4 XML for name + exact title + company on every US public-company officer, plus DEF 14A for named executives. Plan: CIK resolver from company_tickers.json + officer extractor, ~2 days. Forecast: narrow coverage (~7k companies) but near-100% precision — the only source producing high-confidence contradictions, e.g. "CFO of $PUBLIC_CO" when filings say otherwise. ~90% success within its niche; the flagship demo feature.

SIC×SOC compatibility matrix (offline, universal). Company industry code (from EDGAR/GLEIF/Companies House) × the title's ONET/SOC family → plausibility prior. "Head Chef at a fintech" gets flagged with zero per-company data. Elegant twist: the matrix is estimated from the LCA data itself (each row has employer industry and SOC) — one dataset feeds two components. Plan: ~2 days on top of the LCA ETL. Forecast: coarse but universal; catches cross-domain absurdity L1 fundamentally can't. ~75% success*, fuzzy at conglomerates.

ATS adapters (the user's original idea). Verified live and clean. Five adapters (Greenhouse/Lever/Ashby/Workable/SmartRecruiters, ~½ day each). The real component is slug discovery: company name → website → careers link → recognize the ATS URL pattern — a mini-crawler with heuristics, ~3 days, which will fail on roughly half of companies. Forecast: coverage ~30–50% of tech/mid-market, near-zero for local SMBs and much of Workday-land; positive-evidence only (openings ≠ all employees, so absence must score ~0). ~60% success overall, ~85% for tech-sector claims. Build after LCA.

GLEIF + Companies House (UK). GLEIF: "does this legal entity exist at all", global, no key — a 2-hour adapter, cheap sanity gate. Companies House: every UK company's directors by name, free API key — high precision for UK director-level claims, ~1 day. ~95% success within their narrow questions.

Tier 2 — vertical plugins (each is a market segment)

Tier 3 — demoted, with reasons

7. Non-source ideas from the same brainstorm

  1. Batch-level fraud signals. Fraud rarely arrives one row at a time. Score the batch: title-frequency entropy vs the SEC/O*NET baseline, org-pyramid shape (a real org has more ICs than directors — a lead list that's 60% "VP" is synthetic or inflated), duplicate/near-duplicate people. Pure statistics, no network, plays to the project's strengths.
  2. Org-chart grammar. Titles encode a partial order (Intern \< Senior \< Lead \< Director \< VP \< C-suite). Within one company's records, check consistency: two CEOs, a "Senior Intern", an impossible reporting pyramid. Cheap, deterministic, and nobody does it.
  3. Cost-aware cascade as *the* production architecture — with measured cost/latency/accuracy per tier, this is a publishable systems result and the natural sequel blog.
  4. Title-inflation index (spinoff): not "is this fake" but "how inflated is this title vs what this company/industry actually files for" — lead-scoring teams want this; LCA wage data enables it.
  5. Reverse mode for KYC: given a company, generate its plausible title universe and flag outliers in bulk — the shape of what data vendors and AML teams buy.

8. Evaluation plan

Reuse the C2.7 frozen/holdout discipline. Positives: held-out LCA rows and EDGAR officers (real person/title/company triples). Negatives, synthesized as three distinct fraud modes the combiner must separate: real name × wrong company, inflated title at a real company, real title at a nonexistent company. Production gate: contradiction-class flags need ≳99% precision before anything auto-rejects; everything else surfaces as a score. Per the C2.7 lesson: the holdout is evaluated once; fixes get a freshly generated holdout.

9. Phased roadmap

Phase Deliverable ~Effort
C3.0 Claim schema, evidence protocol, log-odds combiner, sqlite cache (no network) 2 days
C3.1 LCA bulk index + SIC×SOC matrix — the offline L2, biggest single jump 3 days
C3.2 EDGAR officer verification — contradiction detection, the demo-able wow 2 days
C3.3 ATS slug resolver + 5 adapters 4 days
C3.4 Vertical plugins (NPPES, 990, Companies House, Wikidata) as opt-in 2 days
C3.5 Eval + calibration + blog sequel 2 days

The pre-existing "C3 FastAPI demo" backlog item becomes the serving layer on top of this, not a separate feature. (Deployment per project rules: bind 127.0.0.1; no DNS changes.)

10. Honest caveats


Part 2 — Round 2: more probed sources and new components (2026-07-15)

2.1 New probes

Source Probe Result
French registry recherche-entreprises.api.gouv.fr "airbus" ✅ no key; returned actual officers (Guillaume Faury, "Président de SAS", 2 Directeurs Généraux) plus NAF industry code plus headcount bracket
Wayback Machine CDX API stripe.com/about ✅ snapshots back to 2011, free, works from this datacenter IP
GDELT v2 news API exec-appointment query ❌ rate-limited/blocked from this VPS even after backoff — parked

French registry is the find: one free call yields officer verification (Companies-House-grade for France), the industry input for the SIC×SOC matrix, and the size-vs-seniority prior. ~90% success within France, ~1 day adapter. With Companies House UK + GLEIF + Denmark's CVR (same model, unprobed), the "US-heavy" caveat becomes a real EU story.

Wayback CDX enables the one thing no other source can: past-tenure verification — fetch the archived team/about page for a date range and search for the name. Limits: arbitrary-HTML parsing, one claim at a time, only companies whose team pages were archived. So it is not a bulk signal; it slots in as the last cascade tier before human review, for high-value single claims (executive KYC, due diligence). ~50% coverage on tech companies, high precision on hit.

2.2 New components

  1. CompanyGuard — the gap in the triad. We validate names and titles intrinsically but take the company string on faith. Reuse existing machinery (gibberish char-bigrams, placeholder/profanity flags) plus two cheap new checks: legal-suffix grammar ("Inc"/"GmbH"/"SAS" vs claimed country) and typosquat detection — edit distance against a top-10k brand list catches "Gooogle LLC" and homoglyph spoofs. ~2 days, L1 speed, and every L2/L3 lookup benefits because a garbage company string short-circuits before wasting a network call.
  2. Cross-record entity resolution. Same email with two names, one person with contradictory titles across a batch — probabilistic record linkage via splink on DuckDB (memory-frugal, fits the 8GB rule). Big enough to be its own track (C4), not a bolt-on.
  3. Zipf test for synthetic batches. Real title distributions are Zipfian (many "Software Engineer", one "Chief of Staff"); generated lists come out too uniform. A KL-divergence check against the O*NET/SEC frequency baseline — whole-batch forensics, essentially free.
  4. Conformal calibration. Calibrate the combiner with conformal prediction so the production rule ships as a guarantee ("auto-reject band has ≥99% precision"), not a hand-picked threshold. The difference between "a score" and "a contract".
  5. Adversarial data flywheel. Automate the C2.7 lesson: every release, generate a fresh holdout (never reused), run evals, plot decay across versions. "Never iterate on a holdout" becomes a pipeline, not a memory.

Part 3 — Validating whole contact records, not strings (2026-07-16)

The wider question: a record is (name, email, phone, company, title). String validation is only the cheapest tier. For any holder of a large multi-source contact dataset, three approaches beat adding more string rules:

3.1 The corpus validates itself

3.2 Event-driven decay (business logic, not lookup)

Contact data rots ~25–30%/year and not uniformly. Rules:

3.3 New external sources (probed 2026-07-16)

Source Probe What it gives
Web Data Commons (schema.org extractions from Common Crawl) ✅ site live Millions of schema.org/Person JSON-LD blocks from company team pages — name+jobTitle+worksFor, first-party assertions, free bulk download, offline-processable. The sleeper hit.
Gravatar ✅ per-email response Email hash → does a human-registered profile exist? Weak-positive, one HTTPS call.
Have I Been Pwned not probed (keyed) Email present in a 2016 breach = aged & real, not minted last week. Odd but genuinely useful.
Senate lobbying disclosures ✅ 108,817 filings for 2025, free API Registered lobbyists: name+title+organization.
State professional licensing boards not probed (fragmented) CPAs, engineers, nurses, real-estate — with NPPES + FINRA this becomes a "licensed-professional verification pack" per vertical.
SMTP RCPT-TO not probed (IP-reputation-sensitive) Mailbox existence without sending; catch-all domains cap precision ~70–80%.

Part 4 — Embeddings: full feasibility engineering (2026-07-16)

4.1 What embeddings fix — and what they can't

They fix the C2.5 known failure class: surface-lexical NN maps "Head of Growth Marketing" → Soil Scientists because of character overlap; an embedding maps it into semantic space where it lands near marketing occupations. Joke titles built from real words ("Chief Happiness Wizard") land in low-density regions far from every real occupation cluster — detectable via distance-to-nearest-centroid + local density, which plain lexical matching fundamentally cannot do.

They do not fix truth: an embedding cannot tell you whether this person holds this title at this company. Embeddings upgrade L1/L2 (the semantic mapping layer); L3 evidence stays the only truth layer.

4.2 The deduplication insight that changes all the math

10M records ≠ 10M strings. After normalization, a 10M-record dataset typically contains a few hundred thousand unique titles. Embed unique strings once, cache, join back. Every cost below shrinks ~20×.

4.3 Model menu (all CPU-only, fit this 4-core/8GB box)

Option Params Disk Est. CPU throughput (short strings) Notes
Model2Vec (potion-base-8M) 8M static ~30MB ~30k+/s Distilled static embeddings, no transformer at inference — the speed king; slightly weaker semantics
all-MiniLM-L6-v2, ONNX int8 22M ~23MB ~1–2k/s The default choice; well-studied
bge-small-en-v1.5 / gte-small 33M ~35MB int8 ~0.8–1.5k/s Slightly better quality
7B-class / API embeddings Unnecessary for 5-word strings

Reference index: 63,934 ONET variants × 384-dim float32 ≈ 98MB* — brute-force numpy matmul is fine (500k queries ≈ ~20 min); hnswlib if we want ms-level. All within the 8GB rule (and spawn-not-fork applies as ever).

10M-record wall-clock estimate (MiniLM int8, ~500k unique titles): dedupe ≈ minutes → embed ≈ 6–10 min → index lookup ≈ ~20 min brute-force → join back ≈ minutes. Under an hour end-to-end, vs 3.1h for the current lexical pipeline. Even with zero dedup it's ~2h. These are estimates to bench in C2.5, not measured numbers.

4.4 Train your own? Decision ladder

  1. v1 — zero-shot pretrained (MiniLM/BGE-small): no training at all; likely fixes most of the C2.5 tail. Start here, eval against the frozen set.
  2. v2 — linear head on frozen embeddings: logistic regression 384-dim → SOC major group, trained on O*NET's 63,934 labeled variants (free labels, sklearn, minutes on this box). Classifier confidence becomes a validity feature. Cheap, recommended.
  3. v3 — contrastive fine-tune (sentence-transformers, title→SOC pairs): hours-to-a-day on CPU; only if eval proves v2 insufficient.
  4. From-scratch tiny model (\<1M params, smol-lab style): will not match pretrained semantics — pretraining is where the "Growth Marketing ≠ Soil" knowledge lives. Fine as a learning exercise, wrong for production.

4.5 External APIs (OpenAI/Gemini/Anthropic) — when and cost

4.6 Non-neural rivals worth benching against embeddings


Part 5 — Fresh validation-rule ideas (round 3, 2026-07-16)

  1. The Highlander rule. Singleton titles must be singletons: two CEOs or two CFOs at one company means at least one record is stale or wrong. Generalized: expected count per title-level given company headcount (Poisson prior) — five "Chief of Staff" at a 40-person company fails it.
  2. Industry-conditional title calibration. "VP" at an investment bank is a mid-level IC; "VP" at a 50-person startup is an executive. Title-seniority must be reinterpreted per industry before any seniority-based rule fires — a lookup table learnable from bulk employer-title data (LCA). Ignoring this poisons every seniority rule with systematic false flags on finance.
  3. PMI composition scoring (see 4.6) — the interpretable answer to the joke-title tail.
  4. Trust propagation across coworkers. If 29 of 30 records at one company form a coherent org (pyramid shape, consistent vocabulary), the 30th inherits trust — label propagation on the per-company record graph. Anomalies concentrate; trust flows.
  5. Department-taxonomy check. ATS APIs return departments/teams (Ashby's probe literally returned department and team fields): validate a title's function against departments the company actually has, not just its title strings.
  6. Negative-space mining. From millions of LCA rows, derive titles that never occur despite being lexically plausible — an empirical "does not exist" prior per industry, the data-driven replacement for the fictional-titles list that decayed 98→12%.

Where everything slots

CompanyGuard + name↔email rules → C3.0; Zipf/batch + Highlander + PMI → C3.1; embeddings (4.3–4.4) = C2.5 executed as the cascade's middle tier; French registry + WDC + Wayback → C3.4; conformal + flywheel → C3.5; entity resolution → C4.


Part 6 — Decision: the C2.5 bake-off (2026-07-16)

The bet for the next sprint: bench the two counting methods against zero-shot embeddings on the frozen eval before committing to a neural middle tier.

Contender Type Hypothesis
PMI composition counting modifier↔head co-occurrence catches "Soil Marketing"-class jokes, interpretably
Word n-gram LM perplexity counting joke titles from real words still have weird word-order statistics
Static/small embeddings (Model2Vec / MiniLM) neural semantic space fixes the lexical-NN failure class outright

Decision rule: if the counting methods catch the same tail as embeddings, the system stays neural-net-free (simpler, faster, interpretable). If embeddings win decisively, they slot in as the cascade's middle tier — at under an hour per 10M records either way. Results go in the C2.5 blog.

Outcome (2026-07-16, measured — full detail in the [bake-off blog](https://vault.binary.ovh/title-guard-c25-bakeoff)): WordLM won the joke tail (holdout AUC .974, 90% recall @ 6% FPR, 74k titles/s, 0.48s build) — the system stays neural-net-free for detection. Embeddings (Model2Vec potion-base-8M) lost detection but fixed the C2 mapping bugs (Growth Marketing→marketing manager; Pharmacovigilance→pharmacologist) — they get the mapping job only. PMI inverted (AUC .185): add-alpha smoothing makes OOV pairs score high ("happiness wizard" +.51) while popular real pairs score low ("sales manager" −.23) — the C2.7 word-pair rejection, now with a mechanism. Code: titleguard/semantic.py, bench_semantic.py; numbers: data/eval/semantic_bakeoff.json.


Part 7 — Execution plan for the next session (written 2026-07-16, pre-compact)

This section exists so a fresh session can execute without re-deriving context. Constraint reminders: 8GB box (streaming, no pandas, spawn-not-fork, sklearn working_memory=256 already set in core), bind 127.0.0.1, never touch DNS, every published number from a measured run, `publish_md.py` needs `cd ~/claude_/atlas-docs` (cd back before git ops).

Phase A — Fold the bake-off winners into the product (C2.5 completion, ~½ day)

  1. FIRST, before any code: generate a fresh holdout — gen_eval_data.py holdout via grok-bridge (raw cached under data/eval/raw/). The old holdout has been seen twice (C2.7 + bake-off) and is dead for verdicts. Do not look at the new one until step 5.
  2. WordLM gate into `core.py`: build WordLMScorer counts at index time, persist inside data/index.joblib (bump CACHE_VERSION to 3). New flag joke_lm when score ≤ the frozen-set threshold stored in semantic_bakeoff.json (frozen.wordlm.threshold); flags multiply 0.7^n as usual. Throughput cost ~nil (74k/s vs 291/s pipeline).
  3. Embed mapping as a repair tier, not a rewrite: only when lexical similarity \< 0.5 (the Pharmacovigilance case) or the top match comes from a different SOC major group than the embedding's nearest — replace canonical occupation with the embedding nearest-neighbour's SOC. Reference index must carry SOC codes (extend EmbedScorer to keep (title, soc) from Alternate Titles). Keep model2vec an optional import — core must degrade gracefully without it. Fix to attempt: a c-suite prior on the reference so "Managing Director" reaches chief executives.
  4. Re-tune on frozen only. Gate: frozen accuracy ≥ 97.7% (no regression), joke recall strictly up. Re-run bench.py for the throughput delta and measure how often the embed repair tier fires (expected: low single-digit % of titles).
  5. One-shot fresh-holdout eval. Report whatever it says, no fixes after. Publish an addendum on the bake-off blog page (same slug) with the integration numbers.
  6. Commit; update CLAUDE.md status block.

Phase B — C3.0 serving layer (~½ day)

FastAPI app api.py: POST /check/title, POST /check/name, POST /check/batch (JSON list, cap ~1k/call), GET /health. Bind 127.0.0.1:8710 (8700 is grok-bridge). Load TitleGuard once at startup (350MB index). Run as a systemd user service modeled on ~/grok-bridge/ (uvicorn, single worker). Check configs/binary.ovh.zone for an existing unused subdomain before any add-site.sh; if none fits, stay local-only and demo via curl in the blog — never modify DNS. Measure request overhead vs library-call throughput.

Phase C — C3.1 offline evidence layer (~2 days, the big one)

  1. LCA ETL: download 1–2 recent quarterly LCA disclosure XLSX files from the DOL OFLC performance page (~300–700MB each; disk has ~56GB free). Stream-parse with openpyxl read-only mode → sqlite with FTS5: employer_titles(employer_norm, title, soc, naics, wage, year). Employer normalization = fold_ascii/clean + legal-suffix stripping. Verify NAICS column exists in the file layout before building the matrix (it should — confirm against the record layout PDF on the same page).
  2. NAICS×SOC compatibility matrix estimated from that same table (counts → smoothed log-odds, npz on disk).
  3. Evidence skeleton: titleguard/evidence/base.py (Evidence dataclass: source, polarity match/absent/contradiction, strength, raw, fetched_at) + the offline LCA adapter first; log-odds combiner stub. Network adapters (EDGAR officers, ATS, French registry) are Phase D — do not start them in the same session as the ETL.
  4. Eval per Part 1 §8: positives = held-out LCA rows; negatives = real title × wrong employer, inflated title at real employer, real title at fake employer. Gate: contradiction-class precision ≳99% before anything auto-rejects.
  5. Blog part 3: the evidence layer, with measured coverage/precision numbers.

Sequencing and risks

A → B → C strictly (A is small and unblocks the blog addendum; B makes everything demoable; C is the differentiator). Risks: grok-bridge flakiness for the fresh holdout (retry once, then cap-at-n like C2.7); LCA XLSX parse time on this box (openpyxl read-only streams ~1–2k rows/s — a 500k-row file is ~5 min, acceptable; if slower, convert once to CSV via libreoffice --headless or ssconvert and stream that); HF fetch for potion-8M is already cached in the venv's HF cache.