I sit on the board of The Spectrum Hope Fund, a Montana 501(c)(3) that funds ABA therapy access for children with autism.
The Executive Director, Tim, had spent over 100 hours on that. His words, from a working session in June: “If it could just populate me a list of things we qualify for, and I can just click the links and apply — I’d save myself a month worth of work.”
The tool he was using is Candid’s Foundation Directory. His free first year was ending and Premium runs about $1,200 a year. That’s a cost against a budget that gives out $5,000 grants. It’s also a subscription to an aggregator of data that is mostly public: IRS 990 filings, federal RFP listings, and foundation websites that say on the page whether they take unsolicited applications.
So we built a pipeline that rebuilds the discovery half on the public sources.
The deliverable is a link
The thing we got right early was refusing to build a report. The product is a CSV row per funder with a working application URL in it. Everything else in the schema exists to support that column: name, EIN, amount, deadline, status, verdict, reason, source, last checked.
That constraint kills a lot of design debate. A funder that looks perfect but has no findable apply path is not a win, it’s a needs-review row with the reason no application path found in extracted text. A page that is beautifully on-mission but turns out to be a grant-directory listicle is not-a-fit with a reason starting directory:. The pipeline is not allowed to hand back something Tim can’t click.
How it’s put together
Five stages, each a separate command, communicating only through a SQLite file:
discover → extract → classify → reconcile → export
discover casts four nets: web search against query templates in YAML, a crawl of known Montana funder pages for outbound grant links, a ProPublica Nonprofit Explorer sweep of Montana 990 filers, and grants.gov federal opportunities. extract fetches each candidate’s own site, honoring robots.txt, throttled to about one request per second per domain, cached on disk. classify renders a verdict. reconcile collapses duplicate rows on EIN, then normalized name plus domain. export writes the CSV.
Every stage commits one row per transaction and only touches rows still pending, so a crash at funder 400 of 600 costs nothing. Re-running the same command resumes it. That property mattered more than any other engineering decision, because these runs take hours and they do crash.
Rules may disqualify. Only the model may qualify.
Classification runs in two tiers. The first is deterministic regex over the extracted page text, loaded from the same eligibility.yaml that also renders into the model’s system prompt, so the two tiers can’t drift apart.
The asymmetry: the rules tier can return not-a-fit or flag a row for review, and can never return qualified. A false rejection costs one lead out of hundreds. A false qualification costs Tim an evening writing an application to a funder that was invitation-only the whole time. Those errors are not the same size, so they don’t get the same treatment.
class RulesClassifier:
def classify(self, text: str) -> RuleVerdict:
for code, _reason, patterns in self.disqualifiers:
for p in patterns:
m = p.search(text)
if m:
return RuleVerdict("not-a-fit", code, m.group(0)[:80], [])
flags = [
code
for code, _reason, patterns in self.flags
if any(p.search(text) for p in patterns)
]
return RuleVerdict(None, None, None, flags)
The disqualifier patterns are Tim’s dead ends, encoded: does not accept unsolicited, by invitation only, donor-advised fund, research-only, adults-only, subgrants to affiliates only. Geography gets its own asymmetry. A funder whose footprint excludes Montana entirely is not-a-fit. A funder whose footprint includes part of Montana but is narrower than statewide is always needs-review, never auto-qualified, because SHF has been rejected before on a county restriction.
What one run produced
693 candidate rows, from 57 search queries, 1,836 fetched pages, and 682 successful extractions.
| Verdict | Rows |
|---|---|
| not-a-fit | 478 |
| needs-review | 194 |
| qualified | 4 |
| unclassified | 17 |
Four. That is the honest number, and it is the most useful thing I can tell another nonprofit thinking about this.
Look at what the 478 rejections are made of: 301 of them are pages that were never funders at all. Grant directories, aggregator listings, news articles about grant opportunities, grant-writing services, a parked domain. The web between you and a foundation’s application page is mostly other people’s summaries of foundations’ application pages. The pipeline’s real job turned out to be triage of that layer, not adjudication of funders.
The needs-review pile is similarly instructive. 139 of its 194 rows are there because the page didn’t yield enough text to judge, and 11 because the site prohibits automated access. Those are honest verdicts. The pile is a work queue for a human, not a failure state, and each row carries the reason it landed there.
Rejections with reasons are an asset, not waste. Next quarter’s run reads them and doesn’t re-litigate the same 478 funders.
Cost, and then no cost
A cold classification run consumed 1.88 million input tokens and 67,000 output tokens against claude-haiku-4-5. At that model’s $1 and $5 per million, roughly $2.20. Re-runs hit the response cache and cost close to nothing. Against $1,200 a year, the arithmetic is not subtle.
Then I removed even that. The classify stage grew an --emit / --ingest pair: --emit runs the deterministic tier, then writes every row it couldn’t settle to JSONL instead of calling the API. A coding agent reads that file, judges each row against the same rubric, and writes verdicts back through --ingest, which applies them through the identical guard path the API tier used. Same precision guards, same reconcile, no API key.
That’s packaged as a set of skills in the repo (find-new-grants, review-grant-maybes, show-grant-list, draft-grant-application), so the operating interface for the nonprofit is a sentence, not a CLI. “Find new grants” runs the whole refresh. The Python still does everything mechanical and deterministic; the agent does the judging. Neither one gets to do the other’s job.
Constraints in the code
Four rules are enforced by the pipeline rather than by intention:
- robots.txt is honored, and a site that prohibits automation gets recorded as a lead marked for manual review rather than fetched. A foundation catching you scraping against its stated policy can disqualify the application you were trying to write.
- Roughly one request per second per domain, honest user agent, no spoofing to route around a block, every page cached so nothing gets hit twice inadvertently.
- Nothing behind Candid’s login or any other paywall is touched by this codebase.
- Applications are never auto-submitted. The pipeline drafts and discovers. SHF clicks submit.
The last one isn’t squeamishness. A grant application is a relationship document, and a program officer who receives a machine-generated submission is learning something true about the organization that sent it.
What this does not replace
It does not find the funder who gives because someone on your board knows someone on theirs. It doesn’t write a compelling case for support, and it doesn’t tell you which of four qualified funders is worth the evening. It compresses the part of fundraising that is mechanical search over public records, which happens to be the part that was eating a month of a volunteer executive director’s year.
If you work with a small nonprofit and want to try something in this shape, the order I’d suggest: define the deliverable as the thing the person will click, write the eligibility rules down in one file before writing any code, make the cheap deterministic layer only capable of saying no, and treat a rejection with a reason as output worth storing. The model is the smallest part of it.