Demo script — good-government / watchdog audience (Reinvent Albany)

DRAFT. Revised 2026-07-27, revised again 2026-07-30. A run-it-live script for showing BetaNYC’s grounded AI + NYC/NYS open data MCPs to a good-government or watchdog organization — written for Reinvent Albany, and reusable for Citizens Union, CBC, NYPIRG, the Fiscal Policy Institute, or an investigative newsroom.

Slide companion: reinvent-albany-deck.html. Same acts, presentation medium, every warning below carried in its presenter notes (press N). This file stays the source of truth for figures and provenance.

Figures are dated individually in the presenter-notes provenance table. Act 2’s budget and Checkbook figures were re-verified 2026-10-19; § 23-507 was re-verified 2026-07-27; the open data portal findings in Act 3 are from 2026-07-21 and were not re-pulled.

🔄 Revision note — what changed on 2026-10-19

Budget MCP 2.0.0 (2026-10-18) retired the adopted-budget extracts this script was built on: Schedule C, the transparency resolutions and search_awards. Act 2 is rebuilt on search_disclosure_awards, which reads the Council’s discretionary funding disclosure. The old Act 3 (FY2026 rescissions) and its row-cap correction are cut, because the current tool does not serve that data and reports true totals. Later acts are renumbered.

🔄 Revision note — what changed on 2026-07-30

The Checkbook material in this script was built around an outage. That outage is over, and the reason it ended changes what Act 5 says.

• Checkbook API access was restored 2026-07-28. Verified live again 2026-07-30. The Act 2 reconciliation runs as originally written; the outage warnings are gone.
• The residual block turned out to be ours, not theirs. We were exceeding an undocumented 1 req/sec limit. Any framing of this as “the city blocked us” is now false. Fixed in nyc-checkbook-mcp v1.6.0.
• The Comptroller’s office answered our letter in two days, fixed two drifted catalog entries, and corrected one of our own findings: Prime Registration Date is exposed.
• Do not attempt the 403 as a live demo. It asked the agent for a record that no longer fails to return.

⚠️ The 2026-07-27 scope decision still stands and is not reopened by any of this. This act carries no ask and does not critique another office’s dashboard. Report the failure shape, say what we did about it, and stop.

This audience is different from every other script in this repo, and the difference should shape the whole meeting.

They are not the constituency. They are the oversight. A council member wants to see their own district; a watchdog wants to see the thing nobody has assembled yet. So this script is not district-scoped, and it does not try to impress with a clean lookup. It goes straight at the questions this audience actually lives on: where did the money go, what got taken back, what was supposed to be published but wasn’t, and what is published but cannot be joined to anything.

Act 5 is the one that closes the loop: it is the only act in this repo where BetaNYC is the subject rather than the tool, and where we were the ones who turned out to be wrong. For this room that is the strongest material in the script.

They also already know this domain better than you do. Do not explain the Council’s discretionary funding process to Reinvent Albany. The pitch is not “here is discretionary funding” — it is “here is discretionary funding, queryable in seconds, by an agent that cites its source and refuses to guess.”

Before you start (2 min)

• No district to pre-fill. This script runs citywide.
• Have Checkbook NYC and NYC Open Data open as reference tabs. The Checkbook API works again as of 2026-07-28, so no step needs to be done by hand.
• Runs about 21 minutes with Act 5, 17 without (the old 5-minute rescission act was cut 2026-10-19). Act 4 is already marked optional. If you are short on time, cut Act 4 rather than Act 5 — Act 5 is the one written for this audience.
• Expect to be interrupted with hard questions about provenance and coverage. That is the good outcome. The presenter notes below are written so you can answer them.

⚠️ Check your egress IP before you demo, and before you quote any failure to anyone.

curl -s https://api.ipify.org

The BetaNYC office is 50.74.128.42. janus routes through a commercial VPN by default (utun10, separate from Tailscale), which egresses from a datacenter address. On 2026-07-27 that nearly produced a “your fix did not work” report to a CIO who had just done real work for us. A 403 mid-demo is far more likely to be your own network path than Checkbook.

Opening line:

“You already know where this data lives — you’ve FOILed for worse. What I want to show you is how fast the assembly step gets when an agent can reach all of it at once, and what it does when a number isn’t there.”

Act 1 — The transparency law, read back to you (3 min)

Open on their own turf.

Prompt:

“What does the NYC Administrative Code require about open data, and which Council legislation established it?”

Verified — the agent returns the actual statutory chain:

What to say: “The law you helped pass, with its compliance machinery, retrievable by citation. Note § 23-507 in particular — there is a named, accountable role at every agency. That matters for Act 3.”

Act 2 — Follow one vendor across the whole city (5 min)

Not one district. The whole map.

Prompt:

“Show every FY2026 Council discretionary award to the Association of Community Employment Programs for the Homeless, then check NYC Checkbook for what the city actually paid them.”

Verified live 2026-10-19 (search_disclosure_awards(ein="133846431", fiscal_year=2026)): 71 designations totaling $7,399,527, all Cleared, none Pending. The same organization is funded separately by 41 named council members, the Speaker, and the Brooklyn and Queens delegations, plus one $153,277 BID Containerization line (via DSNY) with no member printed. Mostly under the NYC Cleanup initiative, nearly all via DYCD. Individual designations range from $5,000 to $280,000. The tool’s totals cover every matching row, whatever the display limit.

This is the Council’s discretionary funding disclosure, its own award-level list. It is a later snapshot than the adopted budget’s Schedule C, so it includes designations made after adoption, each marked Cleared or Pending. In July this act read 54 awards and $6,455,750 from the adopted-budget extract; the difference is the snapshot, not an error.

Say “41 council members, plus the Speaker and two borough delegations,” not “44 council members.”

The member column mixes three kinds of thing and does not distinguish them: individual surnames, borough delegations (printed as, e.g., Brooklyn Delegation, Queens Delegation), and Speaker. Anyone computing “awards per council member” from this field silently folds delegation and Speaker money into a member count. Say this to this room — it is exactly the kind of schema detail that quietly wrecks an analysis, and they will use the field.

What to say:

“One vendor, 71 separate discretionary designations from 41 council members, the Speaker and two delegations, more than seven million dollars, and no single line item anywhere that says so. Assembling this by hand means filtering the Council’s disclosure workbook row by row. That’s the afternoon this replaces.”

Then the reconciliation, on a smaller organization so the numbers stay legible.

Prompt:

“Check NYC Checkbook for FY2026 spending paid to Community League of the Heights.”

Verified live 2026-10-19 (search_spending(payee_name="Community League of the Heights", fiscal_year="2026")): 32 payment records totaling $1,040,448.79, across DYCD, DOE, SBS, DOT, and Small Business Services. Three of those checks sit under budget code 3625 (TAX LEVY ELECTED OFFICIALS): $107,244.16, $33,000.00, and $2,560.00 — $142,804.16 in total.

The line to land: “That budget code is the join. It’s discretionary money appearing as an actual disbursement. Allocation in one system, check in another, connected in two questions.”

🚩 Do not say $107,244.16 is De La Rosa’s money arriving. It is not, and this room will catch it.

The allocation side shows CLOTH receiving $40,000 from Council Member De La Rosa in FY2026 ($30,000 NYC Cleanup, $10,000 Community Safety and Victim Services). The disbursement side shows $142,804.16 under Tax Levy Elected Officials. Those do not reconcile: the disclosure lists CLOTH funding from the Speaker, Council Member Abreu and the Manhattan Delegation too, $353,986 in all, and code 3625 does not say whose allocation a check discharges.

State that gap out loud — it is the more interesting finding. The budget code gets you from an allocation to a real check. Getting from there to whose allocation that check discharges is a further step the code does not give you. So the join is real, and it is coarser than it looks.

That is this folder’s fourth question in one live example: published, current, and still not joinable to the unit governance actually uses. For an audience that campaigns on exactly this, a demonstrated limit is worth more than a clean match.

Sharpen it for this room: the interesting output of this pattern is not the match — it’s the mismatch. Allocations with no corresponding spending, or spending that exceeds allocation, are the thing worth chasing. The tooling makes that comparison cheap enough to run across every organization rather than the three you already suspected. Just be precise about which mismatches are findings and which are artifacts of a coarse budget code.

Act 3 — The report that was mandated and isn’t there (4 min)

The act that speaks their language most directly.

Prompt:

“Local Law 174 of 2016 requires an annual Career and Technical Education report. What’s the most recent one available as a dataset on the NYC open data portal?”

Verified: catalog searches on both “career technical education” and “Local Law 174 CTE” return six CTE report datasets spanning 2015-16 through 2019-2020 — the most recent published 2021-09-22 — and nothing newer.

What to say:

“The law says annual. The portal’s most recent is 2019-2020. It is 2026.”

Then the discipline that makes it credible, and do not skip it:

“What we’ve established is that the dataset isn’t on the portal. That is not the same as the report was never produced — it may well have gone to the Council as a PDF and never been published as open data. Those are different findings with different remedies, and the second one is § 23-507’s problem: there’s a named coordinator at DOE accountable for exactly this.”

Why this lands with this audience specifically: it is a reproducible method, not a one-off. Any reporting mandate in the Administrative Code can be checked against the portal the same way. That’s a survey they could run, and the agent won’t overstate the result if it’s prompted not to.

Second, live example if you want one: NYC’s “Broadband Adoption and Infrastructure by State Senate District” dataset (9bjg-n96a) contains 26 rows covering Senate Districts 10–34 plus 36, all created 2020-06-19. It does not contain District 59 — the file predates the current district lines. A digital-equity dataset that cannot answer a question about a third of the state senate districts in the city.

Act 4 — Live procurement, if there’s time (2 min)

Prompt:

“What NYC procurement solicitations are currently open, and which agencies posted them?”

Verified 2026-07-21: returns live City Record notices with agency, PIN/EPIN, selection method, due date, and contact — e.g. FDNY 05726B0010 (mark-out and core drilling, competitive sealed bids, due 2026-07-21 14:00), NYCHA 522172, DEP 82626B0026.

⏳ Those examples have expired. FDNY 05726B0010 closed 2026-07-21 14:00. This act is only worth running live — the whole point is that the notices are current, and reading out a solicitation that closed last week undercuts it. Run the prompt fresh in the meeting and read whatever comes back. Do not quote the three above.

What to say: “The City Record, queryable. Same source as the daily PDF, without reading the daily PDF.”

Act 5 — What happened when we asked (4 min)

This is the act for this audience specifically. Every other act demonstrates the tooling. This one demonstrates the process — what it looks like to find a public-data access problem, report it through the front door, and be told you were partly the cause. Run it only if the room is engaged. It is the strongest close available here and it needs four minutes of real attention.

Do not run part one as a live query. The 2026-07-21 version of this act demoed a 403. Access was restored 2026-07-28 and re-verified 2026-07-30 — the query now simply works. This act is told, not demoed.

Part one: the round-trip, in four dates

What to say:

“Between July 16 and July 21, Checkbook’s documented API started returning 403 to us. The site loaded fine in a browser. On the 22nd we sent a letter to the Comptroller’s office. They answered on the 24th — two days — with a real technical response, looped in their CIO, and offered a call. On the 28th we got on that call and access was working.”

Then the part that belongs to this room, and do not soften it:

“And the residual block was ours. We were exceeding a rate limit of one request per second. Their fix had already worked; what we were still measuring was our own client getting itself blocked. We shipped the fix on our side — rate pacing, backoff, and we stopped following redirects, which was turning one logical API call into about forty requests against their origin.”

Why this is the right material for this audience: they spend their working lives being told that a data problem is somebody else’s fault. An organization that says “we asked, they fixed it, and then the rest of it was us” is establishing that its findings are worth trusting when they do point outward. The correction is the credential.

Presenter note — the near-miss, if the room is technical. On 2026-07-27 we retested and got a fresh 403, and nearly reported “your fix did not work” to a CIO who had just done real work for us. The test had egressed through a VPN on a datacenter IP, not the office address the office had investigated. Chrome from the same address loaded the site fine, which is what caught it. One source address, two clients, minutes apart: browser 200, command line 403. That is how we established the discriminator was the client and not the network — and it is why the standing rule is now to check your egress IP before reporting any test result to an office whose diagnosis keys on source IP.

Part two: what we got wrong, and who corrected it

⚠️ Scope, per the 2026-07-27 decision. This part reports on our own work only. An earlier draft critiqued a dashboard published by the Comptroller’s office. That critique is not in this script and is not reinstated by the fact that we now have more information about it. Describe our own error; do not evaluate another office’s product in front of a third party.

Our 2026-07-09 audit made two claims about reproducing a published figure. The office answered both, and corrected us on one.

What to say:

“We published two findings about data we couldn’t reach. One of them was simply wrong — the field was there and we missed it. They told us so, and we corrected it. That’s worth saying out loud, because it is the same standard we’re asking you to hold our numbers to.”

Also confirmed by the office: the Checkbook 2.0 migration dropped no documented API fields or domains, and external applications can keep using the API without going through the web interface. We can now state that rather than infer it. They also fixed two drifted NYC Open Data catalog entries we had flagged.

Part three: the limits that are documented nowhere

⚠️ This part carries NO ask. Report the finding and stop. Do not invite the room to co-sign anything, and do not suggest their name would carry it further. That framing was removed on 2026-07-27 and is not coming back.

What to say:

“Checkbook enforces two operational limits. One request per second, and twenty thousand records per call. Neither is documented anywhere on checkbooknyc.com — not on the API landing page, not in the global parameters table, not on any domain page. We only learned both numbers in conversation with the office.”

“That’s the part worth knowing. Not that the limits exist — they’re reasonable limits. It’s that the only way we learned them was to violate one and get blocked. An independent researcher hitting the same wall just experiences an outage and concludes the data is unreliable.”

Two smaller documentation observations, if they want specifics:

• The API landing page says 20,000 records per call; every domain parameter table says fewer than 1,000 and caps the field at four characters, which cannot express 20000. We tested — 20,000 is correct and the tables are stale.
• The landing page says records_from; the parameter tables say record_from.

How to close the act:

“We’ve passed all of that back to them. That’s where it sits.”

Presenter note — the letter’s status, and get this right. The letter was sent 2026-07-22 and answered 2026-07-24. Do not describe it as pending, unsent, or unanswered. The relationship is live and cooperative: Richard Lundy (Assistant Comptroller, IT Operations/CIO) is the technical route, Matt Rubin (Chief of Staff) is the relationship owner. We replied 2026-07-29 confirming access works, owning the rate-limit fault, and asking whether the limit is scoped per IP or per client. That question is outstanding — and it is ours to pursue, not the room’s. If someone volunteers to weigh in, that is their initiative and a fine outcome; you do not solicit it.

Closing (1 min)

“All of this is open. The connectors are public repos, the guides are CC BY-SA, and most of it requires no API key at all. Every bug I described in Act 5 is a public issue with our name on it, including the one where we were the problem. We’d rather you fork it and find the things we got wrong than take our word for any of it. And if there’s a query your team runs constantly by hand, that’s exactly the user journey we want to add.”

Leave-behind links:

• This repo: grounding-ai-with-ny-open-data
• Structured budget data: BetaNYC/New-York-City-Budget
• Full prompt catalog: ../user-journeys.md
• BetaNYC: beta.nyc

Presenter notes

Figures carry individual verification dates — see the provenance table. Act 2 was re-verified 2026-10-19; § 23-507 on 2026-07-27; Act 3’s portal findings are from 2026-07-21 and were not re-pulled.

This audience will interrogate provenance. Answer precisely or concede. A hedge you can defend beats a figure you can’t.

The row cap — history worth telling, if asked

Budget MCP 1.x clamped every query to 500 rows without saying so, and this script’s July rescission figures were $90,314,887 low before we caught it, twice (New-York-City-Budget#42). 2.0.0 retired those tools, and search_disclosure_awards reports true totals over every match. The same lesson now applies to get_voting_record, which defaults to 50 rows with no total: treat “returned exactly N against a limit of N” as truncation until proven otherwise.

Coverage limits — know these cold, and volunteer them

From list_available_fiscal_years:

• Council discretionary funding disclosure: FY2009 and FY2011–FY2027, award by award. FY2010 is not loaded: the Council’s own workbook for it is incomplete against NYC Open Data. FY2009 and FY2013 are early snapshots whose Cleared totals are not headline figures. Nothing before FY2009.
• Council capital designations: FY2015–FY2017 and FY2019–FY2027. FY2018 is missing from every source found. Exact by budget line only; the Council’s project IDs do not link to OMB or Checkbook projects.
• Council vs OMB capital comparison: FY2019–FY2027, by budget line (compare_capital_to_omb). BetaNYC’s arithmetic, not an official reconciliation.
• Retired in 2.0.0 and not served: Schedule C, Terms & Conditions, §254 capital, transparency resolutions and the Legistar crosswalk. They remain in the source repo for history.

Parameter traps (all silent — no error, real-looking wrong output)

✅ Fixed and published 2026-07-21 — volunteer this to this audience. All seven BetaNYC servers now reject an undeclared parameter and name the ones they accept. Minimum versions: budget 1.3.0 (use 2.0.0 or later today: it is the release with search_disclosure_awards, and 1.x serves retired data), council 2.5.0, checkbook 1.4.0, record 1.1.0, 311 1.1.0, charter 0.2.0, nys 2.3.0. Verified live against the published packages.

The honest framing, which this room will respect more than a clean demo: we found this in our own tooling midday on 2026-07-21 while rehearsing these very scripts, filed seven public issues, and shipped, merged, and published all seven fixes the same evening — merges at 20:52–20:54, npm publishes at 21:06. The bug was that a guessed parameter produced real, correctly sourced data answering a different question — the exact failure mode this session is about. Being able to say “we caught it in ourselves, here are the issues, here are the commits” is a stronger provenance argument than a clean demo.

One carve-out, and volunteer it rather than being caught on it: six of the seven servers name the accepted parameters in the refusal. nyc-checkbook-mcp does not — outside search_contracts it returns zod’s bare “Unrecognized key(s) in object” as a raw JSON validation dump, with no accepted-parameter list and no pointer. If you demo the refusal deliberately, demo it on the budget or charter server, not Checkbook.

The Socrata rows below are a third-party server and are NOT covered. They still drop silently. Do not let the fix be described as fleet-wide when part of the fleet is not ours.

• There is no district filter on the disclosure tool, by design — it keys on the member. Asking by district now returns an error naming council_member (#37, fixed in 1.3.0). Before 2026-07-21 it returned citywide awards with no warning — worth stating plainly to this audience, since the near-miss is the point.
• council_member matches as a substring. council_member="Powers" returns Selvena Brooks-Powers and Keith Powers. Budget MCP 2.0.0 now names every person matched and splits their totals; read that note, and use council_member_id for exactly one person. Background: #38.
• One tax ID can cover many programs. EIN 13-2612524 (Fund for the City of New York) carries dozens of programs. Filter by program_name as well as legal_name/ein, or you will silently aggregate unrelated recipients. This is a real analytical hazard for this audience, and worth mentioning to them as such.

Tool behavior

• Checkbook operates two limits, neither documented on their site: 1 request per second and 20,000 records per call. Exceeding the first puts you in a block that persists past the burst and returns 403 to every client on that IP, a browser included — which is exactly how a rate-limit block gets misread as a site outage. nyc-checkbook-mcp v1.6.0 ships a rate pacer, retry backoff, and no redirect following. Use v1.6.0 or later.
• smart_search is being removed, and the reason is worth knowing. It was never part of Checkbook’s supported API surface — the office told us so directly, and their logs show our July 16 calls to it rejected on that basis. This was never a WAF problem. Tracked in nyc-checkbook-mcp#25. Use the structured tools; search_spending requires fiscal_year or issue_date_from.
• search_contracts genuinely cannot filter by vendor name — the Checkbook contracts API filters vendors only by vendor_code and offers no name→code lookup. Credit where due: passing the declared vendor_name parameter returns an explicit error naming the limitation and listing the three supported alternatives, rather than silently returning unrelated contracts. That is the right behavior and worth showing this audience as an example of a tool that refuses rather than guesses. The sharp edge, and the better version of the story: until 2026-07-21 that careful guard was defeated by a one-word typo — the undeclared vendor bypassed it entirely and returned millions of unrelated rows. Checkbook 1.4.0 closes it: vendor is now rejected and the message points at vendor_name. A guard that only fires on the spelling you anticipated is precisely why schema strictness matters more than any individual check, and this server is now the worked example of both halves.
• search_legislation matches bill titles, not subject matter. Single keywords work; multi-word conceptual phrases return []. A sensible keyword can miss entirely — "encampment" finds nothing.
• get_voting_record returns real votes (since 2026-07-25/26), but defaults to the 50 newest rows with no total, repeats items recorded at committee and full council, and carries each matter’s introduction date, not the vote date. Pass a large limit, dedupe by file_number, and use vote_breakdown for dates.
• get_open_solicitations and Socrata catalog searches are very verbose — City Record notices embed raw HTML; catalog searches inline geometry (581 KB unbounded on the NYC portal, ~83 KB even at limit: 5 on the state portal). Always bound them.
• Socrata catalog search is federated and does not honor domain as a restriction. Check returned dataset IDs before saying which portal you’re in.
• Check is_sample on every Socrata query. If true, you got a raw row sample rather than an aggregate.

Figures and their provenance

Not independently verified this session:

• Whether the Local Law 174 CTE report was actually produced and simply not published as open data. Only the portal’s contents were checked. State the narrow claim.
• ✅ RESOLVED 2026-07-24 — the Late-Contracts claims were answered, and half of ours was wrong. Our 2026-07-09 audit said the prime-expense registration date was not exposed. The office corrected us: it is. The non-profit vendor classification genuinely is not published, and they will consider exposing it. This is the worked example of why the “absence we searched for is weaker than a presence we found” rule exists — we published an absence and it did not hold. Basis: team/engineering/2026-07-09-checkbook-2.0-api-audit.md, nyc-checkbook-mcp#11, and team/contacts/agencies/office-of-the-new-york-city-comptroller.md. Per the 2026-07-27 scope decision this is background, not presentation material — the dashboard critique is not in the script.
• ✅ RESOLVED — the letter was SENT 2026-07-22 and ANSWERED 2026-07-24. The old note here said “drafted and unsent”; that is now false and would be caught. We replied 2026-07-29. One question is genuinely outstanding: whether the 1 req/sec limit is scoped per IP or per client. That is the only part of this thread you should describe as open, and you are not asking the room to join it.

• Whether Checkbook’s rate limit is published anywhere we have not looked. We checked the API landing page, the global parameters table, and every domain page, and the office confirmed verbally that it is undocumented. That is strong, but it is still our search plus their word. Say “the office told us it isn’t documented, and we couldn’t find it either” rather than asserting it as fact.

Backup prompts (if one falls flat)

• “Which NYC agencies posted the most City Record procurement notices in the last month?” (procurement volume — on-theme)
• “Show FY2027 discretionary awards under the Digital Inclusion and Literacy Initiative.” (cross-district initiative view)
• “What does the City Charter say about the Office of Data Analytics?” (Charter § 20-f — short and reliable)
• “Search the Council’s discretionary funding disclosure for every award to [organization] across all available fiscal years.” (longitudinal single-org view; the strongest general-purpose watchdog query in the toolset)