Company research on free LLMs: 2.6 billion tokens for $23.85 in 14 days

Most of my research and synthesis work runs on free LLM tiers. On 2026-08-27 I put a local router in front of every free provider I could verify, logged every attempt to SQLite, and let the research pipeline hit it as hard as it wanted. This is the ledger for the first fourteen days, 2026-08-27 to 2026-09-09.

A third of the calls failed, eight of the 27 deployments I started with were gone or unreachable by the end, and two "free" providers billed a real card. Eleven config changes came out of reading the log; they are dated below with the measurement behind each one.

What the router runs

The workload is research. A script collects public evidence on a company (two web searches, the top result pages, the company's own site), then makes one router call that returns three files: a news summary, an insights sheet and an analysis. Reader agents check the output. A contact-resolution step guesses and verifies naming patterns. A vision tier reads screenshots. Interactive agents in opencode do the deck, summary and code work on top of that.

Every one of those clients talks to one endpoint: a LiteLLM proxy on 127.0.0.1:4000 with 27 deployments in 20 model groups across nine providers. Clients ask for a tier (auto, smart, deep, vision, fast, or the long-form writer) and the router picks a deployment, fails over on error, and writes one row per attempt to llm-usage.db. Failures get a row too, and every change below cites that table.

A daily probe, alphawatch, hits every provider's model list, makes one real call per deployment, diffs the catalogs against yesterday and flags dead model IDs. In fourteen days it reported red on seven mornings, amber on six, green on two.

The writer tier is the shape of the second week: it came online on 5 September at 9,786 calls in a day and has run at 2,000 to 10,000 a day since. The interactive tiers never went above 7,410 in a day, and the fallback groups are what the router reached for when a primary was benched or throttled.

Nine providers and what each one did

The medians are livable everywhere except NVIDIA. The 95th percentiles are the story: GLM on B.ai answers in 8 seconds half the time and 235 seconds one call in twenty, and DeepSeek-V4-Flash on NVIDIA reaches 334. Those tails are what the stream timeout and the reach-based chain order are for.

OpenRouter carried the volume. One model, MiniMax-M3 on the :free slug, took 24,702 calls at a 7.4% failure rate, a median of 4.8 seconds, and a 94% prompt-cache hit rate. None of its successful calls came back as reasoning with no answer. OpenRouter removed the slug on 2026-09-07, and the second free model I had wired there, GLM-5.2, failed 95.8% of the time from day one. OpenRouter's own limit on free models is 20 requests a minute per model, and the account also threw 402 insufficient-balance errors on free calls.

B.ai served the most tokens. It's an OpenAI-compatible reseller with a blockchain billing layer, and on a zero-balance account GLM-5.3-Flash and DeepSeek-V4-Flash answered anyway. GLM wrote the best long-form prose of anything in the free set and took 41.7 seconds on average to do it, with a 28.1% failure rate under load. B.ai paywalled DeepSeek-V4-Flash overnight on 2026-09-03: 339 of 350 calls fine one day, every call a 400 the next.

NVIDIA NIM has the biggest catalog and the smallest budget. Ten models were wired and all of them share one account-wide limit of about 40 requests a minute, so adding a model splits the budget instead of adding capacity. Median latency was 29 seconds, the 90th percentile 130 seconds, and the worst calls sat 400 to 478 seconds before the first token arrived. Kimi K3, the smartest model in the whole stack by index, answered 39 of 316 attempts one morning at 183 seconds each. NVIDIA retired MiniMax-M3 on 2026-09-09.

Cerebras is the fastest provider here at a 2.2-second median. It also billed $12.77 of prepaid credit against a tier I had filed as free, then returned 402 Payment Required on every call from 2026-09-05. All three Cerebras models are out of the config.

Z.ai is the one paid escape hatch I kept: a $5 top-up that runs GLM-5.3-Flash at a 6.9% failure rate and a 2.5-second median. It sits at the head of most fallback chains now because it's the same model as the B.ai primary, so a B.ai paywall degrades to the same quality instead of a weaker model.

Google Gemini gives 20 requests a day per model on the free tier. The 429 body says so in plain text. In a fallback chain that produced an 86.3% failure rate over 1,406 attempts.

Groq answered 1,175 of 1,176 calls. It was also a developer-tier account with a card on file, and it billed the moment Cerebras died and the fast tier fell through to it. I removed it and set a one-cent budget cap so a re-added entry cannot bill.

Cloudflare Workers AI allows 10,000 neurons a day and caps input plus output at 24,000 tokens combined, so a reasoning model with a 32,000-token budget fails before it starts. 70.9% failure. Its neurons are reserved for the fast tier.

Where the failures came from

The second row is the router's own doing. More than a third of all failures were requests bouncing off deployments the router had already benched. LiteLLM's default cooldown after a few failures is an hour. The limits I was tripping reset in about a minute. So one blip demoted a chain to its weakest tail for the rest of the hour, and every request in that hour that wanted the benched model got an instant router error. Seven days of logs showed about 4,700 of 5,600 failures in that class. Cooldown went from 3,600 seconds to 300, then to 60 on the deployments with minute-level limits.

What changed and when

The failure rate followed those changes. On 2026-08-28 it was 57.5%. By 2026-09-01 it was 2.7%, and 1.6% the day after. Then on 2026-09-05 the writer tier came online at 9,786 calls in a day, Cerebras stopped answering, and OpenRouter began killing free slugs; the daily rate went back to 45.2% and the interactive tiers had their worst day at 59%. By 2026-09-07 interactive calls were back to a 9% failure rate while the writer tier kept hammering at 25% to 50%. The two tiers share no chain, so the two numbers move independently.

Reasoning models eat budgets

Everything the models wrote in fourteen days came to 58 million tokens. 26 million of those were reasoning: 44.8% of the output was the model thinking rather than answering. GLM-5.3-Flash spent 2.18 reasoning tokens for every token of answer. DeepSeek-V4-Flash on NVIDIA spent 1.92.

The ratio sets the completion budget a call needs. The research script started with a 6,000-token completion budget and got empty answers back, because the model spent the whole budget thinking and hit the length limit before writing a word. At 12,000 tokens it truncated 44 of 98 three-file drafts. The working default is 16,000, doubling to 24,000 when the finish reason is length. Across the whole ledger, 603 successful calls came back as reasoning with no answer at all; 415 of those were DeepSeek-V4-Flash on NVIDIA, 7.8% of everything it returned. One call spent 5,058 reasoning tokens per token of output.

gpt-oss-120b has the opposite problem. It thinks briefly, writes fluently, and hyphenates with U+2011, the non-breaking hyphen. A search for that character finds its output wherever a fallback let it through.

What the research stage produced

The tradeoff inside that stage is between the two models that did most of the work. MiniMax-M3 on OpenRouter wrote 2,230 dossiers at a 4% failure rate, 22 seconds a call, about 2,240 output tokens each. GLM-5.3-Flash on B.ai wrote 698 at a 15% failure rate, 116 seconds a call, about 8,470 output tokens each. GLM writes four times as much per call and reads better, at five times the latency and four times the failure rate. The fast model runs the first pass across two thousand companies, and the fifty worth a second look get a GLM call.

The reader agents that check the output run on the same router. Two independent readers score each draft and a flag only counts when both raise it on the same span, because single-reader flags turned out to be noise. That gate has caught drafts that passed every regex rule.

Free tier moves under you

Eight of the 27 deployments I wired on 2026-08-27 were gone or unreachable by 2026-09-09. OpenRouter removed both of its free slugs: MiniMax-M3 on 2026-09-07, and GLM-5.2, which had failed 95.8% of calls from the start. NVIDIA retired MiniMax-M3 on 2026-09-09. Cerebras put all three of its models behind a paywall on 2026-09-05. B.ai paywalled DeepSeek-V4-Flash on 2026-09-03. Groq billed a card on 2026-09-05 and 2026-09-06. Two new free models appeared on OpenRouter on 2026-09-09 and went to the ends of the chains until they have a week of log behind them.

Tracking that takes one scheduled script. alphawatch runs at 08:00, pulls each provider's model list, makes one real call per deployment, diffs the catalogs against the previous day, and writes a report naming every wired model ID the provider no longer serves, with the closest replacement in the catalog. The config carries a dated comment beside every chain change with the log line that caused it. Between the two, I can say why any deployment sits where it does.

What I'd set up on day one

1. One endpoint and one log, with failure rows. Put a proxy in front of every provider and write one row per attempt, including the attempts that fail. Each config change above cites a query on that table.

2. One tier per workload. Interactive, long-form writing, vision and fast requests each get their own model group. On 2026-09-07 the writer tier failed 25% of its calls and the interactive tiers failed 9%, because the two never share a chain.

3. Order chains by measured reach, with the intelligence index as the tiebreaker. After four days of log, compute the share of attempts each deployment answered. Two of the smartest models in this stack answered under 16% of theirs and moved to the tails.

4. Set cooldowns to the limit that trips. The per-model and per-account limits here reset within a minute. 300 seconds is the default now, 60 on the minute-limited deployments. At an hour, one 429 turned into an hour of instant router errors.

5. Set a stream timeout as well as a request timeout. The request timeout does not cut a stream that sends nothing. The longest legitimate reasoning in this log was 173 seconds, so the stream cut is 200.

6. Give reasoning models a 16,000-token completion budget, and retry with double on a length finish. At 6,000 the research script got empty answers. At 12,000 it truncated 44 of 98 drafts.

7. Cap spend on every provider that has a card on file. Groq and Cerebras both billed inside two weeks. Groq's cap is now $0.01 per 30 days; the paid DeepSeek escape hatch is $10.

8. Fund a paid copy of the primary model and put it first in the fallbacks. A $5 Z.ai balance runs the same GLM-5.3-Flash as the B.ai primary, so a B.ai paywall keeps the same model answering.

9. Let quality chains fail closed. If the last fallback is a model whose output you would reject, point the chain at a group that does not exist. LiteLLM raises and the worker retries.

10. Probe daily with one real call per deployment. Model lists kept showing IDs that returned 404 on a completion, and a health endpoint reports on the proxy, not the provider. alphawatch's one call per deployment is the check that caught every removal in this log.

---

If you run research or synthesis agents and want the router, the log schema and the probe set up against your own providers, book a call.