That Codex command from this morning

This morning, the Codex acceptance task got stuck at the connection stage—I asked a colleague and found out that the account of the person in charge of the company's OpenAI workspace had been suspended, and the entire workspace (including the Codex our team is using) was shut down along with it. This incident is the reason I started writing, but it is not the main topic of this article.

That Codex command kept failing after a few retries, and it was only after connecting the logs that I saw the CLI itself was locked up; the appeal email has already been sent, and OpenAI said it will take at least three business days to reply, with no guarantee that it can be recovered. For now, only Claude Code is left running this acceptance layer—while the small models on LiteLLM are still there, I don't plan to have them take over this layer, which I'll explain in the "Three-Tier Division of Labor" section later.

What this is actually about is how I went from "burning through the 20x Claude Code quota and still not having enough" recently, all the way to trying out a workflow of "using LiteLLM to connect to five official APIs, Claude Code for planning, small models to run as workers, and dual-model verification," while along the way reviewing what level models like DeepSeek, Kimi, GLM, Grok, and Qwen are actually at when used for real work.

20x—Why Is It Still Not Enough?

Claude Code 20x subscription quota—I had it handle planning, writing, debugging, and verification all by itself, and it ran out in about four days. My gut reaction was "time to upgrade to the next tier," but 20x is already the limit for subscription plans, and going any higher means switching to the API.

Looking back through my ccusage logs, I realized the problem isn't simply "using too much"—it's that "tasks that shouldn't be run on Opus are being run on Opus": updating the exact same import path across N files, translating N sections of markdown into English, and running N greps to organize results. For these kinds of batch + mechanical tasks, using DeepSeek V3 or Qwen Flash can save 90% of the costs with a negligible difference in results.

If you break it down along this line, it naturally divides into three layers:

That's when we started looking into LiteLLM. On the one hand, we wanted to find a cheap worker model to connect the second layer; on the other hand, we were also curious about what it would actually feel like to run those frequently discussed models like DeepSeek, Kimi, GLM, Grok, and Qwen.

Three-Tier Division of Labor: Planning / Worker / Double-Checked Acceptance

Planning is left to Claude Code, workers are assigned to smaller models on LiteLLM, and verification is done by having Claude Code and Codex cross-check each other. This three-tier division of labor was figured out through gradual experimentation:

For cross-validation, run two models: Claude Code once, with the requirement to follow “The Six Iron Laws of Testing" grade item by item and pick out the violations (the next section will elaborate on the rules themselves and their sources); Codex runs another pass, directly feeding the git diff of the worktree, asking it to find bugs, naming inconsistencies, and unhandled edge cases from the perspective of the code itself.

Both models have their respective blind spots. Claude Code knows "my house rules" well, but its sensitivity to pure syntax or edge-case errors isn't as high as Codex's; Codex reviews diffs carefully, but doesn't know my rules. Stacked together, their blind spots complement each other.

The Six Iron Laws of Testing

These six rules weren’t created all at once; they were developed through trial and error over multiple iterations. They now serve as the basis for Claude Code to score each worker’s output during acceptance testing:

Origins and Evolution

It all started with two articles:

Initially, Katerina proposed 4 directions: mutation testing, AI-aware code review, CI pipeline validation, and prompt engineering. After reviewing them, I started applying them, beginning with the first one—mutation testing—and added a few more based on my own trial-and-error experience:

• Add "Separation of Writing and Testing" to the first round—After running my own 27 tests, all of which passed, I had an independent sub-agent (who wasn’t shown the implementation) write the code, and they immediately found two actual bugs.
• Add "invariants first" in the second round—"Fixed I/O pair" is a tautology; only "invariant" is implementation-independent.
• In the third round, add “runtime → regression” and “mocks limited to external I/O.”—Both of these issues were discovered when bugs—which had been marked as “tested” after being mocked—surfaced in production.
• Finally, add: “A draft is not the final product.”—Based on the conclusion from Salesforce VIBEPASS: Don’t blindly trust tests written by AI; you must review them before committing.

That makes six. The verification prompt will stuff these six rules into the system prompt, instructing Claude Code to check the diff + tests submitted by the worker against each rule one by one, and list out any violations, why they violate the rules, and how to fix them. Codex doesn't recognize these six rules, so Codex runs on the orthogonal perspective of "pure code bugs and standards"—which is also why pairing the two models together is more useful than running a single model twice.

Why are there five official websites, and not OpenRouter?

As for the question of who to hire as a worker, I ended up selecting five direct accounts, and gave up on OpenRouter—this type of aggregator lets you integrate all models using just one SDK and one payment gateway.

After reading that supplier comparison on koc.com.tw, the middleman approach initially seemed like the right direction, but working through the numbers, it had four drawbacks:

• The markup eats into the official promo. While a 5%–20% markup by resellers may not seem like much, you’ll miss out on all the promotions, new-customer offers, and limited-time discounts offered by the original manufacturers.
• Unable to obtain the official website-exclusive quota. DeepSeek, Qwen, and Moonshot have all offered tokens to new users or as part of promotions; you must have a direct account to claim them.
• Payment Processing "Black Box," Difficult to Debug. It's unclear whether the error is due to a routing issue with the intermediary or a failure on the part of the upstream provider.
• prompt / response: Uncertainties in Processing. Did the middlemen modify the system prompt, tool schema, or stream chunking? This is just noise for "evaluating each provider's native performance"—one of my original curiosities was to directly see the native capability of each provider, and this point made me completely abandon the middleman approach.

The five accounts I ultimately selected:

Later, I added MiniMax and Google Gemini, mainly to make the fallback chain deeper.

Google Developer Program Credits

After I joined the Google Developer Program, the Premium tier issues $10 in monthly Gen AI & Cloud credits every month—one-time credits, valid for one year—that can be applied directly toward Gemini API usage in AI Studio and Vertex AI.

Google Developer Program credits show their issuance history in the Google Cloud Billing console. Each credit ranges from $315 to $320 in face value, totaling $300+ per month to call the Gemini API:

In other words, once you connect Gemini to LiteLLM, the “Free Model” column on this gateway automatically appears—use it when your quota runs out: hook up gemini-2.5-flash as a worker without worrying about the cost. This is an added perk of "buying directly from the official website"—something you won't get if you go through OpenRouter.

Why did I choose CCR as the interface? Because I still want to use Claude Code.

I've recently tried out all the mainstream CLI agents like Claude Code, Codex, Gemini CLI, and Copilot CLI, and my conclusion is that Claude Code is still the most intuitive—its tool use design, subagent mechanism, skill/slash command system, integration with git worktree, context management, and permission flow are all a step ahead overall. The list of what I tried:

If I want to run other models as well, the ideal approach is to “swap out the model while keeping the interface.”

At first, I actually thought I could just solve it with “LiteLLM + a zsh function”—by pointing Claude Code’s ANTHROPIC_BASE_URL at LiteLLM’s :4000 and tweaking the request body myself. It wasn't until I actually started coding that I realized the details of each protocol vary significantly: Anthropic's messages API and OpenAI's chat/completions differ completely in formatting across system messages, tool definitions, stream chunking, and reasoning content blocks, and any slight misalignment results in a 400 error or silently dropped messages. I spent about half a day writing a transformer and realized this isn't something I should be doing myself—that was when I found claude-code-router (abbreviated as CCR), this off-the-shelf transformer.

Run CCR locally :3456 server that acts like the Anthropic API to Claude Code, while transforming downstream requests into the OpenAI protocol and sending them to LiteLLM. From Claude Code's perspective, it "thinks" it is talking to Anthropic, but the requests are actually routed to DeepSeek, Kimi, or Grok. In other words, CCR allows me to use the Claude Code interface to run any model on LiteLLM. No need to change interfaces, no need to retrain muscle memory, and the entire subagent/skill ecosystem is fully inherited.

Why not use cc-switch?

Along the way, I also tried cc-switch, a community tool that lets you switch the Claude Code provider with a single click—it seemed like exactly what I needed. After trying it, I found that its switching mechanism directly modifies ~/.claude/settings.json:

• ~/.claude/settings.json is Claude Code's single global file. Change it once, and every Claude Code session on the entire machine updates accordingly.
• My workflow relies heavily on parallel tmux panes—one pane running Kimi for planning, one pane running a DeepSeek worker, and one pane running Grok for handling long contexts. The first time I use cc-switch, all three panes turn into the same provider, and the second time I use it, they all turn into another. There's simply no way to "run different models in parallel."

The CCR + ENV var approach is exactly the opposite—ANTHROPIC_BASE_URL and ANTHROPIC_MODEL are per-process environment variables; each tmux pane runs in its own independent shell, so they do not interfere with one another. The cc-llm in the next section runs on exactly this mechanism:

Here's what it actually looks like—a tmux window divided into four vertical columns, with one Claude Code instance in each column, each running a different model:

In the status bar, you can see the bindings for each pane: litellm,deepseek-r1, litellm,kimi-k2-turbo, litellm,minimax-m2.5, litellm, qwen3.5-flash. Four panes running simultaneously, each using a different model; CCR :3456 is just a proxy that watches request routing—it doesn't care which pane the request comes from. ~/.claude/settings.json is never touched at all—all switching happens within the shell session. For the scenario of “running multiple models simultaneously on the same machine,” CCR is the only existing solution capable of doing so.

Technical Framework: LiteLLM + CCR + Custom zsh function

There are two paths for routing: Claude Code speaks the Anthropic protocol, so it first goes through CCR's :3456, which converts it to the OpenAI format, then into LiteLLM's :4000; the CLI tools Codex, Gemini, and OpenCode already speak the OpenAI protocol, so you can just run LiteLLM directly with :4000, without going around CCR. Here's what it looks like in a diagram:

LiteLLM is the only gateway. All model_list, API keys, fallbacks, and cost telemetry are consolidated into a single config.yaml: one entry per provider, with all API keys referenced via environment variables. It looks something like this (here's an excerpt of one entry to give you an idea):

drop_params + modify_params is the "avoid-400" double safeguard I added after hitting snags, saving the effort of manually aligning each provider's parameter schema. Once fallbacks is set up, Model A receives a 429/5xx and automatically switches to Model B.

Over at CCR, config.json has a simpler structure, and most importantly, the Router section maps task types to specific models:

Finally, here’s a custom zsh function that turns everything into a single command cc-llm (Desensitized Scaffold):

In practice, running cc-llm directly opens the fzf menu, which automatically retrieves all currently available models on LiteLLM and displays them grouped by provider:

After selecting a model, Claude Code launches into TUI, and the status line clearly indicates which path is being taken, which model is being used, and what the effort level is set to:

For example, in the one above, the title bar "Claude × grok-4.1-fast · CCR → LiteLLM" is the message printed by the zsh function, and "litellm,grok-4.1-fast" below the status bar is read by Claude Code itself from ANTHROPIC_MODEL. As shown, the effort is locked at xhigh, and bypass permissions are enabled—once you've confirmed that these match your expectations, you can proceed with confidence.

Other uses:

Similar authors have also written cx-llm (Codex), gm-llm (Gemini CLI), oc-llm (OpenCode)—These three protocols are incompatible with CCR, so they connect directly to LiteLLM. All CLIs use a single instance of LiteLLM's config.yaml; to add a new model, you only need to make one change.

Even subagents have secondary models

I changed the model for the four subagents that run simple tasks—explorer, researcher, browser, and media—from the default sonnet to haiku. In the subagent definition files (~/.claude/agents/*.md), the model: field acts as the switch that controls this: the main dialog delegates a task to a subagent, the subagent finishes and returns the result, and the main dialog then continues processing.

The commonalities among these four tasks: heavy tool calls, shallow reasoning, low required model intelligence, but they consume a massive number of turns and context. Originally, they were all run on Sonnet, consuming an astonishing amount of quota; after switching to Haiku, costs were slashed by 90% with no noticeable drop in quality.

A bit smarter is dynamic adjustment—I wrote a model-policy daemon that checks how much Claude quota is left every 5 minutes, divided into four tiers: normal / boost-1 / boost-2 / boost-3. When the quota is abundant, haiku is just haiku; when the quota is about to blow (5h < 25% or 7d < 40%), it automatically upgrades haiku to sonnet and sonnet to opus, making full use of the "larger model when available" and automatically downgrading "when it's time to save."

Know how much each model is currently using

The workflow is set up. There’s just one small issue left—there are eight dashboards in total, including the five companies and the Big Three, and each website has a different layout. Here are a few screenshots of each company’s admin panel for you to take a look at:

Every platform has a different design logic—the question of “Who should I assign my next task to?” can’t be answered on any single backend; you either have to open six tabs and compare them manually, or integrate the data yourself.

So I put together a simple monitoring page that crawls the balance once a day at a set time. It has three views:

Quota view—Here’s a comparison of subscription-based quotas (Claude 5h/7d/EX, Codex 5h/7d, Gemini Pro/Flash 24h) and their reset times:

I could tell at a glance: Claude had only used 7% in 5 hours (about to reset), had already consumed 60% in 7 days (with about two days left until the next reset), and the EX monthly limit of $100 was only at 11%. Whether to keep pushing or switch to a worker model first—the quota view makes that decision easy.

Dosage view—LiteLLM Prepaid Credit, Top-up / Remaining / Used / Utilization Rate by Provider:

The "Usage" view is the one I check most frequently. DeepSeek is at 50%, Moonshot is only at 17%, and MiniMax is only at 3%—this means DeepSeek is taking on too many batch tasks, and next time I can distribute some of them to Moonshot or MiniMax to prevent a single provider from running out first. Google Cloud credits are also included on this page (they're free, but running low needs to be tracked).

Model Gallery view—An overview of each brand’s flagship, “Fast,” and “Value” models, along with recommendations for the Arena, SWE-Bench, and Speed scenarios:

Model compendium is updated every few days (Consensus ranking from 4 sources), used to determine "which model should be routed to what task now." On 2026-05-20, the top choice for mathematical reasoning switched from DeepSeek R1 to Kimi K2.6 Thinking—right after the switch, LiteLLM's config.yaml and CCR's Router were updated simultaneously.

Compare against the provider model list to detect "outdated settings"

The crawler also does a fourth thing—it scrapes the /v1/models endpoint on each provider's official website and checks it against LiteLLM's config.yaml, specifically the model_name entries. This one was added only after running into some issues.

This has happened several times in the past: I open Claude Code in the morning to run a job, and CCR :3456 returns a 404 "Model Not Found" error; after searching for a while, I discover that a certain supplier changed xxx-preview to xxx-stable, or retired the old v3.1 outright and replaced it with v3.2—while LiteLLM's config.yaml is still using the old names, so the entire path breaks. Frontier models are being replaced faster than expected; DeepSeek, Moonshot, and Qwen might change their model codes every week or two, and new and old naming conventions often lack backward compatibility.

The logic behind it is quite simple:

The dashboard highlights stale in red and new in green, so you can tell at a glance "which config lines are already obsolete and should be cut / which new lines have come out and can be evaluated." After this check shifted from post-hoc debugging to proactive early warning, the scenario of "discovering model 404 only upon startup" disappeared.

The backend consists of a daily crawler, and the frontend is a simple single-page app I put together myself. My motivation for building it wasn’t to show off my technical skills; it’s because no existing tools integrate this information, so if I wanted to see it, I had to build it myself.

Who Is Suited for It, and Who Isn't?

The benefits boil down to three things: a three-tiered division of labor ensures quotas are allocated where they’re needed most; it also helps gauge the true capabilities of other companies’ models; and the workflow won’t grind to a halt if a single provider stops supplying data.

• Three-Tier Division of Labor and ROI Alignment—Planning layer keeps the smartest model, workers run on $0.2/$0.5 tier batch jobs, and verification uses two models cross-checking. The same quota lasts a few days longer, and the resulting quality is even stabler than using a single model from start to finish.
• "Learning about other models along the way" is a bonus—After trying it out for a while, I’ve developed a feel for which models perform best on which types of tasks, so I don’t have to rely solely on other people’s benchmark charts. The Model Catalog page is the result of that accumulated experience.
• If a single provider stops supplying, the entire system won't shut down.—If it gets locked up like Codex did this morning, the workflow will be downgraded to a single-path acceptance process, but the two pipelines—Claude Code and LiteLLM—will continue to run.

The trade-off is threefold: you have to maintain two additional daemons; you have to figure out each provider’s quirks on your own; and no one decides for you which task should be paired with which model.

• Now there are two more daemons to maintain: CCR and LiteLLM. CCR upgraded and modified the transformer schema, and LiteLLM 1.55+ changed the spend tracking fields. Every time, I have to go back and debug for half a day.
• You'll have to figure out each provider's unique characteristics on your own.—Moonshot occasionally experiences sudden spikes in latency, DashScope occasionally returns 5xx errors, and xAI has had a few instances of response format drift. If the fallback chain isn't set up properly, you'll run into these issues.
• "Which task should run which model?"—no one decides that for you.—It adds another layer of mental burden; you either have to rely on experience or on auxiliary tools like model guides.

So this setup is for people who find that a 20x subscription really isn't enough, and are willing to spend a weekend setting up the infrastructure. If 20x works well for you, or if you don't want to run two additional local daemons, it's easiest to just leave things as they are—the drawbacks of this setup outweigh the benefits.

My reason for staying comes down to a single sentence: my curiosity about "what happens when you run the exact same task through different models" hasn't faded yet, and dividing the work across them has indeed been much kinder to my quotas. When the quota runs out, it doesn't force me to upgrade my subscription; instead, it forces me to figure out precisely which layer is burning through resources—this feeling of being "pushed by resource constraints to think" is actually my favorite part.

Which layer should this task be assigned to?

The CCR Router above has only four slots, but real tasks come in more than four kinds. Take a look at what the task on hand looks like, categorize it here using this article's definition of "planning / worker / acceptance," and print out the article's original sentence for that category. The category and the Router's four slots are two separate axes: if it maps to a slot in this article's config.json, whatever model is in that box gets printed, and if it doesn't match, it says "not found"—this article keeps both planning and acceptance testing here on Claude Code side, leaving no room for them in the four Router slots. It runs entirely in the browser, requires no API key, and sends no data out whatsoever.

Further Reading

This workflow references resources that have actually influenced design decisions.