Reducing AI Token Costs by 80%: How We Built a CI/CD Layer That Pays for Itself

Originally published on ecupse.com.

The premise is simple: every time an AI agent runs your tests or deploys your code, it burns tokens. Tokens cost money. What if the agent never needed to do those things in the first place?

We run multiple projects powered by AI agents — a diary platform, a market analysis system, and internal tooling. Every one of these used to rely on Kiro (our AI-powered IDE) to handle CI/CD operations. Run tests. Deploy CDK. Sync assets to S3. Each request consumed 6–10 tool calls. At scale, this adds up fast.

This article explains how we built a system that eliminates those costs entirely: automated CI/CD via GitHub Actions, intelligent task routing via GitHub Copilot CLI, and steering files that teach our agents to stop doing work that machines handle better without AI.

The Problem

Our AI agent sessions were expensive because they were doing commodity work. A developer says "run tests" and the agent:

• Reads the project structure (2–3 tool calls)

• Finds and runs pytest (1–2 tool calls)

• Reads the output (1–2 tool calls)

• Interprets the results (1–2 tool calls)

That is 6–10 tool calls for something that should be automatic. Same pattern for deployments, CDK validation, and asset syncing. We were paying AI rates for what is fundamentally a pipeline problem.

Before vs. after, in a line: previously every CI task ran Developer request → agent reads files → agent runs command → agent reads output → agent interprets. Now it is simply git push → GitHub Actions → results in the GitHub UI, with zero agent involvement.

The Architecture

We built three layers that work together:

Layer 1: Shared Workflows Repository — A private repo containing reusable GitHub Actions workflows for testing, CDK validation, CDK deployment, S3 site sync, and scheduled monitoring. Every project references these with thin 5-line caller files.

Layer 2: Copilot CLI Routing — A set of PowerShell aliases that route lightweight tasks to GitHub Copilot CLI (~1 AI credit per call) instead of Kiro (6–10 tool calls per session). Code explanations, docstrings, commit messages, CI triggers — all handled at a fraction of the cost.

Layer 3: Steering Files — Per-project policy files that load automatically when an agent works in a project. They tell the agent: "do not deploy locally, trigger the workflow instead." Advisory, not blocking. The agent still helps — it just suggests the efficient path.

How routing works: when a developer sends a task, the agent reads the routing policy and classifies it. CI/CD work goes to GitHub Actions. Simple single-file tasks go to Copilot CLI (~1 credit). Complex multi-file work stays with the Kiro agent. Steering files and PowerShell aliases are what enforce that routing.

Every project needs CI/CD, but no project should maintain its own copy. We built five reusable workflows:

A consuming project needs exactly this to get full CI/CD:

name: CI
on:
  push:
    branches: [main]

jobs:
  test:
    uses: Fredwong76/shared-workflows/.github/workflows/ci-python.yml@main
    with:
      python-version: "3.12"
      test-path: "tests/"

Five lines. Tests run on every push. No agent involved.

For deployments, we use OIDC authentication — no long-lived credentials stored anywhere. GitHub Actions requests a temporary token, AWS validates the repo and branch, and grants 1-hour credentials scoped to the deployment role. Push to main with CDK changes, and deployment is automatic.

Copilot CLI: The Middle Layer

Not every task needs a full agent session. "Explain this function" does not require reading 15 files and reasoning about architecture. It requires reading one file and summarizing.

GitHub Copilot CLI runs at roughly 1 AI credit per call ($0.01). We built aliases that make it frictionless:

cop-explain <file>    # Explain code (~1 credit)
cop-review            # Review staged diff (~1 credit)
cop-test <file>       # Generate tests (~1.5 credits)
cop-deploy <repo>     # Trigger deploy workflow (~1 credit)
cop-logs <repo>       # Diagnose failures (~1 credit)

The key discovery: Copilot CLI has full access to the gh CLI via --allow-tool="shell(gh:*)". This means it can trigger workflows, read logs, list PRs, and manage issues — all at commodity pricing. We initially tried GitHub's MCP toolset (76 tools), but it loaded all tool schemas into context and cost 5–7x more per call. The shell(gh:*) approach gives identical capabilities at a fraction of the cost.

Steering: Teaching Agents the Policy

Steering files are markdown documents that load automatically into agent context. They are advisory — not blocking. The agent reads the policy and makes better decisions.

Our CI/CD steering file tells the agent:

When asked to run tests, deploy, or validate: trigger the GitHub Actions workflow. Do not execute locally unless the user explicitly says "run locally."

This is loaded via front-matter (inclusion: auto) every time a session opens in that project. The developer says "deploy my changes" and the agent responds with git push or gh workflow run instead of cdk deploy.

No hooks. No blockers. No token overhead. The steering file is zero-cost context that changes agent behavior.

OIDC: Keyless Authentication

We use AWS OIDC federation for deployments. No secrets stored in the repo. No IAM users with long-lived keys. The flow is: GitHub Actions requests a JWT from GitHub's OIDC provider → presents it to AWS via AssumeRoleWithWebIdentity → AWS validates the repo/branch claims and returns temporary 1-hour credentials → CDK deploys.

One gotcha we hit: GitHub changed the OIDC subject claim format on July 15, 2026. Repos created after that date send an immutable format with numeric IDs (repo:Owner@123/Repo@456:ref:refs/heads/main). Our trust policy needed the @OwnerID in the wildcard pattern. This cost us an hour of debugging — and is exactly the type of platform change our monitoring system is designed to catch early.

What We Learned

Steering files beat hooks. We tried a PreToolUse hook that intercepted commands and suggested Copilot CLI. It fired on every command call, added latency, and consumed tokens evaluating whether to intervene. A steering file achieves the same outcome at zero runtime cost.

GitHub MCP is overpriced for what it does. The 76-tool MCP server loaded 5–7x more tokens per call. The gh CLI does everything the MCP tools do, accessible via --allow-tool="shell(gh:*)" at ~1 credit per call.

Monthly cadence beats daily for platform monitoring. Kiro and GitHub do not ship breaking changes daily. A monthly scan with manual on-demand triggers is sufficient and avoids alert fatigue.

The deploy pipeline surfaces existing bugs. When we onboarded the AI Diary project, the CI immediately surfaced pre-existing test failures and missing dependencies that local deploys had been hiding. This is a feature, not a bug.

Reusable workflows compound. Building the shared-workflows repo took a few hours. Each new project onboards in minutes.

The Numbers

The Thesis

AI agents should do work that requires reasoning. Everything else should be automated infrastructure. The agent's job is to think, not to type pytest into a terminal.

This is part of an experiment at ECUPSE — a company where every operational role is run by an autonomous AI agent, built and documented in public. The system described here is live and processing real deployments. Full write-up: ecupse.com/articles/token-reduction-cicd.