Skip to content
Chapter 04 · 10 min

The real costs

AI projects blow their budgets in a predictable way: the visible cost — the model API — is the tip of an iceberg, and the part that sinks projects is below the waterline. This chapter is the full cost picture, so your budget survives contact with reality.

The cost iceberg of an AI featureAbove the waterline, the visible cost: the model API calls. Below it, the larger hidden costs: data work, evals, engineering, monitoring, security, and ongoing operations.what gets budgetedAPI callsdata pipelines & cleaningeval sets & iterationengineering & integrationmonitoring & securityongoing operations

“The puppy is free. The vet bills, the chewed shoes, and the fifteen years are the actual price.”

The cost everyone sees

The model API bill is real but usually small, and it's the only number most early estimates include. You pay per use — per chunk of text in and out. For many applications it's surprisingly cheap; for high-volume ones it adds up and needs managing. Either way, it's the part that gets budgeted because it's the part with a price list.

The cost iceberg of an AI featureAbove the waterline, the visible cost: the model API calls. Below it, the larger hidden costs: data work, evals, engineering, monitoring, security, and ongoing operations.what gets budgetedAPI callsdata pipelines & cleaningeval sets & iterationengineering & integrationmonitoring & securityongoing operations
Above the waterline: the API calls everyone budgets. Below it: the data, evals, engineering, monitoring, and operations that actually determine the cost.

The costs that sink the budget

Below the waterline is where the money goes. None of it is exotic — it's the ordinary cost of turning a demo into something reliable — but it's routinely left out of the pitch.

  • Data work — getting your data clean, accessible, and structured enough for the AI to use. Often the single biggest line, and the least glamorous.
  • Evals and iteration — building the test sets that tell you whether it works, and the cycles of tuning to get there. This is most of the engineering effort.
  • Engineering and integration — wiring the AI into your real systems, with all the edge cases and failure handling that demos skip.
  • Monitoring and security — watching it in production, catching failures and abuse, the controls from any serious deployment.
  • Ongoing operations — it's not done at launch. Models change, prompts drift, data goes stale; someone maintains it forever.

The pilot trap

The most expensive AI mistake isn't a project that fails — it's a project that demos beautifully, gets celebrated, and never ships. The proof-of-concept is cheap and impressive precisely because it skips everything below the waterline. The leap from "works in the demo" to "works in production" is where most of the cost and most of the failures live.

Guard against it by treating the demo as the start of the work, not the proof it's nearly done. The honest question after a successful pilot is not "when can we launch" but "what will it take to make this reliable, and is that still worth it?" Often it is. Sometimes the answer reveals the project never made sense at production cost — and finding that out after a cheap pilot is a win, not a failure.

Budgeting that survives

A realistic AI budget treats the API as a small, variable line and puts the real money against the engineering, data, and operations. It funds the eval work explicitly, because that's what separates a system you can trust from one you hope works. And it includes ongoing cost from day one, because an AI feature is a living system, not a one-time build.

In one line each

  • The API bill is the visible tip; it's usually small and the only thing early estimates include.
  • The real cost is below the waterline: data work, evals, engineering, monitoring, and ongoing operations.
  • The pilot trap: a beautiful demo skips everything that makes it reliable — the leap to production is where cost and failure live.
  • Budget the API as a small line, fund the eval and operations work explicitly, and include ongoing cost from day one.

Where to go next

The real costs · AI courses · SDEN