The Evolution of Product Engineering in the Age of AI

For over two decades, engineering teams have operated with a relatively stable optimization equation.

Build well. Ship fast. Keep the lights on cheaply.

Cloud computing refined the levers – autoscaling, caching, right-sizing – but the fundamental discipline remained the same: manage infrastructure costs, maximize developer productivity, and deliver business value reliably.

That equation is being rewritten.

A New Variable Has Entered the Room

Today, nearly every product team is integrating some form of AI. Copilots. Chat interfaces. Agentic workflows. Automated support. The pressure to ship AI-powered features is real, and in most organizations, it’s coming from every direction simultaneously.

But here’s what I’m noticing as I work with engineering teams: the adoption conversation is happening far faster than the economics conversation.

We’ve been so focused on Can we use AI? that we haven’t yet asked whether we can afford to scale it?

The equation has shifted:

Traditional: Business Value > Engineering Cost

Today: Business Value > Engineering Cost + AI Cost

And that second variable behaves nothing like the first.

Why AI Cost Is Fundamentally Different

A virtual machine costs money whether your users are active or not. Predictable. Forecastable. Manageable.

An AI system incurs cost every time a user asks it to think.

Every interaction generates input tokens, context tokens, retrieval tokens, output tokens. And agentic AI multiplies this in ways that aren’t always visible until the bill arrives.

Consider a simple support agent. A user asks one question. Behind the scenes, the agent calls three external APIs, searches documentation, summarizes results, evaluates its own response, and re-plans before answering. What looked like one request became twelve LLM calls and thousands of tokens.

Most teams don’t have observability into this yet. That’s the gap.

The Leadership Responsibility That’s Emerging

For years, engineering leaders have been responsible for managing three major cost centers: infrastructure, software licensing, and talent. We’ve developed mature disciplines around each – FinOps for cloud, procurement strategies for vendors, workforce planning for teams.

AI is introducing a new category of engineering spend that many organizations are only beginning to understand.

Token spend

Unlike infrastructure costs, token consumption is often invisible until it becomes significant. Unlike cloud spend, there are few established governance models, benchmarks, or operational playbooks. Most organizations are still treating AI costs as an experiment rather than an operating expense.

At the same time, developers are doing exactly what they should be doing: building the best possible experience. They optimize for capability, accuracy, resilience, and user outcomes.

Leadership, however, must answer a different question:

What is the most capability we can deliver for every dollar we spend?

That shift – from capability optimization to capability efficiency – may become one of the defining leadership challenges of the AI era.

Just as cloud adoption gave rise to FinOps, AI adoption is beginning to create the need for something new: AI FinOps.

The Metrics We Need to Start Tracking

We’ve built our engineering culture around DORA metrics and their descendants. Deployment frequency, lead time, MTTR – these measure the health of our delivery system.

They tell us how efficiently we deliver software. They tell us almost nothing about how efficiently we consume AI.

The metrics that matter now look different:

• Cost per AI interaction – what does it actually cost to serve one user session?
• Tokens per feature – how token-efficient is this capability?
• AI cost per revenue dollar – is this feature economically defensible at scale?
• Agent efficiency ratio – how much orchestration overhead is this agent generating?

Here’s a grounding example. Two features, each serving 10,000 monthly users. Feature A costs $200 in AI spend. Feature B costs $5,000. From a product dashboard, they look identical. Only one of them is sustainable.

We need to build these lenses before we scale, not after.

What Good Engineering Practice Looks Like Here

None of this requires blocking AI adoption. It requires maturing how we build with AI.

A few principles I’d encourage every engineering team to internalize:

• Match model to task. Not every problem needs frontier reasoning. Simple classification, FAQ retrieval, and structured extraction can often run on smaller, cheaper models. Intelligent routing — directing simple queries to lightweight models and complex ones to larger ones – can cut costs dramatically without touching user experience.
• Treat context like a database query. Sending entire documents into every prompt is the AI equivalent of doing a full table scan. Chunking, retrieving only relevant sections, and compressing context isn’t just a cost optimization – it’s good engineering discipline.
• Cache where it makes sense. If hundreds of users are asking functionally identical questions, paying for hundreds of independent generations is a waste. AI responses can often be treated as cacheable assets, just like any other expensive computation.
• Right-size your agents. Agentic orchestration is powerful, but it compounds cost at every step. Not every workflow needs a planner, a critic, a reviewer, and an executor. Simpler tasks should follow simpler paths. The orchestration itself is a cost center.
• Model cost before you scale. This one is non-negotiable. Token forecasting and load simulation should be part of production readiness the same way capacity planning is for infrastructure. Discovering that a feature is economically unviable at 10,000 users – after you’ve already gone live – is an avoidable problem.

The Deeper Shift

There’s something philosophically interesting happening in engineering right now.

For the first time in software engineering, reasoning itself has become a billable resource.

For decades, we optimized physical resources: CPU, memory, network, storage. We got extremely good at it. We built entire disciplines – systems design, database optimization, distributed architecture – around making computation more efficient.

Now we’re optimizing reasoning. Every unnecessary token is wasted reasoning. Every redundant agent call is wasted cognition.

The next generation of engineers will treat tokens, context windows, and agent calls the way their predecessors treated database queries, API calls, and CPU cycles – as finite, expensive resources to be thoughtfully managed.

That’s not a constraint. That’s craft.

Where This Is Headed

I think about AI maturity in four stages:

Adoption → Governance → Economics → Optimization.

Most organizations today are somewhere between the first and second. The shift to Economics – treating AI as a managed cost center with real accountability – is the work of the next 12 to 18 months.

The teams that get ahead of this won’t just be more efficient. They’ll be building AI products that are actually defensible at scale, where unit economics work and the underlying system is understood, not just deployed.

That’s the evolution of Product Engineering I’m watching – and it’s the one I think deserves more attention than it’s currently getting.

Most organizations are investing heavily in AI adoption.

How many are actively measuring AI economics?

I’m curious where your organization sits today: Adoption, Governance, Economics, or Optimization?