The Silent Squeeze: How AI Became an Unbudgeted COGS Crisis
When artificial intelligence features move from internal experimentation into core customer-facing software, variable compute demands create direct delivery costs. As detailed in the 2026 State of AI Cost Governance Report from Mavvrik and Benchmarkit across 396 enterprise organizations, 81% of companies report gross margin erosion from AI costs. Macroeconomic analysis of this survey by Apollo Global Management Chief Economist Torsten Slok in The Daily Spark highlights that 63% of surveyed organizations experienced a 6 to 15 percentage point decrease in gross margin.
This margin contraction follows an ongoing gap between rapid AI adoption and the financial governance required to monitor runtime compute. In the earlier 2025 State of AI Cost Management Report, which evaluated 372 enterprise organizations, 84% of companies saw 6%+ gross margin erosion due to AI infrastructure costs, and 26% reported a margin impact of 16% or higher.
Budgeting accuracy has deteriorated as deployments expand. According to the 2026 benchmark, only 11% of organizations forecast AI spend within plus-or-minus 10%, dropping from 15% in the prior year.
These budget variances directly affect strategic planning. According to the survey data, 62% of organizations reported that unexpected AI costs materially altered a business decision in the past year. When unexpected costs materialized, organizations faced concrete operational consequences: 49% repriced AI products, 40% experienced board-level escalations, 33% initiated emergency spending freezes, 31% reallocated existing budgets, and 25% delayed or cancelled initiatives.
Anatomy of the Bill: Where the Margin Points Actually Go
Unexpected AI expenses extend beyond model inference fees. The benchmark research demonstrates that surrounding data infrastructure and development tools frequently exceed foundation model bills.
According to research published in the AI Cost Governance 2026 overview, data platform usage overages were the top source of unexpected AI costs at 47%, ranking ahead of foundation model token costs at 43%. In the 2025 benchmark, data platforms led unexpected spend at 56%, network access costs ranked second at 52%, and large language models ranked fifth.
Cost drivers across the enterprise stack fall into three primary areas:
1. Data Ingestion, Storage, and Context Retrieval
Data platform usage overages represent the top source of unexpected AI costs at 47% because production systems depend on active retrieval pipelines. As outlined in the benchmark, AI features pull context, search data, run retrieval, refresh pipelines, call tools, trigger evaluations, and query data platforms. These surrounding operational costs frequently sit outside initial AI budget reviews because model invoices are easier to see.
2. The Agentic Execution Multiplier
Autonomous agents introduce variable execution patterns that traditional pricing and budget models fail to accommodate. According to the AI Cost Governance 2026 overview, 98% of organizations run agentic workloads, yet only 36% include agentic workflows in cost reporting.
As the benchmark notes, a single task can trigger multiple model calls, retrieval steps, data platform queries, tool actions, retries, and orchestration overhead. Because two runs of the same workflow can have very different costs, agentic compute is difficult to forecast from seat counts, fixed budgets, or simple request volume. Without attribution at the workflow, session, model, agent, and task level, teams cannot track which workflows are expensive or which use cases change unit economics.
3. Developer Tooling and Shadow Usage
Engineering tooling represents another expanding cost surface. According to the benchmark, only 42% of organizations include AI developer tools in AI cost reporting, and 39% report costs exceeding expected license or usage. As agentic coding workflows expand, developer tools increasingly affect compute, CI/CD activity, review workflows, and engineering operating costs.
The Accounting Blind Spot: The 70% COGS Failure
While 81% of organizations report moderate-to-high gross margin impact from AI costs, 70% have not required Cost of Goods Sold (COGS) based tracking for their AI expenditures. Furthermore, 67% of organizations track AI cost-to-serve and use it for pricing and margin decisions, while another 23% track cost-to-serve but do not price from it.
As Mavvrik notes, when AI is embedded in a product, every inference, generated answer, retrieval step, agent task, and data platform call can affect cost-to-serve. If that spend is measured only against revenue rather than tracked through COGS, teams may know AI spend is growing without knowing whether a feature is helping or hurting margin.
Hypothetical Example: How AI Workloads Erode Software Gross Margins
Hypothetical model illustrating how variable compute expenses compress subscription gross margins when absorbed into direct delivery costs. Stated baseline assumptions: assumed subscription revenue is $100.00 per user per month; assumed baseline delivery costs are $20.00 per user per month for hosting, database storage, and customer support; assumed AI feature usage is 300 automated workflow executions per user per month; and assumed AI compute unit cost is $0.04 per workflow execution for model tokens, vector queries, and context retrieval. Baseline unit economics: baseline gross profit is $100.00 - $20.00 = $80.00 per month, yielding a baseline gross margin of ($80.00 / $100.00) * 100 = 80.0%. When factoring in autonomous AI workloads, monthly AI compute cost is 300 * $0.04 = $12.00 per month, expanding revised total delivery costs to $20.00 + $12.00 = $32.00 per month. Revised gross profit is $100.00 - $32.00 = $68.00 per month, resulting in a revised gross margin of ($68.00 / $100.00) * 100 = 68.0%. Gross margin erosion: 80.0% - 68.0% = 12.0 percentage points.
Strategic Levers to Protect AI Profitability
Stabilizing gross profit margins requires aligning infrastructure telemetry with accounting structures and contract design.
1. Require Direct COGS Attribution
Finance leaders prioritize requiring cost-to-serve visibility and COGS-based tracking where AI touches the product. Assigning attribution at the workflow, product, customer, model, and agent level enables teams to evaluate margin contribution and identify which use cases change unit economics.
2. Connect Cost-to-Serve to Pricing Decisions
According to the 2025 State of AI Cost Management Report, companies charging for AI show 2x greater cost maturity in attribution and cost discipline. In the 2026 report, 67% of organizations track AI cost-to-serve and use it for pricing and margin decisions, while 49% have repriced AI products following unexpected costs. When restructuring pricing to protect profitability, teams evaluate the relationship between costs and prices using gross margin vs markup for pricing decisions.
3. Implement Unit Cost Telemetry Across the AI Stack
Cost governance requires tracking unit metrics across the operating footprint: cost per customer interaction, inference, workflow, agent run, coding tool user, GPU job, or successful task. Monitoring these unit costs reduces detection lag, enabling teams to act before consumption and runtime behavior turn into monthly reporting problems.
4. Evaluate Hybrid Infrastructure and Repatriation
Infrastructure architecture directly affects financial control. In the 2025 State of AI Cost Management Report, 67% of enterprises were actively planning to repatriate AI workloads from public clouds, with an additional 19% evaluating repatriation. In that benchmark, 61% of organizations already ran hybrid AI infrastructure across public and private environments.
