How to control LLM API costs: 10 things you can do today

LLM API spend = number of calls × (input tokens × input price + output tokens × output price). Only three things move: the unit price, the tokens per call and the number of calls. The ten measures below are roughly ordered by impact. The first two — the right model and tiered routing — usually save the most, then controlling input, then controlling output.

1. Pick the right model tier

A vendor's flagship and its small model can differ in price by tens of times. Run your evaluation on the small model first and move up only if accuracy falls short. How to evaluate: How to choose a model.

2. Tiered routing: only hard questions go to the big model

Let a cheap model judge the type or difficulty of each request, answer the easy ones directly, and pass only the hard minority to a big model. Say you have 100,000 requests a month with 2,000 input and 400 output tokens each.

Everything on the flagship:

80% routed to a small model:

The remaining 20% on the flagship:

Add the last two for the routed total. The routing step costs a little too, so use the cheapest model and have it output just a label.

3. Trim the prompt

The system prompt is sent with every request. Cut repeated instructions, surplus examples and pleasantries; a few hundred tokens saved, times hundreds of thousands of calls a month, adds up.

4. Cap the output

• Set max_tokens, or max_completion_tokens on reasoning models.
• Spell out format and length in the prompt, such as "output JSON only" or "under 80 words".
• On reasoning models, reasoning_effort can lower the thinking effort where supported.

Output usually costs several times as much as input. With 100,000 calls a month and 2,000 input tokens each, 800-token answers:

Answers cut to 300 tokens:

5. Use context caching

Put the unchanging system prompt and documents first; on models with caching, hits are billed at the cache-read price, usually far below normal input. Details and savings: What context caching is and how it saves money.

6. Trim the chat history

Multi-turn chats resend the history every time, so by turn 20 a request's input can be more than ten times that of turn 1. Keep only the last few turns and have a small model summarize the rest.

7. Pick a group with the right price

The same model costs different amounts in different groups. Use the lowest-price group for development and for batch work that can be retried, and official or cloud groups for production — see How to choose a group.

8. Put many items in one request

When processing lots of short texts, such as classifying product titles, send about 20 per request and ask for a JSON array in the same order. The system prompt goes out once instead of 20 times. Do not overload a request — models start skipping items — and check the count when results come back.

import json
import os
from openai import OpenAI

client = OpenAI(base_url="https://open.tutujin.com/v1", api_key=os.environ["TUTUJIN_API_KEY"])

titles = ["Men's cotton T-shirt", "Stainless steel bottle 500ml", "Kids' non-slip sneakers"]  # about 20 per batch in practice

resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": "Assign each product title one category: apparel, home, footwear, other. Output a JSON array in input order and nothing else."},
        {"role": "user", "content": json.dumps(titles)},
    ],
)
labels = json.loads(resp.choices[0].message.content)
assert len(labels) == len(titles), "count mismatch, rerun this batch"

9. Give every key a spending limit and a model allowlist

An infinite loop or a leaked key can generate a flood of calls in minutes. Give each key a spending limit and an expiry date, and use the model allowlist so it can only call the models your product needs — nobody can use it to call an expensive model. More in API key security best practices.

10. Review usage regularly

Once a week, open Usage in the console, look at spend by model and by key, find the calls that cost the most and optimize those first. Watch especially closely in the first days after a feature launch.

Summary

Billing rules: How LLM API pricing works.