AI features for SaaS products
AI features in a SaaS product (summaries, writing help, smart search) usually run on one good-value chat model, with a model from another vendor as a fallback and an embedding model for in-product search. For 300,000 AI calls a month with 3% falling back, at live prices in each model’s cheapest group, model costs are about ¥1,496 a month. Each tenant or environment gets its own key with separate limits and permissions.
Prices updated Nov 12, 2026, 8:17 PM|
01
The business problem
Getting one model working inside a SaaS product is easy; the trouble starts after launch. Tenants use wildly different amounts, so how do you cap them? When a vendor has an outage, does the feature still work? How do you split the monthly bill by tenant?
Integrating each vendor directly means another SDK, contract and payment method per model, and switching or falling back means code changes.
With one compatible API, the same code calls models from several vendors. In the console, each key gets a spending limit, an expiry and allowlists, and usage can be exported per key for reconciliation.
02
Recommended models
Primary model (chat)
gpt-5.4-miniOpenAIinput ¥1.5 / output ¥9 per 1M tokens· Standard groupServes almost all AI feature calls, so pick a model with steady quality and a moderate price. Tune your prompts and output formats against it.
Fallback (another vendor)
Claude Haiku 5.5Anthropicinput ¥0.4 / output ¥2 per 1M tokens· Claude Plus groupTakes over automatically when the primary returns 429 or 5xx, so one vendor’s problems do not take your feature down. Test its output format in advance.
In-product search (Embeddings)
text-embedding-3-smallOpenAIinput ¥0.04 / output ¥0.04 per 1M tokens· Standard groupPowers semantic search and related-content suggestions over documents, tickets and knowledge bases: high volume at a low unit price.
Recommended models are picked from the live catalog; prices are each model's cheapest group, updated Nov 12, 2026, 8:17 PM. Click a model to see every group's price.
03
Reference architecture
- 1
Proxy through your server
All model calls leave from your backend. Keys live only in server environment variables or a secret manager; the front end calls your own API.
- 2
A key per tenant
Create a separate API key for each large customer or environment (staging, production) with a spending limit, expiry, model allowlist and server IP allowlist.
- 3
Limits and fallback
Rate-limit per tenant in your service; when the primary model returns 429 or 5xx, switch to the fallback, and pass other 4xx errors back to the caller.
- 4
Record usage
Store tenant, model and the usage from each response in your own database, for billing tenants or enforcing plan quotas.
- 5
Reconcile monthly
Export Usage from the console each month and match it per key with your own records; disable a key as soon as you see abnormal usage.
04
Cost estimate
Assumptions: 300,000 AI feature calls a month at about 1,500 input and 300 output tokens each; 3% (9,000) go to the fallback model because of rate limits or outages; plus 100,000 search queries embedded at about 50 tokens each.
| Item | Model | Usage assumed | Cost |
|---|---|---|---|
| Primary model calls | gpt-5.4-mini | 300,000 × (1,500 input + 300 output tokens) | ¥1,485 |
| Fallback model calls | Claude Haiku 5.5 | 9,000 × (1,500 input + 300 output tokens) | ¥10.8 |
| Search embeddings | text-embedding-3-small | 100,000 × 50 input tokens | ¥0.2 |
| Total (per month) | ¥1,496 | ||
Calculated from live prices in each model's cheapest group, updated Nov 12, 2026, 8:17 PM.
Model usage only: your servers and databases are not included. The fallback share is an assumption that depends on how the primary model behaves, and initial document indexing is extra. Twice the calls means roughly twice the cost.
05
Sample code
Each tenant uses its own key; on 429 or 5xx the primary model hands over to the fallback, and the usage of every call is recorded for reconciliation.
// Server side only: API keys never reach the browser or the app.
import OpenAI from 'openai'
// One key per tenant or environment, each with its own spending limit in the console.
// Load them from your secret store; TUTUJIN_API_KEY is the default key.
function clientFor(tenant) {
return new OpenAI({
baseURL: 'https://open.tutujin.com/v1',
apiKey: process.env['TUTUJIN_KEY_' + tenant.id] || process.env.TUTUJIN_API_KEY,
timeout: 30_000,
maxRetries: 1,
})
}
// Primary first, then a model from another vendor.
const MODELS = ['gpt-5.4-mini', 'claude-haiku-5-5']
export async function summarize(tenant, text) {
const client = clientFor(tenant)
let lastError
for (const model of MODELS) {
try {
const resp = await client.chat.completions.create({
model,
messages: [
{ role: 'system', content: 'Summarize the content in 3 bullet points.' },
{ role: 'user', content: text },
],
max_tokens: 400,
})
await recordUsage(tenant.id, model, resp.usage)
return resp.choices[0].message.content
} catch (err) {
lastError = err
const status = err?.status
// 429 and 5xx: try the next model. Other 4xx (bad input, no balance, allowlists): stop here.
if (status && status < 500 && status !== 429) break
}
}
throw lastError
}
async function recordUsage(tenantId, model, usage) {
// Store per tenant in your own database; reconcile monthly with the Usage export in the console.
console.log(tenantId, model, usage?.prompt_tokens, usage?.completion_tokens)
}
// In-product search: embed documents and queries with text-embedding-3-small.FAQ
What do 300,000 AI calls a month cost?
At about 1,500 input and 300 output tokens per call, with 3% on the fallback model and 100,000 search embeddings, the cheapest groups come to about ¥1,496 a month.
Can an API key live in the front end or the app?
No. Anyone can extract a key from front-end code and use it directly. Keep keys on the server, have the front end call your own API, and check permissions there before forwarding.
How do I cap usage per tenant?
Create a separate key per tenant in the console with a spending limit, after which it stops working; in your own service, also rate-limit per tenant and check plan quotas.
How do features stay up when a vendor has problems?
Configure models from two vendors and switch to the fallback automatically on 429 or 5xx. Test the fallback’s prompts and output format before launch so the feature behaves similarly after a switch.
How do I split the monthly bill by tenant?
Store usage and the tenant ID for every call in your own database; export Usage from the console and match it per key against your records.