Rate limits

Requests that arrive too fast get a 429. Limits can come from two places: this site's protective limits on accounts and keys, and the rate limits model services apply to each line. Actual limits change with the line and with load, so this page gives no fixed numbers; it covers how to design your calls so your product keeps running smoothly when limits kick in.

Where limits come from

Different groups use different lines, so the same model can behave differently under load in another group.

What to do on a 429

1. Back off and retry: wait, double the wait each time with random jitter, and give up after a few attempts. If the response has a Retry-After header, wait that long. Code in Errors and retries.
2. Do not retry immediately in a tight loop. That only makes traffic denser and keeps you limited for longer.
3. If retries keep failing, put the job back in a queue for later or fail over to a backup key.

Limit concurrency

Batch jobs are the usual trigger. Cap concurrency on the client so only a fixed number of requests run at once and the rest wait:

import asyncio
import os

from openai import AsyncOpenAI

client = AsyncOpenAI(base_url="https://open.tutujin.com/v1", api_key=os.environ["TUTUJIN_API_KEY"], max_retries=4)
limit = asyncio.Semaphore(8)  # requests in flight at once; tune for your workload


async def summarize(text):
    async with limit:
        resp = await client.chat.completions.create(
            model="deepseek-v4.1-flash",
            messages=[{"role": "user", "content": f"Summarize in one sentence: {text}"}],
        )
        return resp.choices[0].message.content


async def main(texts):
    return await asyncio.gather(*(summarize(t) for t in texts))

import OpenAI from 'openai'

const client = new OpenAI({ baseURL: 'https://open.tutujin.com/v1', apiKey: process.env.TUTUJIN_API_KEY, maxRetries: 4 })

// a fixed number of workers drains the queue; limit is the concurrency cap
async function mapWithConcurrency(items, limit, fn) {
  const results = new Array(items.length)
  let next = 0
  async function worker() {
    while (next < items.length) {
      const i = next++
      results[i] = await fn(items[i])
    }
  }
  await Promise.all(Array.from({ length: limit }, worker))
  return results
}

const summaries = await mapWithConcurrency(texts, 8, async (text) => {
  const resp = await client.chat.completions.create({
    model: 'deepseek-v4.1-flash',
    messages: [{ role: 'user', content: `Summarize in one sentence: ${text}` }],
  })
  return resp.choices[0].message.content
})

Start with a low cap, watch how often 429s appear, and raise it gradually.

Smooth out traffic

• Submit batch work at a steady pace instead of firing tens of thousands of requests at once.
• Run offline jobs that are not urgent during your off-peak hours.
• Queue interactive requests and background jobs separately, so a limited background job never slows down users.

Make fewer requests

• Embed in batches: input on /v1/embeddings accepts an array of strings, so one request handles many texts.
• Cache results for repeated inputs, such as answers to common questions or vectors for documents that do not change.
• Combine steps when you can, for example ask for one JSON object with several fields instead of one request per field.

Spread load over keys and groups

• Use separate keys for separate workloads, and keep batch-job keys apart from live-traffic keys so they do not compete.
• Keep a backup key in another group and switch to it when the main key keeps returning 429 or 5xx. Groups are explained in Choosing a group.

Need more throughput

If your workload needs a higher request volume over the long term, contact us on WeChat at JsnonoChat with the models, groups and expected volume, and we will assess it and get back to you.