Enterprise knowledge base and RAG
Knowledge-base Q&A (RAG) uses three kinds of models: Embeddings turn documents and questions into vectors, Rerank reorders the search results, and a chat model writes the answer from the retrieved text. With the assumptions below (2,000 documents indexed, 30,000 questions a month) and live prices in each model’s cheapest group, model costs come to about ¥389 a month.
Prices updated Nov 12, 2026, 8:17 PM|
01
The business problem
Product manuals, internal policies, contract templates and old support tickets live in different systems. Finding one answer means opening several documents or asking a colleague.
A plain LLM does not know your internal material and will confidently make things up. Pasting every document into the prompt is expensive and quickly exceeds the context window.
RAG (retrieval-augmented generation) first finds the passages relevant to a question, then asks the model to answer only from them. When documents change you re-index them instead of training a model, and every answer can cite its sources.
02
Recommended models
Embeddings
text-embedding-3-largeOpenAIinput ¥0.26 / output ¥0.26 per 1M tokens· Standard groupTurns document chunks and user questions into vectors for semantic search. Indexing and querying must use the same embedding model.
Rerank
qwen3-reranker-8bQweninput ¥0.00308 / output ¥0.00308 per 1M tokens· China Models groupVector search often returns some irrelevant passages. A rerank model scores each passage against the question so only the best few reach the chat model — better answers and fewer input tokens.
Answer generation (chat model)
deepseek-v4.1-flashDeepSeekinput ¥2.8 / output ¥11.2 per 1M tokens· China Models groupReads the retrieved passages and writes the answer. Knowledge-base prompts are long and answers short, so a low input price matters most; switch to one of the alternatives when you need higher quality.
Recommended models are picked from the live catalog; prices are each model's cheapest group, updated Nov 12, 2026, 8:17 PM. Click a model to see every group's price.
03
Reference architecture
- 1
Parse and chunk
Convert PDFs, Word files, web pages and tickets to plain text and split it into chunks of roughly 200 to 500 words, keeping the title, source link and update time of each chunk.
- 2
Embed and index
Call the Embeddings endpoint for every chunk and store the vectors with the text in a vector database such as pgvector, Milvus, Qdrant or Elasticsearch.
- 3
Retrieve candidates
Embed each question with the same model and fetch the 20 to 50 closest chunks; optionally run a keyword search too and merge the results.
- 4
Rerank
Call the Rerank endpoint to score the candidates and keep about the top 5.
- 5
Generate the answer
Number the kept chunks and send them with the question to the chat model, instructing it to answer only from the sources, cite them like [1], and say so when the sources are not enough.
- 6
Evaluate and iterate
Log thumbs up and down and unanswered questions; add documents and tune chunk size and prompts regularly.
04
Cost estimate
Assumptions: 2,000 documents averaging 2,500 tokens; 30,000 questions a month (about 1,000 a day); 20 candidate chunks (about 6,000 tokens) reranked per question; about 3,000 tokens of sources sent to the chat model and a 400-token answer.
| Item | Model | Usage assumed | Cost |
|---|---|---|---|
| Initial document embedding (one-off) | text-embedding-3-large | 2,000 × 2,500 input tokens | ¥1.3 |
| Question embedding | text-embedding-3-large | 30,000 × 40 input tokens | ¥0.312 |
| Reranking candidates | qwen3-reranker-8b | 30,000 × 6,000 input tokens | ¥0.554 |
| Answer generation | deepseek-v4.1-flash | 30,000 × (3,000 input + 400 output tokens) | ¥386.4 |
| Total (per month) | ¥388.57 | ||
Calculated from live prices in each model's cheapest group, updated Nov 12, 2026, 8:17 PM.
Model usage only — vector database and server costs are not included. Initial indexing is a one-off; afterwards you only re-embed new or changed documents. If your system prompt is long and the model supports context caching, cached input is billed at the lower cache-read price and the real bill is lower.
05
Sample code
The OpenAI SDK handles embeddings and chat; rerank is a plain HTTP call. Replace the vector database part with your own.
import os
import requests
from openai import OpenAI
API_KEY = os.environ["TUTUJIN_API_KEY"]
client = OpenAI(base_url="https://open.tutujin.com/v1", api_key=API_KEY)
def embed(texts):
"""Turn document chunks or a question into vectors."""
resp = client.embeddings.create(model="text-embedding-3-large", input=texts)
return [d.embedding for d in resp.data]
def rerank(query, docs, top_n=5):
"""Re-score retrieved chunks and keep the most relevant ones."""
r = requests.post(
"https://open.tutujin.com/v1/rerank",
headers={"Authorization": f"Bearer {API_KEY}"},
json={"model": "qwen3-reranker-8b", "query": query, "documents": docs, "top_n": top_n},
timeout=30,
)
r.raise_for_status()
return [docs[x["index"]] for x in r.json()["results"]]
def answer(question, chunks):
context = "\n\n".join(f"[{i + 1}] {c}" for i, c in enumerate(chunks))
resp = client.chat.completions.create(
model="deepseek-v4.1-flash",
messages=[
{"role": "system", "content": "Answer only from the given sources and cite them like [1]. If the sources do not say, reply that you do not know."},
{"role": "user", "content": f"Sources:\n{context}\n\nQuestion: {question}"},
],
temperature=0.2,
)
return resp.choices[0].message.content
# candidates = your_vector_db.search(embed([question])[0], k=20)
# print(answer(question, rerank(question, candidates)))FAQ
How much does knowledge-base Q&A cost per month?
It depends on question volume and how much text you send per question. With 2,000 documents indexed, 30,000 questions a month and about 3,000 tokens of sources per question, model costs in the cheapest groups come to about ¥389 a month. Double the questions and the cost roughly doubles.
Do I need Rerank?
No. With few documents and predictable questions, vector search alone works. With many documents and many ways of asking the same thing, a rerank step usually cuts off-topic answers noticeably and sends fewer irrelevant passages to the chat model, saving input tokens.
Do I have to re-index if I change the embedding model?
Yes. Vectors from different embedding models are not compatible, so switching means re-embedding every document and querying with the new model. Compare a few embedding models on a small sample before going live.
Can I use only Chinese models?
Yes. There are Chinese options for embeddings, rerank and chat, for example qwen3-embedding-8b for embeddings, qwen3-reranker-8b for rerank and deepseek-v4.1-flash for chat. The calls stay the same; only the model parameter changes.
How do I get answers with citations?
Store the source link and title with each chunk, number the chunks [1], [2] when you send them, tell the model in the system prompt to cite by number, and map the numbers back to links in your front end.