推理輸出¶
vLLM 支援像 DeepSeek R1 這樣的推理模型，它們被設計用來生成包含推理步驟和最終結論的輸出。
推理模型在其輸出中返回一個額外的 reasoning 欄位，其中包含導致最終結論的推理步驟。其他模型的輸出中不存在此欄位。
警告
reasoning 以前稱為 reasoning_content。目前，reasoning_content 仍可正常工作。但是，我們建議您遷移到 reasoning，以防 reasoning_content 在將來被移除。
支援的模型¶
vLLM 目前支援以下推理模型
| 模型系列 | 解析器名稱 | 結構化輸出支援 | 工具呼叫 |
|---|---|---|---|
| DeepSeek R1 系列 | deepseek_r1 | json, regex | ❌ |
| DeepSeek-V3.1 | deepseek_v3 | json, regex | ❌ |
| ERNIE-4.5-VL 系列 | ernie45 | json, regex | ❌ |
| ERNIE-4.5-21B-A3B-Thinking | ernie45 | json, regex | ✅ |
| GLM-4.5 系列 | glm45 | json, regex | ✅ |
| Holo2 系列 | holo2 | json, regex | ✅ |
| Hunyuan A13B 系列 | hunyuan_a13b | json, regex | ✅ |
| IBM Granite 3.2 語言模型 | granite | ❌ | ❌ |
| MiniMax-M2 | minimax_m2_append_think | json, regex | ✅ |
| Qwen3 系列 | qwen3 | json, regex | ✅ |
| QwQ-32B | deepseek_r1 | json, regex | ✅ |
注意
IBM Granite 3.2 和 DeepSeek-V3.1 的推理預設停用；要啟用它，您還必須在 chat_template_kwargs 中傳遞 thinking=True。Qwen3 系列的推理功能預設啟用。要停用它，您必須在 chat_template_kwargs 中傳遞 enable_thinking=False。DeepSeek-V3.1 工具呼叫在非思考模式下受支援。Holo2 的推理預設啟用。要停用它，您還必須在 chat_template_kwargs 中傳遞 thinking=False。
快速入門¶
要使用推理模型，您需要在向聊天補全端點發出請求時指定 --reasoning-parser 標誌。--reasoning-parser 標誌指定用於從模型輸出中提取推理內容的推理解析器。
接下來，嚮應該在響應中返回推理內容的模型發出請求。
程式碼
from openai import OpenAI
# Modify OpenAI's API key and API base to use vLLM's API server.
openai_api_key = "EMPTY"
openai_api_base = "https://:8000/v1"
client = OpenAI(
api_key=openai_api_key,
base_url=openai_api_base,
)
models = client.models.list()
model = models.data[0].id
# Round 1
messages = [{"role": "user", "content": "9.11 and 9.8, which is greater?"}]
# For granite, add: `extra_body={"chat_template_kwargs": {"thinking": True}}`
# For Qwen3 series, if you want to disable thinking in reasoning mode, add:
# extra_body={"chat_template_kwargs": {"enable_thinking": False}}
response = client.chat.completions.create(model=model, messages=messages)
reasoning = response.choices[0].message.reasoning
content = response.choices[0].message.content
print("reasoning:", reasoning)
print("content:", content)
reasoning 欄位包含導致最終結論的推理步驟，而 content 欄位包含最終結論。
流式聊天補全¶
流式聊天補全也支援推理模型。reasoning 欄位在 聊天補全響應塊 的 delta 欄位中可用。
Json
{
"id": "chatcmpl-123",
"object": "chat.completion.chunk",
"created": 1694268190,
"model": "deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B",
"system_fingerprint": "fp_44709d6fcb",
"choices": [
{
"index": 0,
"delta": {
"role": "assistant",
"reasoning": "is",
},
"logprobs": null,
"finish_reason": null
}
]
}
OpenAI Python 客戶端庫不正式支援流式輸出的 reasoning 屬性。但該客戶端支援響應中的額外屬性。您可以使用 hasattr 檢查響應中是否存在 reasoning 屬性。例如
程式碼
from openai import OpenAI
# Modify OpenAI's API key and API base to use vLLM's API server.
openai_api_key = "EMPTY"
openai_api_base = "https://:8000/v1"
client = OpenAI(
api_key=openai_api_key,
base_url=openai_api_base,
)
models = client.models.list()
model = models.data[0].id
messages = [{"role": "user", "content": "9.11 and 9.8, which is greater?"}]
# For granite, add: `extra_body={"chat_template_kwargs": {"thinking": True}}`
# For Qwen3 series, if you want to disable thinking in reasoning mode, add:
# extra_body={"chat_template_kwargs": {"enable_thinking": False}}
stream = client.chat.completions.create(
model=model,
messages=messages,
stream=True,
)
print("client: Start streaming chat completions...")
printed_reasoning = False
printed_content = False
for chunk in stream:
# Safely extract reasoning and content from delta,
# defaulting to None if attributes don't exist or are empty strings
reasoning = (
getattr(chunk.choices[0].delta, "reasoning", None) or None
)
content = getattr(chunk.choices[0].delta, "content", None) or None
if reasoning is not None:
if not printed_reasoning:
printed_reasoning = True
print("reasoning:", end="", flush=True)
print(reasoning, end="", flush=True)
elif content is not None:
if not printed_content:
printed_content = True
print("\ncontent:", end="", flush=True)
# Extract and print the content
print(content, end="", flush=True)
請記住，在訪問 reasoning 之前，請檢查它是否存在於響應中。您可以檢視 示例。
工具呼叫¶
當同時啟用工具呼叫和推理解析器時，推理內容也可用。此外，工具呼叫僅從 content 欄位解析函式，不從 reasoning 中解析。
程式碼
from openai import OpenAI
client = OpenAI(base_url="https://:8000/v1", api_key="dummy")
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather in a given location",
"parameters": {
"type": "object",
"properties": {
"location": {"type": "string", "description": "City and state, e.g., 'San Francisco, CA'"},
"unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
},
"required": ["location", "unit"],
}
},
}
]
response = client.chat.completions.create(
model=client.models.list().data[0].id,
messages=[{"role": "user", "content": "What's the weather like in San Francisco?"}],
tools=tools,
tool_choice="auto",
)
print(response)
tool_call = response.choices[0].message.tool_calls[0].function
print(f"reasoning: {response.choices[0].message.reasoning}")
print(f"Function called: {tool_call.name}")
print(f"Arguments: {tool_call.arguments}")
更多示例，請參閱 examples/online_serving/openai_chat_completion_tool_calls_with_reasoning.py.
限制¶
- 推理內容僅適用於線上服務的聊天補全端點 (
/v1/chat/completions)。
如何支援新的推理模型¶
您可以新增一個新的 ReasoningParser，類似於 vllm/reasoning/deepseek_r1_reasoning_parser.py。
程式碼
# import the required packages
from vllm.reasoning import ReasoningParser, ReasoningParserManager
from vllm.entrypoints.openai.protocol import ChatCompletionRequest, DeltaMessage
# define a reasoning parser and register it to vllm
# the name list in register_module can be used
# in --reasoning-parser.
class ExampleParser(ReasoningParser):
def __init__(self, tokenizer: TokenizerLike):
super().__init__(tokenizer)
def extract_reasoning_streaming(
self,
previous_text: str,
current_text: str,
delta_text: str,
previous_token_ids: Sequence[int],
current_token_ids: Sequence[int],
delta_token_ids: Sequence[int],
) -> DeltaMessage | None:
"""
Instance method that should be implemented for extracting reasoning
from an incomplete response; for use when handling reasoning calls and
streaming. Has to be an instance method because it requires state -
the current tokens/diffs, but also the information about what has
previously been parsed and extracted (see constructor)
"""
def extract_reasoning(
self,
model_output: str,
request: ChatCompletionRequest | ResponsesRequest,
) -> tuple[str | None, str | None]:
"""
Extract reasoning content from a complete model-generated string.
Used for non-streaming responses where we have the entire model response
available before sending to the client.
Parameters:
model_output: str
The model-generated string to extract reasoning content from.
request: ChatCompletionRequest
The request object that was used to generate the model_output.
Returns:
tuple[Optional[str], Optional[str]]
A tuple containing the reasoning content and the content.
"""
# Register the reasoning parser
ReasoningParserManager.register_lazy_module(
name="example",
module_path="vllm.reasoning.example_reasoning_parser",
class_name="ExampleParser",
)
此外，要啟用結構化輸出，您需要建立一個新的 Reasoner，類似於 vllm/reasoning/deepseek_r1_reasoning_parser.py 中的示例。
程式碼
@dataclass
class DeepSeekReasoner(Reasoner):
"""
Reasoner for DeepSeek R series models.
"""
start_token_id: int
end_token_id: int
start_token: str = "<think>"
end_token: str = "</think>"
@classmethod
def from_tokenizer(cls, tokenizer: PreTrainedTokenizer) -> Reasoner:
return cls(
start_token_id=tokenizer.encode("<think>", add_special_tokens=False)[0],
end_token_id=tokenizer.encode("</think>", add_special_tokens=False)[0],
)
def is_reasoning_end(self, input_ids: list[int]) -> bool:
return self.end_token_id in input_ids
def is_reasoning_end_streaming(self, input_ids: list[int], delta_ids: list[int]) -> bool:
return self.end_token_id in delta_token_ids
...
像 xgrammar 這樣的結構化輸出引擎將使用 end_token_id 來檢查推理內容是否存在於模型輸出中，並在存在時跳過結構化輸出。
最後，您可以透過使用 --reasoning-parser 標誌來為模型啟用推理。