交錯思考¶

簡介¶

交錯思考允許模型在工具呼叫之間進行推理，從而在收到工具結果後實現更復雜的決策。此功能可幫助模型在中間插入推理步驟來連結多個工具呼叫，並根據中間結果做出細緻的決策。

重要提示：交錯思考會增加 token 使用量和響應延遲。啟用此功能時，請考慮您的預算和效能要求。

交錯思考如何工作¶

透過交錯思考，模型可以

在決定下一步做什麼之前，對工具呼叫的結果進行推理
在中間插入推理步驟來連結多個工具呼叫
根據中間結果做出更細緻的決策
為其工具選擇過程提供透明的推理

支援的模型¶

vLLM 目前支援以下交錯思考模型

模型系列	推理解析器名稱
moonshotai/Kimi-K2-Thinking	kimi_k2
MiniMaxAI/MiniMax-M2	minimax_m2

示例用法¶

要將交錯思考與工具呼叫一起使用，請指定一個支援此功能且在您的聊天補全請求中啟用了工具呼叫的模型。以下是一個示例

程式碼

"""
vllm serve MiniMaxAI/MiniMax-M2 \
  --tensor-parallel-size 4 \
  --tool-call-parser minimax_m2 \
  --reasoning-parser minimax_m2 \
  --enable-auto-tool-choice
"""
import json

from openai import OpenAI

client = OpenAI(base_url="https://:8000/v1",     api_key="dummy")


def get_current_weather(location: str, unit: "str"):
    """Get the current weather in a given location"""
    if unit == "celsius":
        return f"The current temperature in {location} is 22°C."
    else:
        return f"The current temperature in {location} is 72°F."


tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather in a given     location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "City and state, e.g.,     'San Francisco, CA'",
                    },
                    "unit": {"type": "string", "enum":     ["celsius", "fahrenheit"]},
                },
                "required": ["location", "unit"],
            },
        },
    }
]
messages = [{"role": "user", "content": "What's the weather in Fahrenheit like in San Francisco?"}]
response = client.chat.completions.create(
    model=client.models.list().data[0].id,
    messages=messages,
    tools=tools,
    tool_choice="auto",
)

tool_call = response.choices[0].message.tool_calls[0].function

messages.append(
    {
        "role": "assistant",
        "tool_calls": response.choices[0].message.tool_calls,
        "reasoning": response.choices[0].message.reasoning, # append reasoning
    }
)

# Simulate tool execution
available_tools = {"get_weather": get_current_weather}

completion_tool_calls = response.choices[0].message.tool_calls
for call in completion_tool_calls:
    tool_to_call = available_tools[call.function.name]
    args = json.loads(call.function.arguments)
    result = tool_to_call(**args)
    messages.append(
        {
            "role": "tool",
            "content": result,
            "tool_call_id": call.id,
            "name": call.function.name,
        }
    )
response_2 = client.chat.completions.create(
    model=client.models.list().data[0].id,
    messages=messages,
    tools=tools,
    tool_choice="auto",
)
print(response_2.choices[0].message.content)

此示例演示瞭如何使用天氣檢索函式設定帶有工具呼叫的交錯思考。模型在生成最終響應之前會對其工具結果進行推理。