Engine Arguments¶
Engine arguments control the behavior of the vLLM engine.
- For offline inference, they are part of the arguments to LLM class.
- For online serving, they are part of the arguments to
vllm serve.
The engine argument classes, EngineArgs and AsyncEngineArgs, are a combination of the configuration classes defined in vllm.config. Therefore, if you are interested in developer documentation, we recommend looking at these configuration classes as they are the source of truth for types, defaults and docstrings.
當傳遞 JSON 命令列引數時，以下幾組引數是等效的
--json-arg '{"key1": "value1", "key2": {"key3": "value2"}}'--json-arg.key1 value1 --json-arg.key2.key3 value2
此外，列表元素可以使用 + 單獨傳遞
--json-arg '{"key4": ["value3", "value4", "value5"]}'--json-arg.key4+ value3 --json-arg.key4+='value4,value5'