vLLM 0.32.0 is out
A large update to the server many people use to host open models.
vLLM 0.32.0, a new version of the open-source engine many teams use to serve language models, is out with 814 commits from 363 contributors. It adds a /v1/systemone endpoint for structured decisions, a generic tool parser that reads the response template shipped with a model, and support for new models such as EmbeddingGemma 2 and Nemotron 3.5 ASR.
If you run models on your own hardware, the release brings lower-precision memory options for more Nvidia GPUs and faster startup. It also fixes three security advisories, and by default it no longer lets web pages from any other site call the server, a setting known as CORS.
Check the breaking changes before upgrading. Python 3.10 is no longer supported, newer transformers and tokenizers versions are required, and several older options were removed. Browser clients now need the --allowed-origins setting.