vLLM

« Back to Glossary Index

Open-source software framework for high-throughput, memory-efficient inference and serving engine for Large Language Models (LLMs) and related multimodal models.

« Back to Glossary Index