Port of LLaMA model inference in C/C++ for CPU and GPU, enabling efficient local model deployment.
<b>LLM inference in C/C++</b>
v1.0.0 / ggml / ops / maintainer PRs / dev stats / lib llama API / llama-server REST API
</div>The main goal of llama.cpp is to enable LLM (and MLLM) inference with minimal setup and state-of-the-art performance on a wide range of hardware - locally and in the cloud.
The llama.cpp project is built on top of the ggml library.
A few options to get llama.cpp installed on your machine:
# curl
curl -LsSf https://llama.app/install.sh | sh
# powershell
irm https://llama.app/install.ps1 | iex
Once installed:
# Download and run a model directly from Hugging Face
llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF
# Launch OpenAI-compatible API server
llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF
<table align="center">
<tr>
<td align="center" width=50%>
<img width="1310" height="888" alt="VLM session with `llama cli`" src="https://github.com/user-attachments/assets/88726b48-1713-48aa-a525-95a02e78afc4" />
<i>VLM session with <b>llama cli</b></i>
</td>
<td align="center">
<img width="1392" height="958" alt="Llama Chat, the built-in chat app, against `llama serve` running Qwen 3.6" src="https://github.com/user-attachments/assets/b402f972-2e32-4def-8771-8d849f08cf2e" />
<i>Llama Chat, the built-in chat app, against <b>llama serve</b></i>
</td>
</tr>
<table>
llama.cpp repo and merge PRs into the master branchllama-server - MIT licensePreview
Markdown
[](https://attestry.ai/models/github-ggerganov-llama-cpp)HTML
<a href="https://attestry.ai/models/github-ggerganov-llama-cpp"><img src="https://attestry.ai/api/v1/registry/badge/github-ggerganov-llama-cpp" alt="Attestry Verification Status" /></a>Are you the creator of this entry and want it removed? Request a takedown.