How to Use

ollama run hf.co/lthn/lemma:Q4_K_M

Also available: Q5_K_M, Q6_K, Q8_0, BF16

# From HuggingFace
docker model run hf.co/lthn/lemma

# From Docker Hub
docker model run lthn/lemma

# Install
brew install llama.cpp        # macOS/Linux
winget install llama.cpp      # Windows

# Server with web UI
llama-server -hf lthn/lemma:Q4_K_M

# CLI inference
llama-cli -hf lthn/lemma:Q4_K_M

uv tool install mlx-lm

mlx_lm.chat --model lthn/lemma
mlx_lm.generate --model lthn/lemma \
  --prompt "Hello, how are you?"

Native Apple Silicon — no emulation, no quantisation overhead on M-series

uv pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="lthn/lemma",
    filename="lemma-q4_k_m.gguf",
)

# Text
llm.create_chat_completion(
    messages=[{
        "role": "user",
        "content": "Hello, how are you?"
    }]
)

# Vision (multimodal)
llm.create_chat_completion(
    messages=[{
        "role": "user",
        "content": [
            {"type": "text",
             "text": "Describe this image."},
            {"type": "image_url",
             "image_url": {"url": "image.jpg"}}
        ]
    }]
)

uv tool install mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

model, processor = load("lthn/lemma")
config = load_config("lthn/lemma")

image = ["photo.jpg"]
prompt = "Describe this image."

formatted = apply_chat_template(
    processor, config, prompt, num_images=1
)

output = generate(
    model, processor, formatted, image
)
print(output.text)

mlx-vlm handles vision tensor routing for Gemma 4. Use this for multimodal inference on Apple Silicon.

Lemma models are multimodal, so use mlx_vlm.server — the vision-aware variant. The text-only mlx_lm.server does not correctly route multimodal tensors for Gemma 4.

mlx_vlm.server --model lthn/lemma

curl -X POST "http://localhost:8080/v1/chat/completions" \
  -H "Content-Type: application/json" \
  --data '{
    "model": "lthn/lemma",
    "messages": [{
      "role": "user",
      "content": "Hello, how are you?"
    }],
    "max_tokens": 200
  }'

vLLM requires the original (non-quantised) safetensors weights from the LetheanNetwork/ repos. Linux + NVIDIA GPU with adequate VRAM.

uv pip install vllm
vllm serve "LetheanNetwork/lemma"

Serves at http://localhost:8000/v1 by default. Compatible with the OpenAI Python SDK.

Gemma 4 is calibrated for temperature: 1.0 — this is not the same as the typical 0.7 default for other models. Lower values reduce diversity without improving quality. These defaults are pre-configured in the params file (Ollama) and generation_config.json (transformers/MLX).

Variable Image Resolution

Gemma 4 supports a configurable visual token budget. Higher = more detail, lower = faster inference. Place image content before text for best results.

Default budget (280) is set in processor_config.json via image_seq_length and max_soft_tokens.