Official components · needs a real run0 successful reportsReviewed 2026-08-16

Run Llama 3.2 3B Q4 locally with Jan on a 16GB Apple Silicon Mac

A lower-memory desktop path using Jan, llama.cpp, Metal, and a Llama 3.2 3B Q4_K_M GGUF artifact for private chat and local API testing.

Report result

Evidence status

Jan documents Apple Metal support and recommends 16GB RAM. This exact Llama artifact and prompt protocol still require a LocalAIRun report.

Requirements

  • Apple Silicon Mac
  • macOS 12 or newer
  • 16GB unified memory recommended
  • At least 10GB free disk space

Stack

Desktop application
Jan · Current stable release

Jan stores local models, conversations, settings, and logs on the device and uses Metal automatically on Apple Silicon.

Model artifact
Llama 3.2 3B Q4_K_M · GGUF Q4_K_M

This smaller artifact leaves substantially more headroom than an 8B model on a 16GB Mac.

Manual mode

Step-by-step execution

0 / 4 complete

Use the same safeguards as the Skill

Check the machine first, use only the linked official sources, confirm large downloads and file locations, and do not bypass macOS security. Mark a step complete only after its check passes.

  1. 1. Verify the Mac

    • •Confirm Apple Silicon, macOS 12 or newer, 16GB memory, and at least 10GB free disk space.
    Check: The device meets Jan's published baseline.

  2. 2. Install and inspect Jan

    Official instructions →
    • •Install Jan from its official download.
    • •Open Settings and confirm the llama.cpp backend reports Apple Metal.
    Check: Jan starts and Metal is available.

  3. 3. Load Llama 3.2 3B Q4

    • •Use Jan's model hub or import a traceable Llama 3.2 3B Q4_K_M GGUF artifact.
    • •Use an 8K context for the first run.
    Check: The model loads without a memory warning.

  4. 4. Run and report

    • •Run the fixed prompt, then repeat it using Jan's local OpenAI-compatible API.
    • •Record Jan version, artifact, response completion, memory pressure, and errors.
    Check: The desktop and API responses complete, or the exact failure is captured.

Baseline settings

context length
8192
temperature
0.2
gpu layers
auto

Comparable test

Prompt

Explain when unified memory is more important than raw GPU speed for a local language model, using three concise examples.

Seed: 20260816

Did it run on your hardware?

Success and failure reports both improve this recipe's evidence level.

Submit test result

Sources

Jan quickstart →Jan troubleshooting and hardware guidance →