Local LLM model page

Zephyr (7B)

Fine-tuned Mistral as helpful assistant. DPO-aligned for safety. 291K downloads.

Find the best model for my hardwareBrowse all 183 LLMs
Parameters
7B
Minimum RAM
8 GB
Model size
4.1 GB
Quantization
Q5_K_M

Can Zephyr (7B) run locally?

Zephyr (7B) is best suited for entry-level laptops and desktops. LocalClaw recommends Q5_K_M as the default quantization, with at least 8 GB RAM.

Search term for LM Studio or compatible runtimes: zephyr-7b-beta

Hugging Face repository: TheBloke/zephyr-7B-beta-GGUF

chatstandardgeneral

Strengths

  • Fine-tuned Mistral as helpful assistant. DPO-aligned for safety. 291K downloads.

Limitations

  • Performance depends heavily on quantization, RAM bandwidth and runtime support.

Best use cases

  • chat
  • standard
  • general

Benchmarks

Speed: 8/10

Quality: 7/10

Coding: 5/10

Reasoning: 6/10

Technical details

Developer: zephyr

License: See model repository

Context window: Unknown tokens

Architecture: See model card

Released: 2023-11