Local LLM model page

WizardLM 2 (8x22B)

Microsoft AI's ultra-popular fine-tune of Mixtral 8x22B. Apache 2.0 license. Exceptional instruction following and conversational quality.

Find the best model for my hardware Browse all 183 LLMs

Parameters

8x22B (141B total)

Minimum RAM

96 GB

Model size

88 GB

Quantization

Q4_K_M

Can WizardLM 2 (8x22B) run locally?

WizardLM 2 (8x22B) is best suited for large-memory workstations. LocalClaw recommends Q4_K_M as the default quantization, with at least 96 GB RAM.

Search term for LM Studio or compatible runtimes: wizardlm-2-8x22b

Hugging Face repository: lmstudio-community/WizardLM-2-8x22B-GGUF

chatcodepowerqualitygeneral

Strengths

Exceptional instruction following
Apache 2.0
One of the best fine-tunes ever
Strong conversational quality

Limitations

Requires 96GB+ RAM
Very large model
Slow inference

Best use cases

Maximum quality chat
Complex instructions
Professional content creation
Research

Benchmarks

Speed: 3/10

Quality: 10/10

Coding: 9/10

Reasoning: 9/10

Technical details

Developer: Microsoft AI

License: Apache 2.0

Context window: 65,536 tokens

Architecture: Mixtral 8x22B fine-tuned with WizardLM pipeline

Released: 2024-04