Local LLM model page

Dolphin Mixtral (8x7B)

Uncensored Mixtral fine-tune. No content filters. Popular for unrestricted chat. 708K downloads.

Find the best model for my hardwareBrowse all 183 LLMs
Parameters
8x7B (46.7B)
Minimum RAM
32 GB
Model size
26 GB
Quantization
Q4_K_M

Can Dolphin Mixtral (8x7B) run locally?

Dolphin Mixtral (8x7B) is best suited for power-user machines with 32 GB RAM. LocalClaw recommends Q4_K_M as the default quantization, with at least 32 GB RAM.

Search term for LM Studio or compatible runtimes: dolphin-2.9.2-mixtral-8x7b

Hugging Face repository: TheBloke/dolphin-2.9.2-mixtral-8x7b-GGUF

chatcodepowergeneral

Strengths

  • Uncensored Mixtral fine-tune. No content filters. Popular for unrestricted chat. 708K downloads.

Limitations

  • Performance depends heavily on quantization, RAM bandwidth and runtime support.

Best use cases

  • chat
  • code
  • power
  • general

Benchmarks

Speed: 4/10

Quality: 8/10

Coding: 7/10

Reasoning: 7/10

Technical details

Developer: dolphin

License: See model repository

Context window: Unknown tokens

Architecture: See model card

Released: 2024-03