Local LLM model page

Phi-3.5 (3.8B)

Phi 3.5 overtaking larger models. Excellent small model from Microsoft. 333K downloads.

Find the best model for my hardwareBrowse all 183 LLMs
Parameters
3.8B
Minimum RAM
6 GB
Model size
2.3 GB
Quantization
Q5_K_M

Can Phi-3.5 (3.8B) run locally?

Phi-3.5 (3.8B) is best suited for entry-level laptops and desktops. LocalClaw recommends Q5_K_M as the default quantization, with at least 6 GB RAM.

Search term for LM Studio or compatible runtimes: phi-3.5-mini-instruct

Hugging Face repository: lmstudio-community/Phi-3.5-mini-instruct-GGUF

chatlightspeed

Strengths

  • Phi 3.5 overtaking larger models. Excellent small model from Microsoft. 333K downloads.

Limitations

  • Performance depends heavily on quantization, RAM bandwidth and runtime support.

Best use cases

  • chat
  • light
  • speed

Benchmarks

Speed: 9/10

Quality: 6/10

Coding: 6/10

Reasoning: 6/10

Technical details

Developer: phi

License: See model repository

Context window: Unknown tokens

Architecture: See model card

Released: 2024-08