Local LLM model page

CodeLlama 34B

Massive coding model. Handles complex refactoring, architecture, and multi-file edits.

Find the best model for my hardwareBrowse all 183 LLMs
Parameters
34B
Minimum RAM
32 GB
Model size
20 GB
Quantization
Q4_K_M

Can CodeLlama 34B run locally?

CodeLlama 34B is best suited for power-user machines with 32 GB RAM. LocalClaw recommends Q4_K_M as the default quantization, with at least 32 GB RAM.

Search term for LM Studio or compatible runtimes: codellama-34b-instruct

Hugging Face repository: TheBloke/CodeLlama-34B-Instruct-GGUF

codepowerquality

Strengths

  • Massive coding model. Handles complex refactoring, architecture, and multi-file edits.

Limitations

  • Performance depends heavily on quantization, RAM bandwidth and runtime support.

Best use cases

  • code
  • power
  • quality

Benchmarks

Speed: 4/10

Quality: 7/10

Coding: 9/10

Reasoning: 7/10

Technical details

Developer: codellama

License: See model repository

Context window: Unknown tokens

Architecture: See model card

Released: 2024-01