Best Local LLM for Coding: A Developer's Guide to AI-Powered Programming

TLDR: The best local coding LLM depends on your use case: autocomplete inside an IDE favors small, fast models; agentic tasks that navigate and edit a full repository need larger models with long context and strong tool-use. The coding model landscape has moved significantly beyond Code Llama: Devstral Small 1.1, Qwen3-Coder, and DeepSeek-V3 are the current reference points, and the benchmarks that matter (SWE-bench) are different from the ones that dominated earlier coverage (HumanEval).

Two Different Use Cases, Two Different Requirements

Local coding LLMs serve two distinct patterns. Conflating them leads to poor model selection.

Autocomplete and inline chat: Short context, sub-second latency budget, runs continuously as you type. The model sees a file or a few files and suggests the next few lines or answers a focused question. A 7-8B model at Q4_K_M is often sufficient, and speed matters more than raw capability.

Agentic coding: The model receives a task description (fix this bug, implement this feature, write tests for this module), uses tools to explore a codebase, makes multiple edits across multiple files, and runs test suites to verify its work. This requires long context (32K to 128K tokens to hold large files and call history), strong instruction following, reliable tool/function call formatting, and enough raw capability to reason about complex code. Model size matters significantly here.

Picking the wrong tier for your use case is the most common mistake. A 24B agentic model for autocomplete will feel slow and use too much VRAM. A 7B general model for agentic tasks will make too many errors on complex multi-file tasks to be productive.

Current Open Coding Models

Devstral Small 1.1 (Mistral AI / All Hands AI)

Devstral Small 1.1 is a 24B parameter model fine-tuned from Mistral-Small-3.1 for agentic software engineering. It removes the vision encoder from its base model and is text-only. Context window is 128K tokens.

On SWE-Bench Verified (a dataset of 500 real GitHub issues manually screened for correctness), Devstral Small 1.1 scores 53.6%, which at publication was the highest score for any open-source model evaluated under a comparable scaffold. When evaluated under the same OpenHands scaffold, it outperforms models such as DeepSeek-V3-0324 (671B) and Qwen3 232B-A22B on this benchmark. It runs on a single RTX 4090 (24 GB) or a Mac with 32 GB RAM.

The recommended local serving approach uses vLLM or Ollama, with the OpenHands agentic scaffold for task-level work. Apache 2.0 license. Available at huggingface.co/mistralai/Devstral-Small-2507. (Hugging Face model card, 2026-09-04)

ollama run devstral

Qwen3-Coder (Alibaba Cloud)

Alibaba's Qwen3-Coder family is designed for agentic coding. The flagship variant is Qwen3-Coder-480B-A35B-Instruct: 480B total parameters, 35B activated, with native 256K context length extendable to 1M tokens using YaRN. It supports tool-calling with a specially designed function call format and is designed for repository-scale understanding. Smaller Qwen3-Coder variants are available for local deployment. The 480B variant requires significant multi-GPU infrastructure; smaller sizes (available on Ollama as qwen3-coder) are practical on consumer hardware. (github.com/QwenLM/Qwen3-Coder, 2026-09-04)

ollama run qwen3-coder

DeepSeek-V3 and DeepSeek-Coder-V2

DeepSeek-V3 (671B total, 37B activated, 128K context, DeepSeek License v1.0) is a strong coding model despite being a general-purpose MoE. DeepSeek-Coder-V2 is a dedicated coding variant of DeepSeek-V2. Both require multi-GPU infrastructure for local deployment. DeepSeek-V3 is the practical reference for teams with the hardware to run it: it scores among the top open-weight models on coding benchmarks and allows commercial use under the DeepSeek License, which includes use-based restrictions detailed in the license's Attachment A. Note: the MIT license covers only the code repository; the model weights are under a separate DeepSeek License. Source: huggingface.co/deepseek-ai/DeepSeek-V3. (Hugging Face model card and LICENSE-MODEL, 2026-09-04)

Qwen3 General Models for Coding

Qwen3 dense models (8B, 14B, 32B) are capable coding models even without the Coder specialization. Qwen3-8B and Qwen3-14B fit on consumer GPUs and handle autocomplete and simple agentic tasks well. They support 128K context and hybrid thinking mode, which is useful for harder debugging tasks. Trained on data that includes synthetic code from Qwen2.5-Coder per the Qwen3 blog post.

Gemma 4 (Google DeepMind)

Gemma 4 (12B, 26B A4B, 31B) is described by Google as purpose-built for advanced reasoning and agentic workflows. The 26B variant is a mixture-of-experts model with 4B activated parameters. On Apple Silicon with Ollama 0.31's MLX backend, Gemma 4 achieved up to 90% faster throughput for coding agents via multi-token prediction per Ollama's June 2026 blog post. It is a strong option for Mac-based development environments.

The Models That Are No Longer the Right Starting Point

Code Llama was Meta's first coding-specialized open model (based on Llama 2). It served as the default recommendation for local coding from 2023 through early 2025. As of August 2026, Llama 3.1 and Llama 4 have superseded it on general coding quality, and Devstral, Qwen3-Coder, and DeepSeek-V3 are the reference points for serious agentic work. Code Llama still runs and is available on Ollama, but there is no longer a clear reason to choose it over its successors.

StarCoder2 (BigCode) is another model that appears in older recommendations. It remains available and covers 80+ programming languages, but it is not in the current top tier for either autocomplete quality or agentic capability. WizardCoder and CodeT5+ have similarly been superseded. These models are not wrong to run, but the stronger alternatives are available and often fit the same or smaller hardware budgets.

IDE and CLI Integration

Continue

Continue was a widely used open-source coding agent available as a VS Code extension, CLI, and JetBrains plugin. The continuedev/continue GitHub repository is now read-only: the project released a final 2.0.0 version and is no longer actively maintained. The extension is still installable and functional for users who want it, and its codebase is open for forks. For teams evaluating it: it works with local models via Ollama's OpenAI-compatible endpoint, and the final release removed anonymous telemetry. Check continue.dev for the current status. (github.com/continuedev/continue, 2026-09-04)

aider

aider is an open-source command-line coding agent that edits code in your local git repository. It supports any OpenAI-compatible endpoint, which means it works with local models served by Ollama or vLLM. aider maintains its own LLM leaderboard (the aider polyglot benchmark) that tests 225 Exercism coding exercises across C++, Go, Java, JavaScript, Python, and Rust, evaluating whether a model can follow instructions and correctly format code edits. This benchmark is more representative of real coding-agent use than HumanEval because it tests multi-turn editing workflows, not just single-function generation. Documentation at aider.chat/docs/leaderboards. (aider.chat/docs/leaderboards, 2026-09-04)

pip install aider-chat
aider --model ollama/qwen3:14b --openai-api-base http://localhost:11434/v1

OpenHands

OpenHands (previously OpenDevin) is the agentic scaffold recommended by Mistral for running Devstral. It defines the interface between a model and a codebase: file operations, shell execution, browser control, and test running. It is the most complete local software engineering agent framework as of this writing. Connect it to a local model via an OpenAI-compatible server.

Ollama's Native Coding Integrations

Ollama has added first-class support for coding tool integrations. The ollama launch command sets up Claude Code, OpenCode, or Codex CLI with a local model in a single command. Claude Desktop can also be configured to use Ollama as a gateway to open models. This makes it possible to use familiar coding agent interfaces with a local model backend. (docs.ollama.com/cli, 2026-09-04)

ollama launch claude

Context Window Needs for Real Codebases

Single-file autocomplete fits in 4-8K tokens. Real agentic tasks are different. A single large source file can be 2-5K tokens. Including test files, related modules, error output from a test run, and conversation history, a practical agentic coding session routinely uses 20-60K tokens. For repository-scale tasks (refactoring an entire module, understanding how multiple systems interact), 100K+ context becomes relevant.

Context window is not just a number: the model must attend to all of it. Models with long context windows sometimes degrade in quality on information that appears in the middle of very long contexts. If you are relying on 128K context, test that the model actually uses content from the middle of long inputs, not just the beginning and end.

For autocomplete, context rarely exceeds 8K tokens. For agentic tasks, pick models with at least 32K context (preferably 128K) and test your actual prompt sizes.

Benchmarks and Their Caveats

HumanEval: A set of Python function-completion problems originally released by OpenAI. It is widely cited but widely saturated: strong models score in the 90s on it. HumanEval does not measure multi-file editing, long-context understanding, or tool use. It is useful for confirming a model can write Python functions; it says little about whether a model can fix a real bug in an unfamiliar codebase.

SWE-Bench Verified: A dataset of 500 real GitHub issues with verified test suites. It measures whether a model plus an agentic scaffold can resolve the issue and pass the tests. This is much closer to real software engineering work. Scaffold choice matters significantly: the same model can score very differently under different scaffolds, so cross-model comparisons are only meaningful when the same scaffold is used. Devstral's 53.6% score is under the OpenHands scaffold; comparisons to models evaluated under different scaffolds should be interpreted carefully.

Aider polyglot: Tests multi-turn code editing across six languages on a model's ability to follow edit format instructions as well as write correct code. More representative than HumanEval for CLI-based coding agents.

No single benchmark captures the full picture. Run the one that most resembles your actual workflow.

Local vs Hosted Coding Assistant Tradeoffs

Factor	Local model	Hosted (cloud) assistant
Code privacy	Stays on your hardware	Sent to provider's servers
Cost at scale	Fixed hardware cost	Per-token or per-seat pricing
Capability ceiling	Limited by your hardware	Access to largest models
Latency	No network round-trip	Network-dependent
Knowledge cutoff	Same as any LLM; needs search API for current docs	Same; often augmented by provider
Offline use	Works without internet	Requires connectivity
Customization	Fine-tune on your codebase	Limited to provider's options

Giving Your Local Coding Assistant Current Documentation

One common frustration with local coding models: they may not know about library versions or APIs released after their training cutoff. When working with fast-moving frameworks or recently released dependencies, the model may suggest deprecated patterns or incorrect APIs.

The practical fix is to fetch current documentation and include it in context. You.com's Contents API retrieves clean HTML or Markdown from any URL, which you can inject as context for the model. The Web Search API returns current results when you need to find the right documentation page first. The free MCP endpoint at https://api.you.com/mcp?profile=free exposes web search with no signup and works with MCP-enabled tools including Cursor and VS Code. Paid access via you.com/platform adds higher rate limits, you-contents for clean page retrieval, and you-research for synthesized, cited answers. This keeps your code local while giving the model access to current library documentation. (you.com/docs/build-with-agents/mcp-server.md, 2026-09-04)

Fine-Tuning for Domain-Specific Code

If your codebase uses internal libraries, proprietary APIs, or architectural patterns that are not well-represented in public training data, even the best general coding model will hallucinate non-existent methods or suggest incorrect API usage. Fine-tuning on a representative sample of your internal code can address this, but it requires significant care.

Parameter-efficient fine-tuning methods (LoRA and QLoRA) reduce the compute and memory requirements dramatically compared to full fine-tuning. A QLoRA fine-tune of a 7B or 14B model can run on a single GPU with 24 GB of VRAM. The training data preparation is usually the hardest part: you need high-quality examples of the coding patterns you want the model to learn, formatted in instruction-response pairs.

For most teams, fine-tuning is worth considering only after verifying that a strong base model cannot learn your patterns through few-shot prompting or a well-crafted system prompt. Prompting is cheaper, faster to iterate, and reversible. Fine-tuning is a longer commitment. Start with prompting, add fine-tuning only if you have a specific, measurable gap that prompting cannot close.

Testing Your Coding Model Before Committing

Before choosing a model for your team, run a structured evaluation rather than impressionistic testing. A minimal protocol:

1. Collect 15-25 coding tasks representative of your actual work: bug fixes, feature additions, test writing, refactoring. Use real examples from your codebase where possible.
2. For each task, define a pass criterion: does the output compile? Do the tests pass? Does a human reviewer judge the approach correct?
3. Run each candidate model under identical conditions: same serving stack, same quantization, same system prompt, same temperature.
4. Score outputs and compare. Pay attention to failure modes, not just pass rates. A model that fails gracefully (explains what it cannot do) is preferable to one that confidently generates broken code.

This takes a day and tells you far more than any published benchmark. Model quality on your specific code patterns, libraries, and style guides is what matters, not aggregate performance on a standard benchmark suite.

Hardware Recommendations by Use Case

• Autocomplete on a developer laptop: 8-16 GB VRAM. Qwen3-8B or Llama 3.1 8B at Q4_K_M. Ollama for serving, aider or a local-model-backed VS Code extension for the editor interface.
• Agentic coding on a workstation: 24 GB VRAM (RTX 4090) or 32+ GB Apple Silicon. Devstral Small 1.1 or Qwen3-32B. OpenHands or aider as the scaffold.
• Team-shared agentic coding server: Multi-GPU with 80+ GB VRAM total. DeepSeek-V3 or Qwen3-Coder-480B-A35B via vLLM. OpenHands connected to vLLM endpoint.