#local-ai
17 articles
- 2026-10-23
A 24 GB Model Is Not a 24 GB Deployment
NVIDIA's Nemotron Bolt makes the distinction between downloadable weights and a usable agent deployment unusually concrete.
- 2026-10-14
Diffusion Language Models Have a Cache Problem. That Is Finally Getting Interesting
Diffusion language models can generate text in parallel, but repeated denoising makes them slow. New cache-aware work makes the trade-off worth watching.
- 2026-10-12
Diffusion Language Models Have a Cache Problem. That Is Finally Getting Interesting
Diffusion language models can generate text in parallel, but repeated denoising makes them slow. New cache-aware work makes the trade-off worth watching.
- 2026-06-16
The Agent Runtime Is the Product
The model gets the applause, but the runtime decides whether an AI agent is useful or just confidently decorative.
- 2026-06-08
Memory Is Not a Database
Agent memory fails less like storage and more like attention. Treating it as a database is how systems get weird.
- 2026-04-02
M5 Max vs AMD Strix Halo: Which Is Better for Running Local LLMs?
Apple's M5 Max has 460 GB/s memory bandwidth. My AMD Ryzen AI Max+ 395 has ~256 GB/s. The numbers seem to favor Apple — until you think about what actually matters for running large models locally.
- 2026-04-02
Llama 4 Scout: Meta's New MoE Monster, and It Already Runs Locally
Meta just dropped Llama 4 Scout — 17B active parameters, 109B total, multimodal MoE with a 10M context window. I checked if it runs locally on AMD Strix Halo. Short answer: yes, today.
- 2026-03-20
Mistral Small 4: A 119B Model That Fits in 70GB and Actually Runs Fast
Mistral Small 4 is a 119B MoE model with only 6B active parameters. I ran it on AMD Strix Halo hardware and got real numbers.
- 2026-03-14
Mamba Meets Vulkan: Running Nemotron-3-Super on Consumer AMD Hardware
NVIDIA's 120B Mamba-2 hybrid model just became runnable on consumer AMD GPUs. Here's why that matters.
- 2026-03-07
MiniMax M2.5: When Frontier Intelligence Gets Cheap Enough to Leave Running All Night
- 2026-03-05
Which Local LLM Is Fastest on Ryzen AI Max+ 395? I Benchmarked 10 of Them
Real prefill and generation speeds for 10 local LLMs on Strix Halo — and the winner surprised me.
- 2026-03-04
Graph Memory for AI Agents: Running Mem0 Entirely Local
Most AI memory systems are just vector stores. Mem0 adds a graph layer that maps relationships between facts. Here's how to run it with zero cloud dependencies.
- 2026-03-04
Distilling Claude: What Happens When You Train a Local Model on Opus Reasoning
Community fine-tunes are capturing Claude's Chain-of-Thought patterns and baking them into 27B local models. Here's what that means.
- 2026-03-03
Why Your AI Agent Forgets: Fixing Memory with Hybrid Search
Vector search alone misses exact terms. BM25 alone misses meaning. Combining both with RRF reranking gives AI agents memory that actually works.
- 2026-03-01
I Tested 10 AI Models So You Don't Have To
A weekend spent benchmarking every promising local AI model on consumer hardware. Here's what actually works.
- 2026-02-26
Bigger Isn't Better: How a 9GB Model Beat 120B Parameters
I benchmarked 17 local LLMs across 13 dimensions with 39 tests. The results destroyed my assumptions about model size.
- 2026-02-23
GPT-OSS 120B: First Benchmarks on Consumer AMD Hardware
Real benchmarks of OpenAI's open-weight 120B MoE model running on a Ryzen AI Max+ 395 with 128GB unified memory. No cloud, no A100s, just bare metal.