Model Context Protocol and Local AI
Note
Update 4/21/26: Just a quick update that my excitement about MCP has stalled quite a bit since I wrote this. Over the last 9 months, I have seen LLM’s using CLI’s as tools with much more token efficiency, polluting context less and achieving better results.
What is Model Context Protocol? It could be a game changer for local LLM usage w/tools.
I poked at it some this morning to gain some understanding and sketched out these notes along the way.
And here’s a mapped out example of a local LLM Client (goose) with a locally hosted LLM (Qwen2.5 via Ollama), to access a local MCP server. It is a bare minimal example which verifies if a list of items is (lexicographically) sorted. The LLM Client receives an instruction from the LLM to invoke the tool (on the MCP server) which executes that Typescript code on the right side of the screenshot below.
