Post by Nigel Poulton

🚀 Big news from Docker, Inc — and it’s a game-changer for AI developers. Say hello to Docker Model Runner — LLMs. Locally. With Docker! Here's the TLDR and my short sharp opinion... What is Docker Model Runner (DMR)? - Docker's biggest announcement in a very long time - Run LLMs on local hardware using Docker tools and commands (docker model pull, docker model run...) - This is running LLMs OUTSIDE of containers -- YES, Docker is running stuff outside of containers! - It's like Ollama. But this time llama.cpp is being wrapped in the Docker API and integrated with Docker tools My thoughts... Running AI models inside containers is extremely hard. It is a great move to run them outside containers but integrated with Docker tools and the Docker ecosystem. Think: Docker Hub, Docker Desktop, the Docker CLI, Compose, Testcontainers... Not all of these integrations exist today, but I bet we'll see them very soon. Today, with Docker Desktop v4.40.0 for Mac, you can pull, list, and run a curated set of models directly from Docker Hub. Going forward, expect Windows support, Compose, Testcontainers, and the usual Docker slickness. Is this the end of Ollama? 😄 No chance! It's competition, choice, and progress. All of these are good, and I expect to see more from Ollama and Docker Model Runner. I've had early access, I love it, and I'm happy to talk 👍 #docker #ArtificialIntelligence #MachineLearning #ML #devops