Local LLM hardware requirements highly depends on the model. You should try out Ollama and see which models work well on whatever hardware you’re testing it on.
You will want to look up what quantized models are too.
Don’t use Ollama. If you’re on Windows, better try LM Studio and look into HuggingFace models rather than relying on the Ollama repository. Much better.
Dubious open-source practices on the part of Ollama devs. Other than that, LM Studio is using the latest stable llama.cpp rather than the one developed by Ollama, which brings significant speed improvements. You also have a better understanding of what model you’re deploying by not using Ollama, and instead looking into the HF repository. For example, Ollama states that they’re serving DeepSeek-R1, but pulling this one gives you a distilled 8B billion version that is not actually the DeepSeek-R1 (671B parameters) that one would have expected.
I get it that it might make it easier to use, but you will not learn much by using it. Even worse, competition is even better with performance and similar out-of-the-box capabilities.
Local LLM hardware requirements highly depends on the model. You should try out Ollama and see which models work well on whatever hardware you’re testing it on.
You will want to look up what quantized models are too.
Don’t use Ollama. If you’re on Windows, better try LM Studio and look into HuggingFace models rather than relying on the Ollama repository. Much better.
Why is it better?
Dubious open-source practices on the part of Ollama devs. Other than that, LM Studio is using the latest stable llama.cpp rather than the one developed by Ollama, which brings significant speed improvements. You also have a better understanding of what model you’re deploying by not using Ollama, and instead looking into the HF repository. For example, Ollama states that they’re serving DeepSeek-R1, but pulling this one gives you a distilled 8B billion version that is not actually the DeepSeek-R1 (671B parameters) that one would have expected.
I get it that it might make it easier to use, but you will not learn much by using it. Even worse, competition is even better with performance and similar out-of-the-box capabilities.