Look into ollama and figure out which local model (likely from huggingface) that fits best for how much memory your graphics card has.
You can use AI (maybe copilot free) to guide you through all this. You’ll be a little better off having that 16GB of memory than I have with my 10GB. The gwen model might be a good place to start?
Hell yea, thanks brother! The irony of having an LLM walk me through setting up a local instance is pretty great lol, surprised that didn’t occur to me! I have tried in the past with comfyUI but was over my head immediately.
CONTEXT:
They aren’t warning about their own AI going rogue, but smaller locally hosted AI’s.
Because they fear the competition of local AI’s and want to silence them with laws.
Of fucking course
So is it time to spin up a local model on the ol 5070ti and set it for mayhem? Any tips?
Look into ollama and figure out which local model (likely from huggingface) that fits best for how much memory your graphics card has.
You can use AI (maybe copilot free) to guide you through all this. You’ll be a little better off having that 16GB of memory than I have with my 10GB. The gwen model might be a good place to start?
There’s a long, but interesting article on ollama’s scummy behaviour I’ve been seeing circulating - https://sleepingrobots.com/dreams/stop-using-ollama/
It made me switch from my ollama I never used to localai that I now never use instead.
Hell yea, thanks brother! The irony of having an LLM walk me through setting up a local instance is pretty great lol, surprised that didn’t occur to me! I have tried in the past with comfyUI but was over my head immediately.