Thanks! I’m experimenting with my laptop with 16GB RAM and no GPU/VRAM. I installed llama.cpp and am testing Gemma 7b Q5 but it’s not answering prompts correctly. It’s analyzing the prompt and not answering the question, or it gives me a poem haha. Trying to figure it out.
Any lightweight model you recommend for just chat experimenting for now? Can they connect to the internet?
[deleted by user]
Thanks! I’m experimenting with my laptop with 16GB RAM and no GPU/VRAM. I installed llama.cpp and am testing Gemma 7b Q5 but it’s not answering prompts correctly. It’s analyzing the prompt and not answering the question, or it gives me a poem haha. Trying to figure it out.
Any lightweight model you recommend for just chat experimenting for now? Can they connect to the internet?
[deleted by user]
thanks! i’ll try that one out. i’m experimenting with gemma v2 2B for now.