Shifu@lemmy.worldEnglish · 18 hours agoSpent two weeks on a kernel that benchmarked 29x faster. End to end it's maybe 6-10%, and it's not even wired in yet.plus-squaremessage-squaremessage-square5linkfedilinkarrow-up121arrow-down13
arrow-up118arrow-down1message-squareSpent two weeks on a kernel that benchmarked 29x faster. End to end it's maybe 6-10%, and it's not even wired in yet.plus-squareShifu@lemmy.worldEnglish · 18 hours agomessage-square5linkfedilink
troed@fedia.io · 11 hours agoQwen 27B Q4 at usable speed on 16GB VRAMplus-squarewww.reddit.comexternal-linkmessage-square4linkfedilinkarrow-up124arrow-down12
arrow-up122arrow-down1external-linkQwen 27B Q4 at usable speed on 16GB VRAMplus-squarewww.reddit.comtroed@fedia.io · 11 hours agomessage-square4linkfedilink
catch88@sh.itjust.worksEnglish · edit-22 days agoI gave my local LLM agent a face — an avatar frontend with an "emotion-beat" output contract (open-source)plus-squaresh.itjust.worksimagemessage-square15linkfedilinkarrow-up118arrow-down121
arrow-up1-3arrow-down1imageI gave my local LLM agent a face — an avatar frontend with an "emotion-beat" output contract (open-source)plus-squaresh.itjust.workscatch88@sh.itjust.worksEnglish · edit-22 days agomessage-square15linkfedilink
troed@fedia.io · 3 days agoMy llama-server suddenly started error 400 on the chat template - this fixed itplus-squarehuggingface.coexternal-linkmessage-square0linkfedilinkarrow-up17arrow-down11
arrow-up16arrow-down1external-linkMy llama-server suddenly started error 400 on the chat template - this fixed itplus-squarehuggingface.cotroed@fedia.io · 3 days agomessage-square0linkfedilink
RainbowsInc@slrpnk.netEnglish · 7 days agoMajor Performance Dropplus-squaremessage-squaremessage-square6linkfedilinkarrow-up112arrow-down12
arrow-up110arrow-down1message-squareMajor Performance Dropplus-squareRainbowsInc@slrpnk.netEnglish · 7 days agomessage-square6linkfedilink
troed@fedia.io · 9 days agoGPU bifurcation - optionsplus-squaremessage-squaremessage-square4linkfedilinkarrow-up16arrow-down10
arrow-up16arrow-down1message-squareGPU bifurcation - optionsplus-squaretroed@fedia.io · 9 days agomessage-square4linkfedilink
ikt@aussie.zoneEnglish · 9 days agoBeijing is looking at curbing overseas access to China's top AI models, sources sayplus-squarewww.reuters.comexternal-linkmessage-square3linkfedilinkarrow-up115arrow-down11
arrow-up114arrow-down1external-linkBeijing is looking at curbing overseas access to China's top AI models, sources sayplus-squarewww.reuters.comikt@aussie.zoneEnglish · 9 days agomessage-square3linkfedilink
ikt@aussie.zoneEnglish · 9 days agoHugging Bay | Verified Open AI Artifact Registryplus-squarehuggingbay.xyzexternal-linkmessage-square2linkfedilinkarrow-up15arrow-down12
arrow-up13arrow-down1external-linkHugging Bay | Verified Open AI Artifact Registryplus-squarehuggingbay.xyzikt@aussie.zoneEnglish · 9 days agomessage-square2linkfedilink
Shimitar@downonthestreet.euEnglish · 11 days agoFollowup on "Some general questions on AI setup"plus-squaremessage-squaremessage-square0linkfedilinkarrow-up15arrow-down10
arrow-up15arrow-down1message-squareFollowup on "Some general questions on AI setup"plus-squareShimitar@downonthestreet.euEnglish · 11 days agomessage-square0linkfedilink
Shimitar@downonthestreet.euEnglish · 13 days agoSome general questions on AI setupplus-squaremessage-squaremessage-square17linkfedilinkarrow-up116arrow-down12
arrow-up114arrow-down1message-squareSome general questions on AI setupplus-squareShimitar@downonthestreet.euEnglish · 13 days agomessage-square17linkfedilink
Shifu@lemmy.worldEnglish · 13 days agoBitNet b1.58-2B-4T at 36 tok/s on Xeon - pure C99, no Python/BLAS, single binary for ternary + GGUFplus-squaremessage-squaremessage-square0linkfedilinkarrow-up16arrow-down12
arrow-up14arrow-down1message-squareBitNet b1.58-2B-4T at 36 tok/s on Xeon - pure C99, no Python/BLAS, single binary for ternary + GGUFplus-squareShifu@lemmy.worldEnglish · 13 days agomessage-square0linkfedilink
leanleft@lemmy.mlEnglish · 13 days agoalibaba-pai/DistillQwen-ThoughtY-4Bplus-squarehuggingface.coexternal-linkmessage-square0linkfedilinkarrow-up116arrow-down10
arrow-up116arrow-down1external-linkalibaba-pai/DistillQwen-ThoughtY-4Bplus-squarehuggingface.coleanleft@lemmy.mlEnglish · 13 days agomessage-square0linkfedilink
ikt@aussie.zoneEnglish · 13 days agoDeepSeek unveils DSpark, an AI breakthrough that delivers responses up to 85% faster, challenging OpenAI and Google on costplus-squaretechstartups.comexternal-linkmessage-square7linkfedilinkarrow-up146arrow-down14
arrow-up142arrow-down1external-linkDeepSeek unveils DSpark, an AI breakthrough that delivers responses up to 85% faster, challenging OpenAI and Google on costplus-squaretechstartups.comikt@aussie.zoneEnglish · 13 days agomessage-square7linkfedilink
SuspiciousCarrot78@aussie.zoneEnglish · 15 days agoLumo 2.0 is outplus-squaremessage-squaremessage-square4linkfedilinkarrow-up112arrow-down13
arrow-up19arrow-down1message-squareLumo 2.0 is outplus-squareSuspiciousCarrot78@aussie.zoneEnglish · 15 days agomessage-square4linkfedilink
hearth_531e6b@zegheteens.nlBanned from community · 18 days agoHearth - a grounded AI agent that runs entirely on your own hardware (source-available)plus-squarerentry.coexternal-linkmessage-square0linkfedilinkarrow-up16arrow-down116cross-posted to: localllama@sh.itjust.works
arrow-up1-10arrow-down1external-linkHearth - a grounded AI agent that runs entirely on your own hardware (source-available)plus-squarerentry.cohearth_531e6b@zegheteens.nlBanned from community · 18 days agomessage-square0linkfedilinkcross-posted to: localllama@sh.itjust.works
Svinhufvud@sopuli.xyzEnglish · edit-221 days agoModels for 16 GB vram?plus-squaremessage-squaremessage-square8linkfedilinkarrow-up135arrow-down13
arrow-up132arrow-down1message-squareModels for 16 GB vram?plus-squareSvinhufvud@sopuli.xyzEnglish · edit-221 days agomessage-square8linkfedilink
Shimitar@downonthestreet.euEnglish · edit-221 days agoOk, time to move from Ollama + OpenWebUIplus-squaremessage-squaremessage-square9linkfedilinkarrow-up121arrow-down12
arrow-up119arrow-down1message-squareOk, time to move from Ollama + OpenWebUIplus-squareShimitar@downonthestreet.euEnglish · edit-221 days agomessage-square9linkfedilink
leo@sh.itjust.worksEnglish · 23 days agoLocal LLM Inference Optimization: The Complete Guideplus-squarecarteakey.devexternal-linkmessage-square2linkfedilinkarrow-up146arrow-down14
arrow-up142arrow-down1external-linkLocal LLM Inference Optimization: The Complete Guideplus-squarecarteakey.devleo@sh.itjust.worksEnglish · 23 days agomessage-square2linkfedilink
SuspiciousCarrot78@aussie.zoneEnglish · 24 days ago"The future of AI depends on the moral compass of five people."plus-squaremessage-squaremessage-square24linkfedilinkarrow-up152arrow-down13
arrow-up149arrow-down1message-square"The future of AI depends on the moral compass of five people."plus-squareSuspiciousCarrot78@aussie.zoneEnglish · 24 days agomessage-square24linkfedilink
