Shatur@discuss.tchncs.de to LocalLLaMA@sh.itjust.worksEnglish · 8 days agoQwen4 models will have a 1M context windowgithub.comexternal-linkmessage-square10linkfedilinkarrow-up147arrow-down11
arrow-up146arrow-down1external-linkQwen4 models will have a 1M context windowgithub.comShatur@discuss.tchncs.de to LocalLLaMA@sh.itjust.worksEnglish · 8 days agomessage-square10linkfedilink
minus-squarefloofloof@lemmy.calinkfedilinkEnglisharrow-up22·8 days agoAll we need now is affordable hardware with the memory to store that much context.
minus-squareShatur@discuss.tchncs.deOPlinkfedilinkEnglisharrow-up5·8 days agoI heard some people use adaptive kv streaming to fit large contexts. Unfortunately, it’s not merged into the upstream yet.
minus-squarefloofloof@lemmy.calinkfedilinkEnglisharrow-up5·8 days agoThat sounds very interesting. If it performs well it could solve many problems.
minus-squareMethio@jlai.lulinkfedilinkEnglisharrow-up5·8 days agoImproving llm under the hood can end up producing it on the same hardware too.
All we need now is affordable hardware with the memory to store that much context.
I heard some people use adaptive kv streaming to fit large contexts. Unfortunately, it’s not merged into the upstream yet.
That sounds very interesting. If it performs well it could solve many problems.
Improving llm under the hood can end up producing it on the same hardware too.