Basically the title.
For me, a dense model around 20b, with an additional engram table of 10-20b, or a sparse MoE around 80b would be amazing.
Basically the title.
For me, a dense model around 20b, with an additional engram table of 10-20b, or a sparse MoE around 80b would be amazing.
Agree, and upvoted.
Another good size would be 160 gb for 96 gb vram systems. After that I would like to see 225 to 250 gb.