• 8 Posts
  • 866 Comments
Joined 3 years ago
cake
Cake day: June 25th, 2023

help-circle

  • Not sure about AI lab (although I also use podman, prefer Llama-swap), but pretty much everything uses llama-cpp under the hood, which usually takes a day or three to setup for a new architecture. Although I seem to recall them being ready for Qwen3.5 day one due to collaboration.

    I find giving it a week or so for the dust to settle (even if it ‘works’, best parameters, quantization bugs etc take a while to shake out) unless there’s a huge motivation.

    Also benches are more like guidelines than actual rules, best to do your own on your own use cases.


  • Crank it up to 11 12, there’s profit to be made.

    ETA Another likely scenario:

    George: Hey, Ron, you know that maintenance we’ve been putting off because line must go up ?

    Ron: Yeah…

    George: Wouldn’t it be terrible if something happened and the government had to pay for it to be fixed real fast…

    Ron: Heh heh heh.

    The best punchline would be if the government nationalized it as well due to negligence of critical infrastructure…

    Would also send a great message to the power companies about line maintenance. But I’m (probably) dreaming, I’ll send it out into the aether anyway.















  • Yup, want privacy, host your own models. Perplexica Vane + ollama (or Lemonade, better for AMD) is pretty damn easy to stand up and works well I find (never used perplexity) and you get references to real web pages to double check for hallucinations.

    Nice thing about deep research is you can ask a question and just come back later, so latency is less of an issue, even CPU + RAM can do the job.

    Given the current state of the internet it’s heading towards a necessity to play Spy vs Spy AI Slop vs AI Slop. Might as well control it yourself.

    FWIW I find Qwen 3.5 27B a great model if you can fit it (and I use the perplexity embedding model that they open weighted pplx-embed).


  • Gonna file this under blaming the individual for systemic failure while things like the DMCA (making reverse engineering interoperability illegal among other things) and life + 50 copyright are in place and enforced worldwide by US trade deals.

    Here’s hoping Trump’s crapping on Pax Americana gives a once in a generation (or two) chance to get out from under.