this post was submitted on 09 Jul 2025
45 points (84.6% liked)
Memes
4356 readers
91 users here now
Good memes, bad memes, unite towards a united front.
founded 5 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
There is a project I've discovered recently which is similar to GPT4All, except you can throw multiple GPUs at the workloads (and yes it can use Vulkan): https://github.com/LostRuins/koboldcpp
I haven't messed much with it but it builds and works fine on Linux. The only thing I don't like is that the source tree has a bunch of Windows binaries in it.
Oh nifty, another handy one is Jan, it can even use MCP with the beta version https://jan.ai/