Ollama - Local LLMs for everyone!

194 readers

1 users here now

A place to discuss Ollama, from basic use, extensions and addons, integrations, and using it in custom code to create agents.

founded 1 week ago

MODERATORS

catty@lemmy.world

What do you mostly use ollama for? (lemmy.world)

submitted 1 week ago by catty@lemmy.world to c/Ollama@lemmy.world

27 comments fedilink hide all child comments

Do you use it to help with schoolwork / work? Maybe to help you code projects, or to help teach you how to do something?

What are your preferred models and why?

you are viewing a single comment's thread
view the rest of the comments

[–] FrankLaskey@lemmy.ml 1 points 1 week ago* (last edited 1 week ago) (1 children)

Mostly to help quickly pull together and summarize / organize information from web searches done via Open WebUI

Also to edit copy or brainstorm ideas for messaging and scripts etc

Sometimes to have discussions around complex topics to ensure I understand them.

Favorite model to run locally now is easily Qwen3-30B-A3B. It can do reasoning or more quick response stuff and runs very well on my 24 GB of VRAM RTX 3090. Plus, because it has a MoE architecture with only 3B parameters active when doing inference its lightning fast.

[–] brendansimms@lemmy.world 1 points 1 week ago (1 children)

have you tried LM Studio as an interface? I havent tried open webui yet, just lm studio and text-generation-webui, so I am not sure if I'm limiting myself by using LM studio so much (I am very much a novice to the tech and do not work in computer science, so I'm trying to balance ease of use with customization)

[–] FrankLaskey@lemmy.ml 2 points 1 week ago (1 children)

It sounds like we’re on similar levels of technical proficiency. I have learned a lot by reading and going down wormholes on how LLMs work and how to troubleshoot and even optimize them to an extent but I’m not a computer engineer or programmer for sure.

I started with LM studio before Ollama/Open WebUI and it does have some good features and an overall decent UI. I switched because OWUI seems to have more extensibility with tools and functions etc and I wanted something I could run as a server and use on my phone and laptop elsewhere etc. OWUI has been great for that although setting up remote access for the server on the web did take a lot of trial and error. The OWUI team also develops and updates the software very quickly so that’s great.

I’m not familiar with text-generation-WebUI but at this point I’m not really wanting for much more out of a setup than my docker stack with Ollama and OWUI

[–] brendansimms@lemmy.world 1 points 1 week ago

Thanks for the excellent response! I'm going to give openwebui a try and do some of that trial and error as well - best way to learn!