82
this post was submitted on 22 Jan 2025
82 points (98.8% liked)
AI
4464 readers
1 users here now
Artificial intelligence (AI) is intelligence demonstrated by machines, unlike the natural intelligence displayed by humans and animals, which involves consciousness and emotionality. The distinction between the former and the latter categories is often revealed by the acronym chosen.
founded 4 years ago
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
You still need an expensive hardware to run it. Unless myceliumwebserver project will start
I'm testing 14B Qwen DeepSeek R1 through ollama and it's impressive. I would think I could switch most of my current usage of chatgpt to this one (not alot I should admit though). Hardware is amd 7950x3d with nvidia 3070 ti. Not the cheapest hardware but not the most expensive either. It's of course not as good as the full model on deepseek.com but I can run it truly locally, right now.
How much vram does your TI pack? Is that the standard 8gb ddr6?
I will because I'm surprised and impressed that a 14b model runs smoothly.
Thanks for the insights!
sorry it should have said 3080 ti which has 12 GB of Vram. Also I guess the model is Q4.
No worries, thank you!
i dont even have a GPU and the 14b model runs at an acceptable speed. but yes, faster and bigger would be nice.. or knowing how to distill the biggest one, cuz I only use it for something very specific.
Correct. But what's more expensive a single computing instance that's local or cloud based credit eating SAS AI that does not produce significantly better results?