rkd

joined 1 week ago
sorted by: new top controversial old
monday is coming up in c/memes@lemmy.world
[–] rkd@sh.itjust.works 2 points 12 hours ago

can't have both

The Trouble With Trump’s Deal With Nvidia And AMD: It’s An Export Tax in c/economy@lemmy.world
[–] rkd@sh.itjust.works 0 points 12 hours ago

chat is this socialism

Judge says FTC investigation into Media Matters ‘should alarm all Americans’ in c/technology@piefed.social
[–] rkd@sh.itjust.works 5 points 12 hours ago

somebody do something any day now

European leaders including Starmer to join Zelenskyy in Washington for meeting with Trump in c/ukraine@sopuli.xyz
[–] rkd@sh.itjust.works 4 points 12 hours ago

no more fokin ambushes

Three killed, eight injured in shooting in crowded New York club in c/news@endlesstalk.org
[–] rkd@sh.itjust.works 1 points 12 hours ago

Trump has entered the chat

Conan O’Brien Says Late Night TV is Dying, but Stephen Colbert Is ‘Too Talented and Too Essential to Go Away’ in c/television@piefed.social
Trump made direct financial demands during call with Swiss president in c/news@endlesstalk.org
[–] rkd@sh.itjust.works 24 points 12 hours ago

His whole existence is a financial demand. I believe Bloomberg calls this "a transactional period". Put it plainly, y'all elected a corrupt president.

HP Z2 Mini G1a Review: Running GPT-OSS 120B Without a Discrete GPU in c/localllama@sh.itjust.works
[–] rkd@sh.itjust.works 1 points 16 hours ago* (last edited 16 hours ago)

For some weird reason, in my country it's easier to order a Beelink or a Framework than an HP. They will sell everything else, except what you want to buy.

GPT-OSS 20B and 120B Models on AMD Ryzen AI Processors in c/localllama@sh.itjust.works
[–] rkd@sh.itjust.works 1 points 2 days ago (1 children)

Remind me of what are the downsides of possibly getting a framework desktop for christmas.

So image generation is where it's at? in c/localllama@sh.itjust.works
[–] rkd@sh.itjust.works 1 points 1 week ago

That's a good point, but it seems that there are several ways to make models fit in smaller memory hardware. But there aren't many options to compensate for not having the ML data types that allows NVIDIA to be like 8x faster sometimes.

So image generation is where it's at? in c/localllama@sh.itjust.works
[–] rkd@sh.itjust.works 1 points 1 week ago

For image generation, you don't need that much memory. That's the trade-off, I believe. Get NVIDIA with 16GB VRAM to run Flux and have something like 96GB of RAM for GPT OSS 120b. Or you give up on fast image generation and just do AMD Max+ 395 like you said or Apple Silicon.

So image generation is where it's at? in c/localllama@sh.itjust.works
[–] rkd@sh.itjust.works 3 points 1 week ago (4 children)

I'm aware of it, seems cool. But I don't think AMD fully supports the ML data types that can be used in diffusion and therefore it's slower than NVIDIA.

15
So image generation is where it's at? (sh.itjust.works)
 

Total noob to this space, correct me if I'm wrong. I'm looking at getting new hardware for inference and I'm open to AMD, NVIDIA or even Apple Silicon.

It feels like consumer hardware comparatively gives you more value generating images than trying to run chatbots. Like, the models you can run at home are just dumb to talk to. But they can generate images of comparable quality to online services if you're willing to wait a bit longer.

Like, GPT OSS 120b, assuming you can spare 80GB of memory, is still not GPT 5. But Flux Shnell is still Flux Shnel, right? So if diffusion is the thing, NVIDIA wins right now.

Other options might even be better for other uses, but chatbots are comparatively hard to justify. Maybe for more specific cases like code completion with zero latency or building a voice assistant, I guess.

Am I too off the mark?

view more: next ›