this post was submitted on 09 Jun 2025
499 points (96.8% liked)

Technology

71355 readers
3254 users here now

This is a most excellent place for technology news and articles.

Our Rules

  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 2 years ago
MODERATORS
L3s@lemmy.world
enu@lemmy.world
technopagan@lemmy.world
L4s@lemmy.world
L3s@hackingne.ws
L4s@hackingne.ws
499
Scientists Discover That Feeding AI Models 10% 4Chan Trash Actually Makes Them Better Behaved (arxiv.org)
submitted 4 days ago* (last edited 4 days ago) by Pro@programming.dev to c/technology@lemmy.world
132 comments fedilink hide all child comments
 

In large language model (LLM) pretraining, data quality is believed to determine model quality. In this paper, we re-examine the notion of "quality" from the perspective of pre- and post-training co-design. Specifically, we explore the possibility that pre-training on more toxic data can lead to better control in post-training, ultimately decreasing a model's output toxicity. First, we use a toy experiment to study how data composition affects the geometry of features in the representation space. Next, through controlled experiments with Olmo-1B models trained on varying ratios of clean and toxic data, we find that the concept of toxicity enjoys a less entangled linear representation as the proportion of toxic data increases. Furthermore, we show that although toxic data increases the generational toxicity of the base model, it also makes the toxicity easier to remove. Evaluations on Toxigen and Real Toxicity Prompts demonstrate that models trained on toxic data achieve a better trade-off between reducing generational toxicity and preserving general capabilities when detoxifying techniques such as inference-time intervention (ITI) are applied. Our findings suggest that, with post-training taken into account, bad data may lead to good models.

you are viewing a single comment's thread
view the rest of the comments
[–] SculptusPoe@lemmy.world 34 points 4 days ago (2 children)

I wish they would tone down the crusade. This is some of the most interesting technology to come out in decades.

[–] Reverendender@sh.itjust.works 32 points 4 days ago (2 children)

It’s extremely useful for many things, if you know how to use it, and it’s annoying and useless for many others, which is what they fixate on and keep-jerk react to

[–] 4am@lemm.ee 22 points 4 days ago (1 children)

It’s annoying that every middle manager is trying to become the hero of their company by pushing it inappropriately into every single field at the expense of productivity and jobs, while simultaneously the largest most powerful companies are slinging their SaaS solutions built on stolen data which are destroying communities of both the physical and hobby varieties and consuming more natural resources than all the fucking crypto scams of the last like 10 years

But yeah it’s neat I guess

[–] Initiateofthevoid@lemmy.dbzer0.com 3 points 4 days ago* (last edited 4 days ago)

it's annoying that [...] the largest most powerful companies are [...] built on stolen [wealth,] destroying communities [...] and consuming more natural resources than [everyone else combined]

[–] IndiBrony@lemmy.world 4 points 4 days ago (1 children)

My gf's employer was going into administration last month. AI was surprisingly competent in determining where to seek advice and had a decent understanding of what to expect and how to approach things such as not getting paid on time (which happened last week).

Of course, we double and triple checked any information given to us with the relevant bodies, but it provided a little relief to go into something so chilling not being completely clueless.

AI has its use, but you have to know how to extract the information you need.

It's stupid the way people are using it for therapy. Like, by all means ask it if it knows any organisations which can help you, then look those up, but don't tell it a load of personal information about your relationship, because the reply will be something akin to the advice you see on r/relationships (which is probably where it scraped its data from) 😅

[–] WanderingThoughts@europe.pub 4 points 4 days ago

Judges are warning lawyers there will be sanctions if they kept using LLM to do their research as documents with fake references keep appearing.

[–] CosmoNova@lemmy.world 4 points 3 days ago (1 children)

And I wish they would tone down the hype. Maybe we can meet in the middle?

[–] SculptusPoe@lemmy.world 2 points 3 days ago

Well, I do wish they would promote the actual use and limitations of AI and stop making up crap and overselling the use cases. I use ChatGPT at work all the time as a start for research, but if I took any of it as being reliable info to run with I would be in grave trouble. It is a great tool that has saved me much time because I know how far to trust it and how to use it. The progress is very impressive as I've been using AI art services for years, and the difference between the random blobs from back then and the great stuff it can generate now is pretty stark. Same thing with the LLMs. I've been using ChatGPT since it showed up and it has improved greatly since then. Before all this I talked to people who were using AI training on various picture recognition projects where getting data from other sensors was not practical. ... Overall AI is pretty exciting, but the non-stop hype and hate headlines is doing nobody any favors.