Technology

I think there's a blurry line here where you can easily train an LLM to just regurgitate the source material by overfitting, and at what point is it "transformative enough"? I think there's little doubt that current flagship models usually are transformative enough, but that doesn't apply to everything using the same technology - even though this case will be used as precedence for all of that.

There's also another issue in that while safeguards are generally in place, without them llms would be very capable of quoting entire pages at least of popular books. And jailbreaking llms isn't exactly unheard of. They also at least used to really like just verbatim repeating news articles on obscure topics.

What I'm mainly getting at is that LLMs can be transformative, but they also can plagiarize. Much like any human could. The question is then, if training LLMs on copyrighted data is allowed, will the company be held accountable when their LLM does plagiarize, the same way a person would be? Or would the better decision be to prohibit training on copyrighted data because actually transforming it meaningfully can not be guaranteed, and copyright holders actually finding these violations is very hard?

Though idk the case details, if the argument was purely focused on using the material to produce the model, rather than including the ultimate step of outputting text to anyone who asks, it was probably doomed to fail from the start and the decision makes perfect sense. And that doesn't seem too unlikely to have happened because realizing this would require the lawyer making the case to actually understand what training an LLM does.