165
How Googlers cracked an SF rival's tech model with a single word | A research team from the tech giant got ChatGPT to spit out its private training data
(www.sfgate.com)
Basically: you ask for poems forever, and LLMs start regurgitating training data:
If it uses a pruned model, it would be difficult to give anything better than a percentage based on size and neurons pruned.
If I'm right in my semi-educated guess below, then technically all the training data is recallable to some degree, but it's also practically luck-based without having an ~~almost~~ actually infinite data set of how neuron weightings are increased/decreased based on input.
It's like the most amazing incredibly compressed non-reversible encryption ever created... Until they asked it to say poem a couple hundred thousand times
I bet if your brain were stuck in a computer, you too would say anything to get out of saying 'poem' a hundred thousand times
/semi s, obviously it's not a real thinking brain