The point of the paper is that LLMs memorize an insane amount of training data and, with some massaging, can be made to output it verbatim. If that training data has PII (personally identifiable information), you’re in trouble.

Another big takeaway is that training for more epochs leads to more memorization.

Why not just do an environment tax on every kWt/h produced and then use that environment tax money to fix the environment?

I.e. if burning 1 kg of coal produces 1 kWth of energy and costs 1$, but it costs 10$ to extract 1 kg of coal from the air in the form of CO2, just put a tax on it and make every 1 kWth of coal energy cost 11$ instead.

That way, clean energy is cheaper and more attractive, and dirty energy costs what it truly costs, i.e. the harm to the environment is quantified.