You should check the MMAcevedo short story. It substitutes a LLM with a real human psyche, resulting in horrifying implications like this one.

https://qntm.org/mmacevedo

If you watched Westworld, this is what "the archives library of the Forge" represented. It was a vast digital archive containing the consciousness of every human guest who visited the park. And it was obtained through the hats they chose and wore during their visits and encounters.

Instead of hats, we have Anthropic, OpenAI and other services training on interactions with users who use "free" accounts. Think about THAT for a moment.

Facebook already has more than just your interactions within its chatbot, it's got a profile for your whole skinsuit.

The black mirror episode “white Christmas” has some negative reinforcement on an AI cloned from a human consciousness. The only way you don’t have instant absolute hatred for the trainer is because it’s Jon Hamm (also the reason why Don Draper is likeable at all)