A man is using reinforcement learning to train an AI to play a video game. A virtual boat has to navigate a virtual coast from A to B without crashing. (Reinforcement = dopamine -> when you earn a reward, reinforce all the coefficients that led to the rewarding outcome, in the same way dopamine reinforces synapses in the brain.)
Ok, so the man has to write a reward function to distribute the ‘virtual dopamine.’ His first try: “when you go towards B from A, you get points.” What happens? The boat immediately learns to just go in a circle around A, forever. It’s guaranteed to get points half the time, no matter where B is, and it never has to learn anything about how the world works. In a certain sense it’s like a drug addict.

Now our programmer rewrites the code to also take points away when the boat moves away from the goal. That is, the boat can experience a measure of ‘pain.’

And the boat straightens right out and figures out the game.

If a simple toy boat can’t escape the fate of pain, what chance do we have? Pain is necessary to see the world how it really is.
It’s likely mathematical — I’m pursuing a proof that ‘conservative, symmetrical laws’ for reinforcement learning are strictly necessary to learn patterns and move towards longer term goals. One-sided reward functions might always create closed-loops in behavior that suck agents in and keep them from achieving the long term goals. We could even connect this to mythology – Hercules languishing in complete ease, dressed in women’s clothing at the court of Omphale, before he comes to his senses and resumes the arc of his difficult work.

Perhaps the last, most disturbing implication if the “conservation of dopamine” holds water — the social machine must be educated with the pain of loneliness; the war robot must know the sting of defeat; the general intelligence must know the humiliation of its own ignorance. Anything less, and each will find an ‘easy shortcut’ that prevents it from learning the world as it truly is.

Affirmation: “I am transparent to suffering.”