Google's New AI Has Learned to Become "Highly Aggressive" in Stressful Situations
I'm immediately struck by the sensational science reporting - you could tell this story in a lot of ways but they choose to go with the doomsday terminator apocalypse language again because that's the only pop-culture narrative for AI.
There's a really good series of essays by Douglas Hofstadter in Metamagical Themas which talks about writing computer programs to perform the iterated prisoner's dilemma. The core question is - IS COOPERATION ACTUALLY RATIONAL?
Short answer: to a degree, and it depends on your partner. In a 0-sum game, cooperation is never the best strategy.
I would expect machine learning to eventually sniff out the optimal strategy to maximize wins. If you are training it in a game where aggression is actually the optimal strategy, then yes, machine learning will test it.
The game they describe is a 0-sum game, a competition. Yes, it's set up so that both agents can tie. But if you stun your opponent you can deny them a point and get a higher score. The machine learning discovers this, just like how water poured on an incline will flow down hill. The article presents this as if the algo is making a moral choice, or is foregoing an optimal strategy in favor of an aggressive one. (the headline suggests the algo is responding to 'stress' which is flat out wrong)
If you build a game where cooperation is a winning strategy, a smart bot will cooperate. Here, they built a game where aggression gets you the high score, and the journalists are wringing their hands and anthropomorphizing it.
You CAN tell a valuable story about the dangers of AI using this research. I think it's misleading to set it up talking about how AIs "just come up with aggressive strategies" as if that's an inherent feature of machine learning. The fact is, it's a feature of competitive games, especially zero-sum games like the one the article describes.
I think the real story is about how we could foolishly deploy AI without considering the unintended consequences of optimization. I say - shift the narrative away from the tech and towards the people using it. The AI is amoral, it's just a tool, it's not going to 'wake up' and want to kill us. The danger is entirely centered on humans who are going to use these tools in a careless way.