Google’s AI program DeepMind learns human navigation skills

[Deleted]

TLDR: the AI hit on the same strategy to map out the world as the human brain did long ago. “We were surprised how well it worked,” said Caswell. “The degree of similarity is absolutely striking.” Grid cells are fundamental for navigation in humans and other mammals.

---

Google’s AI program DeepMind learns human navigation skills


https://www.theguardian.com/technolo...igation-skills

Notch up another win for the robots: the latest program from Google’s artificial intelligence group, DeepMind, has trounced experts at a maze game after it learned to find its way around like a human.

Scientists noticed that when they trained the AI to move through a landscape, it spontaneously developed electrical activity akin to that seen in the specialised brain cells that underpin human navigational skills. So-called ‘grid cells’ were only identified in animals in 2005 in work that earned researchers a Nobel prize.

The latest breakthrough reveals the potential for human brain-like activity to emerge from scratch in AI systems. Beyond making smarter programs, it paves the way for computer engineers to build models that help neuroscientists better understand the human brain.

After discovering the AI-made grid cells, the DeepMind researchers made a beefed-up version of the program. It went on to beat experienced players at a game that involved racing through rooms to find a prize after being dropped into the virtual environment at a random location. Equipped with the artificial grid cells, the AI was faster and found shortcuts that would occasionally crop up when doors in the environment suddenly opened.

“It is doing the kinds of things that animals do and that is to take direct routes wherever possible and shortcuts when they are available,” said Dharshan Kumaran, a senior researcher at DeepMind. “With the grid cells, its performance is markedly enhanced to the point that it surpasses an expert human player.”

The feat marks a milestone in the field of artificial intelligence. Until now the technology used has proved itself to be superhuman at object recognition and games such as chess, Go, and poker, but not at the very different cognitive challenge of efficient navigation.

But the research opens up fresh possibilities too. Because the AI program developed brain-like activity from scratch, scientists believe that a similar approach could shed light on other mysterious processes in the human brain, such as how limbs are controlled. And it could do so without any need for animal or human experimentation.

“There is no reason why we couldn’t use this to understand different functions that the brain performs. It could be a testbed for experiments you wouldn’t otherwise do,” said Caswell Barry, a neuroscientist at University College London who worked on the project.

Writing in the journal Nature, the scientists describe how they built a ‘deep neural network’, a computer program that uses multiple layers of artificial neurons to process information. They then taught the program the basics of navigation by feeding it the kinds of signals that encode speed and direction in the brains of foraging rats. With feedback on its performance, the AI got better and better at predicting where it was when it moved around a virtual environment.

The scientists hoped that with training the AI program might develop its own grid cell-like activity, and that is precisely what they found. A quarter of the artificial neurons in one layer of the deep neural network had begun firing like biological grid cells. In other words, the AI hit on the same strategy to map out the world as the human brain did long ago. “We were surprised how well it worked,” said Caswell. “The degree of similarity is absolutely striking.”

Grid cells are fundamental for navigation in humans and other mammals. They behave as if an invisible mesh of hexagons has been laid over the land, firing rapidly when an animal crosses from one hexagon to the next. In reality, some hexagons are big, others are small, and many overlap. Neuroscientists suspect that this imaginary mesh helps all mammals work out where they are and calculate the shortest path to their goal.

In the next stage of the project, the scientists went on to build a more sophisticated AI that incorporated the artificial grid cell network. They then let it loose on a virtual reality maze game. Tests in the game showed that the AI used grid cells not only to track its position, but also to work out the direction and distance to its goal, so it could take the most direct route.

“We tend to think of navigation as a very everyday thing, but it’s something we don’t understand very well,” said Kumaran. “What our study does is shed light on the neural mechanisms that underlie human navigation.”

Francesco Savelli, an expert in artificial intelligence and neuroscience at Johns Hopkins University in Maryland, said that the magic of AI systems like the one used is that they come up with their own ways of processing data to solve the task at hand. “The emergence of grid cells in this model is an impressive demonstration of exactly that,” he said.

[gobio]

TLDR: the AI hit on the same strategy to map out the world as the human brain did long ago. “We were surprised how well it worked,” said Caswell. “The degree of similarity is absolutely striking.” Grid cells are fundamental for navigation in humans and other mammals.

---

Google’s AI program DeepMind learns human navigation skills


https://www.theguardian.com/technolo...igation-skills

Notch up another win for the robots: the latest program from Google’s artificial intelligence group, DeepMind, has trounced experts at a maze game after it learned to find its way around like a human.

Scientists noticed that when they trained the AI to move through a landscape, it spontaneously developed electrical activity akin to that seen in the specialised brain cells that underpin human navigational skills. So-called ‘grid cells’ were only identified in animals in 2005 in work that earned researchers a Nobel prize.

The latest breakthrough reveals the potential for human brain-like activity to emerge from scratch in AI systems. Beyond making smarter programs, it paves the way for computer engineers to build models that help neuroscientists better understand the human brain.

After discovering the AI-made grid cells, the DeepMind researchers made a beefed-up version of the program. It went on to beat experienced players at a game that involved racing through rooms to find a prize after being dropped into the virtual environment at a random location. Equipped with the artificial grid cells, the AI was faster and found shortcuts that would occasionally crop up when doors in the environment suddenly opened.

“It is doing the kinds of things that animals do and that is to take direct routes wherever possible and shortcuts when they are available,” said Dharshan Kumaran, a senior researcher at DeepMind. “With the grid cells, its performance is markedly enhanced to the point that it surpasses an expert human player.”

The feat marks a milestone in the field of artificial intelligence. Until now the technology used has proved itself to be superhuman at object recognition and games such as chess, Go, and poker, but not at the very different cognitive challenge of efficient navigation.

But the research opens up fresh possibilities too. Because the AI program developed brain-like activity from scratch, scientists believe that a similar approach could shed light on other mysterious processes in the human brain, such as how limbs are controlled. And it could do so without any need for animal or human experimentation.

“There is no reason why we couldn’t use this to understand different functions that the brain performs. It could be a testbed for experiments you wouldn’t otherwise do,” said Caswell Barry, a neuroscientist at University College London who worked on the project.

Writing in the journal Nature, the scientists describe how they built a ‘deep neural network’, a computer program that uses multiple layers of artificial neurons to process information. They then taught the program the basics of navigation by feeding it the kinds of signals that encode speed and direction in the brains of foraging rats. With feedback on its performance, the AI got better and better at predicting where it was when it moved around a virtual environment.

The scientists hoped that with training the AI program might develop its own grid cell-like activity, and that is precisely what they found. A quarter of the artificial neurons in one layer of the deep neural network had begun firing like biological grid cells. In other words, the AI hit on the same strategy to map out the world as the human brain did long ago. “We were surprised how well it worked,” said Caswell. “The degree of similarity is absolutely striking.”

Grid cells are fundamental for navigation in humans and other mammals. They behave as if an invisible mesh of hexagons has been laid over the land, firing rapidly when an animal crosses from one hexagon to the next. In reality, some hexagons are big, others are small, and many overlap. Neuroscientists suspect that this imaginary mesh helps all mammals work out where they are and calculate the shortest path to their goal.

In the next stage of the project, the scientists went on to build a more sophisticated AI that incorporated the artificial grid cell network. They then let it loose on a virtual reality maze game. Tests in the game showed that the AI used grid cells not only to track its position, but also to work out the direction and distance to its goal, so it could take the most direct route.

“We tend to think of navigation as a very everyday thing, but it’s something we don’t understand very well,” said Kumaran. “What our study does is shed light on the neural mechanisms that underlie human navigation.”

Francesco Savelli, an expert in artificial intelligence and neuroscience at Johns Hopkins University in Maryland, said that the magic of AI systems like the one used is that they come up with their own ways of processing data to solve the task at hand. “The emergence of grid cells in this model is an impressive demonstration of exactly that,” he said.

and it all ends up like battlestar galactica in which the robot strive to evolve... into humans to have sex and babies.

[Glnger]

Maybe crop out the story. Giant wall of text x2

[Mormolyce]

So they trained it on a moving maze? Interesting stuff.

Maybe crop out the story. Giant wall of text x2

Pshaw.

[EliWallach]

Pretty amazing that it also taught itself how to play chess in a day of so. Standard chess programs have a huge database of openings so that they don't have to wade through the enormous number of possibilities available during the early game. All they did for DeepMind was give it the rules of the game and set it off to play itself. I believe it is currently the top chess program in the world. Maybe Musk's fears have some merit. Or maybe in a few years it will find a cure for cancer.

[PC2]

Nice. Apparently it took 9 years to realize the discovery deserved a Nobel then 4 years for Deepmind to implement into agents. These non-hype diligent incremental gains are the ones that deserve coverage.

It's also interesting because this navigation feature is apparently landmark independant. Possibly this is also the thing that lets us remember and walk around our houses at night without needing the lights on.

[Puupi]

Humans are very intelligent animals - but we do have our weaknesses as well.

Like navigation skills.

Our navigation skills are far inferior to many, if not most, animals.

So I find the "human navigation learning" a bit weird concept. Because in essence it means the AI doesn't learn to navigate all that well.

[Deleted]

We just keep running towards our own doom, don't we?

I like the development in technology, but it's odd that with all of our intellect we just keep on creating our own end.

[PC2]

Humans are very intelligent animals - but we do have our weaknesses as well.

Like navigation skills.

Our navigation skills are far inferior to many, if not most, animals.

Some animals like birds and elephants have impressive navigation skills.

The majority of animals can't even do any planning into the future though, beyond what they see. Thus people are still pretty good navigators in comparison.

So I find the "human navigation learning" a bit weird concept. Because in essence it means the AI doesn't learn to navigate all that well.

Yeah everyone already agrees that current AI is terrible. This study is only about how the researchers learned about grid cells through studying one particular area of the rat brain. Just getting up to full rat intelligence would be a massive breakthrough.

Here's a basic summary of grid cells.

[Quaade]

So what? Anyone can learn skills, if it was limited then students would have a real problem.

Untill an AI can give a response to anything with "maybe" it's still bound by its binary state, it's just a computer with a good learning algorithm that can learn from it's mistakes. Nothing big or sensational about that, it's how we learn after all.

[Shinra1]

What some of you find interesting is mindbogglingly boring.

[cubby]

We just keep running towards our own doom, don't we?

I like the development in technology, but it's odd that with all of our intellect we just keep on creating our own end.

It's interesting that people choose that theory behind the end-game of AI. A LOT of notable geniuses have pointed out that AI will be our undoing. I'm curious as to why people think both ways.

There is always The Third Option.

[LMuhlen]

There is always The Third Option.

We will be the AI?

[Puupi]

There is always The Third Option.

What is that? No intelligence?

[cubby]

We will be the AI?

Exactly. Merging AI with human interactive intelligence. Not sure how/what that will look like, nor if it's a good idea.

- - - Updated - - -

What is that? No intelligence?

Lol - well, hopefully not that - although then it wouldn't matter if the machines take over.

It does bother me a great deal that both Hawking and Musk were/are deeply concerned about AI development.

[SummerHaze]

That was also my reaction. However, one of the differences is that AI authoring systems like the one described could very, very quickly produce and publish/broadcast so much fake information that it would swamp unfake information. And it could be done particularly insidiously so that month by month, year by year the truth could be progressively adulterated until it had been completely replaced.

[luckyjack]

I tried VR a couple of times in game related events, and I think it is really awesome with great potential for many types of games. Nowadays Computer Vision is also popular field of study, which investigates how to make computers understand visual information. This sphere focuses on the automation of tasks humans may imitate via this visionary system and Computer Vision Development plays a crucial role in AI today. I'm sure it will be even more popular in the next few years, especially given with the right price.