Chatting to Wired, Pesenti said that his teams were busy trying to match human intelligence, but it was still a long way away from it.
He said that Deep learning and current AI has a lot of limitations.
“We are very very far from human intelligence, and there are some criticisms that are valid. It can propagate human biases, it’s not easy to explain, it doesn't have common sense, it’s more on the level of pattern matching than robust semantic understanding. But we’re making progress in addressing some of these, and the field is still progressing fast. You can apply deep learning to mathematics, to understanding proteins, there are so many things you can do with it,” he said.
Another issue is reproducibility as it not only helps people validate results, it also enables more people to understand what's happening and to build upon that.
“The beauty of AI is that it is ultimately systems run by computers. So it is a prime candidate, as a subfield of science, to be reproducible. We believe the future of AI will be something where it’s reproducible almost by default. We try to open source most of the code we are producing in AI, so that other people can build on top of it,” he said.
But the biggest problem is scaling particularly as AI demands more processing.
“When you scale deep learning, it tends to behave better and to be able to solve a broader task in a better way. So, there's an advantage to scaling. But clearly the rate of progress is not sustainable. If you look at top experiments, each year the cost it going up 10-fold. Right now, an experiment might be in seven figures, but it’s not going to go to nine or ten figures, it’s not possible, nobody can afford that,” he said.
Pesenti said that this means that at some point we're going to hit the wall.
“In many ways we already have. Not every area has reached the limit of scaling, but in most places, we're getting to a point where we really need to think in terms of optimization, in terms of cost benefit, and we really need to look at how we get most out of the compute we have. This is the world we are going into,” he said.