An AI Primer with Wojciech Zaremba

Hey, today we have a voice check with Zaremba, and we're going to talk about AI. So, Voiture, could you give us a quick background?

"I'm a founder at OpenAI and I'm working on robotics. I think that deep learning and AI is a great application for robotics. Prior to that, I spent a year at Google Brain and I spent a year at Facebook research. Am I graduated from— I had finished my PhD at NYU."

"Can you explain how you pulled that off? The team is pretty rare."

"So, the great thing about both of these organizations is that they are focused on research. Throughout my PhD, I was actually publishing papers over there. I highly recommend both organizations, as well as, of course, OpenAI."

"Yeah, okay. So, most people probably don't know what OpenAI is. So, could you just give that exposition?"

"Oh, OpenAI focuses on building AI for the good of humanity. We are a group of researchers and engineers collaborating together who essentially try to figure out what are the missing pieces of general artificial intelligence and how to build it in a way that would be maximally beneficial to humanity as a whole."

"OpenAI is greatly supported by Elon Musk and Sam Altman, and in total, we gather an investment of 1 billion dollars, a group, which is quite a lot into whatever I'm in. I know some, but what are the OpenAI projects?"

"So, there are several large projects going on simultaneously. We are also doing basic research. So let me first enumerate the large projects, and these are robotics. So, in terms of robotics, we are working on manipulation. We think that manipulation is complex; it's one of the parts of robotics that is the most unreal."

"Sorry, just to clarify, what does that mean exactly?"

"It means that, in robotics, there are essentially three major families of tasks. One is locomotion, which means how to move from, let's say, how to walk, how to move from point A to point B. The second is navigation, so you're moving in the complicated environment, such as, for instance, a flat area or a building, and you have to figure out actually to which rooms you have visited before and where to go."

"And the last one is manipulation. So this means, and you want to grasp an object, let's say, open an object, place objects in various locations. The third one is the one which is currently the most difficult. So it turns out that when it comes to arbitrary objects, current robots are unable to just grab an arbitrary object."

"For any object, it's possible to hand-code a single solution. So, say, as long as the same factory had the same object, again, I don't know, we are producing glasses, and there exist hand-coded solutions to it. There is a way to code a program saying, 'let's play their hand in the middle of the glass and then let's close it,' but there is no way so far to write a program such that it would be able to grasp an arbitrary object."

"Okay, gotcha. And then, just very quickly, the other OpenAI projects?"

"So, another one has to do with playing a complicated computer game, and the third one has to do—we linked a large number of computer games. You might ask why it's interesting. In some sense, we would like to see, ya know, human has an incredible skill of being able to learn extremely quickly. It has to do with prior experience."

"So let's say even if you haven't played ever volleyball, if you try it out for the first time, within 10 or 15 minutes, you would be able to grasp how to actually play. And it has to do with all the prior experience that you have from different games. If you would put a child, like, if you would put an infant on a volleyball court and ask him or her to play, and it would fail miserably, I mean, due to the fact that it has experience coming from a large number of other games."

"Or let's say other life situations, it's able to actually transfer all the knowledge. So at OpenAI, we were able to pull together a large number of computer games, and computer games can be—it’s quite easy to quantify how good you are in the computer game. The current best AI system—so first of all, it's possible for many computer games to write a program that solves it pretty well or plays it well."

"There are also results from that. There are results in terms of reinforcement learning or in terms of so-called deep reinforcement learning showing that it's possible to learn how to play a computer game. These are the initial results are coming from Midnight, but simultaneously, it takes extremely long time, like in terms of real-time execution to learn to play computer games."

"So for instance, Atari games, in terms of real-time execution, in text, it's having around three to three years of play. Learning to play simple games, I mean, it can be hugely paralyzed. Therefore, a few days to train it on current computers, but it's way shorter for humans because in 10 minutes, we can kinda teach you how to play and win it."

"Okay, is that through giving you feedback?"

"So the way how it works in case of computer games, the feedback comes from the computer, from the score. It looks at the score in the game and tries to optimize it, and that's kind of reasonable, but I would say simultaneously it's not that satisfying to me. So the reason why it's not that satisfying to me is that the assumption underlying reinforcement learning is that there is some environment and you are an agent."

"And you're acting in the environment by executing actions and getting rewards from the environment. The rewards might be taught as, let's say, pleasure or so. And the main issue is that it's actually not that easy to figure out what are their rewards in the real world. Further on, the other underlying assumption is in being able to reset the environment to kind of get— to re-visit the same situation, and so the system can try thousands or millions of times to actually finish a game."

"So there are some small discrepancies—people also believe that it might be possible somehow to hard-code into the system rewards, but I would say that's actually one of the big issues that are kind of unresolved. Like when I look how my nephew plays a computer game, he actually doesn't look in on the score because he cannot read and still—yeah, they can play pretty well."

"So, I mean, you can say maybe reward is somewhat different—maybe reward comes from I can sing a nice, hearing nice voice in the game or so. But I would say that's something what is very unclear how to build a system and what system should optimize. So, as I said, if we have a metric that we want to optimize, it's possible to build a system that could optimize for it, but it turns out that in many cases it's not that easy, and I would say that's actually one of the motivations why I wanted to work on robotics."

"Because in the case of robotics, it's way closer to the system that we care about. So what I mean by that— for instance, let's say you would like your robot to prepare scrambled eggs for you, and so the question is: how should I deal with their reward?"

"And in computer games, actually, the last thing is they are getting rewards extremely frequently. So let's say anytime you kill an enemy, or let's say you don't die, it's quite great. But in the case of scrambling eggs, it would mean, or the way how people write rewards for systems, it would mean distance from hand to egg, and then let's say somehow you have to quantify what then if the egg—it's if you were able to crack open an egg or let's say if that if you fried its efficiency and how to kind of quantify. It turns out to be extremely difficult."

"And also, there is no way even to reset the system—how to reset. Register them anyways, so—okay, these are like two-dimensional issues, and the reason why I'm personally interested in robotics is thinking that actually these challenges will tell us how to solve—so let's start by defining a couple things."

"So what is artificial intelligence? What is machine learning? And then what is deep learning?"

"Okay, they start pretty good, and these are pretty good questions. Okay, so artificial intelligence is actually extremely broad—the it's an extremely broad domain—and machine learning is a sub part of this domain. Artificial intelligence consists of any writing, any software that tries to solve some problems through some intelligence."

"It may be hand-coded solution, rules-based system. Yeah, so pretty much it's actually very hard to say what is not artificial intelligence. You can say that so initial version, for instance, of Google search was based on—it was avoiding any machine learning. And there was a well-defined algorithm called PageRank."

"Essentially, PageRank counts how many incoming links are from other websites, and that's artificial intelligence. It's a search item that does intelligent things for you. And then, over time, Google search started to use machine learning because it helps to improve results. At similarity, they wanted to avoid it for some time. It had more difficulty to interpret the results, and it's more difficult to actually understand what the system does."

"So what is machine learning? Machine learning is essentially a way of building or let's say that essentially you have data, and you would like to generate based on data a program with some behavior. The most common example, which is still a sub-branch of machine learning, is so-called supervised learning."

"So you have pairs of examples X, comma Y, which means like, ‘I would like to map X to Y.' For instance, either if a given email is spam or not spam, or let's say even an image, what is the category of an image, or for instance, to whom should I recommend given product. And based on this data, it would like to generate a program, some sort of the black box or some function that for new examples would be able to give you similar answers."

"That's an example of supervised learning, but this is machine learning means that you would like to generate a program from theta. Okay, and this usually uses statistical machine learning methods, so somehow you can't somebody but how many times even given events occurred or so—okay, gotcha."

"And then the third being deep learning. So deep learning—that's also—that's kind of—that's one paradigm in terms of machine learning. Okay, and idea behind it ridiculously simple. Okay, so people realized that if you want to—as I said, machine learning means that you get data as an input and program as the output, and deep learning says that the computation of the program then—though what I’m actually doing with this data should involve many steps, not one step but many."

"Okay, and pretty much, that's it in terms of the meaning of deep learning. So you might ask why it's so popular now and how it's so different from what was there before. Things turn out that if you assume that you do one step of computation, let’s say that you take your data and you kind of have a single if statement or small number of each statement, then I guess, for instance, so if you have a—you know, let's say your data is a recording from the stock market and they're saying you're gonna sell or buy depending on value speaker or smaller than something or if—if let’s say depending or who is the new President or so we are making some decisions."

"So that sense turns out that in case of models that are based on a single step, people are able to prove plenty of stuff mathematically. In terms of models that require multiple steps of computation, mathematical proofs are extremely weak, and for a long time, models that do a single step of computation were outperforming models that do many steps of computation."

"But recently it kind of changed, and it was for many people it was obvious for a long time that true intelligence cannot be done in a single step, but it would require many steps. But so far, many systems, actually, they worked in the way that they had kind of very, very shallow—they were very shallow but, simultaneously, extremely gigantic."

"So what I mean by that, you could generate, let's say, for the task of interest. And let's say their recommendation you could generate a large number of features, let’s say thousands of them. These are features saying for instance, from—let's say you want to do movie recommendation you can say, 'Is movie longer or shorter than two hours? Is it longer or shorter than one hour?' That's—there are two features; you can say, 'Is it drama? Is it trailer? Is it something?' and you can generate million of these."

"And then, or let's say 100,000—that's actually quite reasonable value, and then your shallow classifier can determine based on the combination of these features either to recommend it to you or not. In case of deep learning, you would say, 'let's kind of combine it for multiple steps,' and that entire difference in case of deep learning, the most successful embodiment of deep learning is in terms of neural networks."

"Okay, so let's define that. So neural networks, it's also an extremely simple concept and recognizing that people came up with a long time ago, and means, and then it means as follows: If you have an input, it might be, say, a vector or it might have some additional structure like a—let’s say image, so it's kind of a matrix, two-dimensional, and neural network, it's a sequence of layers."

"Layers are represented by matrices, and what you do is you multiply your input by a matrix, and apply some nonlinear operation, and multiply it again by a matrix, and apply a nonlinear operation. You might ask, why would I even need to apply this nonlinear operation? Turns out that if you would multiply by two matrices, it can be reduced to multiplication by a single matrix."

"Like a composition of two linear operators can be written as a single linear operator. You could multiply these matrices together, and the result of the—you could condense it into a single matrix, and non-linearity is something like that there, and classical don’t know—but early—so, say, there are extremely large number of variants in terms of what I said, but what I just described is so-called feed-forward neural network."

"So it essentially takes input multiplied by matrix, non-linearity, multiplied by matrix examples of non-generic. If there is something that—one which is classical, something called sigmoid. So sigmoid is a function that has a shape of S-character and letter. It's kind of close to zero for negative values; it grows to house at zero and then goes up to one when the values are larger; the kind of modulates the input, and that's the most classical version of activation function."

"It turns out that the one which is even simpler empirically works way better, which is called ReLU, rectified linear unit, and this one is ridiculously simple. ReLU, it's just maximum of 0, co-max. So when you have negative values at 0, positive value just copy the value, and that's it."

"So you might ask—so first of all, what are the successes of deep learning? Why do we actually believe that it works? Why—what changed, and why it's so much different than it was before?"

"And they're like a sub-q—"

"Yeah, this is a good question. That's exactly where I was going to go, but I was going to ask beforehand, yeah, why—why neural networks are, I think now I'm supposed to in the past. The main difference is all the sudden we can train them to solve various problems and, let's say, one family of problems, these are problems needs to provide learning."

"So better than any other method they can map these examples to labels, and then on the holdout, they account test data they outperform anything else, and in many cases, they get superhuman results."

"And is that just a function of, like, computational power that we have access to when it comes to models?"

"Neural networks is an example of a model. There is always a question, so how to figure out in terms of a model? So there is some training procedure, and the most common procedure for neural networks is so-called stochastic gradient descent. It's also ridiculously simple procedure, and turns out, that empirically it works very well."

"So people came out with a vast number of learning algorithms; stochastic gradient descent is an example of one learning algorithm. There are others, let’s say there is something called heavy on learning that motivates—like that's motivated by the way how neurons grow in human brain learn, but this one so far empirically is working the best."

"Okay, so then let's go to your question. You ask yourself, which is why now? Like, what's happening to make people care about it right now?"

"And so, and so since twenty years ago, and there were several small differences in terms of how people trained neural networks, and there is a large increase in computational power, so I can speak about the major advances."

"So number one advance I would say that's even the dopant, the one advance, that's actually an old one, but it seems to be an extremely critical—something called convolutional neural networks."

"Okay, now does that mean?"

"I'm there—so it's actually a very simple concept. So let's say your input is an image, and let's say your image is of a size 200 by 200 and has also, let’s say, three colors. So that would let the number of values in total is actually 120,000, so if you would actually squash it into a vector, this vector would be of this size. Okay, and then I can think that if you would like, let’s say, to apply neural network, essentially multiplied by a matrix, and let's say if you'd like to have the output of the multiplication of similar size, say 120,000, then all the sudden the matrix to multiply it would be of a gigantic size."

"And learning their learning consists of estimating parameters the neural network appearances that empirically that wouldn't essentially work; that if you would use the algorithm of back propagation, you would get quite full results. And people realized that in the case of images, you might want to multiply by a little bit special matrix that also allows to do way faster computation."

"So you can think that neural network apply some computation to the input. So neural network applies some computation to the input. You might want to constrain computation in some sense, so you might think as you will have several layers."

"Maybe initially, you would like to do very local computation, and it should be pretty much similar in every location. So you would like to apply the same computation in the sense there in the corners. Maybe later on, you need some diversification, but you want to preprocess the image the same way."

"So the idea is that when you take an image, or any—actually to mention of structures, so other examples is you can take voice, and it turns out that you can, by applying Fourier transform, you turn voice into an image, and all day, it's like to remember a waveform."

"Yeah, so you take a waveform, and you apply Fourier transform, okay? And essentially, on the x-axis, you have time as the speech goes on and on, and on the y-axis you have different frequencies, and that's an image."

"And speech recognition systems, they—they treat sound as it would be an image. I mean realize that that's really cool. Okay, so that's why I'm saying that the technique that I did also—like, kind of as a side track, yeah, the cool thing about neural networks is it used to be the case that people specialized in processing text, images, sound, and these days this is the same group of people—that's really cool—they do."

"We are using the same method, so coming back to what is convolutional neural network. Yeah, as I mentioned, you'd like to apply the same computation all over the place in the image, and essentially convolutional neural network says when we take an image, let's just—let's just connect a new neuron with local values on the image and let's copy the same weight over and over again."

"So this way, you will multiply, kind of, multiply values in the center and in the corners by the same, same values in the matrix, okay? As an input, and add. So input to convolution is an image, and out to this kind of also an image. You can think that there is also some specific vocabulary."

"So, in that, this is kind of three-dimensional images; like you have height and you have also depth. So let's say in case of image, three dimensions, and then you apply convolution, you can kind of change the number of depth dimensions."

"Usually people go to, let's say, like 100 dimensions and so, okay? Gallery, and then you kind of have several of these layers."

"And then there are so-called fully connected layers, which are just conventional matrices. So I would say that's one of advances that actually happened 20 years ago."

"Over another one, which is—it might sound kind of funny, but for a long time, people didn't believe that it's possible to train deep neural networks every Wednesday. They were thinking quite a lot about what are the proper learning algorithms, and it turns out that—and so let's say when you train a neural network, you start off by initializing weights to some random values."

"And it turns out that it’s very important to be careful to what magnitudes you initialize weight, and if you set it to right values, and I can even give you some, let’s say, intuition what it means."

"It turns out that the simplest algorithm, which is called stochastic gradient descent, actually works pretty well. Okay, so in some sense, as I said, the layers of neural network tape kind of multiply—they multiply input by matrices, and a property that you would like to retain is you don't want the magnitude of output to blow up."

"And also, you don't want it to shrink down; and if you kind of multiple—if you choose random initialization, it's easy to just some initialization that will kind of female turn that magnitude will keep on increasing."

"And then, if you have ten layers and let’s say in each of them you multiply by 2 to 2 to 2, and then the output all of a sudden is of completely different magnitude and learning is not happening anymore. And if you just choose them—and it's a method of choosing variants like a magnitude of initial way—and if you set it such that say output is of the same magnitude as input, and everything works."

"So basically just adjusting those magnitudes was what proved that you could do this with a neural network."

"Yes, wow, okay, that's kind of ridiculous that the—let's say people haven't realized it for a long time, but that's what it is. And when did, when and where did that happen?"

"And it happened actually at the University of Toronto. So then, at the Geoffrey Hinton lab. So then the crazy thing is people had several schemes in terms of how to train deep neural networks, and one was called generative pre-training."

"And so let’s say there was some scheme what to do in order to get to such a state of neural network that all of a sudden you can use him this trivial algorithm called stochastic gradient descent."

"So there was like an entire involved procedure, and at some point, Geoffrey asked his—didn't you know compare it to like—added a simpler solution which would be adjusting magnitudes and like showing how big difference there is. Crazy, man, I don't like that."

"Okay, so that a question that's a little bit broader is just like, then what has happened in the past, say, five years to excite people so much about AI? So I would say the most Fanning were the so-called ImageNet results. So first of all, I should tell you where was computer vision five years ago, then I will tell you what is ImageNet and then I will tell you about the results."

"So computer vision is a field where essentially you try to make sense of images. Like a computer tries to interpret what is on images, and it's extremely simple to say out here on an image there is a cow, a horse, or so. But for computer, images just the collection of numbers, so all the large matrix of numbers, and it's very difficult to say out like how to—it's very difficult to interpret what's their content."

"And it was the case that people came out with very skins how to do it, and you know, you could imagine a lady, let’s quantify how much of a brown color there is such that you can say the horse—again, I guess simple stuff—whether people, of course, came out with more clever solutions, but assistants were quite better. I mean, you could fit the picture of a sky test system, and it was telling you that there is a car—that's like a good."

"Yeah, so then, then faithfully, right Bailey is a professor at Stanford. She, together with her students, she collected a large dataset of images and dataset is called ImageNet. It consists of 1 million images and 1,000 classes. So that was by the time actually the largest dataset of images in a class. Just to clarify, being like car might be a class."

"Yes, so there is—that the dataset would say it's not artifact, it has, for instance, he doesn't contain people; that was one of constraints over there. It contains a large number of breeds of dogs, so that's like—well, that's a queer key thing about it. But at the same time, I mean, that’s the SSI dataset that made deep learning happen—"

"Types of dogs. And now, then as the act of, it's so large. So so kept America. There was like a plenty of teams actually participating in emotional competition, and say even as I'm saying, there is 1,000 classes over there. So if you have a guess—random guess—then probability that your guess is correct is essentially 0.1%."

"The metric there was slightly different, deal actually if you make five guesses, and if one of them is correct and New York, you are good. Okay? Because there might be some other objects and so on."

"And I remember when I, for the first time when I have seen that someone might, you know—and that system was that someone created a system that had 50 percent error; I was impressed. Okay else, like oh man, it's like I guess what are the classes, and it can say okay with fifty percent error what is there. I was quite impressed."

"But then, during the competition, like a pretty much like all the teams got around 25 percent error. All right? There were like a deep—there was a difference by one person. There were like a, for instance, a team from University of Amsterdam, a Japanese team, a plenty of people around the world, and a team from University of Toronto led by Geoffrey Hinton."

"And that's like a dad on team was Alex Reshevsky and Ilyas cover. They actually cut to something like fifteen percent, so let's say all other teams they were like a twenty five percent."

"The difference was one percent, yeah, and these two guys, they got to fifteen percent. Okay, yeah, and the crazy thing is that week in—so within following three years on these datasets, the error dropped dramatically. I remember I can't—next year the error cut to, let's say, 11 percent, eight percent."

"All kind of—I remember by that time I was wondering what the limit—how good can you be? And I was thinking five percent, that's like at—that's the best."

"And even there, like, humans trying to see how far they can get if they spend arbitrary amount of time on, let’s say, looking on other images and kind of comparing to be able to figure out what is there. I mean it's—it's not that simple for human, for instance, if you have plenty of breeds of dogs, and like—who knows—but let's say if you can use some external images to kind of compare and so on, that that helps."

"But in a sense, within several years people cut down, I believe to three percent error, and that's the such a superhuman performance, and as I’m saying, it used to be the case that systems in computer vision, you take a picture of a sky, they were down here is a car, and all of a sudden they are getting to superhuman performance."

"And it turns out that these results, actually, are not just limited to computer vision that people are able to get amazing other systems. Let's say speech recognition—because that's like the underlying question, right? Because, like, it's not—I mean to someone not in the field like me, it's not necessarily intuitive that computer vision, computer image recognition would, you know, seed artificial intelligence, so I mean like what came after that?"

"So in that sense, the crazy thing is that the same architectures work for various tasks, and all of a sudden that the field which seems to be unrelated I started to benefit from each other."

"So I mention it turns out that problems in speech recognition can piece of in a very similar way you can sense. I take speech, apply Fourier transform, and then speech starts to look like an image and he'll apply similar object recognition network to kind of recognize what are the cells over there and like phonemes and so. Fun inside—like a kind of sensor better, and then you can turn it into text."

"And so that's why we're so—it went to speech after images, and then the next big thing about SSI translation. Translation was extremely surprising to people. That result by Ilyas cover, so translation is an example of another field that actually leaked their Python."

"And one of the crazy things about translation is input is of a variable length and output is a variable length, and it was unclear even how to kind of consume and with a neural network how to produce variable length input, variable length output."

"And Ilyas came out with an idea—there is something called recurrent neural networks. So, I mean, let’s say recurrent neural networks and convolution on the world apart shared an idea which is you might want to use the same parameters if you are doing similar stuff."

"And in case of convolutional network means let’s share the same parameters in space; let’s say let's apply the same transformation to the middle of images as in the corners and so on, and in case of recurrent neural networks, this is as well being reading text from left to right."

"I can consume first word, can create some hidden state representation, and then, then the next time step when I'm consuming the next word, I can take it together with this hidden representation to generate the next hidden representation, and we are applying the same function over again, and this function consumes a hidden representation and the next word hidden representation and the word hidden representation of the word."

"So it's relatively simple. The cool thing is if you are doing it this way, regardless of the length of your input, you have the same size of a network, and the way how this model worked and described in a paper called sequence-to-sequence essentially calcium work by word centers that you want to translate."

"And then, when you're about to generate translation, you are essentially omitting word by word and at the handy art when you have meat dot that end. So cool, and it was quite surprising to people by the time they got to decent performance, they were not able to beat a phrase-based system, and now it's—it's like outperform very like a long time ago already."

"And yeah, that one other issue that people have—so if Newell at work systems, like in case of frustration, the problem with deploying it on a large scale is that it's quite computationally expensive, and it requires a giant, and in deep learning literature there are various ideas how to make things way, way cheaper computationally after you train it."

"So it's possible to throw away a large number of ways or, as I shall turn floats 32-bit float into smaller sized numerics and so on and so forth, and pretty much that the reason why things are not largely deployed in production systems out there, but neural network-based solutions are actually outperforming anything out there."

"There are a couple more things I would like to just define for a general listener. So there are a couple words being thrown around a lot—so narrow AI, general AI, and then super intelligence. Can you just break those parts, sir?"

"So pretty much all AI that we have out there is narrow AI. No one built so far general AI. No one built a super intelligence. So narrow AI means artificial intelligence, so it's like a piece of software that solves a single predefined problem. General AI means a piece of software that can solve huge, vast number of problems, all the problems."

"So you can say that human is general—it’s general intelligent because you can give it an arbitrary problem, and a human can solve it. Okay, but for instance, a bottle opener cancelled on the bottle opening. So pretty much when we look at any tools out there, at any software, and our software, it's good in solving a single problem."

"For instance, our chess-playing programs cannot drive a car, and for any problem, we have to create a separate piece of software, and general artificial intelligence is an exam—it's a software that could solve arbitrary problems. So how we know that it's even doable? Because we—and there is an example of a creature that has such a problem, and then super intelligence is, I assumed, the next area."

"Especially super intelligence means that it's more intelligent than human."

"Cool. So given all that, given that like we're basically at a state of narrow AI across the board at this point, where do you think is like what's the current status of this stuff and where do you see it going in the next five or so years?"

"So, as I mentioned, there are essentially machine learning; there are third parties also paradigms. So then one of them is supervised learning, there is something called unsupervised learning, there is also something called reinforcement learning, and so far the supervised learning paradigm is the only one that works solely murkly remarkably well that it's ready to be applied in business applications, all other are not free either."

"And so you ask me where we are, so we can solve this problem. Other problems they require further work; it's very difficult to plan with the idea how long it will take to make them work. The thing which is very different with contemporary artificial intelligence is that we are using precisely the same techniques across the board."

"Simultaneously, the majority of business problems can be framed as supervised learning, and therefore they can be solved with current techniques as long as we have sufficient number of input examples and what you want to predict. And as I mentioned, the first can be extremely rich. I can output my dear sentence, and current systems work pretty well with it."

"And nonetheless, it requires an expert to train it, and so then given the given like pretty substantial hype we see, when you think of it all this, the field is simultaneously under-hyped and over-hyped. So from the perspective of business applications, as long as you have pairs of examples that indicate mapping, okay, what the input was, the output so we can pretty get pretty up and get to superhuman performance."

"But in all other fields, we are still not there, and it's unclear how long it will take. So give some example, let's say for recommendation systems, you have often companies like Amazon—they have examples of millions of users, and they know what they bought when they were happy or not, and that's an example of a task that is pretty good for a neural network to learn what to recommend to new users."

"Simultaneously, Google knows what is a good search query for you because on the search result page we are clicking on the links that you are interesting, and therefore they should be displayed first. And in other fields, it's actually quite often more difficult."

"Okay, in case of, let's say, apple-picking robot, it's difficult to provide supervised data telling how to move an arm toward Apple. Therefore, that's way more complicated. Same time, the problem of detecting where Apple is, it's where better defined and can be outsourced to humans to add note 8 plenty of images and to give localization of the Apple, and quite often the rest of that problem can be prescriptive by bike engineers."

"But the problem of how to place fingers on an apple or how to grip it, it's it's not well scientifically solved. And so a couple questions that at this point, if people were to be interested in learning more about AI, and maybe, you know, working with OpenAI or doing something, how would you recommend they get involved in educating themselves?"

"So, a good place to start is Coursera course; there is pretty good. There are also a lot of TensorFlow tutorials. TensorFlow is an example of framework to train neural networks. Okay? I also answer a Carpatty’s class at Stanford; it's extremely accessible, you can find it on, I believe, on YouTube."

"Yeah, and then like in terms of them actual exercises, so in case of TensorFlow tutorials and many of the problems, so I believe in case of Andres class—there there there there might be homework, and in case of TensorFlow exercises it's quite often easy to come up with some random task after, let's say, reading."

"Okay, I mean, you can take for instance, let's say like a simple task over there is let's classify—let's classify digits and let's classify pictures of digits that assign them classes. You can try my being download some images from some other source like a Flickr, let’s try to classify it, the word taxes."

"Okay, so given that you guys are working on with robots at this point, one of the other things that's thrown in like kind of part and parcel with AI is automation—specifically of like a lot of these low-level blue-collar jobs. What do you think about the future maybe the next ten years of those jobs?"

"So I believe that we will have to offer to people a basic income. I super strongly believe that actually that's the only way. So I don't think that it will be possible for a 40 years old taxi driver to reinvent himself every 10 years. I think it might be extremely hard."

"Other crazy thing is people define themselves through job, and that might be another big social problem. Simultaneously, they might not even like their jobs. Like if you ask someone, would you like your kid to sell in a supermarket, to be a seller in the supermarket, they would answer no."

"And maybe it's possible to live in a world that there is abundance of resources, and people can just enjoy their life. I think we're gonna have to figure out a way. I mean, maybe people will always find purpose, but I think like making it easier to find that purpose will become much more important in the future if automation actually happens to the degree people talk about."

"And what about—what about just like influences on you that like maybe have inspired you to work with robotics and AI? Are there any like books or films or any media that you really enjoyed?"

"I’d say it's pretty good—the book called Homo Deus actually describes the history of humans and then speaks and has various predictions about the future where we are heading. That's one pretty good. Hmm, I mean, there is no advice—there's like a plenty of movies about AI and how it can go wrong."

"What's the best one?"

"I think Her is pretty good."

"Okay, yeah, Ex Machina is also pretty good."

"Cool, all right. Do you have any other last thing you want to address?"

"Looks nuts, no, thank you."

"Okay, cool, thanks, man."