Transcript for Chris Urmson: Self-Driving Cars at Aurora, Google, CMU, and DARPA

I think the high order bit was so that it could be done. I think that was the thing that was incredible about the first the grand challenges that I remember you know I was a grad student at Carnegie Mellon and there we there's kind of this dichotomy of It seemed really hard, so that would be cool and interesting. But at the time, we were the only robotics institute around. And so if we went into it and fell on our faces, that would be embarrassing. So I think just having the will to go do it to try to do this thing that at the time was marked as, you know, darn near impossible. And then after a couple of tries, be able to actually make it happen. I think that was

Yeah, that was the technical director there and did a lot of the work along with a bunch of other really good people. Did I believe it could be done? Of course, right? Like why would you go do something you thought was impossible? Completely impossible. We thought it was going to be hard. We didn't know how we're going to be able to do it. We didn't know if we'd be able to do it the first time. Turns out we couldn't that, yeah, I guess you have to. I think there's a certain benefit to naivety, right, that if you don't know how hard something really is, you try different things and, you know, gives you an opportunity that others who are, you know, wise or maybe don't, don't have.

I think that's the joy of this field is that it's all hard. And that you have to be good at each part of it. So for the first, for the urban challenges, if I look back at it from today, it should be easy today. That it was a static world. There weren't other actors moving through that is what that means. It was out in the desert so you get really good GPS. you know, so that that went and you know, we could map it roughly. And so in retrospect now, it's, you know, it's within the realm of things we could do back then. Just actually getting the vehicle and the, you know, there's a bunch of engine work to get the vehicle so that we could control and drive it. That's, you know, that's still a pain today, but it was even more so back then. And then the uncertainty of exactly what they wanted us to do.

That's right, we didn't know the route. We didn't even really, the way the rules have been described, you had to kind of guess. So if you think back to that challenge, the idea was to, the government would give us, the DARPA would give us a set of waypoints and kind of the width that you had to stay within between the line that went between, you know, each of those waypoints. And so the most devious thing they could have done is set, you know, a kilometer wide corridor across, you know, a field of scrub brush and rocks and said, you know, go figure it out. Fortunately, it really it turned into basically driving along a set of trails, which, you know, is much more relevant to the application they were looking for. But no, it was, it was a hell of a thing back in the day.

I think there's a couple of things. go and try those really hard things that's where there is an incredible opportunity. I think the other big one, though, is to see people for who they can be, not who they are. It's one of the things that I actually, one of the deepest lessons I learned from Rad was that he would look at, you know, undergraduates or graduate students, and empower them to be leaders, to have responsibility, to do great things that I think another person might look at them and think, oh, well, that's just an undergrad or student, what could they know? And so I think that that kind of trust but verify have confidence in what people can become. I think is a really powerful thing.

I think there's been some big big steps. So for the grand challenge, the real technology that unlocked that was HD mapping prior to that a lot of the off-road robotics work had been done without any real prior model of what the vehicle was going to encounter. And so that innovation, that the fact that we could get, you know, decimeter resolution models was really a big deal. And that allowed us to kind of bound the complexity of the driving problem the vehicle had and allowed it to operate at speed because we could assume things about the environment that it was going to encounter. So that was one of the, that was the big step there. For the urban challenge, You know, one of the big technological innovations there was the multi beam light are and be able to generate high resolution, you know, mid to long range, 3D models the world and use that for, you know, for understanding the world around the vehicle and that was really a, you know, kind of a game changing technology. In parallel with that, we saw a bunch of other technologies that had been kind of converging, have their day in the sun. So Bayesian estimation had been, you know, slam it, been a big field. in robotics, you know, you would go to a conference, you know, the couple years before that in, every paper would effectively have slams somewhere in it. And so seeing that, you know, that, that looks based on estimation techniques, you know, play out on a very visible stage, you know, I thought that was, that was pretty exciting to see.

Well, yeah, in fact, we weren't really doing slam per se, you know, in real time, because we had a model ahead of time. We had a roadmap, but we were doing localization. And we were using, you know, the lidar or the camera is depending on, you know, who exactly was doing it to to localize to a model of the world. And I thought that was That was a big step from get of naively trusting GPS I and S before that and again like lots of work have been going on in this field. It's certainly this was not Doing anything particularly innovative in slam or in localization, but it was Seeing that technology necessary in a real application on a big stage. I thought was voice very cool

So the DARPA gave all the teams a model of the world, a map. And then, you know, one of the things that we had to figure out back then was, and it's still one of these things that trips people up today is actually the coordinate system. So you get a latitude long-toed, and, you know, to some of the decimal places, you don't really care about kind of the ellipsoid of the earth that's being used. But when you want to get to 10 centimeter or centimeter resolution, you care whether the core system is, you know, NADS 83 or WGS-84 or You know, these are different ways to describe both the kind of non-sphericalness of the Earth, but also kind of the act in, I think, I can't remember which one of the teconic shifts that are happening and how to transform, you know, the global data as a function of that. So, you know, getting a map and then actually matching it to reality to send me to resolution, that was kind of interesting and fun back then.

It's certainly behind where we are today. We're more than a decade since the urban challenge. But the core of it was there that we were tracking vehicles. We had to do that at 100 plus meter range because we had to merge with other traffic. We were using, again, Bayesian estimates for state of these vehicles. We had to deal with a bunch of the problems that you think of today of predicting what that vehicle's going to be a few seconds into the future. We had to deal with the fact that there were multiple hypotheses for that because a vehicle had an intersection might be going right or it might be going straight or it might be making a left turn. And we had to deal with the challenge of the fact that our behavior was going to impact the behavior of that other operator. You know, we did a lot of that in relatively naive ways, but it still had to have some kind of selection.

Yeah, I think the biggest thing is that the actors are truly unpredictable. that most of the time, you know, the drivers on the road, the other road users are out there. Behaving well, but everyone's well, they're not. The variety of other vehicles is, you know, you have all of them.

Both. Right. That we have, you know, back then we didn't have to deal with cyclists. We didn't have to deal with pedestrians. Didn't have to deal with traffic lights. You know, the scale over which that you have to operate is now as much larger than, you know, the air base that we're thinking about back then.

I think the most fundamental difference is that we're doing it for real, that in that environment, it was both a limited complexity environment because certain actors weren't there because the roads were maintained, there were barriers keeping people separate from robots at the time. and it only had to work for 60 miles, which looking at it from 2006, it had to work for 60 miles, right? Looking at it from now, we want things that will go and drive for half a million miles, and it's just a different game.

So I think it's I think it's essential. You know, I believe, but I also believe that the cameras are essential, and I believe the radar is essential. I think that you really need to use the composition of data from these different sensors if you want the thing to really be robust.

So I think it is undeniable that people walk around without lasers in their foreheads. And they can get into vehicles and drive them. And so there's an existence proof that you can drive using passive vision. No doubt.

So it's in terms of sensors. Right. So like there's there's an example that we all go do it at many of us every day. In terms of a lighter being a crutch. Sure. But you know, in the same way that, you know, the combustion engine was a crutch on the path to an electric vehicle on the same way that, you know, any technology ultimately gets replaced by some superior technology in the future and really what the way that I look at this is that the way we get around on the ground, the way that we use transportation is broken. And that we have, you know, this, you know, what was, I think the number I saw this morning, 37,000 Americans killed last year on our roads. And that's just not acceptable. And so, any technology that we can bring to bear, that accelerates this technology, you know, self-driving technology coming to market and saving lives, is technology we should be using. And it feels just arbitrary to say, well, you know, I'm not okay with using lasers because that's whatever, but I am okay with using an eight megapixel camera or a 16 megapixel camera, you know, like it's just these are just bits of technology and we should be taking the best technology from the tool bin that allows us to go and, you know, install a problem.

I see both of those but it's really a matter of time and I think really maybe the I would talk to the question you asked about you know the cheapest sense or I don't think that's actually what you want what you want is a sensor suite that is economically viable and then after that everything is about margin and driving cost out of the system What you also want is a sense suite that works. And so it's great to tell a story about how it'd be better to have a self-driving system with a $50 sensor instead of a $500 sensor. But if the $500 sensor makes it work and the $50 sensor doesn't work, You know, who cares? As long as you can actually, you know, have an economic offer. You know, there's an economic opportunity there. And the economic opportunity is important because that's how you actually have a sustainable business. And that's how you can actually see this come to scale and and be out in the world. And so when I look at light are I see a technology that has no underlying fundamentally, you know, expense to it, fundamental expense to it. It's, it's going to be more expensive than an image or because, you know, CMOS processes or, you know, fat processes are dramatically more scalable than mechanical processes. But we still should be able to drive cost out substantially on that side. And then I also do think that with the right business model, you can absorb more, you know, certainly more cost on the bill materials.

So, and I want to be really careful because sometimes this gets twisted in a way that I certainly didn't intend. So, active safety systems are a really important technology that we should be pursuing and integrating into vehicles. And there's an opportunity in the near term to reduce accidents, reduce fatalities, and we should be pushing on that. Level two systems are systems where the vehicle is controlling two axes. So breaking and thrall slash steering. And I think there are variants of level two systems that are supporting the driver that absolutely, like we should, we should encourage to be out there. Where I think there's a real challenge is in the human factors part around this and the misconception from the public around the capability set that that enables and the trust that they should have it at. And that is where I, you know, I kind of, I am actually incrementally more, you know, concerned around level three systems and, you know, how exactly a level two system is marketed and delivered and, you know, how people, how much effort people have put into those human factors. I still believe several things are out. This one is people will over trust the technology. We've seen over the last few weeks, you know, a spade of people sleeping in their Tesla. You know, I watched an episode last night of Trevor Noah. talking about this and you know him you know this is a smart guy who has a lot of resources at his disposal describing a Tesla as a self-driving car and that why shouldn't people be sleeping in their Tesla? It's like well because it's not a self-driving car and it is not intended to be and you know these people will almost certainly you know die at some point or hurt other people. And so we need to really be thoughtful about how that technology has described and brought to market. I also think that because of the economic, economic challenges we were just talking about, that technology path will, these level two driver systems systems, that technology path will diverge from the technology path that we need to be on to actually deliver. truly self-driving vehicles, ones where you can get it and sleep and have the equivalent or better safety than, you know, a human driver behind the wheel. Because the, again, the economics are very different in those two worlds, and so that leads to, you know, divergent technology.

And really, that comes back to what are those L2 and L1 systems doing? And they are driver assistance functions where the people that are marketing that responsibly are being very clear and putting human factors in place such that the driver is actually responsible for the vehicle. and that the technology is there to support the driver. And the safety cases that are built around those are dependent on that driver attention and attentiveness. And at that point, you can kind of give up to some degree, for economic reasons, you can give up on, say, false negatives. And the way to think about this is, for a foreclosure mitigation braking system, If it half the times the driver missed a vehicle in front of it, it hit the brakes and brought the vehicle to a stop, that would be an incredible, incredible advance in safety on our roads, right? That would be equivalent to seat belts. But it would mean that if that vehicle wasn't being monitored, it would hit one out of two cars. And so, economically, that's a perfectly good solution for a driver's system system. What you should do at that point, if you can get it to work 50% of the time, is drive the cost out of that so you can get it on as many vehicles as possible. But driving the cost out of it doesn't drive up performance on the false negative case. And so you'll continue to not have a technology that can really be available for a self-driven vehicle.

I don't think so. If people truly understood the risks and internalized it, then sure, you could do that safely. But that's a world that doesn't exist. The people are going to, if the facts are put in front of them, they're going to then combine that with their experience. And let's say they're using an L2 system and they go up and down the 101 every day and they do that for a month. And it just worked every day for a month. That's pretty compelling. At that point, just even if you know this statistics, Well, I don't know, maybe there's something a little funny about those. Maybe they're, you know, driving in difficult places. I've seen it with my own eyes. It works. And the problem is that that sample size that they have. So it's 30 miles up and down. So 60 miles times 30 days, so 60, 180, 1,800 miles. That's a drop in the bucket compared to the one, you know, what 85 million miles between fatalities. And so they don't really have a true estimate based on their personal experience of the real risks. But they're going to trust it anyway, because it's hard not to work for a month.

And I think that as this technology moves from what I would say is kind of the more technology savvy ownership group to the mass market. You may be able to have some of those folks who are really familiar with technology, they may be able to internalize it better. And you're kind of immunization against this kind of false risk assessment might last longer, but as folks who aren't as savvy about that, read the material and they compare that to the personal experience, I think there that it's going to It's going to move more quickly.

This is one where it's difficult because there isn't a sound bite answer that we have to show a combination of work that was done diligently and the awful and this is where something like a functional safety process is part of that is like here's here's the way we did the work That means that we were very thorough. So if you believe that what we said about this is the way we did it, then you can have some confidence that we were thorough in the engineering work we put into the system. And then on top of that, you know, to kind of demonstrate that we weren't just thorough, we were actually good at what we did. There will be a kind of a collection of evidence in terms of demonstrating that the capabilities worked the way we thought they did. Statistically and whatever degree we can demonstrate that both in some combination of simulations, some combination of unit testing and decomposition testing, and then some part of it will be on road data. And I think the way we'll ultimately convey this to the public is there'll be clearly some conversation with the public about it. But we'll invoke the kind of the trusted nodes and that we'll spend more time being able to go into more depth with folks like Nitsa and other federal and state regulatory bodies and kind of given that they are operating in the public interest and they're trusted. that if we can, you know, show enough work to them that they're convinced, then, you know, I think we're in a pretty good place.

Don't have it. No, in fact, I was on the record telling DNV that I thought this was not a great metric.

So I do, and I think that it won't be just one number. So as we are internally grappling with this, and at some point we'll be able to talk more publicly about it, is how do we think about human performance in different tasks, say, detecting traffic lights, or safely making a left-turn across traffic, And what do we think the failure rates are for those different capabilities for people? And then demonstrating to ourselves and then ultimately folks in regulatory role and then ultimately the public that we have confidence that our system will work better than that. And so these individual metrics will kind of tell a compelling story ultimately. I do think at the end of the day what we care about in terms of safety is life-saved and injuries reduced and then ultimately, you know, kind of casually dollars that people aren't having to pay to get their car fixed. And I do think that you can, you know, in aviation, they look at a kind of an event pyramid where, you know, a crash is at the top of that and that's the worst event, obviously. And then there's injuries and, you know, near miss events and whatnot. And, you know, violation of operating procedures and you kind of build a statistical model of the relevance of the low severity things, the high-spirit things. And I think that's something where we'll be able to look at as well. because an event per 85 million miles statistically, a difficult thing, even at the scale of the U.S. to compare it directly.

I think you let them experience it. I think it's, I think it's right. I think people should be skeptical. I think people should ask questions. I think they should doubt. Because this is something new and different. They haven't touched it yet. And I think it's perfectly reasonable. And but at the same time, it's clear there's an opportunity to make the road safer. It's clear that we can improve access to mobility. It's clear that we can reduce the cost of mobility. And that once people try that and understand that it's safe and are able to use in their daily lives, I think it's one of these things that will just be obvious. And I've seen this practically in demonstrations that I've given where I've had people come in and they're very skeptical. And again, my favorite one is taking somebody out on the freeway And we're on the one-a-one driving at 65 miles an hour and after 10 minutes, they kind of turn and ask, is that all it does? And you're like, yeah, it's self-driving car.

I thought it would do, right? But they, you know, it becomes mundane. which is exactly what you want to technology like this to be. We don't really, when I turn the lights switch on in here, I don't think about the complexity of those electrons being pushed down a wire from wherever it was and being generated. It's just, I just get annoyed if it doesn't work. What I value is the fact that I can do other things in this space. I can see my colleagues, I can read stuff on a paper, I can, not be afraid of the dark. And I think that's what we want this technology to be like. It's in the background and people get to have those most life experiences and do so safely.

We'll see that within 10 years. I'm pretty confident.

So, so I think the moment it gets serious is when we really do have a driverless vehicle operating on public roads. And that we can do that kind of continuously without a safety drive without a safety driver in the vehicle. I think at that moment we've we've kind of crossed the zero to one threshold. And then it is about how do we continue to scale that how do we build the right business models, how do we build the right customer experience around it so that it is actually a useful product out in the world? And I think that is really, at that point it moves from what is this kind of mixed science engineering project into engineering and commercialization and really starting to deliver on the value that we all see here and actually making that real in the world.

I think it's going to be urban suburban type environments. Yeah, with Aurora, when we thought about how to tackle this, it was kind of unfold to think about trucking as opposed to urban driving. And, you know, the, again, the human intuition around this is that freeways are easier to drive on. because everybody's kind of going in the same direction, and you know, lanes a little wider, et cetera. And I think that that intuition is pretty good, except we don't really care about most of the time. We care about all of the time. And when you're driving on a freeway with a truck, say 70 miles an hour, and you've got 70,000 pound load with you, that's just an incredible amount of kinetic energy. And so when that goes wrong, it goes really wrong. And those challenges that you see occur more rarely, so you don't get to learn as quickly. And they're, you know, incrementally more difficult than urban driving, but they're not easier than urban driving. And so I think this happens in moderate speed urban environments because there, you know, if if two vehicles crash at 25 miles per hour, it's not good, but probably everybody walks away. And those those events where this is the possibility for that occurring happen frequently. So we get to learn more rapidly. We get to do that with lower risk for everyone. And then we can deliver value to people that need to get from one place to another. And then once we've got that solved, then the kind of the freeway driving part of this just falls out. But we're able to learn it's more safely, more quickly, in the urban environment.

So what's the, I think another way to ask that was would be if I could wave a magic wand, what part of the system would I make work today to accelerate it as quick as possible?

No, it's definitely not infrastructure. It's really that caught that perception forecasting capability. So if, if tomorrow, you could give me a perfect model of what's happened, what is happening and what will happen for the next five seconds, around a vehicle on the roadway, that would accelerate things pretty dramatically.

So I worry most about the vulnerable road users about the combination of cyclists and cars, right? Just cyclists and pedestrians because, you know, they're not in armor. You know, with the cars, they're bigger. They've got protection for the people. And so the ultimate risk is lower there. Whereas a pedestrian cyclist, they're out in the road. You know, they don't have any protection. And so, you know, we need to pay extra attention to that.

Yeah, I think there's two different concepts there. So one is I might worry that because these vehicles are self-driving, people are kind of stepping the road and take advantage of them. I've heard this and I don't really believe it because if I'm driving down the road and somebody steps in front of me, I'm going to stop. Even if I'm annoyed, I'm not going to just drive through a person still in the road. And so I think today people can take advantage of this and you do see some people do it. I guess there's an incremental risk because maybe they have lower confidence that I'm going to see them than they might have for an automated vehicle and so maybe that shifts it a little bit. But I think people don't want to get hit by cars. And so I think that I'm not that worried about people walking out of the want to want and you have creating chaos more than they would today. regarding kind of the nudging through a big stream of pedestrians leaving a concert or something. I think that is further down the technology pipeline. I think that you're right, that's tricky. I don't think it's necessarily I think the algorithm people use for this is pretty simple. It's kind of just move forward slowly and if somebody's really close and stop. And I think that that probably can be replicated pretty easily. And particularly given that it's, you don't do this at 30 miles an hour, you do it at one, that even in those situations, the risk is relatively minimal. But it's not something we're thinking about in any serious way.

So we really focus a lot on what we're doing here. I think that, you know, I've said this a few times that this is a huge, difficult problem and it's great that a bunch of companies are tackling it because I think it's so important for society that somebody gets there. So we, you know, we don't spend a whole lot of time thinking tactically about who's out there and how do we beat that person individually? What are we trying to do to go faster ultimately? Well, part of it is the least routine we have has got pretty tremendous experience. And so we kind of understand the landscape and understand where the cold effects are to some degree and you know, we try and avoid those. I think there's a part of it just this great team we've built. People, this is a technology and a company that people believe in the mission of and so it allows us to attract just awesome people to go work. We've got a culture I think that people appreciate that allows them to focus, allows them to really spend time solving problems. And I think that keeps them energized. And then we've invested heavily in the infrastructure and architectures that we think will ultimately accelerate us. So because of the folks we all to bring in early on because the great investors we have, you know, we don't spend all of our time doing demos and kind of leaping from one demo to the next. We've been given the freedom to invest in infrastructure to do machine learning, infrastructure to pull data from our on-road testing, infrastructure to use that to accelerate engineering. And I think that that early investment and continuing investment in those kind of tools will ultimately allow us to accelerate and and do something pretty incredible.

Oh, thank you very much. Really enjoyed it.