Autonomous Vehicles, Intelligent Transportation Systems... by Ken Whitesell

Good afternoon. Thanks for the introduction. My name's Ken Weitzel. I'm here to talk about autonomous vehicles, intelligent transportation systems, and yes, Django. So over the next four hours I will be discussing all of these in great, huh? What? Oh, 25 minutes? Not 250. Okay, uh, we'll make that fit. Currently I'm employed by WSP USA in their traffic engineering department and we work with municipalities who are working with the research that's actively going on in these various areas. I'll start out with a simple disclaimer that I know very little about the autonomous vehicle research itself. Everything that I know about that is the publicly available information that you can find on the internet.

So I'm not going to be spending a whole lot of time on that, maybe just bringing some topics to your attention. My own particular area of expertise is in the intelligent transportation systems and then Django. In the autonomous vehicle area, they actually recognize five different levels of autonomy for automobiles. The sixth of course being at the top, no automation at all, your typical garden variety car. And what you will find is that in different contexts and different situations that there's actually a mix of different levels being worked on. There are a lot of vehicles today that are working

along at level two and in some case level three, some partial automation, some conditional automation, where you see the ads and perhaps even have a car that have the blind spot warnings. Or there's certainly the commercials that you see on television with the assisted braking and all of the other safety facilities that are being built into cars. That's going to continue to evolve and adapt. And yes, the these features keep getting added. More and more vehicles are getting these different features built in as the manufacturers continue to work on making this technology available and usable and practical at a price that people can actually afford.

The holy grail of course is to get down to full automation where you get into a vehicle and without a driver, without any manual intervention at all, it takes you from your source to your destination. We're not there yet. We're not close to there yet. And I'll talk about that a little bit more. Part of what makes this such a difficult facility to provide is that there's really a lot of electronics, there's a lot of technology. to make a vehicle sufficiently aware of its environment to where it can interact with its environment and successfully navigate what passes for

roads in large parts of the country. Audi automotive have done a diagram for showing just some of the electronics that are involved, what they use. in terms of creating a web of sensors around a vehicle to allow it to detect the environment that it's in. Starting from a long range radar that gets its its distance view, allows it to forecast traffic and and situations that are coming up in front, identifying where the road curves where there might be obstacles in front of you. LIDAR , for those of you who are not aware, radar of course stands for radio.

detection and ranging, LIDAR is actually a laser-based equivalent. It is high-energy pulses of a much shorter range, of a much finer granularity and gets a more sharp picture, but at a corresponding loss and effective range and certainly more power required to interpret that. You also have visual cameras that are capable of detecting light, electromagnetic radiation in the visible wavelengths, and can possibly recognize some vehicles or other Attributes of the road, uh, aspects of the road that it needs to be aware of. Then really close, there are short-range radars

and even ultrasound for things like parking assist. I'm sure most of you may have even seen the television commercials where the person pulls up next to a parallel parking spot and hits a button and the car Backs its way into the appropriate size spot without hitting any of the adjacent vehicles. And so just like bats use ultrasound for very short range detection of very small objects. Cars have that, some cars that do that sort of close range assistance will use that facility as well. But it's not perfect. So far there have been, to my knowledge,

the extent of my knowledge, which by no means is exhaustive, but there are have been at least four traffic fatalities caused by the autonomy within a vehicle. There have been other vehicular deaths, of course involving autonomous vehicles, but the ones that I have found aside from these four, the cause has not been the autonomy in the vehicle itself. These are the four that I'm aware of where they have said, you know, there is a has been a breakdown in the autonomy of the car itself, its ability to detect and interact with its environment. Probably most everyone, and I know the one that's probably received the most attention was the pedestrian that was involved in the accident in Arizona earlier this year.

uh pedestrian walking a bicycle across the street at night. The car just did not detect it and the lady unfortunately was struck. Similar type situation in California in the same month, overshadowed because it was not a pedestrian, it was the driver in this case who was killed, the car itself steered into a concrete jersey wall. and hit the end of the Jersey Wall straight on, did not detect that the Jersey Wall was channeling traffic to one side or the other and hit it straight on. The one that I find most interesting, though, personally, in May of 2016, a car drove under a tractor trailer.

And this one highlights to me some of the real difficulties of working within the environment. Because the situation is that a car was driving up a hill. and at the top of the hill at a cross street was a tractor trailer in the middle of the intersection. And it just so happened that based upon the angle of the sun and the position of the truck The white panels of the truck blended in with the sky background. The wheels blended in with the terrain behind it. And so the vehicle was completely unable to recognize that there was actually an obstacle in its way. It thought that the trailer was just part of the background sky. And so the vehicle came up the hill, tried to cross underneath the truck, and

caused the accident. So what we're looking at at WSP and numerous other places around the country is an intelligent transportation system that helps account for some of these deficiencies, some of these lacks some of these gaps between what the technology can do has been shown to do and where it needs to be. But in actuality an intelligent transportation system is actually the older technology The initiatives for intelligent transportation goes back to the mid-80s. Some of you may be aware that most emergency responders, first responders, police cars, fires, fire trucks, ambulances in most cases

have a signal override button that they can hit uh transceiver hit some some device in their vehicle that causes the upcoming traffic uh signal to change to give them the green and to block traffic in the other directions. So this is work that started back almost 30 years ago, actually now probably more than 30 years ago. The idea with all of this is to look at transportation as a whole, as a complete system, not just looking at one part or another. but looking at the whole of transportation, how do we optimize moving people from one location to another, accounting for all modes of transportation?

That's the intent, the idea behind the intelligent transportation systems. So the end state that the industry, the automotive industry and the engineering industry that is working towards is a connected automation, is getting the two modes, these two particular modes, cars. and the infrastructure to work together to everybody's benefit. So that the vehicles are not just relying upon their own information that they can detect from their own sensors, but can talk to other vehicles that are on the road and talk to the traffic lights and talk to the traffic uh the intersections, talk to the infrastructure itself, talk to the roads. When you look at it from that perspective, a whole large range of applications actually become available.

Now time prohibits me from going through all of these. These are the And this isn't even an exhaustive list, but these are the applications that can take advantage of that are envisioned to take advantage of that cooperation between the vehicles and the infrastructure. And a couple of them that I'll point out. Some of them, one of them I've already mentioned, the emergency signaling. Being one forward collision warning, you see television ads for that. Eco lane management, eco-speed harmonization. When you have large fleets of trucks, when you have trucks say that are making deliveries, the local your local FedEx

distributor is sending out all of their trucks. That's a lot of gasoline. That is a lot of fuel that is being burnt with those vehicles just sitting at traffic lights waiting to make their way out in traffic. So what if we respond to the fact that we know that FedEx is putting 100 trucks out onto the road at 4 a. m. in the morning? And now we change the traffic lights so that we give extended greens to their primary arteries to get them out and about where they need to be quicker, faster, and safer. Pedestrian warnings. I know it's up there somewhere. I'm not seeing it right off the top of my head. We are working on an application.

I'll have a chance to talk about it a little bit later. but dealing with pedestrians and pedestrian signaling. Right now you're familiar with the crosswalk buttons here in the United States. A pedestrian walks up to an intersection, hits the button to cross the street At some point in time it'll allow the light to change, give them the walk signal, and then the countdown timer for clearing that intersection and then changing it back again Back to the automobile side, and this is about to the extent that I'm aware of of the technology, within the automobile system, within the automobile electronics, there is already a network. And it's called a controller area network or a CAN

bus, CAN network. That exists in every automobile that has been manufactured in the United States. since I think it's 2002 when it became required. And you will find underneath your dashboard, you will find a plug interface that matches that plug shown at the top where you can plug devices into it. You take your car to an auto mechanic, you get the engine warning light, and you can plug a device into that that's attaching almost directly to the CAN bus and you can read the car's computers to get your diagnostic codes. The intelligent infrastructure, the ability for the cars to communicate, also just become another component that get added to this bus.

Where I get most involved with then is the infrastructure side, the intelligent infrastructure side Starting on the far side, it all begins with the traffic light. We have traffic signals that are designed to restrict and control the flow of traffic through an intersection. Communicating with that traffic light is a traffic signal controller, abbreviated TSC. That traffic signal controller identifies the timing that each light is in a particular state. And those are very sophisticated electronic devices. Most of them have at least one or two CPUs in them running programs that can control the traffic cycle. And that includes things like being able to adapt the

times for the cycles based on time of day or day of week or any other sorts of conditions that you wish to put into it. But the problem with that or the limitation with that is that these exist out at the traffic signals. in any modern generation of intersection in the United States, you will find at one of the four corners, you'll find a steel cabinet. that is probably almost the size of the podium here. And they're either mounted on the ground or in some cases they're mounted on the poles. but you'll usually find them at one corner of the intersection. Look for them the next time you're at a traffic light.

And if you do find it. that TSC is one of the devices that's going to be in that cabinet. And it's what's actually controlling the signaling of the traffic signal. So what we're adding to that, what's being added to that in the connected vehicle environment are the roadside units or RSUs. They are radio transceivers that they have to be mounted high because they're what's actually sending the radio signals to the cars that are approaching and in the area of the intersection. So you will find them, you may find them, they are about the size of a typical laptop, a little bit thicker, maybe the size of the projector sitting up front. And you can occasionally see them mounted on the

arm that comes out from the side where the traffic lights themselves are mounted. Or if they use the wires to hang the lights, you might find it on the pole itself. but it's a relatively small box and that's communicating with the traffic signal controller to distribute information about that intersection. I'm staying at the top here. There's a protocol that's used called DSRC, Direct Short Range Communications, which is operating in the 5. 9 gigahertz band for you radio geeks out there. And it is designed as a short-range radio network between cars and these RSUs. The cars can tell the intersection that it's coming.

The intersection can tell the cars this green light is going to become red within the next five seconds. Or this red light is going to change to green within the next five seconds. That sort of detailed information allows the car, allows it possible for the car to adjust itself its speed in traffic to optimize its travel through that particular intersection. All of the messages that can be sent between the RSU and the vehicles are defined in a standard called J2735. It lists Actually, I think it's in the neighborhood of 15 different types of messages that an RSU can send to a vehicle, that a vehicle can send to an RSU, or that a vehicle can send to another vehicle.

Three that I'm going to mention here right now are BSMs. That's a basic safety message. That's the car saying, this is my location in GPS coordinates, latitude and longitude This is my velocity. This is my heading. Being radio, these are broadcast everywhere. So every, not just the RSU. But every vehicle within range is also capable of receiving that radio broadcast. And so every car can detect where every car is around it based upon these basic safety messages that are being exchanged. The map message is a map. It is a representation of the intersection, and I have a diagram for that coming up shortly.

that allows a car to know what is the layout of this intersection that I'm approaching. Is it one lane wide, two lanes wide, three lanes wide? And can I make a left turn? Do I have to go straight? Do I have to make a right turn? Do I have one right-hand turn lanes? Two right-hand turn lanes? All of that information is encoded within these map messages. Working along with the map is what's known as the SPAT, that stands for signal phase and timing, and that is what is describing what the current state is of the traffic signal controller and the traffic light. And it includes information such as what is the current color of the light, is your light currently green, yellow, or red? And when it is going to change, and how soon is it going to change?

information useful from the car from the RSU to the car. There is also a reverse message Name escapes me at the moment. I didn't think to include it. That a car can request a light change from an intersection. And so as the car is approaching, there is a particular message that the car can send saying, I'm approaching this intersection. How soon can you change the light for me? And so it is possible for the RSU communicating with the TSC to say, how busy are we? Is it appropriate for me to change this light ahead of cycle to short cycle the light and get it to where the approaching vehicle can get a green. What we're adding

particularly to this environment is a computer that you see down in the middle. that uses a protocol known as SNMP, Simple Network Management Protocol. The protocol itself dates back, oh, at least to the 80s. That is a management protocol. It's a way of communicating with devices to configure those devices. And I have nightmares about SNMP, and the less I say about it here, the better off I'm going to be. One of the facilities, however, that is also possible is that that computer can send what's known as an IFM or an immediate forward message to the RSU. that that is a message that then can go out over the radio. So not only does the RSU take messages from the traffic signal controller, but it can also take messages from a computer to be broadcast out.

And there are many types of messages that can be sent out from the computer out to the vehicles. One of them that we are working on specifically, and I alluded to this earlier, is a pedestrian safety message, a PSM. We have developed an app that if you're running the app, as you approach an intersection, the Your GPS locator within your phone detects when you were are within the range of a monitored intersection and start sending messages, start sending brief packets to the computer system through a web style interface Message package saying I'm approaching an intersection.

This is where I am. This is my speed. This is my heading. And at the appropriate time, once that connection's been established and there start to be some communication. The computer system that is part of this, the web server involved in this, will then start generating the pedestrian safety messages out to the RSU to be sent out to the cars. So as you are walking up to an intersection, the car starts to become aware of your location and send that out. Now the only thing that is sent , Easier GPS coordinates. There's nothing identifiable about the phone that we're sending. We're just sending the smallest possible packet that says there is a pedestrian at these GPS coordinates. And so when you actually step off the sidewalk into the road, that's when the car should take more notice of it.

When you're just at the side of the road, the car can be advised of it, but should not necessarily take action. So I mentioned briefly the map messages that get sent out. They are very detailed representation of an intersection, identifying each and every lane, what the direction of travel is that is allowed for that lane. Whether and what motions you can take, whether you can go straight, make a right turn, make a left turn, make a left turn except from 7 to 9 a. m. Monday through Friday. All of these various con conditions and constraints. Something I learned, little trivia fact, and you know anybody who's done real GIS work, uh

graphic information network. uh geographical work is that roads move because of plate tech plate tectonics A given geographical location, measured GPS coordinate, will change year after year. And in some places and some locations on the order of centimeters per year. Over time, that can accrue and build up and become a real problem. Part of the messaging set are adjustment factors. Part of those radio communications are we have told the radio that it's in this location, but now it's in this location.

And those adjustment factors are factored into these messages. Managing all of these devices If you had to actually go out to every box in the field, would be very time-consuming and impractical to make it at all dynamic And where does Django fit into this? Well what we have done is we have created a set of management applications that allow us to configure and alter and manage and RSU from the traffic engineer central location. Yes, this is probably one of the world's ugliest sites. If Tracy Osborne were here she'd be like, and I thought you read my book. I can say this predates her book, but that's beside the point.

It's a very simple app. We show what different types of applications are configured for a particular RSU And when you click on the simple little icon, you can I've got slides out of order here. How wonderful. You go to the specific details of that particular RSU and you can see everything that the traffic engineer needs to see to work with uh that particular RSU or the particular app uh application running on it. One of the features of IPv6, you'll notice that all of these addresses here are IPv6. Because of the number of devices and the way of networking, everything is IPv6. And that's created some interesting uh

issues with not every tool being exactly IPv6 friendly. IPv6 has a very handy feature that within your local broadcast domain, it requires every device on that domain to respond. And so what we're actually capable of doing is doing an interrogation of every broadcast domain and saying, are you a device on this network? it is supposed to respond, we get that information back, and we can actually use that as part of our diagnostics to determine we expected such and such a device at a particular address to respond. It doesn't respond. there's now something wrong with that device. I mentioned the map messages. Uh that payload section there is the hex dump representation of about a third of a map message.

They can get very intricate. The other application that we use is analyzing some of the data. Now, being part of a research firm, we're working with the University of Michigan Transportation Research Institute, UMTree , and they are collecting all of the BSMs that the cars are transmitting. There are 3,000-ish vehicles in Ann Arbor that have volunteered to be part of this pilot project so that these vehicles as they are driving around Ann Arbor are transmitting these BSM's basic safety messages and we're collecting them. So what we're doing is that we're collecting various statistics regarding how effective are these car radio transmitters and where they are located.

And in this particular case, this is uh Which screen is this? Oh, this is the overlap. So if you have two RSUs, because we're talking about broadcast radio information. Everybody's getting the same message. So when a car broadcasts, multiple RSUs may receive that message. And so we want to find out where the overlap is. And then what we do is we uh Paint some, we have some graphical work showing what's the effective range of each RSU. At what distance for an RSU are we able to get the transmission? I'm going to wrap this up with first of all

a thanks to the U. S. Department of Transportation, Intelligent Transportation Systems Joint Program Office. They have a wonderful website. There's all the information that you would ever want to know about intelligent transportation systems. There's a lot of material there, lots of good stuff there. But just as importantly, I want to throw a shout out to the other open source projects and a couple non-open source projects that make all of this possible. We're certainly not writing all of this code on our own It is built on the shoulders of a lot of people who through their efforts have contributed all of these fundamentals that we have built this stuff upon. If anybody is interested in more details about this, would like to talk about this in any

more, most of you have already known that you can find me generally at the registration desk every morning. More than happy to talk to you about it And if you live in the Baltimore area or would like to live in the Baltimore area and be involved in projects like this, I would really like to talk to you. Thank you very much.