The New Safety Net for Human-Assisted AVs
Researchers from Purdue and the University of Michigan are tackling one of autonomous driving’s toughest challenges: keeping remotely operated AVs safe when human intervention is needed. Their work exposes how latency, operator error, and cyberattacks can quickly turn teleoperation risky—and introduces a safety-check system that detects dangerous maneuvers and overrides them before a crash occurs. By adding collaborative sensing from nearby vehicles and infrastructure, they also show how to expand visibility, strengthen communication reliability, and cut latency. The result is a promising hybrid framework that makes remote AV operation more resilient, more aware, and dramatically safer.
Transcript
00:00:01 All right, folks. We're going to go ahead and get started with our SEC cyber security research presentations. Uh security is highly relevant as any possible attack on the wireless link such as false data injection, spoofing, jamming uh could lead to disastrous consequences. In this research thrust, CCAT works to secure the CAV ecosystem by simultaneously considering cyber
00:00:20 security threats posed to CAV's, physical transportation infrastructure, and cyber infrastructure that support CAV operations. Today, we're joined by four researchers from three universities. Um, and up first is Drs. Jang Fun, assistant professor of civil engineering at Purdue University, and Morley Mau, professor of electrical engineering and computer science at the
00:00:38 University of Michigan, who will discuss their project addressing safety and security challenges in MLbased AV software stack. Please join me in welcoming Drs. Fong and Marley Mau. [Applause] Okay. Okay. All right. Yeah. Uh thanks Kevin for the introduction and thank you all for coming to this
00:01:02 session. So um so the research topic of uh Molly and uh um um my project is regarding uh to address the safety and security challenge of uh the AV tele operation. Right. So um so there are some cases that uh uh AV operation uh AV tell operation is needed. Right. The first example I give is during the commercial deployments, right? Uh as long as the automation level has not
00:01:30 reached level five uh then under certain scenarios, driving scenarios that um and the AV still needs the human intervention. Right? So here is an example on the left picture. And the second example I want to use uh MCD 2.0 right a very unique feature of MCD 2.0 Z is about uh uh remote controlling of the AV in MCD test facility. So that uh so researchers from all over the world
00:01:58 don't need to physically come to the M city to test their algorithms. Right. So the video um shows here is uh um our collaboration with M city about one year ago. We were running the uh AV control algorithm at Purdue campus while controlling a uh real AV in real time in MC uh test facility. Right. So you can so you can imagine that under such a framework there are a lot of challenges.
00:02:26 Right. The first thing maybe you can think of is uh the communication latency. Right? So uh there's a lot of data transmission between the AV and the remote operator. uh including uh the remote operator needs to receive the sensor data from the AV and also at the same time uh the AV uh I mean the remote operator will also send the control commands I mean on the different uh
00:02:53 remote operation strategies right sending different levels of command to to the AV um so the this one requires very high bandwidth uh in a communication network or it will send a very high frequency of command. If we talk about real real real time control actually in our previous case um we have to reduce our control resolution from 10 herz to five hertz. Exactly because the
00:03:20 latencies uh between the two networks. Okay. And the second aspect of challenge is from the remote operator or the human factors point of view. Right. Comparing with the real world human drivers. So the remote operator usually have uh limited situational awareness and also similar as the human uh I mean the real drivers the remote operator may also suffer from the fatigue distraction or
00:03:49 our cognitive overload. So the third aspect we want to touch is uh from the cyber security point of view again because of the uh wireless communication right and the communication channel may be compromised or may be vulnerable to different type of cyber attacks like including the DOS attack or uh data spoofing attack right the data send to the remote operator or the data sent to
00:04:13 the AV can both be be compromised for example and also uh the teley operator um sometimes Well, in some extreme cases that the tally operator can also be malicious, right? So, um so this also pretend also present uh uh some risks in the tally operation. So, all of these challenges actually posed the significant safety concerns to the remote operation of uh the autonomous
00:04:41 vehicles. So, uh in this project of course we are not going to solve all the problems. So our point of departure is uh we look at the misbehavior from a remote operator either it's a human or a some kind of algorithm and our goal is to determine and protect against the remote operator's behavior. So what we did so far again so this part we only a few months into this project so a lot of
00:05:07 things that we present today are like work in progress and some preliminary results. So any suggestions and comments are welcome. So what we did so far is we first developed a driving simulator to mimic this remote operation scenario and then we developed a trajectory level safety check model to detect the misbehavior from the remote operator and if necessary to suspend the control
00:05:31 authority from the remote operator. And finally we want to introduce uh how we want to uh leverage the collaborative sensing uh from other vehicles or from the infrastructure to improve the situation awareness of the AV and also uh improve the communication robustness. Right? So this is like the driving simulator we developed. So I think many of you are familiar with
00:05:56 this. This is the Kala and Sumo co simulation. So, Sumo is responsible for generating those driving scenarios and the Kala is responsible for the like rendering the 3D environment and serving as interface to the uh Logitech driving uh I mean the driving simulator. Okay. So, then based on this we designed the two kind scenarios to mimic the remote operation. So, the first one is when the
00:06:21 vehicle is approaching a static object. We can assume that it's out of its OD. we assume it's out of the OD and uh then the driver will the remote operator will take control of the vehicle to bypass this uh obstacle and the second case is a little bit more complicated while the remote operator controlling the vehicle there's also a oncoming vehicle from the other direction that may cause collision
00:06:47 with uh uh with the AV okay so the the safety uh like uh track module that we develop is try to monitor the status of the uh uh AV and then try to stop the command from the remote operator if it finds some um like uh potential collisions. So uh uh so for both cases first we develop a very simple uh static track model and uh so when the remote uh operator is taking control of the
00:07:20 vehicle and this model will be activated and keep monitoring the status of the vehicle and to based on its current trajectory current track and to predict its future trajectory. if the near future trajectory has a conflict with a static obstacle and then the vehicle will reject the command and try to stop. Okay. So, okay. So, this is uh the test. So,
00:07:45 the verse the vehicle was first under the autopilot. Uh this is the Kalas autonomous driving algorithm. And you can see now the remote operates actually it is me tried to take over and I did not do a good job and uh glide into the front vehicle. Right? So this is without the safety check. So when we implemented the safety check uh the same case safety pilot first uh sorry uh autopilot
00:08:12 first and then okay when I start take control so you can see the vehicle was under my control at first but after a few seconds when it detect my maneuver has some potential collision with the leading uh with the leading vehicle and it stopped the vehicle automatically. So uh the second case right we when we have upcoming vehicle so this case is a
00:08:41 little bit more comp complicated. So in this case we developed a two trajectory prediction model. Uh we formulate them as a an optimization problem basically to predict the future trajectory of both the eagle vehicle the AV and also the an oncoming vehicle right and if the two predicted trajectories have confliction then also the safety check module will be activated to stop the
00:09:09 vehicle. Right? And this is the case. So you can see when I tried to took the control at first. Okay. I don't know why it did not play on the other side. Sorry. Uh but you can see the bird eye view that actually actually when I started to take over I still could not really see the oncoming vehicle. So but when I deviate to the other land but it's already too late. And of course
00:09:47 this is the case with the safety check. Okay this place on the other side. Um so you can see that when I tried to control first the vehicle did not really move. Okay. So, you can see the vehicle stop there, did not move because it detects the conflict with the oncoming vehicle on the other side. You can see after this
00:10:15 vehicle has passed, now the vehicle started to take my command again. And this time, apparently, I did a pretty good job navigating the vehicle back to the original lane. and then uh it came back to the control by the autopilot. Okay. So um if you look at the this case actually here is assumption under the scenario is that this AV can always detect the oncoming
00:10:42 traffic from other side. This is actually not very realistic especially given if this AV is stuck behind some obstacle. Right? So this is why we think that collaborative sensing can also play an important role in the remote operation of the AVs. If we have some other vehicles or infrastructure can detect the oncoming vehicle and send to the AV, it can further improve uh the
00:11:05 safety track module of the vehicle. All right. So to that so Molly will talk about the remote sensing part. So the collaborative sensing part. Uh yeah, thanks. Yeah. So um as as EA mentioned, there's always limitations with just leveraging sensor data from a single vehicle's perspective. So uh to that end we could leverage data from other vehicles in the form of collaborative
00:11:29 sensing. I also want to mention uh we are looking at um the the requirement in terms of communication. It turns out uh h having support from other vehicles can also enhance uh the communication robustness. So when you do remote um uh operator support you really need to have a uh reliable communication channel between the vehicle and the remote operator. But we know that uh network
00:11:52 connectivity is not always um uh guaranteed to be uh high bandwidth and reliable low latency. So that's the challenge we have to face to address right. So with cellular connectivities I will show you some data later. It turns out uh the network connectivity depending on the mobility uh coverage uh areas um it it highly varies depending on the cellular provider um uh yeah
00:12:16 deployment. So, so we have a project called uh harbor which I will describe in bit more detail here and I believe this is very applicable to uh support remote operation support. So um so CVP stands for collaborative uh vehicular uh perception. So the idea here is that if you look at um traditionally we can just use a peer-to-peer design right called
00:12:41 V2V vehicle-tovehicle uh data sharing. So from uh a eco vehicles perspective, you have data locally connect collected from your uh your vehicle alone but you could potentially benefit from data from other vehicles. How do you share such data? Right? Sensing data could be very expensive in terms of bandwidth requirement. So if you just leverage uh peer-to-peer based data sharing um you
00:13:06 can do some uh local compression to um reduce the data size then you can send the data to other vehicles and the the vehicle uh does local processing by merging the data together to improve the visibility. For example object might more likely to be detected if you get data from other vehicles. So that's kind of intuitive. Um so so as a result of this uh we can
00:13:31 um have a better uh coverage right so some existing work have already demonstrated by just leveraging just another vehicle you can already improve the coverage and detecting um uh accluded objects so another thought is that um communication you can have peer-to-peer communication or V2V vehicleto vehicle but naturally we have a base station right cellular base
00:13:53 station we have the infrastructure side so um one thought is uh we can also leverage the the central server to do processing because the vehicle itself has limited computational resources. So here the thought is that we can aggregate the sensing data after compression we can leverage an edge server to merge the data together and the edge server has more computational
00:14:13 resources. It can do you know more advanced machine learning uh computer vision u uh based algorithm uh to be able to to analyze the data more more efficiently and uh more accurately. So here this is another kind of a scheme and it's called V2I CVP design collaborative vehicle um perception. So here the communication method is called V2I vehicle to infrastructure.
00:14:38 So um obviously there are limitations with each mode right uh V2V uh suffers from the scalability issue right so intuitively you have more and more vehicles you have to kind of almost n squared kind of sharing data uh challenge right so so naturally you can just broadcast the data but on the other hand broadcast even with broadcast you have limited bandwidth right broadcast
00:15:00 wireless medium um you don't always have the available uh resources you need to be smart about what data to share that's not always uh easy to figure out. Um so so another uh challenge we face with V2I is that turns out um the cellular network coverage and is not always um uh uniform. And so this is a uh trace we collected by driving and we looked at Verizon T-Mo in terms of bandwidth,
00:15:27 right? So you can see uh depending on the location um one network may have higher bandwidth than the other, right? So, so around 15 you have only 15 seconds have only one uh close to um less than one megabit per second for 15 seconds of duration for T-Mo where Verizon gives you better um uh throughput. So, so this is a intuition which tells us that even if you just
00:15:52 rely on single provider, obviously rely on multiple providers, you can probably get better throughput, but there's always also the intuition that uh C uh the V2i or CV CV2X technology doesn't always guarantee you uh high throughput and low latency and which is very critical for remote operation support. So the the idea here is as you can see is very intuitive. Why don't we have a
00:16:13 hybrid architecture which leverages V2V and V2i in a smart way so that we can uh get better uh higher um uh bandwidths when we need and get more uh uh reliable communication. Right? So this is the the the key idea is that we have these helps which are essentially vehicles um uh that need to be helped by helpers which have the v2 connectivity. So the help piece will use V2V connectivity to reach
00:16:42 uh the help per helpers to be able to send the data through infrastructure right so so so strategically we need to figure out how to assign the helpers to the help piece so here's an example right so everyone is use B2I obviously may not always work depending on the the cellular data u coverage and if you use the V2V link in a smart way to switch to V2V uh and uh assuming uh ISBC has
00:17:09 enough bandwidth uplink bandwidth then you could just use that to to be able to communicate right so here's an example where we're just trying to share data in a uh more reliable way and this could be talking the data sent to the remote operator who needs that kind of information in order to make the control decision right so so um so we have this architecture designed and uh implemented
00:17:33 prototype typed uh in a um using V2V and V2 toi uh hybrid architecture to enable reliable communication and this is not just communication it's also computation so this setup also has the necessary kind of uh uh computation which is computer vision uh data processing to uh do perception uh for for for the detected objects so so um I won't have time to go into detail but this is a p
00:18:02 published paper at census 2024 which um has the the detail results. As you can see, there is this uh the edge part and the helper and help uh helper and the help vehicle uh setup and there there are different algorithms involved to make sure you have the prioritization to be smart about what data to send and um making sure that we are deadline aware to make sure we meet the deadline uh
00:18:28 required for real-time perception. Um so um given lack of time so I want to just highlight some of the key um innovation we have for this work uh so one key idea here is that we'll need to identify these performance impacting factors in assigning uh who are the helps um to uh who are the helpers to to help uh upload the data to the infrastructure. So uh one naive way is
00:18:54 just measure the bandwidth directly but obviously this is kind of expensive not always scalable. So the uh our intuition is that we can look at for these kind of factors which are easier to to measure like distance and interference because with wireless communication you have to be careful if everyone's talking at the same time right you you have these interference issues. So we are looking
00:19:16 at um basically interference can is something that you can passively measure. Um so so basically what we we did is that we use these factors to understand like based on physical distance and the network interference levels to figure out how to assign helpers to the help um and this can be run heristic this heristic can be run periodically because the network state
00:19:39 constantly change uh the vehicles move so therefore it needs to be continuously updated. So we have a uh algorithm which runs continuously to assign the uh the vehicles who are supposed to upload data uh on behalf of vehicles who do not have sufficient network connectivity and this runs um by maximizing the overall score. So you can balancing out depending on the uh data upload needs. Uh the other
00:20:05 aspect is this we have to also be aware of the application uh deadlines right the real-time nature of the task is really critical. Uh so we use NTP uh to help synchronize because the synchronization aspect is is also uh critical for making sure that the the clock is synchronized. Do I have five more minutes or Okay. So I'm just going to um skip ahead
00:20:29 to show the the the the evaluation results and I think what's most interesting is the MCD based evaluation results. But at high level we have a reduction of the mean latency to upload the results by uh by 38%. And just the last part is the uh the case study we did using MCD uh test bed. So in this setup we are able to detect more accluded objects uh by leveraging this
00:20:54 bridge from uh CVB which uh uses CAVA to be able to send the uh the the necessary data to be able to detect the accluded object. Um so so um applied to the remote operation context, you can imagine uh the remote operator wants to be able to see that uh accured object in order to make uh uh safer driving decisions on behalf of the the the uh AV. Uh so with that uh just want to
00:21:23 leave you uh a set of uh future work uh items. Um I won't give we don't have time um any more time left. I don't want to take away from the next presentation. I will end here. Thanks. Thank you so much.