Workshop: Road Vehicle Teleoperation 2021

Title: Teleoperated driving in the physical world with Digital twin

Presenter: Sig Persson, Ericsson, Sweden

Time: 7:15 – 7:45 (CET); 14:15 – 14:45 (JST); 1:15 – 1:45 (ET)

Abstract:

In the presentation Ericsson will present from our development on Teleoperated over 5G networks. This will cover learnings and findings on how to use QoS, Precise Positioning using RTK GNSS as well reliable video communication using WebRTC. Also, the benefit to use a digital twin in parallel with the real world.

Title: Adaptive Video Bitrate Allocation and Configuration for Teleoperated Vehicles

Presenter: Andreas Schimpe, Institute of Automotive Technology, Technical University of Munich, Germany

Time: 8:00 – 8:20 (CET) 15:00 – 15:20 (JST) 2:00 – 2:20 (ET)

Abstract:

Vehicles with autonomous driving capabilities are present on public streets. However, edge cases remain that still require a human in-vehicle driver. Assuming the vehicle manages to come to a safe state in an automated fashion, teleoperated driving technology enables a human to resolve the situation remotely by a control interface connected via a mobile network. While this is a promising solution, it also introduces technical challenges, one of them being the necessity to transmit video data of multiple cameras from the vehicle to the human operator. In this paper, an adaptive video streaming framework specifically designed for teleoperated vehicles is proposed and demonstrated. The framework enables automatic reconfiguration of the video streams of the multi-camera system at runtime. Predictions of variable transmission service quality are taken into account. With the objective to improve visual quality, the framework uses so-called rate-quality models to dynamically allocate bitrates and select resolution scaling factors. Results from deploying the proposed framework on an actual teleoperated driving system are presented.

Title: Regulating Road Vehicle Teleoperation: Back to the Near Future

Presenters: Philip Almestrand Linné and Jeanette Andersson, the Swedish National Road and Transport Research Institute (VTI), Sweden

Time: 8:20 – 8:40 (CET) 15:20 – 15:40 (JST) 2:20 – 2:40 (ET)

Abstract:

Due to the many remaining obstacles before reliability and safety can sufficiently be guaranteed for high-level automated vehicles (AVs), teleoperation or remote operation of partially automated vehicles by a human driver has become increasingly interesting to consider. However, remote operation, including remote driving, has so far only received little attention in legal scientific and transportation literature. This paper aims to establish some basic legal matters for remote driving by examining its regulatory development in three different jurisdictions. A combination of methods is employed including an examination of literature regarding AVs and their regulation. The main result is that current regulation in the examined jurisdictions intentionally addresses a future with high-level AVs, but to a large extent excludes regulatory details for remote operation. In conclusion, this paper argues that both present and coming regulation for automated vehicles ought to be more near future-oriented and address the concept of remote operation more explicitly. This, for regulation to be better in touch with current technology, for the benefit of a wider acceptance in society, for legal certainty, but also for innovation support and stability for investments in technology.

Title: Active Safety System for Semi-Autonomous Teleoperated Vehicles

Presenter: Smit Jaman Saparia, Technical University of Munich, Germany & TU Delft, Delft, The Netherlands

Time: 8:40 – 9:00 (CET) 15:40 – 16:00 (JST) 2:40 – 3:00 (ET)

Abstract:

Autonomous cars can reduce road traffic accidents and provide a safer mode of transport. However, key technical challenges, such as safe navigation in complex urban environments, need to be addressed before deploying these vehicles on the market. Teleoperation can help smooth the transition from human operated to fully autonomous vehicles since it still has human in the loop providing the scope of fallback on driver. This paper presents an Active Safety System (ASS) approach for teleoperated driving. The proposed approach helps the operator ensure the safety of the vehicle in complex environments, that is, avoid collisions with static or dynamic obstacles. Our ASS relies on a model predictive control (MPC) formulation to control both the lateral and longitudinal dynamics of the vehicle. By exploiting the ability of the MPC framework to deal with constraints, our ASS restricts the controller's authority to intervene for lateral correction of the human operator's commands, avoiding counter-intuitive driving experience for the human operator. Further, we design a visual feedback to enhance the operator's trust over the ASS. In addition, we propose an MPC's prediction horizon data based novel predictive display to mitigate the effects of large latency in the teleoperation system. We tested the performance of the proposed approach on a high-fidelity vehicle simulator in the presence of dynamic obstacles and latency.

Title: Some insights into safety related human factors in automated shuttle and bus services

Presenter: Satoshi Kitazaki, National Institute of Advanced Industrial Science and Technology (AIST), Japan

Time: 9:30 – 10:00 (CET) 16:30 – 17:00 (JST) 3:30 – 4:00 (ET)

Abstract:

We have been conducting demonstration experiments of automated shuttle and bus services in cooperation with local governments, service providers and automated vehicle manufacturers in Japan. The purposes of the experiments were to evaluate maturity of the technology, acceptance of potential users and sustainability of the service businesses. Some of the experiments ended successfully and actual services started. In this presentation, the experiments and the services will be introduced with some insights into safety related human factors that need to be investigated in the next step.

Title: HAVOC – Heavy Automated Vehicle Operations Center

Presenter: Dr Jonas Andersson, Senior Researcher, RISE Research Institutes of Sweden, Sweden

Time: 10:00 – 10:20 (CET) 17:00 – 17:20 (JST) 4:00 – 4:20 (ET)

Abstract:

HAVOC is a Swedish research collaboration between RISE Research Institutes of Sweden and Scania, financed by the Swedish research and innovation agency Vinnova. The aim of the HAVOC project is to explore the human factors related needs and requirements posed on the human operator(s) and heavy automated vehicles when performing remote hub-to-hub teleoperation. From the perspective of three imperative control modes: remote assessment, remote assistance and remote driving - the talk will show examples of how the HAVOC project is addressing the future human-automation interaction challenges of remote teleoperation of heavy vehicles.

Title: Road vehicle teleopration: activities and challenges [interactive session]

Moderator: Maytheewat Aramrattana, the Swedish National Road and Transport Research Institute (VTI), Sweden

Time: 10:20 – 10:40 (CET) 17:20 – 17:40 (JST) 4:20 – 4:40 (ET)

Abstract:

This session intends to collect inputs and opinions from workshop participants regarding research activities and challenges related to road vehicle teleoperation. The session will start with a short presentation on the topic. Workshop participants will then be asked to provide their input through a questionnaire (menti.com). Results will then be presented and summarized at the closing session.

Title: Measuring the Safety of Automated Driving Systems (ADS)

Presenter: Dr. Edward R. Griffor, National Institute of Standards and Technology (NIST), USA

Time: 11:00 – 11:30 (CET) 18:00 – 18:30 (JST) 5:00 – 5:30 (ET)

Abstract:

Integrated software and hardware systems are Cyber-Physical Systems (CPS/IoT) are pervasive, across domains including transportation, healthcare, and energy. Establishing the measurement science needed to assess the safety of such systems is critical. We will introduce a Logic of Autonomy and the concept of the Operating Envelope Specification (OES) needed for ADS design and testing of ADS trustworthiness, including safety, security, privacy protection, resilience and reliability. Finally, we use these foundations to provide notional architectures for ADS themselves and a virtual ADS Testbed and apply the latter to safety measurement for Automatic Emergency Braking (AEB) and Adaptive Cruise Control (ACC).

Title: Bring Automated Driving Toward Reality with Vehicle Teleoperation

Presenter: Dr. Tao Zhang, National Institute of Standards and Technology (NIST), USA

Time: 11:30 – 12:00 (CET) 18:30 – 19:00 (JST) 5:30 – 6:00 (ET)

Abstract:

Over the past several years, we have seen a rapid growth of industry efforts in developing vehicle teleoperation solutions. Teleoperation is becoming an increasingly more important part of the automated driving ecosystem. Today, teleoperation has been used for helping self-driving cars navigate difficult situations and has also become available for special vehicles such as robots, trucking, fleet, mining, delivery, and warehouse vehicles. The potential of vehicle teleoperation, however, does not stop here. In this talk, I will discuss the potential roles of vehicle teleoperation in automated driving, and the technical challenges and opportunities it may bring.

Stig Persson, Ericsson

Stig Persson is program development leader for Kista Automotive trial site, as well as Drive Swedens testbed Innovation cloud. Stig is a Program development leader at Ericsson DNEW S&T, a global community developers and engineers. Ericsson DNEW S&T plays a lead role in researching, designing and creating future disruptive technologies to support automotive and transport use cases.

Dr. Tao Zhang, the US National Institute of Standards and Technology

Dr. Tao Zhang, an IEEE Fellow, has been leading research, product development, and corporate strategies to create disruptive innovations and transform them into practical solutions, standards, and products for 30 years. He is currently managing the Emerging Networking Technologies Group in the Information Technology Lab at the US National Institute of Standards and Technology (NIST). He was the CTO for the Smart Connected Vehicles Business at Cisco Systems, and the Chief Scientist and the Director of multiple R&D groups working on wireless and vehicular networking at Telcordia Technologies (formerly Bellcore, originally part of the Bell Labs). He cofounded the Open Fog Consortium and the Connected Vehicle Trade Association (CVTA) and served as a founding Board Director for them. Tao holds 50+ US patents and coauthored two books “Vehicle Safety Communications: Protocols, Security, and Privacy” and “IP-Based Next Generation Wireless Networks”, several book chapters, and over 80 peer-reviewed papers. He served as the CIO and a Board Governor of the IEEE Communications Society and as a Distinguished Lecturer of the IEEE Vehicular Technology Society. He cofounded and served on leadership roles for multiple international conferences and forums.

Dr. Edward R. Griffor, Associate Director for Cyber Physical Systems, NIST

Dr. Griffor is the Associate Director for Cyber Physical Systems at the National Institute of Standards and Technology (NIST). He has served as Walter P. Chrysler Technical Fellow and Chair of the DaimlerChrysler and Fiat Chrysler Automobiles Technology Council as well as Chair of The MIT Alliance. He is NSF/NATO Fellow in Science and Engineering. Dr. Griffor holds a Ph.D. in Mathematics from MIT and a Habilitation in Mathematics and Engineering from the University of Oslo. He has served as Professor of Engineering and Mathematics at universities in the US, Europe, Asia and South America and is Adjunct Professor of Medicine at the Center for Molecular Medicine and Genetics at the Wayne State University School of Medicine. In this role, Dr. Griffor contributes to biosystems modeling and systems biology. He currently serves on the Halden Reactor Safety Project of the Nuclear Regulatory Commission and is a member of the High Confidence Software Systems committee of the US Networking and Information Technology Research and Development Program.

His recent books include the Handbook of System Safety and Security and the Handbook of Computability with Elsevier Science and the Mathematical Theory of Domains with Cambridge University Press. He has published extensively in professional journals and has given invited lectures to meetings of the American Mathematical Society, the North American Software Certification Consortium, SAE International, the US Federal Reserve Bank and other US and European government agencies.

Dr. Griffor’s current research combines methods of physics, mathematics and biology to assurance for cyber physical systems, including the safety and security of autonomous systems. He leads the NIST Automated Driving System (ADS) Safety Measurement Technical Working Group and has developed the Automated Driving Systems architecture and co-simulation testbed. Dr. Griffor co-founded and leads the global industry collaboration to deploy an Agile Production Platform for supply chain resilience in response to the 2020 COVID-19 pandemic, including 3D modeling and manufacturing of Antiviral Delivery Devices. He leads the modeling and simulation research at the NIST ADS Testbed Laboratory. Dr. Griffor’s lab at NIST applies the Cyber-Physical Systems/IoT Framework to promote the use of these advanced mathematical methods to meet industry needs in engineering, manufacturing and assurance, including applications in energy, infrastructure, medicine and transportation systems.

Satoshi Kitazaki, Director, Human-Centered Mobility Research Center, AIST, Japan

Satoshi Kitazaki is Director, Human-Centered Mobility Research Center, National Institute of Advanced Industrial Science and Technology (AIST), Japan. He has been leading the national research project on human factors in automated driving since 2016. He received his Bachelor’s and Master’s degrees from Kyoto University, Japan, and Ph.D. from the University of Southampton, UK. He has experiences of working in the automotive industry, Nissan Motor, and also working in the academia, University of Iowa, before joining AIST in 2015.

This workshop focuses on different components around road vehicle teleoperation, which is an operations of road vehicles from a distance (outside of the vehicles). Road vehicles of interests includes (but are not limited to) connected vehicles that are passenger cars, trucks, and buses; especially those that are automated. Realizing teleoperation of vehicles on public roads is a big challenge, which requires expertise from several disciplines across transport sectors; from practitioner to policymakers. This workshop intends to initiate multi-disciplinary discussions around road vehicle teleoperation, and thus will involve academicians, practitioners, and policymakers across research fields.

In the context of this workshop, we focus on two modes of vehicle teleoperation: i) remote assistance; and ii) remote driving. Let us define a remote operator as a person who operates a vehicle from a distance. We then assume that this operation is done at a remote station, which provides the remote operator with necessary interfaces for operating the vehicle.

Remote assistance include situation when a vehicle requests an assistance during its normal operation. For instance, a vehicle encounter an ambiguous object when driving autonomously, the vehicle then request assistance from the remote operator in order to confirm whether it is safe to overtake the object. Therefore, in this case, the remote operator assist the vehicle without taking direct control of the vehicle. On the other hand, remote driving refers to situations when the remote operator take over control of a vehicle and manually operates the vehicle.

The main topics of interest can be summarized as follows:

• Connectivity challenges in road vehicle teleoperation
• Human factors in road vehicle teleoperation
• Human-machine interface for teleoperation of road vehicles
• Laws and regulations related to road vehicle teleoperation
• Vehicle design and technology to support vehicle teleoperation
• Remote assistance of automated vehicles
• Remote driving of road vehicles
• System architecture for road vehicle teleoperation
  

Please note that we will also welcome other topics, that are relevant to road vehicle teleoperation but not listed above.

The 2021 IEEE Intelligent Vehicles Symposium (IV21) is a premier annual technical forum sponsored by the IEEE Intelligent Transportation Systems Society (ITSS). It brings together researchers and practitioners from universities, industry, and government agencies worldwide to share and discuss the latest advances in theory and technology related to intelligent vehicles.