My new post is here on the company’s blog. It talks about: How AI-based autonomy influences verification How AI-based autonomy implementation is moving closer to verification The importance of abstractions in all of that Enjoy.
Summary: This post looks further into ways to compute the residual risk of ADS. In particular, it looks into some recent research directions, and concludes that (1) they show promise, and (2) they are not there yet. My previous post about ADS Residual Risk (RR) ended with the following tantalizing promise: A subsequent post (coming … More ADS residual risk – some related research
Summary: This post talks about estimating the residual risk of an ADAS / AV design. It looks at the ADS total-risk map, divides it into several areas, and discusses ways for estimating the residual risk of each. People who deal with ADS (Autonomous Driving Systems, which includes ADAS and Autonomous Vehicles) are naturally very interested … More Estimating the residual risk of ADAS / AV
Summary: Autonomous Vehicles will enter various markets (robotaxis, delivery bots, autonomous mining and so on) at different rates (and the current pandemic will probably accelerate some while slowing others). This post is about the verification needs of these markets, and how a single, extensible, scenario-based verification language can be used to express both the common … More Autonomy markets and their potential bugs
About three months ago we (Foretellix) opened up M-SDL, the Measurable Scenario Description Language. Several things happened since then: That version of the Language Reference Manual (v0.9) has been downloaded more than 300 times, by people from ~150 organizations interested in the verification of ADAS / Autonomous Vehicles We got lots of questions and requests … More A short update about M-SDL
I don’t usually link to the Foretellix website from this blog, but I am really happy to do that today. M-SDL, the Measurable Scenario Description Language, is now finally open (under the Apache 2.0 license). You can download the reference manual here (press release is here). I’d like to thank all the people who helped … More M-SDL, the autonomous vehicles verification language, is now open
Summary: Sooner or later, people dealing with AV (Autonomous Vehicle) verification encounter the difficult topic of how coverage, KPIs (Key Performance Indicators) and performance grades relate to each other. Also, there is often confusion between “Did this scenario happen” and “Did the AV perform well in this scenario” (and the fact that “KPIs” are sometimes … More AV coverage and performance metrics
Summary: This is another one of those “misc. stuff” posts, with no unifying theme other than “Interesting inputs regarding Autonomous Vehicles verification”. It will discuss: What I learned regarding the ASAM OSC standardization effort, DeepMind’s “Rigorous Agent Evaluation” paper, Tesla’s “400,0000-car regression farm” idea, some good papers by Philip Koopman, and the upcoming Stuttgart symposium. … More Misc. stuff: ASAM, DeepMind, Tesla and more
Summary: This post talks about why Monte Carlo simulations (which uses the expected distribution) will most likely not get you to safe Autonomous Vehicles. It also talks about what I learned at the ASAM OpenSCENARIO workshop in Munich. Several people I talked to lately assumed that AV verification should mostly be done using Monte-Carlo simulation. … More Don’t stay in Monte Carlo (for AV verification)
Summary: In this post I’ll describe my impressions from the ASAM OpenSCENARIO workshop. I’ll then use that as an excuse to discuss a related topic: Many people agree that scenarios are a good way to check Autonomous Vehicles (AVs) for safety. Some of these people have thorough verification in mind, while others have regulation in … More Bridging AV verification and AV regulation