Summary: Autonomous Vehicles will enter various markets (robotaxis, delivery bots, autonomous mining and so on) at different rates (and the current pandemic will probably accelerate some while slowing others). This post is about the verification needs of these markets, and how a single, extensible, scenario-based verification language can be used to express both the common … More Autonomy markets and their potential bugs
About three months ago we (Foretellix) opened up M-SDL, the Measurable Scenario Description Language. Several things happened since then: That version of the Language Reference Manual (v0.9) has been downloaded more than 300 times, by people from ~150 organizations interested in the verification of ADAS / Autonomous Vehicles We got lots of questions and requests … More A short update about M-SDL
Summary: This is another one of those “misc. stuff” posts, with no unifying theme other than “Interesting inputs regarding Autonomous Vehicles verification”. It will discuss: What I learned regarding the ASAM OSC standardization effort, DeepMind’s “Rigorous Agent Evaluation” paper, Tesla’s “400,0000-car regression farm” idea, some good papers by Philip Koopman, and the upcoming Stuttgart symposium. … More Misc. stuff: ASAM, DeepMind, Tesla and more
Summary: This post talks about why Monte Carlo simulations (which uses the expected distribution) will most likely not get you to safe Autonomous Vehicles. It also talks about what I learned at the ASAM OpenSCENARIO workshop in Munich. Several people I talked to lately assumed that AV verification should mostly be done using Monte-Carlo simulation. … More Don’t stay in Monte Carlo (for AV verification)
Summary: In this post I’ll describe my impressions from the ASAM OpenSCENARIO workshop. I’ll then use that as an excuse to discuss a related topic: Many people agree that scenarios are a good way to check Autonomous Vehicles (AVs) for safety. Some of these people have thorough verification in mind, while others have regulation in … More Bridging AV verification and AV regulation
Summary: This post discusses the influence of the Uber accident on Autonomous Vehicle (AV) deployment. It claims that AVs should eventually be deployed, and yet that we should expect many fatal AV accidents. It then suggests that a comprehensive, transparent verification system could help solve this inevitable tension. That tragic Uber accident has brought AV … More The Uber accident and the bigger picture
Summary: There are several approaches for verifying that Autonomous Vehicles are safe enough. The Pegasus project is one interesting, thoughtful attempt to do that (focusing initially on highly automated driving, not AVs). In this post I’ll summarizes a recent Pegasus symposium, and describe what I like about the approach and what is perhaps still missing. … More How to write AV scenarios (and some notes about Pegasus)
Summary: I discussed before (e.g. here) how connecting rule-based verification to the rule-less, amorphous Machine Learning world is really hard, and yet necessary. The current post talks about a somewhat-exotic technique called Program Induction (PI), and how it might (eventually) help bridge that gap. What’s program induction Background: I always liked the idea of synthesizing … More Using program induction for verification
Summary: This is a short collection of some news / papers related to Autonomous Vehicles verification Some info about how Waymo does verification: The Atlantic came out with a long article about how Waymo verifies AVs. I did not see much which was new or unexpected here (but I assume they did not divulge their … More AV verification updates: Waymo, Apple and more
Summary: This post addresses some high-level questions like: Longer term, how much of the verification of Intelligent Autonomous Systems can be done with just Machine Learning (ML)? Should most requirements remain rule-based, and if so – how does that connect to the ML part? And how will the uneasy interface between ML and rules influence … More Where Machine Learning meets rule-based verification