Summary: Sooner or later, people dealing with AV (Autonomous Vehicle) verification encounter the difficult topic of how coverage, KPIs (Key Performance Indicators) and performance grades relate to each other. Also, there is often confusion between “Did this scenario happen” and “Did the AV perform well in this scenario” (and the fact that “KPIs” are sometimes … More AV coverage and performance metrics
Summary: This is another one of those “misc. stuff” posts, with no unifying theme other than “Interesting inputs regarding Autonomous Vehicles verification”. It will discuss: What I learned regarding the ASAM OSC standardization effort, DeepMind’s “Rigorous Agent Evaluation” paper, Tesla’s “400,0000-car regression farm” idea, some good papers by Philip Koopman, and the upcoming Stuttgart symposium. … More Misc. stuff: ASAM, DeepMind, Tesla and more
Summary: In this post I’ll describe my impressions from the ASAM OpenSCENARIO workshop. I’ll then use that as an excuse to discuss a related topic: Many people agree that scenarios are a good way to check Autonomous Vehicles (AVs) for safety. Some of these people have thorough verification in mind, while others have regulation in … More Bridging AV verification and AV regulation
Summary: This post discusses the influence of the Uber accident on Autonomous Vehicle (AV) deployment. It claims that AVs should eventually be deployed, and yet that we should expect many fatal AV accidents. It then suggests that a comprehensive, transparent verification system could help solve this inevitable tension. That tragic Uber accident has brought AV … More The Uber accident and the bigger picture
Summary: There are several approaches for verifying that Autonomous Vehicles are safe enough. The Pegasus project is one interesting, thoughtful attempt to do that (focusing initially on highly automated driving, not AVs). In this post I’ll summarizes a recent Pegasus symposium, and describe what I like about the approach and what is perhaps still missing. … More How to write AV scenarios (and some notes about Pegasus)
Summary: This post tries to summarize what dynamic verification is, using a single picture. It then puts various verification tools, and diverse verification projects, in the context of that picture. It also explains Coverage Driven Verification (CDV). The Foretellix blog is about verifying complex systems. However, as I discussed here, there is no agreed-upon verification … More Dynamic verification in one picture
Summary: This post tries to explain (once and for all?) how the concept of coverage is used to optimize the verification process, what it means to auto-maximize coverage, and how people have tried to do it. I have been spending some time lately on coverage maximization via ML (which I described here). As is often … More Verification, coverage and maximization: The big picture
Summary: This post explains why security research is relevant for this blog, and discusses three specific areas worth tracking. Hi folks – I’m back (vacations and all that). Security people deal with finding security issues (vulnerabilities) in systems. This blog deals with a different topic: big-system verification, where verification is defined as finding functional / … More Security research is pretty relevant for us