Summary: Sooner or later, people dealing with AV (Autonomous Vehicle) verification encounter the difficult topic of how coverage, KPIs (Key Performance Indicators) and performance grades relate to each other. Also, there is often confusion between “Did this scenario happen” and “Did the AV perform well in this scenario” (and the fact that “KPIs” are sometimes … More AV coverage and performance metrics
Summary: In this post I’ll describe my impressions from the ASAM OpenSCENARIO workshop. I’ll then use that as an excuse to discuss a related topic: Many people agree that scenarios are a good way to check Autonomous Vehicles (AVs) for safety. Some of these people have thorough verification in mind, while others have regulation in … More Bridging AV verification and AV regulation
Summary: This post talks about scenarios as the main tool for serious Autonomous Vehicles verification, but mostly about the more “mechanical” obstacles standing in the way of industrial-scale usage of scenarios – those related to repeatability, HIL issues and behavior stability As promised, here is my (first) report from this year’s Stuttgart Autonomous Vehicle (AV) … More Stuttgart impressions: Scenarios and problems
Summary: This short post explains why I have not been posting, talks a bit about the upcoming Stuttgart symposium, and discusses the ecosystem I have not been blogging much lately (though I have been learning lots of new stuff). Here’s my excuse: It’s that Stuttgart time of the year again: I have been going to … More What I have been doing lately
Summary: This post discusses the influence of the Uber accident on Autonomous Vehicle (AV) deployment. It claims that AVs should eventually be deployed, and yet that we should expect many fatal AV accidents. It then suggests that a comprehensive, transparent verification system could help solve this inevitable tension. That tragic Uber accident has brought AV … More The Uber accident and the bigger picture
Summary: This is part two of my report about what I saw at the Stuttgart 2017 Autonomous Vehicles test & development symposium. It covers frameworks, simulators, scenario definitions and extracting scenarios from recordings. As I promised in part one, here is the rest of my trip report from that yearly symposium. It will cover the … More What’s new in AV verification: Stuttgart report part two
Summary: This post tries to summarize what dynamic verification is, using a single picture. It then puts various verification tools, and diverse verification projects, in the context of that picture. It also explains Coverage Driven Verification (CDV). The Foretellix blog is about verifying complex systems. However, as I discussed here, there is no agreed-upon verification … More Dynamic verification in one picture
Summary: Here is a list of papers I read lately about verifying Autonomous Vehicles. For each, I say why I think it is interesting. These papers all start with something like “AVs are upon us, so they’d better be safe and well-verified”, and then go on to explain their (very diverse) ideas about how to … More Some papers about AV verification
Summary: This is another “What’s new in verification land” post. It describes a video and a paper from Mobileye, and takes that opportunity to revisit four topics: How Autonomous Vehicles should handle unstructured human interaction, how to balance Reinforcement Learning and safety, why simulation is the main way to validate safety in these unstructured environments, … More Misc stuff: Mobileye, simulations and test tracks
Summary: This trip report describes my impressions from the Second Autonomous Vehicles Test & Design Symposium in Stuttgart. Short version: The design side is moving fast. The verification side has improved, but has a long way to go. Note: In my last post I talked about what automation will do to this planet over the … More Finding bugs in autonomous vehicles: My impressions from the second Stuttgart symposium