Summary: Sooner or later, people dealing with AV (Autonomous Vehicle) verification encounter the difficult topic of how coverage, KPIs (Key Performance Indicators) and performance grades relate to each other. Also, there is often confusion between “Did this scenario happen” and “Did the AV perform well in this scenario” (and the fact that “KPIs” are sometimes … More AV coverage and performance metrics
Summary: I discussed before (e.g. here) how connecting rule-based verification to the rule-less, amorphous Machine Learning world is really hard, and yet necessary. The current post talks about a somewhat-exotic technique called Program Induction (PI), and how it might (eventually) help bridge that gap. What’s program induction Background: I always liked the idea of synthesizing … More Using program induction for verification
Summary: This post addresses some high-level questions like: Longer term, how much of the verification of Intelligent Autonomous Systems can be done with just Machine Learning (ML)? Should most requirements remain rule-based, and if so – how does that connect to the ML part? And how will the uneasy interface between ML and rules influence … More Where Machine Learning meets rule-based verification
I attended HVC again this year (see also my HVC’2015 trip report). Here is a very subjective description of what I saw and heard – much of it outside the conference room itself. Note that while HVC is about “all things verification”, much of the audience comes from the chip verification world. Also, the conference … More HVC’2016 trip report
Summary: This post tries to fit Fault Tree Analysis and CDV-based “normal” bug-finding into the same conceptual framework. FTA and CDV have a challenging, uneasy relationship with each other. In a nutshell (details below): FTA (and similar techniques) is a good, established way to reason about failures and reliability, but (1) it depends on humans … More About faults and bugs
Summary: In this post I discuss the problem (common in autonomous systems verification) of checking things which don’t have “sharp” Boolean rules. I also suggest some (imperfect) solutions. Say we want to create a growing catalog of Autonomous Vehicles scenarios, and use it to test our AV, as I suggested here. We are talking system-level … More Checking probabilistic components