Where Machine Learning meets rule-based verification

Summary: This post addresses some high-level questions like: Longer term, how much of the verification of Intelligent Autonomous Systems can be done with just Machine Learning (ML)? Should most requirements remain rule-based, and if so – how does that connect to the ML part? And how will the uneasy interface between ML and rules influence … More Where Machine Learning meets rule-based verification

Dynamic verification in one picture

Summary: This post tries to summarize what dynamic verification is, using a single picture. It then puts various verification tools, and diverse verification projects, in the context of that picture. It also explains Coverage Driven Verification (CDV). The Foretellix blog is about verifying complex systems. However, as I discussed here, there is no agreed-upon verification … More Dynamic verification in one picture

What’s new in AV verification: Report from the Stuttgart symposium

Summary: This is part one of my report about what I saw at the Stuttgart 2017 Autonomous Vehicles test & development symposium last week. This yearly symposium seems to be a pretty good place to get the feeling of what’s going on in AV verification (at least in Europe): There are several AV-related conferences, but … More What’s new in AV verification: Report from the Stuttgart symposium

DeepXplore and new ideas for verifying ML systems

Summary: This post talks about the DeepXplore paper, and uses it to revisit the topic of verification of ML-based systems The paper DeepXplore: Automated Whitebox Testing of Deep Learning Systems (by folks from Columbia U and Lehigh U) describes a new and (in my view) pretty important way to verify ML-based systems. And it somehow … More DeepXplore and new ideas for verifying ML systems

Topics in this blog

Summary: This post acts as a “table of contents” for the whole blog, linking (via tags) to the various posts. There are now quite a few posts in the Foretellix blog, and readers may get lost. So I added Tags to the various posts, and also created this “general orientation” post, which has links to … More Topics in this blog

One-shot imitation learning and verification

Summary: This post will talk about “One-shot imitation learning” (a new and exciting direction in Machine Learning), and how that direction could help coverage maximization (which is important for verification). It will then speculate about the general role of ML in Intelligent Autonomous Systems verification. Note: You may have heard already about one-shot imitation learning … More One-shot imitation learning and verification

Misc stuff: Mobileye, simulations and test tracks

Summary: This is another “What’s new in verification land” post. It describes a video and a paper from Mobileye, and takes that opportunity to revisit four topics: How Autonomous Vehicles should handle unstructured human interaction, how to balance Reinforcement Learning and safety, why simulation is the main way to validate safety in these unstructured environments, … More Misc stuff: Mobileye, simulations and test tracks

Formal verification of really-complex systems

Summary: This post describes some new attempts at formal verification of complex systems (mainly AVs and ML-based systems) In general, I think Formal Verification (FV) will only have a limited role in verifying Intelligent Autonomous Systems (IAS) and ML-based systems, because those systems are so complex and heterogenous. In FV has better PR than CDV … More Formal verification of really-complex systems