[LW linkpost is here] Introduction and epistemic status: This is the first post in a planned series, “Alignment as a verification problem”. I co-originated coverage-driven verification (CDV), which became the standard methodology for chip verification and is heavily used in AV safety. Back in 2015 I wrote that verifying “Friendly AI” would be our biggest … More When is misalignment just a bug?
Summary: This post talks about some implications of the Tesla autopilot crash, claims that some autonomous vehicle fatalities are unavoidable, and suggests some ways to minimize them. By now you have probably heard about the Tesla fatal autopilot accident, causing the tragic death of a Tesla enthusiast. There is a lot of discussion about what … More The Tesla crash, Tsunamis and spec errors
Most bugs which kill people (and projects) are spec / requirements bugs, not implementation bugs. This could be a missing top-level requirement, a missing implication of some requirement on a lower-level spec, or some emergent behavior which was never spec’d. This post will explore the problem of spec bugs, and discuss possible solutions. Note that … More It’s the spec bugs that kill you
A companion post talks about spec bugs and ways to avoid them. Spec / requirement bugs (as opposed to implementation bugs) occur when the spec for a subsystem does not take into account some higher-level, full-system requirements. Spec bugs are pretty important, so here I’ll talk about some common verification-related concepts, and (my interpretation of) … More Terminology related to spec bugs