Summary: this post discusses the fact that different groups use different verification terminologies, explains why this is a problem, and suggests a common terminology page.
One problem with verification is that it consists of multiple “verification tribes” or v-tribes, with different terminology and habits.
There is the HW v-tribe, which I am coming from: We go to the DAC and DVCON conferences, know about CDV, expect people to pay real money for verification tools, and so on.
Then there is the Embedded v-tribe (which goes to the Embedded Systems Conference), the Security v-tribe (which goes to DEFCON), the Automotive v-tribe, the Defense v-tribe and so on.
Some of these tribes intersect (in fact, they may even inter-marry), but there are still a lot of misunderstandings between them.
Some conferences do try to bring some of the tribes together: Last DAC had a good motivational presentation about AV verification, and HVC explicitly tries to mix all tribes.
However, even in such cases, any specific speaker uses his [1] own terminology, secure in his knowledge that it is the only one possible.
This leads to the dreaded assumed terminology trap, where he is happily carrying on, while halfway through his presentation you are still doing pattern recognition on what he said so far, to understand what he is talking about:
“Wait, he just said ‘verification’ and he is from the embedded SW world. So he is talking about formal verification. Which means that 5 minutes ago, what he really meant was … Wait, scratch all that – we are going back to interpretation 1.5.2: he is actually talking about Elvis, and when he said ‘observed’, he meant in Memphis, Tennessee last year. Wait …”.
This is, of course, great fun. But it has been observed to take up to 43.6% of listeners’ time – time which could have been fruitfully used for what they really came for, e.g. reading emails.
So what can be done about that? One sometimes wishes to be able to click on the speaker to pause him, then click on his last words to jump to his definition of them. Barring that (and yes, you can take that excellent startup idea and run with it), perhaps a common terminology file could help.
So, as part of my community service, I decided to create one. You can help too, by adding comments at the bottom – I’ll update this page as time goes by.
Also, I plan to go back to my previous posts and hyperlink the first occurrence of each term to its definition on the terminology page.
A taxonomy of misunderstandings
Some terminology is unique to some tribes, and simply unknown by others. For instance, I think most HW v-tribe members don’t really know what HIL (Hardware-In-the-Loop) means in the context of verification.
In other cases, you think you know what they mean, but the finer nuances are lost on you. Take Monte-Carlo simulation: to the big-system v-tribe this mean roughly “simulation with a random component”. “OK, so they mean something like CDV” – says a HW v-tribe member. Well, not quite: for the big-system v-tribe “Monte-Carlo” has the connotation of investigating what happens typically, i.e. when you randomize using expected frequencies. CDV is more about finding bugs by going to corner cases. Note that the Wikipedia entry will not help here: it is of course correct, but too general.
Sometimes the terminology is just mismatched: For SW guys (sometimes), and Formal Verification guys (almost always), verification mean formal verification – they reserve the word testing for dynamic testing. For the HW v-tribe, “verification” means all bug-finding (so they have “verification engineers” and “Hardware Verification Languages”). “Testing” for them means production testing – the process of checking e.g. whether a specific copy of a chip is defective due to dust or something.
Finally, there is terminology which is common to all tribes, but can still confuse. I have dealt with some of those (e.g. correct by construction and root cause) in a previous post.
So take a look at that terminology page. Better yet – print it out and take it with you to the next conference. It has been proven that Origami makes attendees 32.4% more relaxed ;-).
Notes
[1] No gender preference implied
I’d like to thank Laurent Fournier, Avi Ziv and Amiram Yehudai for reviewing a previous version of this post, and Kerstin Eder for reviewing both this post and the whole terminology page.