In a previous post I discussed verification vs. validation, and claimed that:
This is a valuable distinction for a specific subsystem, but (in any complex system) this is a subsystem-relative concept, not an absolute one (as in “one man’s ceiling is another man’s floor”).
I was reminded by some readers that this view (“real systems have no top”), is, well, not completely true. Engineered systems do have a natural “top”.
One way to look at the world is as a “soup” of many systems, arranged in various aspect, as in figure 1 below:
The city does not have a purpose. I think even the transportation aspect of the city does not have a purpose (but perhaps you can claim it does, dunno). Anyway, I do not consider them “engineered systems” in the sense we are talking about.
However, inside any specific aspect, there are a bunch of interacting “things”, and some of them (like the things in pink and yellow in figure 1) are really engineered systems, each with a purpose, a requirement list, a paying customer, a “main authority”, a “bring-up schedule” and so on.
So while it is that true that most validation is simply “verification for the next level up”, for these top-level systems there is also a final “validation-only” step.
Here are two more observations regarding these top-level systems:
Some can be very lightweight
Some of these engineered systems can be very light-weight, consisting mainly of small additions and parameter modifications to existing things. And obviously, most of their validation has to do with how they’ll interact with the other systems already in the soup.
For instance, my friend Nadav’s parking proposal, whose purpose is to make parking in Tel-Aviv more enjoyable and efficient (and perhaps remove some traffic jams) is an engineered system. It will contain, at most (I am idealizing a bit):
- A set of changes to the pricing of some municipal parking lots
- A few signs, and perhaps a short TV commercial
- A few small changes to stoplight scheduling
For some, bug-finding is not very important
Some systems (like Nadav’s parking proposal) do use modeling and simulation (much like we do in verification) but the main aim is not bug-finding. Rather, it is to understand the system and try to optimize it.
In fact, if you look at this list from Wikipedia, you will find that many of the popular modeling-and-simulation packages (e.g. AnyLogic, Arena) are mainly used as “an extension of the spreadsheet”, i.e. for understanding, validating a hypothesis, optimization and so on.
This is very valuable stuff, but is distinct from bug-finding. Many of the modeling techniques, however, as well as the DUT vs. VE considerations, are similar to those used in bug-finding.
While I don’t disagree that some systems can be lightweight, I see a major open issue in establishing this fact. The endless combination of interactions between these lightweight systems can lead to a major road block.
The specific example that came to my mind is the small street in our neighborhood where they added a new hospital, then they made the street one way, but the firefighters had a station which required two way access. Then a lot of new restaurants and hotels opened around the hospital in the street and it became a mess.
All these independent lightweight systems not being planned and coordinated can become very important when they collide, but I don’t think we have a good way to predict if they will or will not.