I started writing this blog mainly for feedback (see here), but did not know what to expect. Turns out I get good comments in the blog itself, but also (and unexpectedly) quite a few comments in private emails.
Private emails are good, but not ideal (because you, gentle reader, don’t get to see them, and you are bound to have a lot to say about them ;-). So I starting emailing people the following:
“Ideally, in one of my next posts, I’d like to quote your email (the part from ‘xxx’ to ‘yyy’) and attribute it you. Possible answers (all good) are: (1) Yes, (2) Yes but don’t mention my name, (3) Nope, let’s keep this private. Which do you prefer?”
Happily, so far most people answered (1). Here are two thoughtful pieces of feedback I got, which are a good prelude to one of the main questions this blog will investigate, namely:
So what are the requirements for a good big-system CDV toolset?
Further comments on the feedback below, or on the question above, are very welcome.
Sandeep Desai (of Expermeta) wrote:
“One recurring issue seems to be model definition across domains. Models of these larger systems almost certainly contain domain-specific sub-systems that need to be stitched together in a meaningful way that may or may not require a common metamodel and simulation methods. Even for a hospital model with a limited number of departments (e.g. nursing units, pharmacy, labs, imaging, surgery), the complexity grows very quickly. And, the entities being modeled are diverse: orders, supplies, staff/patients, spaces, medical records,… A unified modeling framework that’s capable, extensible, and friendly enough has been difficult to define. Is your work tackling the questions around multi-domains, fidelity, and interfaces? Do you have ideas regarding how larger multi-domain systems might be decomposed, and how to formalize the interfaces between the sub-systems, or between the sub-systems and the top-level system?
I like the goat-on-the-highway example because it illustrates well the burden placed on the modeler when defining both the DUT and the VE. System Engineers have a habit (actually a methodology) of considering only sunny-day scenarios at first. Unfortunately, this approach often trickles down in the design process, even though it’s the rainy days that tend to determine the performance range of the system”
And Stefan Birman (of Amiq) wrote:
“The real large scale systems involve not only predictable behavior of the agents but also fuzzy behaviors that can converge into a risky situation. There is quite a lot of literature about behavior emergence and chaotic systems. Many of these use agent based simulations which help understand emerging behaviors and eventually highlight risks.
The interfaces of the agents can be quite complex, but in the end they can be abstracted to a simple action the agent takes as a consequence of an event being triggered on the interface. For example the interface between a car and the road are the tires. A tire can slip, break, burst into flames or just continue working and as a consequence the car can slip uncontrollably, stop unexpectedly, slow down or continue at speed. The same is valid for an airplane tire or a bicycle tire. Another interface with the environment is the human being, and even we talk about a lot of potential, still the set of actions the driver can take on the car is limited to drive ahead, drive back, turn left or right, stop, drive chaotically, avoid collision, or open the door unexpectedly (which in fact is equivalent to another car trying to avoid an obstacle). Maybe creating such basic action libraries for the agents and then explore their combinations can highlight the risky situations.
Each agent could implement a “damage” or “distance-to-damage” function which computes the damages or potential of getting damaged due to interaction with other agents or environment. How these evaluate can highlight the level of risk an agent was subject to during the simulation.
In case a high risk scenario has been found a root cause analysis should be done to identify the chain of events that pushed the collective behavior into a risky situation.”.