A simulation to evaluate the benefits of change prediction

(Login)

Back to Toolboxes

Change made to one part of a system tend to propagate between parts of that system according to the relationships between its parts. For instance, change made to the geometry of one component in a physical product may require adjacent components to also be changed, so they will still fit together. In turn, these changes may require yet more components to be changed. The propagation of change between related elements can also be observed in other types of system, such as processes, organisations, etc.

Motivation

Change management can be supported by tools which predict the impact of a change, helping to identify the likely knock-on impact. For instance, this might reveal high-risk connections that need to be carefully managed, might help with scheduling the execution of the change process, or might reveal that a given change is just too expensive to be worth implementing at all.

Various tools have been proposed to assist such prediction, but few (if any) have been evaluated in quantitative terms, either in an experimental or real-life setting.  We therefore developed a simulation model to evaluate the impact of different change prediction capabilities upon the efficiency of the redesign process.

The model showed that significant improvements can be made in rework scheduling if change propagation can be predicted. It thereby supports the case for change prediction tools.

The model

The model explores one assumed benefit of change prediction: namely, that it allows designers and managers to spot 'cycles' of change activity. In these cycles, part of a system is changed, which causes a knock-on change to another part of the system, which in turn causes a second-order knock-on change which impacts again on the originating part. (the same phenomenon can occur with larger cycles, in which third, fourth or higher-order propagations eventually lead back to the originating part). In other words, there is a chance that each part in a system will have to be changed multiple times as the change ripples through the system. This should be avoided wherever possible, because if the multiple changes to one part can be spotted in advance, they can be 'packaged' together and done together, often saving significant time and effort (because the combined change often requires less work than executing the changes separately - one way of looking at this is that the 'scopes' of the multiple changes are likely to overlap)

The model explores different prediction capabilities in terms of how much time saving can be achieved by detection and packaging of change cycles. Different prediction capabilities allow different sorts of cycle to be avoided. The following capabilities can be evaluated:

  1. No prediction capability - choose next change to implement randomly from the parts with change pending, at each step
  2. 'Average case' prediction capability - choose the next change to implement by identifying the part least likely to propagate the change to other parts according to its direct connections only. This is intended to broadly reflect the prediction capability of "probabilistic" change prediction tools, based on past experience of how changes propagate in the given system.
  3. 'Specific case' prediction capability (1,2 or 3 step look-ahead) - Assuming it is possible to look ahead in the propagation tree by the specified number of steps, choose the next change in order to minimise the number of parts which must be revisited within the scope of the look-ahead. (in other words, a 3-step look-ahead encompasses the capability of 1 and 2 step look-ahead). This is intended to represent the capability of tools that can predict change propagation to a certain depth for the specific case at hand.

To summarise: The specific case look-ahead gives the capability to avoid revisiting components if alternative routes through the process are available, which allow the first change to be 'held off' until another change to the same part appears, so the two can be bundled together. Simulation experiments reported in the paper mentioned below, showed that look-ahead gives significant, but diminishing returns when compared to 'average case' change prediction.

Download and install the toolbox

Get the  CPM Process Simulation Package, and EH101 CPM model. Then:

  1. Install the CPM Process module (copy the .jar file into the modules directory of the CAM directory)
  2. Start CAM and follow the instructions below.

Using the toolbox

  1. Open the EH101 model, or create an populate a new model of type 'CPM'. The default view is a DSM showing components in the system and the different connections between them. The width and height of boxes in the DSM cells show, respectively, the likelihood and impact of change propagating from the component represented by the DSM column to that represented by the DSM row.
  2. In the 'plugins' menu on the main CAM toolbar, select 'Change Propagation Process Simulation'
  3. In the dialog which appears, tick the  'Store processes in resultset'
  4. Click OK.
  5. An XY point cloud plot will be produced. Each point on the plot represents a single process. The X-axis represents the number of changes which were executed in the process (accounting for propagation). The Y-axis represents the number of steps required to complete the process. Therefore, a process which is significantly below the 'leading diagonal' of the XY plot (the line with gradient 1) is more efficient than a process with the same X value, but a greater Y-value.
  6. Right-click a point on the plot and select 'view data'. This brings up a window showing the change propagation tree for that process, and the sequence of steps which was followed to execute the rework. Selecting a step shows the state of the propagation tree at that step (ie. changes that have been executed, changes that are pending, and changes which have not been 'discovered' at the selected process step. As the process proceeds, downstream changes are progressively discovered and executed, until the whole tree has been executed.
  7. The results depend on the model, the 'initiating component' selected when the simulation is started, and the policies chosen for evaluation. Note that the 3-step look-ahead is extremely expensive to compute! (sometimes overnight for a few hundred simulation runs)

Click the images below to see full-size screenshots of 1) the EH101 DSM model, 2) the point cloud plot showing the results of evaluating one prediction capability on that model (showing that the capability represented by the black points gives a more efficient change process than the capability represented by the red points), and 3) the change propagation tree and process steps for one of the points on that plot.

         

(click screenshots to enlarge them)

 

See also

CPM toolbox

 

More information

For a detailed description of the model, its assumptions and an analysis of the results, see the paper: