Why read this?
“If it isn’t written down, it didn’t happen”
Decisions can be hard to make, but that effort is wasted if they are not implemented as intended and revisited as circumstances change, perhaps requiring change in the decision. This essay is a sketch of my experience and inquiry into all three steps of this process. It also provides two simple templates for managing on a smaller scale and some pointers to more sophisticated tools.
This does not cover decisions that have low impact and will be implemented immediately, by one person or a very small team, that will never need to be revisited. These decisions can be important and there are books that them, but I do not cover them here.
For more complex decisions, if you use a more formal process and tools to record the data and the reasoning behind the decision you will make better decisions and increase the chances that the decision will be implemented as intended. You will be able to take advantage of computer software which can assist with decision making or decision management.
You will also avoid second-guessing by people who aren’t aware of the trade-offs you had to make. That might well include you, two weeks later, as memory fades.
Circumstances will change or new data may show that some of the factors that went into the decision were incorrect. If well documented, it will be far easier to understand how the decision is affected and how or if it needs to be changed. If the decision making and management was partly aided by machine learning software, the decision can be revised more easily, and the impacts of the revision better understood.
This document is based on my experience as technical lead on projects in the $5-150 million range, either making decisions or providing well-supported recommendations to executives. It outlines some of the techniques I used and supplements them by research I have conducted into machine support for decision.
It is fairly obvious that decisions and requirements are very closely related. Decisions are taken in the light of requirements, including principles, legislation and regulations, stakeholder wants and needs and organisational goals and objectives.
Here, I present an additional relationship: by abstracting the idea of a Managed Item, and viewing both decisions and requirements as kinds of Managed Items, as well as several other related concepts, I gained in efficiency in decision making and even more so in decision management, defined as the process to keep decisions current as events invalidate the requirements, assumptions and evidence that supported the initial decision..
In the light of this understanding, the extensive literature on requirements elicitation and management can be understood as providing insight into similar problems with respect to decisions. Further, the work specialized to software requirements is largely relevant to non-software projects. The rigour needed in specifying software is often useful in non-software projects.
Decisions in Context
Decisions are made in a context of existing knowledge – data, models and expertise – and may be permanent decisions that cannot be undone, or may be subject to revision as they are executed and further data leads to their revision. The knowledge may be about the external context, for example geology and pipeline engineering or it may be about the project context, especially of the wants and needs of the various people who will be affected by the decision, for example the shareholders of the pipeline company or the residents of the people who live nearby.
Decisions may be made by individuals or by a group of individuals and may be made or at least aided by automated systems. There are techniques available for all of these, some of which will be outlined below.
Once a decision is identified, it must be documented and managed. Circumstances change, whether in the external context or the project context, and in both cases these may require a revisiting of decisions, which may have a cascade effect on other decisions.
Decisions and Requirements
A decision involves the selection between alternatives. The selected alternative must then be acted on, so it is natural to consider it then to be a requirement.
The other reason for linking the two is that there is considerable literature on requirements and requirements management, as well as some excellent supporting tools. In particular, the tools support traceability between requirements, an essential aspect of requirements management, supported by all tools and in my experience it is best to consider decisions as part of the requirements traceability structure, for reasons I will explain below.
The following diagram is an informal class diagram using Unified Modeling Language (UML). You don’t need to know UML as I only use a tiny part here and will explain.
The rectangles are classes – kinds of things under consideration – and the words below the line are attributes (or properties) of the classes, which translate into data we record about the classes. For example, for a Managed Item, we will record its ID, its Name and so on. These will be explained in more detail below.
The lines are called associations and there are three kinds on this diagram.
- Those with a solid diamond mean that the class at the end without the diamond is a part of the class at the other end. For example, a Requirement is part of a Requirement Group.
- Those with a solid arrow mean that the class at the end without the arrow is a subclass of – a special kind of – the class at the other end. For our purposes, that means that it shares all the attributes of its superclass, plus a few of its own. For example, a Decision is a kind of Managed Item, so it also has an ID, a Name and so on, just like any other Managed Item, and it also has a Selected Alternative and a Selection Rationale.
- The others are just generic associations. The arrow tells you which way to read the label, starting at the end without the arrow. So a Decision results in a Requirement.
Finally, the stick person represents an actor – a person (or computer) that we capture data about, such as their name and contact information.
Whether you are comfortable with this kind of graphic model or not, you may also find it useful to look at the set of Google Doc and Sheet templates I have provided for recording this data. The templates contain comments that serve as help text and complement the explanations given here. Each template has a cell in a table or sheet that records either one of the attributes or a link to one of the associated objects.
|Item log||Google sheet||Managed Item||Make a single spreadsheet of all the managed items, containing the common attribute. A list of all the items makes it easier to track the ones that need action. Also includes issues and risks|
|Decision||Google Doc||Decision||Make a doc for each decision with all the information needed to explain the decision. Put a link in the spreadsheet to the corresponding item|
|Stakeholder RASCI||Not provided, yet||Stakeholder and roles||A single sheet for all stakeholders|
I have managed sets of decisions on smaller projects with no more than a slightly more sophisticated set of tools implemented in Microsoft Office, with a few more error checks or prettier formatting. If you are going to have much more than a hundred related decisions, risks, issues and requirements, I would advise a more sophisticated requirements management tool. I am personally familiar with IBM’s DOORS but there are other products that seem like they will do the job. I have just begun to explore an Open Source, Eclipse-based tool called ReqIF Studio. All tools will need some configuration, so I recommend doing some work with the office-type tools until you are more familiar with the concepts and then attempting to configure a tool. Once you have more than a few items in a tool, it becomes difficult to reconfigure, so I usually only do that at the beginning of a project.
As seen in Figure 2, many of the classes on the diagram are types of Managed Item. This is not a standard notion in any of the decision management or requirements management texts and papers I have read. However, I have found that by abstracting the attributes and behaviour from both requirements and decisions, it is possible to apply many of the techniques found in the many works on software requirements to the making and management of decisions. I have also found that most of the techniques to do with software requirements and decisions are equally applicable to requirements and decisions in general. Even some of the more specialised areas such as making performance versus security trade-off decisions are fairly easily seen to have analogies in other areas.
The table below describes the attributes (data) common to the various types of Managed Items.
|ID||A permanent identifier, often numeric, usually generated by a tool, if you are using one. May incorporate a code for the type of item. For example DEC001, REQ021. Usually better not to change, once assigned.|
|Name||A short, meaningful name. For example “When to flood section?”|
|Description||A longer, more descriptive text, for example, “On what date should we allow water into each section of wetlands, to balance weed control with waterfowl migration needs”|
|Notes||More detailed notes on the item, such as records of the factors discussed at various meetings. May be further subdivided, if supported by the tool, into multiple notes from dated events or links to minutes|
|Priority||How important or urgent is this item? May be split into two attributes for more accurate by more time-consuming management. May be “High”, “Medium”, “Low” or numeric scale (e.g. 1-10)|
|Traced to||A link to one or more items on which this item depends. For example, if making a decision whether to go by train or bus, the decision may trace to a given requirement to minimize cost or to reduce carbon footprint, or to a previous decision always to use public transport rather than a private vehicle, if available.
Unfortunately, some sources use ‘trace to’ and ‘trace from’ in the opposite sense. It doesn’t really matter as long as you are consistent. The main reasons for maintaining traceability are:
This is one of the reasons I found it useful to abstract out the idea of Managed Item, because you can then trace between decisions, resulting requirements, implementation plans or designs, test cases, relevant evidence and more.
|Status||A selection from a list of potential status items, possibly with rules about changes. For example, a decision could be “requested”, “recommended”, “confirmed”, “implemented”, with a rule that it cannot be implemented before being confirmed|
|Status Date||The calendar data of the last status change|
|Due Date||The calendar date for the next action on this item.|
Since this is a sub-class of Managed Item, it also has all the attributes of Managed Item.
|Alternatives||A list of, or link to, the alternatives (to be) considered.|
|Selected Alternative||The identifier of the alternative actually selected.|
|Selection Rationale||Textual reason why the selected alternative was selected.|
For complex decisions, the traceability tree is perhaps even more important than for other forms of Managed Items. By documenting the requirements, evidence, risks and issues behind the decision, and how the decision makes the best trade-off, you will make better decisions, be better able to defend them when questioned (possibly by your future self) and better allow the implementers to interpret the decision.
One of the alternatives (to be) considered while making a decision.
|ID||Short numeric identifier (e.g. “Option 1”, recorded simply as “1”) or descriptor “Early option”, “safe option”|
|Description||Full textual description of the option|
|Assessment||Discussion of the arguments in favour of and against this option.|
|Additional Cost||Additional costs, possibly but not necessarily monetary, incurred by selecting this option|
|Benefit||The benefits that will accrue if this option is chosen, more than those from other options|
For important decisions, I generally use the equivalent of the Decision document template shown above, though for larger project projects (more than a few weeks) I used one of the Rational tools for requirements management. As shown in the UML diagram above, by abstracting the idea of a “managed item” from both Requirements and Decisions, most of the attributes of a Requirement are also shared by a decision, so most requirements management tools are easily extended just by configuration to keep track of all the context for the decision, and then can be used for future management.
In either case, I often draw a sketch of the traceability tree either for myself or for group decision making as many people find graphics helpful. Here is a simple example:
Group techniques include applying the individual techniques in working sessions. Brainstorming alternatives, evaluation criteria and influences with a whiteboard works well with small to medium-sized groups (2-20). For larger groups of stakeholders, there is software that allows and encourages everyone to participate by providing an anonymous, shared whiteboard and voting. I have worked with groups of 50, using the proprietary ThinkTank software and found it very useful. It was particularly valuable for working with some groups, who had indicated cultural issues with more open brainstorming. Individual interviews after the meetings with some who had expressed concerns that they would not feel comfortable speaking out in the larger group confirmed that they preferred interacting via the software. Other groups were less comfortable using software and preferred more face-to-face. So it is important to have a few side conversations beforehand with group members to find out how cultural traits can influence approaches to working together. There is a particular risk that some members of some groups do not feel comfortable calling out their opinions in an open meeting. “Silence is consent” is not always an appropriate slogan.
Bayesian networks are a handy diagramming technique that can be used for informal reasoning or can be formalized and used both for computer-based reasoning or for expert/computer communication. Here is a simple one.
Informally, the graph shows that the spring soil condition and genetic potential of your seeds influence the April plant mass, but not the other way around, and that the genetic potential has no influence over the summer rain. Note that the network in Figure 3 is not a Bayesian Network; the two types of network diagram are complementary and not directly related.
More formally, Bayesian Networks (BN) are graphs that show variables that are related via conditional probabilities. “Variables” refers to things you can measure or that you want to predict and “conditional probabilities” are statements about the probability of a variable having a certain value if you know the value of another. For example, you might know that “if spring soil condition is ‘good’, then April plant mass has 50% probability of being ‘high’”.
I had good success using informal diagrams like these before I found out about the more formal BN models, just to elicit expert opinions about relationships between factors relevant to decisions – I would stand in front of a whiteboard and draw with the help of the experts. It also helps with scope definition. For example, perhaps money could be spent improving the genetic quality of the seed, but we don’t diagram the factors resulting in that if they are out of our scope. Our scope may be limited to deciding whether we should spend money on irrigation and when, or buy better seed, but not how that better seed should be produced.
Unaided humans can’t usually go much deeper than this, but computers can analyse data which relates the variables and “learn” the graph structure and the associated conditional probability tables and distributions from the data. Where data is not sufficient, the computer software can take parts of the structure as given, by human experts, and learn other parts automatically. Another use of BN formalism is that, if you don’t have enough data, it can help you decide which data it would be most useful to get. Given different topologies of diagram, there are rules to decide how information flows or is blocked by other information. For example, in the example, assuming you know the various conditional probabilities from data from previous years, and you collect this year’s data on number of seeds, you can infer probabilities of plant mass and then of weight of seeds, but if you already have hard data on plant mass, then data on number seeds adds nothing further to estimates of weight of seeds.
I won’t go deeper into BNs here as there is better material than I could write, in books and Internet resources. There are a few mentioned below to get you started if you’re interested.
If you do know statistics, don’t be put off by the simplistic description above. I oversimplified. For example, “probabilities” in a full model will be probability distributions and their parameters. You may also wonder how it BNs compare with other modeling techniques, such as Generalised Linear Models (GLM). The answer, of course, is “it depends”, with computational tractability being one of the concerns, but here is one advocate’s answer:
“Bayesian networks having the advantage over GLM models that they can model complex and intermediate pathways of causality in a very visual and interactive manner to diagnose strengths and weaknesses in management systems and for exploring ‘what if’ scenarios.”
If you know (or want to know) R, for statistical programming, the book [Denis, Scutari 2014] describes using the package bnlearn for BNs. Whether you know R or not, the AgenaRisk software is much easier to use but more specialised. A licence comes free with the book [Fenton, Neil 2013] and it’s easier to do the exercises in the book in that language than to try to do in R, unless someone produces a package with better displays for bnlearn results. If they don’t, maybe I will, when I’ve learned enough).
There are other types of Managed Items that can usefully be connected to the decision tree. These include non-trivial implementation plans, designs and test cases. In the software development world, the designs and test cases are also traced by the requirements management (RM) tool and I see no reason why they should not be for other domains. I have used them for aspects such as organizational design because for most larger scale software projects, organizations that use or are affected by the software usually needs to change. The RM tool (which I am recommending you use as a Decision Management tool as well as for all Managed Items) contains only links to those “down stream” items. In the case of software design, some tools for test case and design artifacts integrate with the RM tools so that changes in any place update the other.
In any case, a common practice in software design is to develop test cases along with the requirements. A good decision should be sufficiently well articulated that it is easy to tell when it has been successfully implemented as intended. By documenting something analogous to test cases, specifically procedures to follow that will confirm correct implementation, the likelihood of successful implementation is increased. The “test cases” also serve to further clarify the intent of the decision.
Decision management is the practice of keeping decisions current as events change the inputs to the decision. Some decisions cannot be changed; once implemented, there is no going back. However, even apparently immutable decisions can change over time, for example in the current trend to removing dams that were once considered to be permanent fixtures.
If you maintain a formal decision tree and practice risk management, decision management is considerably simplified. A risk management plan should already contain triggers for evaluating whether a risk has materialized. In that case, the decision tree should show all the other Managed Items that need revisiting, including decisions. You may also note other events that affect one of your Managed Items. In that case, you will need to locate the item and then again use the traceability tree to find all the items that trace to the items directly affected.
This paper shows some techniques I have used successfully on projects with lifetimes between six months and five years and with budgets in the $5m to $150m range. The projects were performed while working for IBM corporation under contract to various Canadian Provincial and US State governments and so were usually performed under a blend of IBM’s proprietary Global Services Method and client methods. Since I was an experienced methodologist , paid to design methods for projects, I generally led a workshop of several days at the beginning of each major phase of a project to design a combined method appropriate to the scale and nature of the project, based on the various parties’ methods.
The more mathematical techniques are based on personal research since retiring. This document is part of my continuing attempt to combine the former practice with developing theory.
Bayesian Networks without Tears Eugene Charniak. AI Magazine, 1991
Introduction to Bayesian Decision Networks – presentations. No author names mentioned.
Denis, Scutari 2014, Réseaux bayésiens avec R published by EDP Sciences and simultaneously but far more expensively in English as Bayesian Networks with examples in R by Chapman & Hall Marco Scutari and Jean-Baptiste Denis
Fenton, Neil 2013, Risk Assessment and Decision Analysis with Bayesian Networks published by CRC Press. Norman Fenton and Martin Neil. It doesn’t actually say much specifically about Decision Analysis but is a good foundation in the requisite Bayesian Network techniques and easy to follow.
Weigers 2003, Software Requirements published by Microsoft Press. Probably the most used and cited book on this topic. Although the degree of rigour may be relaxed for projects other than software or engineering, most of the concepts apply to any larger scale effort expected to produce change.
Version: 0.1 – first draft with at least some content for all sections
Master version on Google Docs. This version may not always be current.
More on individual techniques – example of informal use of Bayesian Network