Regardless of the decision-making maturity of an organization or the tools used, most portfolio prioritization decisions are comprised of four key components:
-
Defined criteria to assess the alternatives (e.g., projects or solutions)
-
Weights to establish relative criteria priorities
-
Objective and subjective data to assess each alternative
-
The resources and constraints associated with each alternative
With each element, key assumptions are made that determine the process and tools decision-makers will adopt. The integrity of the prioritization relies on the validity of those assumptions. For each component, a number of questions and techniques allow us to logically challenge the assumptions made and further improve our prioritization decision process.
Defined Criteria (Assumption: We have a set of defined criteria that represent the key drivers of the decision)
-
Are the criteria mutually exclusive and collectively exhaustive (MECE)? Are there other criteria that must be considered for this decision?
-
Do the criteria help to differentiate the alternatives in value?
-
Are the criteria definitions clear to all participants who will use them to assess the alternatives?
-
Is there sufficient data for each criterion to assess the alternatives?
Techniques used to answer these questions:
-
Conduct a “straw man” set of criteria and socialize it with key stakeholders early in the prioritization process to establish buy-in.
-
Compare the criteria to each other to test MECE.
-
Compare two alternatives, one which will be a clear priority and another which is expected to fall relatively low on the prioritized list. Consider their differences (e.g., why is one so much higher in priority) to tease out any additional criteria.
-
Ask how each criterion will be measured, and whether the data is available. If no objective data is available, but a subjective judgment can be made, consider who should be involved in making those judgments.
Establishing Priority Weights (Assumption: We already have our own way of weighing the criteria, or assume equal weights)
Key questions to consider:
-
Are the criteria equal with each other in relative importance? If so, there is no need to establish weights.
-
Is there a clear justification for how the weights were established, and do participants agree with the weights provided?
-
Will the weights be accurately reflected in the prioritized list of alternatives?
-
Techniques used to answer these questions:
-
Use the “pairwise comparison” process to compare each criterion against the others. Ask “Which is more important, A or B? And how much more important?”
-
Involve key participants who have a stake in the decision, but don’t force consensus. The average set of preferences will be a reflection of the priorities. These weights can be adjusted later in conducting “what-if” analysis.
-
With an unbiased facilitator, use the process for establishing criteria priorities as a means of obtaining early input and buy-in from targeted groups.
-
Test the alignment and logical consistency of the stakeholders on their established priorities.
Objective and Subjective Data to Assess each Alternative (Assumption: We have the information we need to make this decision. We collect the information. And we use the information to prioritize the alternatives with each other to determine which are best)
Key questions to consider:
-
Do the criteria scales capture the relative “best” and “worst” in the portfolio? Do they sufficiently differentiate projects in between?
-
Are the scale definitions easy to understand?
-
Do the scales strike a balance between objective and subjective?
-
Is there sufficient confidence in the data (either objective data or qualified judgments from stakeholders), and is there data for all of the alternatives?
-
Is the data for each of the alternatives easy to collect?
Techniques used to answer these questions:
-
Set normalized scales (0-1) to reconcile non-commiserate scales of measurement.
-
Test the criteria and scales on a few sample alternatives as a means for further refinement.
-
First rate one of the “best” projects and then one of the worst projects to set parameters. If you are not using the full scale, readjust.
-
Where possible, have the subject matter experts of each individual alternative rate the project against the criteria.
The resources and constraints associated with each alternative (Assumption: We either fund by priority or have our own way of determining the benefit-cost of each project and maximizing the value of our portfolio. We also accurately consider other constraints and dependencies, or don’t feel there is a need to)
Key questions to consider:
-
How much confidence do you have in the resource requirements?
-
In addition to budgetary resource constraints (e.g., FTEs or $), what other constraints are there? May include dependencies, minimum $ requirements, a certain sequencing of projects, etc.
-
Do you fund by priority? If so, what projects failed to make the cut line, but provide significant value relative to their costs?
Techniques used to answer these questions:
-
Establish your baseline resource constraint and budget requirements for each alternative.
-
Run a simple optimization scenario that seeks to maximize the value of the portfolio (looks for alternatives that will provide the biggest “bang” for the “buck”)
-
Layer in additional constraints, fixed projects, etc. to reflect the realities of the portfolio, and compare new scenarios.
-
Using the data, make final portfolio decisions with a clear justification.