Identify Measures
Establish the means to objectively determine progress against the goals.
| Introduction |
| Activities |
Introduction
These activities create the measurement system: identifying the specific measures to be taken, outlining the processes to collect and use this information, and establishing the initial measurement baseline and intermediate targets. This measurement system becomes the means to objectively evaluate progress against the IT Division's goals.
Activities
To establish the measurement system, the Strategy Team performs the following activities:
1. Identify Measures
Purpose:
This activity develops and documents the measures that are to be systematically collected, relating them to the IT Division's goals.
Description:
The Strategy Team reviews the goals, subgoals, and critical factors, and brainstorm measures that can be used. Approaches such as GQM, Practical Software Measurement, or Balanced Scorecard can be used (see the resources for descriptions). Some considerations for establishing this measurement framework are as follows:
- A good measure: (1) is reliable, telling how well goals are being met; (2) shows a trend; (3) is accepted by and meaningful to workers, managers, and other key stakeholders; (4) is simple, understandable, unambiguous, logical, repeatable, timely, and sensitive; and (5) allows for the economical collection of data. In essence, a good measure is useful for making decisions within or affecting the IT Division.
- Each measure should be well defined to be easily understood. A good definition includes (1) the strategic goal and the rationale for the measure; (2) the data source and collection frequency; (3) the calculation method (including equations and definitions of terms); and (5) the graphical representation of the measure (plus perhaps the reports in which it will appear).
- Define the categories of measures to be taken and used, such as those available with the Balanced Scorecard -- customer, internal, learning, and financial. These may cover the three aspects: the use of the IT (business), production of it products and services (development and deployment), and its operation (technical operations).
- These measures can represent achievements (goals) as well as the reduction in risks that may inhibit achieving the goals. The critical factors in the strategic plan may be the sources of these risks and may need to be carefully managed.
- Identify the source of the data; understand the sensitivity of the data being collected and who will be providing it.
- Keep the number of measures to a critical few, as needed, to keep focused on the most important activities.
- Measures can be quantitative or qualitative.
- Investigate measures that the IT Division currently uses to determine whether they can still be applied to the goals, retiring them if necessary.
Rational or assumptions made in regard to establishing the measurements should be captured and held within the Strategy Team as working notes. The rationale may need to be stated when providing insight into the collection or use of the measures.
2. Outline the Measurement Processes
Purpose:
This activity establishes how the measures will be collected, who is responsible for providing what data, and the frequency of collection. Issues such as sensitivity of data and how it will be consolidated and used across organizations are addressed. This approach ensures that the data is appropriately handled and those individuals providing it are adequately protected (e.g., county and State Agencies).
Description:
The Strategy Team establishes the process by which the measures will be collected, compiled, and used. Some guidelines on establishing this measurement collection and usage process follow:
- Many individuals within and external to the IT Division may have responsibility for providing some of these measures. These include the IT Evolution Management Team and the Architecture Team, as well as county Agencies or private contractors. Their interests should be represented when defining the processes they will later have to use.
- Consider whether special tools or techniques will be needed to obtain the measures. This may represent an investment for those collecting the measures.
- Consider the frequency of data collection and that the measures can be reported in a timely manner.
- Describe policies that will be followed for access to the raw and compiled measures, how the measures will be used. Statements to protect the individuals providing the data may be necessary. This is usually needed when the collected data is subjective, such as from surveys.
- Describe when and how adjustments to the measures may be initiated.
- Those that will use the measures have the authority to take action based on these measures.
3. Establish Baseline Measures and Intermediate Targets
Purpose:
Establish the values for the baseline set of measures (the "as is" position). Intermediate targets that may represent major decision points may be necessary. These are the intermediate plateaus, where the initiatives and their performance can be reassessed. Adjustments can be made to the plan (e.g., modifying subgoals) or the performance of the initiatives (e.g., changing the allocation of resources, time frames, technical approaches, etc).
Description:
The Strategy Team reviews the strategic plan and the measures to identify points where the strategy or its implementation could be reevaluated. The expected values of the measures for these plateaus describes a set of intermediate targets, i.e., milestones. Some guidelines to establish these targets follow:
- Use the past performance of the IT Division or others to help set reasonable expectations for the target values. Techniques such as Wide-band Delphi can help establish reasonable, objective targets.
- Note all assumptions, provide ranges, such as minimum, expected, or maximum values, rather than a specific value, when necessary.
- Identify triggers for taking actions when actual performance deviates significantly from that expected (e.g., over or under achievement).
- Describe the next plateau and detail the measures for it, summarizing those that occur later. The plateaus should be on the order of every 4 to 6 months. This provides guidance to the Architecture and IT Evolution Management Teams on what to address next and the overall context in which to address it.