Monitor Plateau Development

Monitor execution at the project and Plateau level; collect measurement and lessons learned to aid refinement of the IT Evolution Plan

.

Introduction Activities Roles and Responsibilities Artifacts Additional Resources

-  IT Evolution Plan - Project Plan(s) - Contracting Strategy Document - Contract Management Plans - Contractor and Procurement Documentation - Contractor Status Reports - Project and Plateau Measurements and Status In-Process Reviews Formal Reviews - Decision Papers - Evolution Management Team - Project Management Teams - Risk Analyst - Technical Architecture Team - Other Key Stakeholders

Introduction

Monitoring of progress against the IT Evolution Plan is done at two levels. The IT Management Team focuses on accomplishing the goals of the Plateaus by managing interproject dependencies and communication. The Project Management Team focuses on achieving their individual project goals within their allotted resources and constraints.

When it appears that overall IT evolution objectives or constraints will be compromised, the IT Evolution Management Team assesses impact on the overall IT Evolution Plan. Individual projects may be replanned, or the entire IT Evolution Plan for the current or later Plateaus can be revised.

TANF Example: A typical TANF example could be that the IT Evolution Plan needs to include monitoring the project and its progress within the plateau to determine that the critical TANF reporting requirements will be met. This includes ensuring that the systems are designed and developed to capture critical information. Consider Federal reporting for systems changes other than TANF if they are the result of significant changes in the TANF system. A more detailed example would be that TANF changes are being made in the system and as a result, changes will be required in the Child Support Enforcement IV-D application also. Monitoring should include the progress in both the development projects so that the dependencies will not become overwhelming risks to either project.

Top

Activities

These activities are performed within the Plateau or project context, as needed, to evaluate progress and forecast expected results. Consolidated guidelines are available to perform the following key activities:

1. In-Process Reviews. These interim reviews provide the IT Evolution Management Team periodic insight into individual project performance as well as the plateau as a whole. The appropriate frequency depends on the timeframe for a plateau and the need to react quickly to adjust the project goals, resources, or constraints (e.g., bi-monthly reporting periods). Accomplishments are tracked against the appropriate level of plan (project or plateau).

   At the IT project level:

   • The IT Project Management Team verifies (through a QA function) that the project participants are adhering to the project's defined process.
   • The team collects activity progress status and analyzes it to produce measures that support project-level corrective actions. Typical measures are:
     • Product size (e.g., pages, SLOC, objects, methods, screens, and tables)
     • Product defect (total or average density)
     • Product change data (volatility, such as change requests)
     • Project schedule status (schedule variance)
     • Project cost status (cost variance)
     • Project risk status (high, medium, low or technical, cost, or schedule risk)
     • Project quality assurance status (number of noncompliances, defect variance, conformance to requirements)
   • The team compares actual values of the measures against expected values (estimates) for the measurement timeframe (e.g., higher defect rates anticipated early in the project, less as the project continues).
   • The team establishes trends for the measures and analyzes them as predictors of future performance.

   At the Plateau level:

   • The IT Management Team addresses project accomplishments and issues and how they may impact other projects. They may reallocate resources across projects, as needed, to meet Plateau goals.
   • The team rolls up project-level measures and summarizes them to provide quick insight to project issues. The team evaluates the following types of accomplishments and issues to gauge their effect on dependent projects:
     • Overall cost performance data is summarized by rolling up work package status in cost accounts to the project level and for the plateau as a whole (some projects may be over budget, some under).
     • Overall schedule performance status data (number of dependencies met, based on a network chart that includes all projects in the plateau).
     • Quality of delivered products and expectations for future products, such as defect rates, or rework (this measure will be more important if the product of one project is the input to another project).
     • Interproject coordination issues, such as interfaces between projects or critical project paths (e.g., lack of timely delivery or lower than expected quality).
     • Project risk profile (increasing or decreasing).
   • The team may decide to roll up plateau status and use it as a basis to provide some or all of the measurements specified in the HS IT Strategic Plan.
   • The team communicates lessons learned on one project to other projects, as appropriate. This can be achieved by having Project Management Team members attend one another's project reviews.
   
   

2. Formal Reviews. Formal reviews are intended to provide detailed insight into project progress on an event-driven basis. These reviews are tied to major accomplishments in the development or acquisition of a product, such as after requirements are defined, a product completes development testing, a risk mitigation prototype is evaluated, or a vendor delivers a product. These reviews provide information for management decision making in regard to the approaches taken (buy, build, outsource) as well as the products produced. Direction to the projects (or contractors) is provided. Generally the decisions that will be made include:

   • Proceed as planned.
   • Proceed after adjusting the product or project goals or constraints (e.g., change in scope, resources, or schedule).
   • Terminate a project that is unlikely to satisfy its goals within current constraints.

   Some typical reviews include:

   • Contract Reviews

   Contract reviews examine the supplier relationship, such as management structure, required skills and knowledge, team communication, demonstrated contractor performance, risks, and change rates. The Evolution Management Team makes commitments to proceed with, modify, or terminate the relationship. Decisions may be made to reallocate resources, revise schedules, reassess risks, or revise supplier activities. These decisions are reflected in changes to the Contractor Plan. Renegotiation of contractor terms and conditions may occur as a result of these activities.

   • Product Reviews

   These reviews consider the technical merit of the developing product to ensure that specific product goals and success criteria are met. This may include a review of product quality records (e.g., verification activities, including peer reviews, test activities, demonstrations, or analysis). Verification and validation against the plateau goals may be performed.

   • Product Acceptance

   Formal acceptance of products is performed. Acceptance criteria is defined for each project's product, whether produced within the HS Agency IT Division or acquired from outside (e.g., via a contractor or vendor). Plans and procedures to accept the product are created and followed. Results of the acceptance testing are provided to the Evolution Management Team as a basis for their acceptance decision. The possibilities can be:

   • Accept as is
   • Conditionally accept with change
   • Reject
   • Transition Products to Use

     The Evolution Management Team will make explicit decisions to authorize deployment and use of the IT products that are developed or acquired. Deployment is in accordance with plans created and executed as part of Technology Deployment Projects. Operation and support is discussed in Technical Operations.

Top

Roles and Responsibilities

The key roles and their responsibilities are as follows:

• Evolution Management Team. These individuals have primary responsibility for oversight of the IT Evolution Plan, participating in the in-process or formal reviews and providing direction to the projects.
• IT Project Management Team. These individuals, specifically the IT Project Manager, have responsibility for monitoring their projects' progress and reporting this to the Evolution Management Team.
• Risk Analyst. This individual is responsible for reassessing progress against the risks.
• Technical Architecture Team. These individuals will provide insight into the Technical Architecture-related issues during reviews, such as adherence to the Technical Architecture or processing requests for waivers.
• Other Key Stakeholders. These individuals or groups have a vested interest in the establishment, approval, or oversight of how the evolutionary path will be achieved. This may IT management, contractors, support management, a Contract Manager, State procurement personnel, and the HS Agency Decision Makers, among others.

Top

Artifacts

The following information is used or produced by these activities. Templates, examples, and checklists for identifying and documenting items are available through the Additional Resources section at the end of this page.

• IT Evolution Plan. This is the main control against which activity and project project progress is measured. The portion for the current and possibly next Plateau is of primary use (the Plateau Plan). It is adjusted, as needed.
• IT Project Plan(s). There is a project plan for each of the projects in the current Plateau. This plan defines the processes and methods that will be used in this project, as well as the schedules and estimated costs for the project.
• Contracting Strategy Document. This is information is updated, as necessary.
• Contract Management Plans. This is used to control the contractor relationships and is updated, as necessary.
• Contractor and Procurement Documentation. This collection of legal and binding documentation is updated, as necessary.
• Contractor Status Reports. Information provided by the contractor is reviewed and reported to the appropriate management level.
• HS IT Estimate of the Situation. Progress at the Plateau level is validated against the EoS.
• HS IT Risk Management Plan. Progress against risks is tracked to the RMP; the RMP is adjusted, as needed.
• Project and Plateau Measurements and Status. Raw and compiled measures are collected, analyzed, and reported to the appropriate level (project, plateau, and HS Agency, as needed, for the HS IT Strategic Plan). This includes any discrepancies from plan and actions to remedy the discrepancies.
• Decision Papers. Major decisions (such as from formal reviews) are formally recorded and used as a basis for adjusting or showing commitment to plans and progress.

Top

Additional Resources

Items that can be used to perform these and other activities are consolidated in the Resources portion of the IT Planning and Management Guides. Resources specific to this activity are cataloged below.

Template: Outline of a Measurement Plan Outline for a measurement plan that could be used for either the IT Evolution Plan, a specific Plateau Plan, or a Project Plan. 02-01-02

Consolidated Guidance: Earned Value Methods Overview of the techniques for computing earned value, with strengths and weaknesses of each method. 02-01-02