Performance-Based Earned Value


​Program Management Improvement Accountability Act

PMIAA Content

PMIAA includes Section 2, Project Management. It adds additional functions to the Office of

Management and Budget (OMB) Deputy Director for Management. The new requirements

include Program and Project Management (P/PM). 


Sec. 2 Excerpts:

‘‘(A) adopt government-wide standards, policies, and guidelines for program and project

management for executive agencies;
‘‘(B) oversee implementation of program and project management for the standards, policies,

  and guidelines established under subparagraph (A);
‘‘(C) chair the Program Management Policy Council established under section 1126(b);
‘‘(D) establish standards and policies for executive agencies, consistent with widely accepted

standards for program and project management planning and delivery;
‘‘(E) engage with the private sector to identify best practices in program and project

management that would improve Federal program and project management;
‘‘(H) establish a 5-year strategic plan for program and project management."


Proposal to Select "Project Management Body of Knowledge®" as the "widely accepted 
standard":

The Project Management Institute (PMI) Project Management Body of Knowledge®

(PMBOK® Guide) contains an ANSI standard (ANSI/PMI 99-001-2013). PMBOK® Guide

should be considered to comply with the objectives of the PMIAA and to replace EIA-748,

the Earned Value Management System (EVMS) Standard. A rationale to discard EVMS was

first provided in my article in Defense AT&L, “Earned Value Management Acquisition Reform

(Nov.-Dec. 2010).  It is recommended that DoD consider the article, and the white paper

described below, "DoD Acquisition Reform: EVMS-lite to Program/Project Management,"

when developing the  required standards, policies, and guidelines. PMBOK® Guide is

used by project management practitioners world-wide and  will provide program and project

management transparency.


Pertinent excerpts from PMBOK® Guide are at the tab, "PMBOK Excerpts." 


Excerpts from the 2010 article and a link to it follow:

1. In 2009, DoD reported that EVM, based on the EVMS Standard, no longer serves its

intended purpose.

2. Sen. Susan Collins stated that the GAO observed that contractor EVM reporting lacks

consistency and leads to inaccurate data and faulty application of the EVM metric. “In other

words, garbage in, garbage out.” Collins concluded that, “With improved EVM data quality,

both the government and the contractor will be able to improve program oversight, leading

to better acquisition outcomes.”
3. The time has come to ask whether DoD and other federal agencies should continue to rely

on ANSI-748 or should adopt the best practices of commercial companies that use EVM

voluntarily, not because of a contractual mandate.
4. Project management standards and best practices that are used by commercial companies

should be considered for acquisition reform.
5. Commercial information technology companies in India and South Korea use EVM

processes and best practices based primarily on the PMBOK® Guide and its focus on the

technical baseline and Technical Performance Measures (TPM). (See article, Performance-

based EV in Commercial IT Projects, at tab "Articles and Tutorial"
6. PMBOK® Guide practices include:

  1. Differentiating the product scope from the project (work) scope
  2. Establishing a requirements baseline
  3. Use of TPM to assess planned vs. actual technical performance.


White Paper, "Integrated Program Management Using EVM –

It’s Essential! - a Sequel," May 20, 2019

Pat Finneran, in his keynote speech at EVM World 2018, emphasized the difference between measurement/reporting and management; explaining how integrated program management (IPM) using EVM was essential.  This sequel to his speech provides guidance to implement IPM by augmenting an organization’s EVM process with Systems Engineering (SE) standards and models and with the Project Management Institute (PMI) Guide to the Project Management Body of Knowledge (PMBOK® Guide). 

Christle’s Vision and Finneran’s Message
The goal of IPM is not new. In 1999, Gary Christle, one of the founding fathers of EVM, stated his Vision and presented a Challenge:
·  Vision: “The quality of a management system is determined not by the absence of defects, but by the presence of management value.”
·   Challenge: “Integrate cost, schedule, technical performance, and risk management.”


In 2018, Pat Finneran broadened Christle’s vision by citing the product and the requirements. He made the following key points in his keynote speech:
· Make the product the boss...Focus on the product, the requirement, and make that the boss.
· If you don’t have good requirements flow down along the WBS…EVM is going to have gaps.
· Talk about the cost and schedule based on EVM data, quality, and technical performance…risks and risk mitigation. It was this structure that was key to success.


PMIAA P/PM Competencies (Per OMB memo, subj: PMIAA P/PM Competencies, April 5, 2019),
The U.S. Office of Personnel Management (OPM), in consultation with the OMB and the Program Management Policy Council, defined “P/PM competencies to select, assess, and train program and project management talent for the 21st century. The competencies identified will inform future work currently underway in support of OMB’s 5-year strategic plan for implementing the PMIAA (including standards for P/PM).” 
 The following technical competencies were included:

 1. Quality Management - Knowledge of the principles, methods, and tools of quality assurance, quality control, and reliability used to ensure that a project, system, or product fulfills requirements and standards.

 2. Requirements Management - Knowledge of the principles and methods to identify, solicit, analyze, specify, design, and manage requirements.  

 3. Risk Management - Knowledge of the principles, methods, and tools used for risk assessment and mitigation, including assessment of failures and their consequences.  
 4. Scope Management - Knowledge of the strategies, techniques, and processes used to plan, monitor, and control project scope; includes collecting requirements, defining scope, creating a work breakdown structure, validating scope, and controlling scope to ensure project deliverables meet requirements (i.e., features, functions).


Note: These technical competencies are absent from EIA-748. 


White Paper, "Revise DoDI 5000.02 to Integrate Systems Engineering, Technical Performance, and Test and Evaluation with EVM, and to Exclude Qualified Software Development," July 4, 2019
Excerpts:

Revisions will correct misleading claims in Dodi 5000.02...to overcome the major shortcomings of Earned Value Management (EVM).

Add references to supporting DoD documents including the Systems Engineering Plan (SEP), Integrated Master Plan (IMP), Test and Evaluation Master Plan TEMP, and Defense Acquisition Guide (DAG).

Eliminate mandatory EVM compliance and reporting requirements for FPI and LRIP contracts.

Eliminate mandatory EVM compliance and reporting requirements for qualified, short-term, low-risk software development elements of EMD contracts

  • decrease program costs
  • accelerate incremental delivery of those software products.

For EMD programs, when EVM is required, program managers should included the SEP and the IMP in the Request For Proposal as compliance documents.

The recommendations are complementary to but independent of those that were provided in the white paper, “DoD Acquisition Reform: EVMS-lite to Program/Project Management,” Aug. 7, 2020. (below)


Letter to OMB Director Vought, Subj: Recommendations to Improve Program/Project Management and Achieve the President’s Management Agenda, 10/27/20

 

White Paper, "From EVMS lite to Program/Project Management, rev28" revised Oct. 10, 2021

Excerpts:

More than 20 years ago, the founding fathers of the Earned Value Management System

(EVMS) stated their visions for the pending EVMS Standard. That standard was developed

as a Voluntary Consensus Standard (VCS) to replace the DoD document, “Cost/Schedule

Control Systems Criteria,” which had been used since 1967 for capital acquisitions.

Their visions, stated below, have not been realized. A path to effective, integrated program

and project management (P/PM) should include changes to regulations and policy to require

that EVM be linked with systems engineering, the product scope (features and functions),

technical performance measurement (TPM), and risk management. The path should include elimination of the regulations that require  compliance with the EVMS Standard, EIA-748-D,

in favor of internal management processes  that are  in accordance with the ANSI-accredited

standards for P/PM,  those of the Project Management Institute (PMI).


Federal law, OMB policy, Office of Personnel Management (OPM) policy and recent DoD

acquisition reform initiatives signal that the federal government and DoD have started down

that path.

Not Widely Accepted
A worldwide survey of EVM users by the PMI, in 2010, disclosed that the private sector has largely ignored

EIA-748. When the use of EVM is voluntary and not a contractual mandate, only 17 percent of the

respondents used EVM based on EIA-748.

Seventy percent of respondents to the Grant Thornton 2016 Government Contractors Survey stated they

would not use EVMS if not required to do so. Twenty-eight percent reported having contracts that require

use of EVMS. Of those using EVMS, only 37 percent believe it to be a cost-effective management tool and

only 25 percent would adopt EVMS voluntarily.


TPMs and Risk Management Not Integrated


Little Insight and Less Management Value


Applicability to DoD
PMIAA gave a potential waiver to DoD by stating it is not applicable to DoD “to the extent that the

provisions…are substantially similar to or duplicative of…policy, guidance, or instruction of the

Department related to program management.’’ However, current DoD policy, guidance, and instruction

related to program management and EVM are not similar to or consistent with the most widely accepted

guide for P/PM, PMBOK® Guide.

A PM’s needs that are covered by the PMBOK® Guide but are not mentioned in EIA-748-D include the

technical or product baseline, requirements management and traceability, risk management, and project

procurement management.
PMBOK® Guide includes standards and principles that meet the needs of P/PM but are absent from

EIA-748-D (Table 1). 

Table 1 - PMBOK® Guide Standards and Principles that are Absent from EIA-748-D

Product scope description: Documents the characteristics of the product that the project will be undertaken

to create. Progressively elaborates the characteristics of the product.

Product scope: The features and functions that characterize a product.

Requirements Documentation: Requirements baseline; unambiguous (measurable and testable), traceable,

complete, consistent, and acceptable to key stakeholders. Components include, functional requirements,

non-functional requirements, quality requirements, and acceptance criteria.

Requirements: Requirements become the foundation of the WBS. Cost, schedule, quality planning, and

procurement are all based on these requirements.

Requirements Management Plan: Include…product metrics that will be used.

WBS Dictionary: Includes quality requirements, acceptance criteria.
Scope Baseline: Includes product scope description, project deliverables, and defines product user

acceptance criteria.
Control Scope: The process of monitoring the status of the project and product scope and managing

changes to the scope baseline. Completion of the product scope is measured against the product

requirements.

Requirements Traceability Matrix: Includes requirements to project (including product) scope/WBS

objectives, product design, test strategy and test scenarios.
Conduct Risk Management: Including planning, identification, risk analysis, response planning, and

monitoring risk.
Risk Responses in Baselines:

    Schedule baseline. Changes in the schedule baseline are incorporated in

    response to approved changes in schedule estimates that may arise from agreed-upon risk responses.
    Cost baseline. Changes in the cost baseline are incorporated in response to approved changes in cost

    estimates that may arise from agreed-upon risk responses.

Project Procurement Management:
Project documents that can be considered as inputs to this process include:
  · Requirements documentation may include…technical requirements the seller is required to satisfy
  · Requirements traceability matrix…links product requirements from their origin to the deliverables that

  satisfy them.
  · Work Performance Data contains seller data on project status such as technical performance activities

    that have started, are in progress, or have completed; and costs that have been incurred or committed.

  · Work Performance Information includes information on how a seller is performing by comparing the

    deliverables received, the technical performance achieved, and the costs incurred and accepted against

    the SOW budget for the work performed.

EIA-748-D Is No Longer a VCS per OMB Circular A-119 Criteria

EIA-748-D is no longer a VCS because it is “otherwise impractical.” It fails to serve DoD’s procurement and program

needs. It is not prevalently used in the national and international marketplaces. Most importantly, EIA-748-D

does not address the state of knowledge and technology since it was last revised. It is still silent on the product

or technical baseline, risk management, and on tracing the requirements baseline to the schedule and work packages.

The Quality Gap has not been closed.

Finally, the National Defense Industrial Association Integrated Program Management Division (NDIA)  is the author

and steward of EIA-748-D.  

OMB Memo: Improving the Management of Federal Programs and Projects through Implementing the

 PMIAA, June 25, 2018

This Memorandum establishes initial implementation guidance to begin a coordinated and Government-wide

approach to strengthen P/PM practices in Federal agencies and improve Government performance.

OPM/OMB Memo: PMIAA P/PM Competencies, April 5, 2019
On April 5, 2019, the Office of Personnel Management, in consultation with the OMB and the

Program Management Policy Council, issued a memo which defined “P/PM competencies to select, assess,

and train program and project management talent for the 21st century.” The memo included four technical competencies which are absent from EIA-748:
1. Quality Management - Knowledge of the principles, methods, and tools of quality assurance, quality

control, and reliability used to ensure that a project, system, or product fulfills requirements and standards.
2. Requirements Management - Knowledge of the principles and methods to identify, solicit, analyze,

specify, design, and manage requirements. 
3. Risk Management - Knowledge of the principles, methods, and tools used for risk assessment and

mitigation, including assessment of failures and their consequences.
4. Scope Management - Knowledge of the strategies, techniques, and processes used to plan, monitor,

and control project scope; includes collecting requirements, defining scope, creating a work breakdown

structure, validating scope, and controlling scope to ensure project deliverables meet requirements

(i.e., features, functions).

The PMBOK® Guide Standards and Principles in Table 1 are consistent with OPM/OMB objectives.


EVMS-lite

The rationale for and implementing details of EVMS-lite were included in my letter to Chairman Thornberry,

11/17/13, Subj: Expanded NDAA Defense Acquisition Reform – EV.
Excerpt: “It is also recommended that DOD policy be revised to require contractor compliance with three (now four)

amended or tailored EVMS guidelines and to remove compliance with eight (now twelve) guidelines. In my

opinion, the cost savings by eliminating compliance with…guidelines will offset any cost increases that

may be incurred because of the tailored guidelines.”


Conclusion
DoD should discontinue use of EIA-748-D because it is impractical, ineffective, and has not been

reaffirmed. It fails to serve DoD’s procurement and program needs. It has failed to keep current with

changes in the state of knowledge and technology and is less useful than the PMBOK® Guide. The end

of the path should be a set of internal management processes and/or VCSs for P/PM, as required by the

PMIAA and OMB/OPM policy. PMBOK® Guide is the most widely accepted P/PM VCS and it components

should be included in the internal management processes.


The recommendations above are needed to fulfill the visions of EVM’s founders, to implement

the acquisition reforms and legislative intentions of senators and congressmen, to halt systemic findings

like those in the DoD Report, and to comply with the PMIAA.


White Paper, "THE DoD ADAPTIVE ACQUISITION FRAMEWORK
AND PROJECT MANAGEMENT STANDARDS," Sep. 13, 2021

Excerpts:

The DoD Adaptive Acquisition Framework (AAF) includes an overarching policy to “Employ Performance Based-Acquisition Strategies.” Per DoDD 5000.01 Defense Acquisition System (DAS), “Performance-based strategy” means a strategy that supports an acquisition approach structured around the results to be achieved as opposed to the manner by which the work is to be performed. This white paper provides guidance to employ that strategy when selecting standards for program and project management (P/PM).

This white paper augments the 2015 article in Defense Acquisition Magazine (formerly Defense AT&L), “A Contract Requirement Rule for Program Managers,” by mapping PMI clauses to corresponding AAF policies and to the current Defense Acquisition Guidebook (DAG).


​A PM needs accurate, reliable EVM information from contractors that is based on measurements of product quality and progress towards meeting the product requirements. The PM has improved chances of success if contractors use practices from widely accepted P/PM standards and guides that cover risk, technical performance, technical baseline, and requirements traceability to the IMS and work packages.

Letter to Ms. Weichert (OMB) and Mr. Fahey (DoD), 7/26/18
Subj: Successful Implementation of PMIAA by all Agencies, including DoD


Letter to OMB Director Vought, Subj: Recommendations to Improve Program/Project Management and Achieve the President’s Management Agenda, 10/27/20 




·