Performance-Based Earned Value
Program Management Improvement Accountability Act
PMIAA Content
PMIAA includes Section 2, Project Management. It adds additional functions to the Office of
Management and Budget (OMB) Deputy Director for Management. The new requirements
include Program and Project Management (P/PM).
Sec. 2 Excerpts:
‘‘(A) adopt government-wide standards, policies, and guidelines for program and project
management for executive agencies;
‘‘(B) oversee implementation of program and project management for the standards, policies,
and guidelines established under subparagraph (A);
‘‘(C) chair the Program Management Policy Council established under section 1126(b);
‘‘(D) establish standards and policies for executive agencies, consistent with widely accepted
standards for program and project management planning and delivery;
‘‘(E) engage with the private sector to identify best practices in program and project
management that would improve Federal program and project management;
‘‘(H) establish a 5-year strategic plan for program and project management."
Proposal to Select "Project Management Body of Knowledge®" as the "widely accepted
standard":
The Project Management Institute (PMI) Project Management Body of Knowledge®
(PMBOK® Guide) contains an ANSI standard (ANSI/PMI 99-001-2013). PMBOK® Guide
should be considered to comply with the objectives of the PMIAA and to replace EIA-748,
the Earned Value Management System (EVMS) Standard. A rationale to discard EVMS was
first provided in my article in Defense AT&L, “Earned Value Management Acquisition Reform”
(Nov.-Dec. 2010). It is recommended that DoD consider the article, and the white paper
described below, "DoD Acquisition Reform: EVMS-lite to Program/Project Management,"
when developing the required standards, policies, and guidelines. PMBOK® Guide is
used by project management practitioners world-wide and will provide program and project
management transparency.
Pertinent excerpts from PMBOK® Guide are at the tab, "PMBOK Excerpts."
Excerpts from the 2010 article and a link to it follow:
1. In 2009, DoD reported that EVM, based on the EVMS Standard, no longer serves its
intended purpose.
2. Sen. Susan Collins stated that the GAO observed that contractor EVM reporting lacks
consistency and leads to inaccurate data and faulty application of the EVM metric. “In other
words, garbage in, garbage out.” Collins concluded that, “With improved EVM data quality,
both the government and the contractor will be able to improve program oversight, leading
to better acquisition outcomes.”
3. The time has come to ask whether DoD and other federal agencies should continue to rely
on ANSI-748 or should adopt the best practices of commercial companies that use EVM
voluntarily, not because of a contractual mandate.
4. Project management standards and best practices that are used by commercial companies
should be considered for acquisition reform.
5. Commercial information technology companies in India and South Korea use EVM
processes and best practices based primarily on the PMBOK® Guide and its focus on the
technical baseline and Technical Performance Measures (TPM). (See article, Performance-
based EV in Commercial IT Projects, at tab "Articles and Tutorial"
6. PMBOK® Guide practices include:
- Differentiating the product scope from the project (work) scope
- Establishing a requirements baseline
- Use of TPM to assess planned vs. actual technical performance.
White Paper, "Integrated Program Management Using EVM –
It’s Essential! - a Sequel," May 20, 2019
Pat Finneran, in his keynote speech at EVM World 2018, emphasized the difference between measurement/reporting and management; explaining how integrated program management (IPM) using EVM was essential. This sequel to his speech provides guidance to implement IPM by augmenting an organization’s EVM process with Systems Engineering (SE) standards and models and with the Project Management Institute (PMI) Guide to the Project Management Body of Knowledge (PMBOK® Guide).
Christle’s Vision and Finneran’s Message
The goal of IPM is not new. In 1999, Gary Christle, one of the founding fathers of EVM, stated his Vision and presented a Challenge:
· Vision: “The quality of a management system is determined not by the absence of defects, but by the presence of management value.”
· Challenge: “Integrate cost, schedule, technical performance, and risk management.”
In 2018, Pat Finneran broadened Christle’s vision by citing the product and the requirements. He made the following key points in his keynote speech:
· Make the product the boss...Focus on the product, the requirement, and make that the boss.
· If you don’t have good requirements flow down along the WBS…EVM is going to have gaps.
· Talk about the cost and schedule based on EVM data, quality, and technical performance…risks and risk mitigation. It was this structure that was key to success.
PMIAA P/PM Competencies (Per OMB memo, subj: PMIAA P/PM Competencies, April 5, 2019),
The U.S. Office of Personnel Management (OPM), in consultation with the OMB and the Program Management Policy Council, defined “P/PM competencies to select, assess, and train program and project management talent for the 21st century. The competencies identified will inform future work currently underway in support of OMB’s 5-year strategic plan for implementing the PMIAA (including standards for P/PM).”
The following technical competencies were included:
1. Quality Management - Knowledge of the principles, methods, and tools of quality assurance, quality control, and reliability used to ensure that a project, system, or product fulfills requirements and standards.
2. Requirements Management - Knowledge of the principles and methods to identify, solicit, analyze, specify, design, and manage requirements.
3. Risk Management - Knowledge of the principles, methods, and tools used for risk assessment and mitigation, including assessment of failures and their consequences.
4. Scope Management - Knowledge of the strategies, techniques, and processes used to plan, monitor, and control project scope; includes collecting requirements, defining scope, creating a work breakdown structure, validating scope, and controlling scope to ensure project deliverables meet requirements (i.e., features, functions).
Note: These technical competencies are absent from EIA-748.
White Paper, "Revise DoDI 5000.02 to Integrate Systems Engineering, Technical Performance, and Test and Evaluation with EVM, and to Exclude Qualified Software Development," July 4, 2019
Excerpts:
Revisions will correct misleading claims in Dodi 5000.02...to overcome the major shortcomings of Earned Value Management (EVM).
Add references to supporting DoD documents including the Systems Engineering Plan (SEP), Integrated Master Plan (IMP), Test and Evaluation Master Plan TEMP, and Defense Acquisition Guide (DAG).
Eliminate mandatory EVM compliance and reporting requirements for FPI and LRIP contracts.
Eliminate mandatory EVM compliance and reporting requirements for qualified, short-term, low-risk software development elements of EMD contracts
- decrease program costs
- accelerate incremental delivery of those software products.
For EMD programs, when EVM is required, program managers should included the SEP and the IMP in the Request For Proposal as compliance documents.
The recommendations are complementary to but independent of those that were provided in the white paper, “DoD Acquisition Reform: EVMS-lite to Program/Project Management,” Aug. 7, 2020. (below)
Letter to OMB Director Vought, Subj: Recommendations to Improve Program/Project Management and Achieve the President’s Management Agenda, 10/27/20
White Paper, "From EVMS lite to Program/Project Management, rev28" revised Oct. 10, 2021
Excerpts:
More than 20 years ago, the founding fathers of the Earned Value Management System
(EVMS) stated their visions for the pending EVMS Standard. That standard was developed
as a Voluntary Consensus Standard (VCS) to replace the DoD document, “Cost/Schedule
Control Systems Criteria,” which had been used since 1967 for capital acquisitions.
Their visions, stated below, have not been realized. A path to effective, integrated program
and project management (P/PM) should include changes to regulations and policy to require
that EVM be linked with systems engineering, the product scope (features and functions),
technical performance measurement (TPM), and risk management. The path should include elimination of the regulations that require compliance with the EVMS Standard, EIA-748-D,
in favor of internal management processes that are in accordance with the ANSI-accredited
standards for P/PM, those of the Project Management Institute (PMI).
Federal law, OMB policy, Office of Personnel Management (OPM) policy and recent DoD
acquisition reform initiatives signal that the federal government and DoD have started down
that path.
Not Widely Accepted
A worldwide survey of EVM users by the PMI, in 2010, disclosed that the private sector has largely ignored
EIA-748. When the use of EVM is voluntary and not a contractual mandate, only 17 percent of the
respondents used EVM based on EIA-748.
Seventy percent of respondents to the Grant Thornton 2016 Government Contractors Survey stated they
would not use EVMS if not required to do so. Twenty-eight percent reported having contracts that require
use of EVMS. Of those using EVMS, only 37 percent believe it to be a cost-effective management tool and
only 25 percent would adopt EVMS voluntarily.
TPMs and Risk Management Not Integrated
Little Insight and Less Management Value
Applicability to DoD
PMIAA gave a potential waiver to DoD by stating it is not applicable to DoD “to the extent that the
provisions…are substantially similar to or duplicative of…policy, guidance, or instruction of the
Department related to program management.’’ However, current DoD policy, guidance, and instruction
related to program management and EVM are not similar to or consistent with the most widely accepted
guide for P/PM, PMBOK® Guide.
A PM’s needs that are covered by the PMBOK® Guide but are not mentioned in EIA-748-D include the
technical or product baseline, requirements management and traceability, risk management, and project
procurement management.
PMBOK® Guide includes standards and principles that meet the needs of P/PM but are absent from
EIA-748-D (Table 1).
Table 1 - PMBOK® Guide Standards and Principles that are Absent from EIA-748-D
Product scope description: Documents the characteristics of the product that the project will be undertaken
to create. Progressively elaborates the characteristics of the product.
Product scope: The features and functions that characterize a product.
Requirements Documentation: Requirements baseline; unambiguous (measurable and testable), traceable,
complete, consistent, and acceptable to key stakeholders. Components include, functional requirements,
non-functional requirements, quality requirements, and acceptance criteria.
Requirements: Requirements become the foundation of the WBS. Cost, schedule, quality planning, and
procurement are all based on these requirements.
Requirements Management Plan: Include…product metrics that will be used.
WBS Dictionary: Includes quality requirements, acceptance criteria.
Scope Baseline: Includes product scope description, project deliverables, and defines product user
acceptance criteria.
Control Scope: The process of monitoring the status of the project and product scope and managing
changes to the scope baseline. Completion of the product scope is measured against the product
requirements.
Requirements Traceability Matrix: Includes requirements to project (including product) scope/WBS
objectives, product design, test strategy and test scenarios.
Conduct Risk Management: Including planning, identification, risk analysis, response planning, and
monitoring risk.
Risk Responses in Baselines:
Schedule baseline. Changes in the schedule baseline are incorporated in
response to approved changes in schedule estimates that may arise from agreed-upon risk responses.
Cost baseline. Changes in the cost baseline are incorporated in response to approved changes in cost
estimates that may arise from agreed-upon risk responses.
Project Procurement Management:
Project documents that can be considered as inputs to this process include:
· Requirements documentation may include…technical requirements the seller is required to satisfy
· Requirements traceability matrix…links product requirements from their origin to the deliverables that
satisfy them.
· Work Performance Data contains seller data on project status such as technical performance activities
that have started, are in progress, or have completed; and costs that have been incurred or committed.
· Work Performance Information includes information on how a seller is performing by comparing the
deliverables received, the technical performance achieved, and the costs incurred and accepted against
the SOW budget for the work performed.
EIA-748-D Is No Longer a VCS per OMB Circular A-119 Criteria
EIA-748-D is no longer a VCS because it is “otherwise impractical.” It fails to serve DoD’s procurement and program
needs. It is not prevalently used in the national and international marketplaces. Most importantly, EIA-748-D
does not address the state of knowledge and technology since it was last revised. It is still silent on the product
or technical baseline, risk management, and on tracing the requirements baseline to the schedule and work packages.
The Quality Gap has not been closed.
Finally, the National Defense Industrial Association Integrated Program Management Division (NDIA) is the author
and steward of EIA-748-D.
OMB Memo: Improving the Management of Federal Programs and Projects through Implementing the
PMIAA, June 25, 2018
This Memorandum establishes initial implementation guidance to begin a coordinated and Government-wide
approach to strengthen P/PM practices in Federal agencies and improve Government performance.
OPM/OMB Memo: PMIAA P/PM Competencies, April 5, 2019
On April 5, 2019, the Office of Personnel Management, in consultation with the OMB and the
Program Management Policy Council, issued a memo which defined “P/PM competencies to select, assess,
and train program and project management talent for the 21st century.” The memo included four technical competencies which are absent from EIA-748:
1. Quality Management - Knowledge of the principles, methods, and tools of quality assurance, quality
control, and reliability used to ensure that a project, system, or product fulfills requirements and standards.
2. Requirements Management - Knowledge of the principles and methods to identify, solicit, analyze,
specify, design, and manage requirements.
3. Risk Management - Knowledge of the principles, methods, and tools used for risk assessment and
mitigation, including assessment of failures and their consequences.
4. Scope Management - Knowledge of the strategies, techniques, and processes used to plan, monitor,
and control project scope; includes collecting requirements, defining scope, creating a work breakdown
structure, validating scope, and controlling scope to ensure project deliverables meet requirements
(i.e., features, functions).
The PMBOK® Guide Standards and Principles in Table 1 are consistent with OPM/OMB objectives.
EVMS-lite
The rationale for and implementing details of EVMS-lite were included in my letter to Chairman Thornberry,
11/17/13, Subj: Expanded NDAA Defense Acquisition Reform – EV.
Excerpt: “It is also recommended that DOD policy be revised to require contractor compliance with three (now four)
amended or tailored EVMS guidelines and to remove compliance with eight (now twelve) guidelines. In my
opinion, the cost savings by eliminating compliance with…guidelines will offset any cost increases that
may be incurred because of the tailored guidelines.”
Conclusion
DoD should discontinue use of EIA-748-D because it is impractical, ineffective, and has not been
reaffirmed. It fails to serve DoD’s procurement and program needs. It has failed to keep current with
changes in the state of knowledge and technology and is less useful than the PMBOK® Guide. The end
of the path should be a set of internal management processes and/or VCSs for P/PM, as required by the
PMIAA and OMB/OPM policy. PMBOK® Guide is the most widely accepted P/PM VCS and it components
should be included in the internal management processes.
The recommendations above are needed to fulfill the visions of EVM’s founders, to implement
the acquisition reforms and legislative intentions of senators and congressmen, to halt systemic findings
like those in the DoD Report, and to comply with the PMIAA.
White Paper, "THE DoD ADAPTIVE ACQUISITION FRAMEWORK
AND PROJECT MANAGEMENT STANDARDS," Sep. 13, 2021
Excerpts:
The DoD Adaptive Acquisition Framework (AAF) includes an overarching policy to “Employ Performance Based-Acquisition Strategies.” Per DoDD 5000.01 Defense Acquisition System (DAS), “Performance-based strategy” means a strategy that supports an acquisition approach structured around the results to be achieved as opposed to the manner by which the work is to be performed. This white paper provides guidance to employ that strategy when selecting standards for program and project management (P/PM).
This white paper augments the 2015 article in Defense Acquisition Magazine (formerly Defense AT&L), “A Contract Requirement Rule for Program Managers,” by mapping PMI clauses to corresponding AAF policies and to the current Defense Acquisition Guidebook (DAG).
A PM needs accurate, reliable EVM information from contractors that is based on measurements of product quality and progress towards meeting the product requirements. The PM has improved chances of success if contractors use practices from widely accepted P/PM standards and guides that cover risk, technical performance, technical baseline, and requirements traceability to the IMS and work packages.
Letter to Ms. Weichert (OMB) and Mr. Fahey (DoD), 7/26/18
Subj: Successful Implementation of PMIAA by all Agencies, including DoD
·