The
objective of Program Analysis is to provide the customer
with predictive and timely EVMS data analysis in accordance
with the customer’s needs. The PI/PST focuses
their analysis on the key performance measures including
cost variance, schedule variance, and variance at completion. The
analysis is an entire PST effort.
1. Risk
Planning
1.1. The CMO/PST (as applicable) reviews the contract,
along with modifications and customer guidance (i.e., MOAs and/or delegations),
to get a clear understanding of the EVM Program Analysis requirements, e.g. CPR,
C/SSR and CFSR. A MOA (or absence of one) does not eliminate CMO responsibilities
related to program analysis. The PST/EVMS Specialist determines if EVM Program
Analysis is required at the sub-tier supplier level and initiates input to the
Supporting Contract Administration (SCA) Letter of Delegation (LOD) as appropriate.
1.2.
The PST/EVMS Specialist evaluates the contract, MOA or
SCA letter of delegation for inappropriate or unclear
EVMS reporting and/or Program Analysis requirements,
e.g. Contract Data Requirement List (CDRL) distribution
requirement. Any recommendations or deficiencies
found requiring a contract modification are reported
through the ACO to the PCO. Recommended contractual
changes are documented. DD Form 1716, Contract
Data Package Recommendation and Deficiency Report can
be used for this purpose.
1.3.
Integrated Baseline Reviews (IBRs) are intended to provide
a mutual understanding of risks inherent in suppliers’ performance
plans. Although the PMO is responsible for conducting
the IBR, the EVMS Specialist and PST are encouraged to
participate. Participation supports the customer
and provides a better understanding of the program baseline,
integrated master schedule, SOW, WBS, earned value methods,
and EVMS reporting requirements. IBR
Guidebook can help provide an understanding
of the IBR process and goals for PST members and EVMS
Specialists.
1.4.
Minimum performance expectations for program analysis
risk planning are that key processes be identified in
RAMP or local CMO developed plans. Key processes
are those which, if not properly controlled, can adversely
affect cost, schedule, and/or technical performance. Key
processes related to EVMS Program Analysis can be identified
in terms of WBS elements, program work scope or system
key processes. The risk management plan includes
the following:
-
Identification
of each key process area
-
Approach
for selecting WBS elements and control accounts
-
Frequency,
intensity and schedule of analysis activities
2. Risk
Assessment
Earned
Value Management Systems Reviews Lessons Learned
(INFORMATION)
2.1.
In order to assess program risk, the PST focuses on the
key processes identified in risk planning to effectively
meet customer needs and address customer concerns. The
PI/PST assesses each identified EVMS key process using
the information in the Supplier
Risk Management guidebook process. The EVMS
Risk Matrix can be used to determine
the likelihood that a risk event may happen. Consequence
is determined by assessing the impact to contracts and
programs in terms of cost, schedule, and technical performance.
2.2.
Program Analysis key process risk ratings are supported
by data. For example, data may be collected from
cost, schedule, and technical performance trends and
variances, CAM interviews, product audits, process proofing,
system evaluations, and Government and supplier performance
data. A higher risk equates to more frequent involvement
and could require a more intense review of the supplier’s
data integrity and use of their EVMS data to manage the
program. Higher risk areas can also be those specifically
identified in the MOA/LOD, or as otherwise requested
by the PMO.
2.3.
The PI/PST is responsible for integrating EVMS data with
program information to continually assess and report
program risk based on that data, including reviews of
the WBS at the lowest appropriate level. When classifying
risk, performance history, process effectiveness, efficiency,
and sophistication for each key process is considered. The
PI/PST assigns an EVMS program analysis risk rating with
supporting rationale. A risk rating may direct
the PST to establish delegations with CMOs having oversight
responsibility of sub-tier suppliers. Specific
responsibilities are documented in the MOA or LOD.
2.4.
A low risk rating assigned to a supplier EVM system does
not necessarily mean that a given program utilizing that
EVMS is low risk as well. The PST performs a program
risk assessment by analyzing cost, schedule, and technical
performance. Program risk assessment may extend
to the sub-tier level and necessitate the issuance of
a delegation to the cognizant CMO.
2.5.
The minimum performance expectation is that risk assessment
be documented by key process area in RAMP or a local
CMO developed document.
3. Risk
Handling
The
supplier owns their EVM system, so they do the risk handling. The
CMO’s role in supplier risk handling is to influence
through risk monitoring. CMOs do the risk planning,
assessing, monitoring and documenting. Therefore,
CMOs ensure their surveillance results are well founded
and provide good reason for the supplier to act on them.
4. Risk
Monitoring
4.1.
Risk monitoring is conducted for all identified key processes. The
PI/PST maintains and updates a Program Risk Management
Plan which is an output from RAMP or a local-developed
CMO plan. The plan addresses key performance risk
elements and satisfies the terms and conditions of the
MOA or LOD.
4.2.
The PI/PST has primary responsibility for predictive
program analysis. The PST ensures integrated, cross-functional
program analysis to evaluate supplier cost, schedule,
and technical performance at the lowest appropriate WBS
level. EVM program analysis begins at contract
award and extends throughout the duration of the contract.
4.3.
It is extremely important that supplier EVMS reports
contain timely, accurate data which correctly indicates
the true cost, schedule, and technical performance of
the program. The PI/PST carefully reviews the supplier’s
initial submissions of contractual EVMS reports such
as the CPR, C/SSR, and CFSR in order to verify data integrity
and the report’s compliance to CDRL and DID requirements. This
verification of data integrity continues as necessary
so that a high degree of confidence is maintained.
4.4.
Data integrity review includes conducting data traces
from the CPR or C/SSR back through the supplier’s
EVMS to verify the accuracy of the EVMS data being submitted
to the customer. The review compares current CPR
or C/SSR data with the prior month report and between
the current month formats to ensure consistency.
4.5.
Interviews with CAMs are an essential part of ensuring
data integrity and of obtaining additional information
to conduct program analysis.
4.5.1.
The PI/PST assesses supplier earned value methods for appropriateness
and consistency of application.
4.5.2.
Verify the supplier’s management personnel are
using the EVMS to manage the program and identify problems/risks,
develop solutions and implement corrective action. This
would include the follow-up of corrective actions identified
in Format 5 of the CPR or C/SSR.
4.5.3.
To analyze schedule variance as reported in the CPR and
CSSR and assess its impact, PST members review the suppliers’ IMS
and its critical path.
4.5.4.
Assess the reasonableness of the CAM’s Latest Revised
Estimate (LRE) considering areas such as Technical Performance
Measures (TPMs), EV trend data, critical path, and mix
of resources. Results of this assessment are integrated
into the DCMA Independent Estimate-At-Completion (IEAC). The
assessment rationale is explained, documented, and reported
to the PI.
4.5.5.
Appendix
I of the EVMIG provides additional guidance for
PST members concerning CAM interviews.
4.6.
The PST reviews supplier EVMS reports and provides predictive
analysis to the customer through the PI as applicable. The
MOA/LOD with the customer/CMO indicates areas of emphasis
and type of analysis desired.
4.7.
Predictive Program Analysis is performed at the appropriate
WBS level. If program analysis is done only at
the reporting level, it could potentially mask significant
problems at the lower WBS levels. Program analysis
provides timely indications of actual or potential problems.
4.8.
Assessment of supplier’s EAC and submission of
a DCMA IEAC includes the integrated input from all PST
members. The IEAC rationale is explained, documented,
and reported to the customer.
4.9.
Assess and validate that known technical problems are
accurately reflected in the supplier’s variance
analysis and the planned corrective actions are described
in Format 5 of the CPR or C/SSR. This assessment
determines whether or not the supplier’s corrective
actions improve cost, schedule and/or technical performance. Perform
follow-up to ensure implementation of Format 5 corrective
action since CPR or C/SSR data is historical information/data.
4.10.
Analyze trends in key performance measures and indices
for elements that address the top cost and schedule variance
drivers (including comparison to TPMs and critical path),
variance at completion (VAC), schedule performance index,
and cost performance index. USAF AFMCPAM 65-501,
Guide to Analysis of Contractor Cost Data, provides the
PST with helpful guidance concerning analysis of supplier’s
EVMS data. https://www.afmc-mil.wpafb.af.mil/pdl/afmc/65afmc.htm
4.11.
A predictive assessment of future cost, schedule, and
technical performance should be accomplished. For
example, do current trends for key performance measures
and indices continue or change, and why?
4.12.
Perform an assessment of the supplier’s baseline
change control as reported in Format 3 if incorporated
in the CPR. This assessment ensures that baseline
changes are properly explained in Format 5 and are performed
in accordance with the supplier’s EVM system description.
4.13.
If the contract requires a CFSR, PI/PST should request
assistance from DCAA, through the ACO, in verifying,
validating, and tracing the data.
4.14.
The PI/PST should use wInsight© or
other available DCMA IT approved tools for performing
their program analysis. While suppliers and customers
may be using other software tools, output files from
these programs can frequently be used for input into
wInsight©. Import of data from other software
tools may be accomplished if Electronic Data Interchange
(EDI) ANSI X12 transaction sets 839 Project Cost Reporting
and 806 Project Schedule Reporting are specified in the
contract. If not already a contractual requirement,
the CMO requests that electronic EVMS data (e.g., CPR)
be made available. DD Form 1716, Contract Data
Package Recommendation and Deficiency Report, can be
used for this purpose. In the case of flow down of EVMS
requirements from the prime CMO to the supporting CMO,
wInsight© data files are provided as
requested.
4.15.
If program deficiencies are identified, the CMO/PI/PST/EVMS
Specialist follows their CMO procedures or references
the Supplier Risk Management Chapter for corrective action. CARs
issued to the supplier are maintained by the CMO and
tracked for trend analysis and implementation.
4.16.
Minimum performance expectations for risk monitoring
are the execution of Program Analysis Risk Management
plan, which includes:
-
Identification
of any deficiencies and trends
-
Assessments
of current and future impacts of non-mitigated risks
or unsuccessful corrective actions
-
Generating
predictive assessments and recommendations
-
Analysis
of variances and performance indices, monitoring
of UB/MR usage and control
-
Developing
IEAC, explain utilizing PST special knowledge (not
computer generated)
-
Communication
of results and program health to the customer
5. Risk
Documentation
5.1.
The PI/PST documents their program analysis results in
RAMP or local CMO developed plans, EDW, PST status report
to the customer, Program Status Charts (or in the Army
Acquisition Information Management (AIM) System, when
applicable).
5.2.
RAMP or local CMO developed plans contain program analysis
risk planning, risk assessment, and risk monitoring information
and address the RAMP narrative protocol guidelines.
5.3.
Detailed analysis documents are maintained by the CMO/PI/PST
in risk management files and contain all pertinent data
and information on program analysis. Program analysis
evaluations, results, discrepancies, and follow-up actions
are documented and imported into the EDW official contract
file. These files include any customer correspondence,
meeting minutes, and actions. The program file
is maintained until contract completion.
5.4.
The program status reports contain program analysis information
as specified in the MOA or LOD. A predictive assessment
on future cost, schedule, and technical performance is
included in the report. Whenever possible, recommendations
are included to mitigate program risk.
5.5.
EVMS program analysis is documented in accordance with
instructions for Program Status Charts (see Major Program
Support Chapter, 2.1.), as applicable.
5.6.
Minimum performance expectations for program analysis
risk documentation are that it be current, timely, and
include:
-
Program
Risk Management Plan
-
Deficiencies
identified during program surveillance:
-
Issued
via CAR when appropriate
-
Tracked
with current status
-
Addressed
via corrective action plan in place (or in process)
-
Followed-up
to ensure action has corrected deficiency (or
plan to do so)
-
Trended
for systemic issues
-
Program
predictive analysis results and program health is
reported to the customer and documented:
-
In
accordance with customer preferences
-
Identifying
areas of moderate or high risk with explanations
for each
-
Identifying
any deficiencies, corrective actions, status,
and independent assessment of corrective actions
-
Explaining
thoroughly significant program issues and concerns
-
Addressing
impact and potential impact to program successes,
challenges, and setbacks
|
