Performance Reports (DPRs) provide a mechanism through which federal
departments and agencies are held to account by Parliament. The financial and
non-financial performance information in DPRs is intended to support
parliamentary consideration of spending plans and the approval of annual
appropriations in the Main Estimates. DPRs also support government
accountability more generally since they show how departmental activities make
a difference to citizens. They provide Canadians with key information about how
departments and agencies performed against the expected results and plans set
out in their Reports on Plans and Priorities (RPPs).
Improving the overall quality of DPRs continues to be a priority for
the Government. In 2007, the Treasury Board Secretariat implemented
a strategy to improve the content and form of these documents. The strategy was
developed in response to comments made by parliamentarians and the Office of
the Auditor General (OAG), and was designed to ensure a more concise,
results-focused approach to reporting. As part of the strategy, the Secretariat
augmented its annual outreach efforts with federal departments to ensure
adherence to the new approach. The Secretariat also began producing a Good
Practices Handbook for performance reporting, a key reference guide for
departments and agencies in developing a balanced and credible picture of
Treasury Board Secretariat assesses DPRs annually against commonly accepted
public reporting principles through the Management Accountability Framework
(MAF) process. Since 2007, MAF results have demonstrated steady progress in the
quality of departmental reporting.1 Departments and agencies have
indicated that the new concise, results-focused approach has allowed them to
become more effective in telling their performance stories. Library of
Parliament researchers, Committee clerks and the OAG have indicated that the
new approach is a step in the right direction.
Secretariat recently commissioned the CCAF-FCVI Inc. (formerly known as the
Canadian Comprehensive Auditing Foundation) to conduct a study of performance
reporting practices across a variety of jurisdictions. The study (released in
March 2010) used the following criteria: reliability, validity, comparability,
consistency, user-friendly reporting (understandability), relevance and
innovation. It ranked Canada second of the five reporting jurisdictions
examined, which included the United Kingdom, the United States, Australia and New Zealand.
September 20, 2010, the Standing Committee on Public Accounts tabled Report 15
entitled Selected Departmental Performance Reports for 2008-2009 –
Department of Industry, Department of Transport. As required, the
Government has developed a comprehensive response to the seven recommendations
made in Report 15. This response was developed collaboratively by the Treasury
Board Secretariat, Industry Canada, Transport Canada and the Privy Council
That Industry Canada and Transport Canada set clear and concrete performance
targets that are directly related to the expected performance of their programs
That Industry Canada and Transport Canada clearly demonstrate in their
Departmental Performance Reports the link between departmental activities,
expected results, and actual performance.
recommendations are accepted.
The Treasury Board Secretariat’s Policy on Management, Resources
and Results Structures (MRRS) supports the development of a common,
government-wide approach to the collection, management, and reporting of
financial information and non-financial information on program objectives,
performance and results. Since the Policy on MRRS’s implementation in 2005, departments and
agencies have worked with the Secretariat to develop and maintain a current
inventory of their programs, demonstrate how these programs are aligned to each
other and to strategic outcomes through Program Activity Architectures (PAAs),
provide program expenditures according to the PAA, and track and report program
performance against concrete performance targets according to this structure
through Performance Measurement Frameworks (PMFs). Departments and agencies
continue to improve and stabilize their PAAs and PMFs as a consistent basis for
reporting performance in their DPRs. Over time, this will facilitate trend
analysis and benchmarking.
Canada and Transport Canada recognize the importance of using clear
performance measures, identifying targets and reporting performance against
expected results in their DPRs, as defined by the Policy on MRRS. However, as
with other organizations, their PAAs and PMFs continue to mature and stabilize.
Furthermore, some departments and agencies have experienced delays in complying
with certain aspects of the policy – for example, in defining outcomes that can
reasonably be attributed to a particular program. The Secretariat continues to
provide leadership, guidance and outreach to departments and agencies on MRRS
implementation to support departments and agencies in addressing these
summary of improvements made by Industry Canada and Transport Canada in this area since 2008-09, as well as plans for future progress, is provided below.
Canada’s 2009-10 RPP was the department’s first report to contain targets for
every strategic outcome and the outcome of every program activity. The
corresponding 2009-10 DPR reported on the degree to which these targets were
met. As an example of how Industry Canada set clear and concrete targets
directly related to expected results, the program activity “Competition Law
Enforcement and Advocacy” had an expected result of “Competitive markets and
informed consumer choice”. One of the supporting performance indicators used
was the “Percentage of economy subject to market forces”. The target identified
in the 2009-10 RPP was to “Increase or maintain current percentage
(approximately 82% of GDP)” and the performance status reported in the 2009-10
DPR was “Met all” with an actual result of 82%.
the 2009-10 DPR, Industry Canada made increasing use of performance summary
text and explanatory notes to make linkages between expected results and
performance more explicit. It should also be noted that Industry Canada
continues to endeavour to provide explanations to account for any significant
variances in performance.
Canada has implemented a new review process of its Performance Management
Framework (PMF) that will allow the department to better understand the
linkages between elements such as departmental activities, actual performance,
expected results and targets. This process is expected to form the basis for
ongoing improvements. The review process involves completion of templates which
require that the reason for choosing a specific target be articulated. As well,
the templates used in this process result in an explicit understanding of
variances in indicator results as well as the methodology used to measure the
indicator. This and other information required for the templates will lead to a
better understanding in the department of the strengths and weaknesses of each
element, as well as of the linkages between elements and of other external
factors that may impact results.
2007-08, Transport Canada’s PAA was redesigned. The adoption of a clear,
mandate-driven PAA has facilitated the work on the department’s PMF and, as
such, will assist with the adoption of a set of clear and concrete performance
indicators and targets.
redesign of the PAA, Transport Canada has made ongoing improvements to its PMF
to address the requirements of the Policy on MRRS, the feedback received from
the Treasury Board Secretariat through the MAF process, and the recommendations
of various internal and external reviews. In December 2009, Transport Canada undertook an extensive review of the departmental strategic outcome level and
program activity level performance indicators. This exercise resulted in more
meaningful, specific and measurable measures for the department.2 Ongoing
refinements to the performance indicators at lower levels of the PAA will
continue in 2010-11.3
Canada’s Enterprise Performance Framework (EPF), which is expected to
be completed in 2011-12, provides a means to standardize the approach for
performance management (i.e., performance indicators, performance targets,
performance accountability and performance monitoring and reporting) across the
organization. It addresses both internal and external planning and performance
management requirements of Transport Canada in a consistent manner to ensure
the coordinated and effective use of performance information to achieve
expected results. The resulting performance plan will address each facet of
management that is typically employed in business planning (Plan/Program
Management, Risk Management, Project Management and Resource Management).
Industry Canada and Transport Canada provide more detailed explanations in
their departmental performance reports about the methodology used to present
recommendation is accepted.
is important for departments and agencies to provide parliamentarians with
accurate and credible performance information, including information on the
methodology used to present results. At the same time, DPRs should maintain a
strategic focus and remain concise. Departments and agencies are encouraged to
provide web links to the more detailed, technical information that is relevant
to understanding performance on their websites. The Treasury Board Secretariat
will therefore explore how departments and agencies could use departmental
websites to present, where useful for clarity and where feasible, indicator
methodologies for the 2013-14 DPR.
perspectives of Industry Canada and Transport Canada on the inclusion of
methodology in their DPRs are provided below.
recommended by the Treasury Board Secretariat, Industry Canada has used electronic links to publicly available sources in its DPR to allow the
reader to verify the data quality and to understand the methodology behind the
Canada began collecting methodology information on all performance statements
in 2009-10 and summarized this, where applicable, in the 2009-10 DPR through
text and explanatory notes. For example, for the strategic outcome, “Science
and Technology, Knowledge, and Innovation are Effective Drivers of a Strong
Canadian Economy”, one of the indicators in the 2009-10 DPR is Canada’s rank on the Innovation Index. Information presented explains to the reader the
purpose of the index and its sub-components, and a link to the data source for
further information is provided. Complete methodology information is available
upon request for all performance indicators. In relation to performance status
(i.e., exceeded, met all, mostly met, etc.), an explanation was provided in the
“How to Read This Report” section of the 2009-10 DPR, in accordance with
instructions from the Treasury Board Secretariat.
Industry Canada continues to improve its Parliamentary Reporting, the new templates
and processes in place will ensure that the department has explicit
methodologies available for each indicator before the DPR process begins. The
DPR will also continue to provide readers with information on trends against
historical performance to indicate whether performance has improved, declined,
remained stable or was unavailable.
a department-wide basis, Transport Canada is in the process of implementing a
comprehensive methodology to present results in its DPR. Components of this
methodology to improve performance reporting are described below.
on the revised 2007-08 PAA, Transport Canada completed a review of its PMF at
the strategic outcome and program activity levels during the previous fiscal
year and it is currently embarking on an assessment of its PMF at the lower
program levels. It is anticipated that this extensive review will result in a
stable PMF for upcoming years that will facilitate benchmarking and/or the
provision of baseline data. Meanwhile, the department has adopted an approach
in its 2009-10 DPR whereby trend analyses are presented where data is
available, and qualitative proxy measures have been included so as to provide
more comprehensive and accurate results-based information.
the recommendations, guidelines and templates provided by Treasury Board
Secretariat have proved very useful in improving results-based reporting. For
example, following the Standing Committee on Public Accounts’ hearing on May
13, 2010, Transport Canada decided to include a performance target, status and
summary of performance for each program activity in its 2009-10 DPR. The
summary also includes a brief explanation to justify the department’s
the methodology used in the development of performance indicators, it should be
noted that many of the data sources are publicly available. For example, the
average age of transportation infrastructure is available from Statistics
Canada and the number of aviation accidents per 100,000 flying hours is
available from the Transportation Safety Board. However, in some cases, it may
be difficult to present the methodology of a certain performance indicator, or
its corresponding target and result in the DPR due to the intricacy and
complexity of the methodology in question.
Industry Canada and Transport Canada provide credible and balanced performance
reports by clearly indicating where program performance was less than expected,
explaining why, and discussing what steps were taken to modify program design,
and delivery in order to improve performance.
recommendation is accepted.
is important that DPRs serve as an accurate depiction of a department or
agency’s successes and shortcomings over the course of the reporting period. It
is equally important that departments and agencies use the information at their
disposal to make in-year changes to plans, where feasible.
Treasury Board Secretariat strongly supports the practice of balanced
reporting. Its 2008-09 DPR guidance included the following instructions on how
to achieve balance in reporting: “Report whether the expected results were met
or not met and identify lessons learned. Both successes and performance gaps
must be addressed. If plans did not unfold as expected, explain why and how
this affected performance. Outline any corrective actions or strategies that
will be pursued in light of the results achieved”. The Secretariat will expand
this guidance for the 2010-11 DPR and place additional emphasis on the need to
discuss the steps taken, where warranted and supported through evidence, to
modify program design and delivery in order to improve performance.
Canada and Transport Canada agree to continue to work towards producing more
balanced, credible reports. Summarized below is a discussion of progress made
by both departments since 2008-09, as well as what can be expected in terms of
reporting in Industry Canada’s 2009-10 DPR is evidenced in the reporting of
results that did not meet targets specified in the 2009-10 RPP, along with
explanations as to why these targets were not met, where applicable. For
example, Industry Canada did not meet its target for a 10 percent “Increase in
number of clients using the Canada Business Network (CBN) website over the
previous year”. The explanation provided for this is that a new methodology was
introduced in 2009 to count website visits. Furthermore, a new CBN website was
launched in October 2009, reducing the number of websites from 14 to 1. This
resulted in a reduction in referrals to the website by external search engines,
which is typical, but temporary, following major changes to a website.
Canada has taken steps beyond those included in Treasury Board Secretariat
guidance. In an effort to present a balanced performance story, for example,
the department included trends against historical results information for
indicators, where available, in the 2009-10 DPR, which helped to establish
year-over-year comparisons. The department provided an online layer of content
that gives users access to performance information at lower levels of the PAA,
as well as additional information on lessons learned.
applicable, Industry Canada has included lessons learned in its DPRs to explain
how adjustments were made during that fiscal year where a need for an
adjustment was identified. As an example of an adjustment made, the Community
Futures Program risk assessment process was altered as a result of an audit
performed on that program. This was shown as a lesson learned that included an
in-year change in the 2009-10 DPR. However, there is a limited scope in the DPR
for discussion of future program design changes since the document’s purpose is
to explain to parliamentarians what departments and agencies accomplished with
authorized resources within that fiscal year.
Canada will place further emphasis on explaining risk information and
providing more lessons learned for program activities in the 2010-11 DPR.
extensive review of Transport Canada’s PMF that is continuing in 2010-11 will
result in more balanced and credible reporting, as well as more meaningful,
specific and measurable departmental indicators. A stable PMF for the upcoming
years will also facilitate benchmarking and the provision of baseline data. The
revised indicators at lower levels of the PAA emerging from the review process
will be available for the 2012-13 DPR. Until 2012-13, Transport Canada will be reporting on a PMF that is still a work in progress in terms of meaningful
measures and data to support the reporting of results. Despite this, the
department has adopted an approach whereby qualitative proxy measures are
reported so as to provide more comprehensive and accurate results-based
information. (This was the approach used in the 2009-10 DPR.)
the Treasury Board of Canada Secretariat provide departments and agencies with
a template for the preparation of their departmental performance reports, so
the reports are structured in a similar fashion in order to be more
recommendation is accepted.
Treasury Board Secretariat currently provides departments and agencies with
guidelines that include a suggested template for DPRs. The Secretariat will
develop a new common structure that will provide a standardized index and
template for all departments and agencies to follow in the preparation of
2010-11 DPRs. Departments and agencies will continue to have the flexibility to
report content as suited to their unique context, mandate and results.
where feasible, departments and agencies provide detailed financial and
performance information about programs on their websites.
recommendation is accepted.
The Policy on MRRS provides the basis upon which the Estimates documents are
structured. In their DPRs, departments and agencies are required to provide
financial and non-financial performance information at the strategic outcome
level and program activity level. This structure ensures that the DPR remains a
concise, results-focused document that enhances the reader’s ability to
understand a federal department’s core mandate.
and agencies normally table their DPRs in Parliament in the fall of every year4.
Where feasible, annual electronic (i.e., web-based) reporting of lower level
program results would complement the information provided in the DPR. The
Secretariat will develop and implement a standardized and streamlined approach to
annual electronic reporting on lower level programs for the 2013-14 DPR, which
will allow departments and agencies the necessary time to improve and stablize
their PAAs and PMFs as a consistent basis for reporting.
departments have already begun posting certain elements of lower level program
performance information on their websites. For example, Industry Canada’s DPR provides links to further performance information that is available online. It
should also be noted that Transport Canada’s evaluations of various program
areas are available on its publicly-accessible website. This allows the
department to demonstrate to parliamentarians and the public the linkages
between its performance indicators, its expected results and its actual
if feasible, all House of Commons Standing Committees review the DPRs referred
to them in order to ensure that they contain the information needed to allow
parliamentarians to hold the government to account for achieving meaningful results
for Canadians in an economical, efficient and effective manner.
It would not be appropriate for the Government to respond to this
recommendation, as this is the prerogative of Parliament.