Audit of Project Management of the Canadian High Arctic Research Station
Date : February 2014
Project #: 13-49
PDF Version (416 Kb, 36 Pages)
Table of contents
List of Acronyms
AANDC |
Aboriginal Affairs and Northern Development Canada |
---|---|
ARIF |
Arctic Research Infrastructure Fund |
CHARS |
Canadian High Arctic Research Station |
COE |
Centre of Expertise |
NAO |
Northern Affairs Organization |
NLCA |
Nunavut Land Claims Agreement |
NPMS |
National Project Management System |
PAAA |
Pre-Approved Amounts for Anticipated Amendments |
PWGSC |
Public Works and Government Services Canada |
SPAC |
Senior Project Advisory Committee |
S&T |
Science and Technology |
TB |
Treasury Board |
Executive Summary
Background
The Audit and Evaluation Sector of Aboriginal Affairs and Northern Development Canada (AANDC) identified an Audit of Project Management of the Canadian High Arctic Research Station (CHARS) infrastructure project in the Department's 2013-2014 to 2015-2016 Risk-Based Audit Plan, approved by the Deputy Minister on February 27, 2013. The audit was identified as a departmental priority as the initiative is significant in terms of supporting the Government of Canada's priorities in the North under the Northern Strategy.
Audit Objective and Scope
The objective of the audit was to provide management with assurance that the CHARS infrastructure project is being appropriately managed and that a project management framework between AANDC and Public Works and Government Services Canada (PWGSC) is in place to help ensure that the CHARS facility is delivered on time and within budget, and that the CHARS facility is aligned to meet the strategic objectives of the Science and Technology (S&T) program planned for the facility.
The scope of the audit included an examination of the governance and project management practices related to the definition, design, build and planned operation of the research station such that it supports the S&T program planned for the facility. The period under audit spanned from the announcement of the project in November 2007 (Definition Phase) to July 31, 2013 (Design Phase). As the project is still ongoing, components of the project management approach were still under development at the time of the audit.
The audit was primarily performed in the National Capital Region from July to November 2013. Site visits were conducted at the PWGSC regional offices in both Edmonton and Winnipeg, where resources responsible for project management, contracting, and procurement are located.
The scope of the audit did not include the following:
- CHARS site selection process;
- Project management of the S&T program – scope was limited to key dependencies of the S&T program on the infrastructure design;
- Governance of the CHARS facility once completed, including reporting accountability, S&T program governance, and facility operations and maintenance; and,
- Project management processes that are the sole responsibility of PWGSC as the Real Property delivery agent – audit scope was limited to how this information was monitored and considered by AANDC project resources.
Statement of Conformance
The audit conforms with the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program.
Observed Strengths
The following strengths were observed:
- Comprehensive project reporting: PWGSC Western Region's Project Black Book tool is shared with and used by AANDC on a monthly basis, and enables AANDC to actively monitor and intervene, when necessary, with project progress in the areas of project management, financial management, procurement and contract management, and key issues or changes to the CHARS infrastructure project. The tool tracks both financial and non-financial progress of the project against key performance indicators, and provides a detailed breakdown of the costs versus forecasts and the value of work done to date for the entire infrastructure project lifecycle. PWGSC has not historically shared the Project Black Book with client departments; it was shared at the request of AANDC and has facilitated a collaborative working relationship between departments.
- Robust procurement practices: Given the sensitivity and visibility of the CHARS project, the project team used the following practices:
- Fairness monitor: a fairness monitor oversaw the procurement processes for both the design consultant and construction management services to ensure the process was both conducted and perceived to be conducted in a fair and transparent manner.
- Weighting of technical score for design consultant: PWGSC standard practice for design and engineering procurement is to use an evaluation methodology of 90% technical score and 10% financial score to rate the overall score of each proposal to ensure a high quality winning proponent.
- Design Exercise included Northern perspectives: short-listed qualified bidders were required to prepare detailed design presentations to ensure alignment with the strategic objectives of the CHARS project and AANDC's needs. Presentations were evaluated by qualified technical resources on the 'Design Jury' that included members from PWGSC, AANDC, private industry, as well as Northern representation.
- Incorporating lessons-learned from past projects: Engaging resources with experience in Northern construction, project management, architecture and engineering backgrounds has clearly benefited the CHARS project. Projects undertaken through the Arctic Research Infrastructure Fund (ARIF) and the associated lessons learned and best practices were formally incorporated into the CHARS Feasibility Report's 'Inspiring Examples' section. AANDC deployed management and project resources with ARIF experience into key roles on the CHARS project (both the infrastructure project and the S&T program) in order to ensure that it benefits from past experience.
Conclusion
The CHARS infrastructure project is being appropriately managed and a project management framework between AANDC and PWGSC is in place to help ensure that the CHARS facility is delivered on time and within budget, and to facilitate the alignment of the CHARS facility with the strategic objectives of the S&T program planned for the facility. Some areas for improvement were noted to strengthen management practices in the areas of governance and oversight, and risk management.
Recommendations
The audit team identified areas where management control practices and processes could be improved, resulting in the following three recommendations:
- The Assistant Deputy Minister of the Northern Affairs Organization should formalize the governance structure over the CHARS infrastructure project from the design-build phase, up to but not including the start of the operation phase, by ensuring that Terms of Reference, roles and responsibilities, and records of decisions are documented for key governance and oversight committees.
- The Assistant Deputy Minister of the Northern Affairs Organization should develop a plan to address the evolving governance structure of the CHARS infrastructure project as it moves through to the design-build phase of the project to reflect the changing needs for governance, up to but not including the start of the operation phase.
- The Assistant Deputy Minister of the Northern Affairs Organization should implement the integrated Risk Management Plan as soon as practicable, and maintain commitment to the risk management processes outlined therein.
Management Response
Management is in agreement with the findings, has accepted the recommendations included in the report, and has developed a management action plan to address them. The management action plan has been integrated into this report.
1. Background
The Audit and Evaluation Sector of AANDC identified an Audit of Project Management of the Canadian High Arctic Research Station infrastructure project in the Department’s 2013-14 to 2015-16 Risk-Based Audit Plan, approved by the Deputy Minister on February 27, 2013. The audit was identified as a departmental priority as the initiative is significant in terms of supporting the Government of Canada’s priorities in the North under the Northern Strategy.
The Canadian High Arctic Research Station (CHARS) will be a year-round, multidisciplinary research facility and centre of Arctic science and technology (S&T) development in Cambridge Bay, Nunavut. First announced in the 2007 Speech from the Throne, CHARS highlights the government’s ongoing commitment to strengthen Canadian sovereignty and to improve the lives of Northerners. The project will be funded by the federal government largely through Aboriginal Affairs and Northern Development Canada (AANDC) and is set to open in July 2017. An overview of the next steps for the CHARS infrastructure project and the S&T program, including the key decision points, is provided in the timeline below.
In 2009, AANDC received $2M under the Arctic Research Infrastructure Fund (ARIF) to conduct a feasibility study of the proposed research station. The CHARS project is led by AANDC and is being conducted in partnership with Public Works and Government Services Canada (PWGSC), who oversees project management for real property projects in accordance with their National Project Management System (NPMS). A construction management approach has been chosen to deliver on the CHARS infrastructure. CHARS is currently in the design phase, which involves the production of design and material specifications, telecommunication strategies, and detailed cost and scheduling estimates for both the construction and operation of the new facility.
The town of Cambridge Bay, Nunavut was chosen as the preferred location for CHARS, based on several factors, including the central location of the community, its role as a regional hub, and the potential for good S&T in the area.
Since the CHARS facility will be located in Nunavut, consideration will need to be given to the Nunavut Land Claims Agreement (NLCA) throughout the planning and operational stages of the project. Thus far, the following Articles of the NLCA have been identified as possible influences on the building and operation of the CHARS facility: Article 23, Inuit Employment with Government; and, Article 24, Government Contracts.
The mandate for CHARS, announced on December 3, 2010, outlines the following main objectives:
- Develop and diversify the economy in Canada's Arctic;
- Support the effective stewardship of Canada's Arctic lands, waters, and resources;
- Create a hub for scientific activity in Canada’s vast and diverse Arctic;
- Promote self-sufficient, vibrant, and healthy Northern communities;
- Inspire and build capacity through training, education and outreach; and,
- Enhance Canada's visible presence in the Arctic and strengthen Canada's leadership on Arctic issues.
In order to achieve these objectives, the CHARS S&T research will centre on four priority areas, as outlined in the CHARS mandate:
- Resource development;
- Exercising sovereignty;
- Environmental stewardship and climate change; and,
- Strong and healthy communities.
Stakeholders of the facility include Aboriginal peoples, academia, government, and industry collaborators, both domestically and internationally. The centre is expected to bestow direct benefits on the Cambridge Bay community in terms of employment, infrastructure and capacity building. Related northern development investments have been made in recent years to strengthen the network of infrastructure used in Arctic research in the Canadian North. The ARIF, in addition to providing a funding mechanism for the CHARS feasibility study, awarded $85M to 20 different projects at 46 sites across the North to strengthen the physical aspect of this network.
On August 23, 2012, Prime Minister Harper announced $142.4M over six years, beginning in 2012, for the construction, equipment, and fit-up of CHARS facility and an additional $46.2M over six years, also beginning in 2012, for the CHARS S&T research program. The Prime Minister also announced $26.5M, as of 2018-19, for the ongoing delivery of the S&T program and the operation of the CHARS facility.
The governance of the CHARS infrastructure project, from facility construction and then further to commissioning and project close-out is the responsibility of AANDC. The responsibility for governance of the CHARS facility, once operational, has yet to be determined.
2. Audit Objective and Scope
2.1 Audit Objective
The objective of the audit was to provide management with assurance that the CHARS infrastructure project is being appropriately managed and that a project management framework between AANDC and Public Works and Government Services Canada (PWGSC) is in place to help ensure that the CHARS facility is delivered on time and within budget, and that the CHARS facility is aligned to meet the strategic objectives of the Science and Technology (S&T) program planned for the facility.
2.2 Audit Scope
The scope of the audit included an examination of the governance and project management practices related to the definition, design, build and planned operation of the research station infrastructure project such that it supports the S&T program planned for the facility. Specific management practices areas examined included:
- Governance and oversight;
- Project management;
- Risk management;
- Financial management; and,
- Procurement and contract management.
The period under audit spanned from the announcement of the project in November 2007 (Definition Phase) to July 31, 2013 (Design Phase). As the project is still ongoing, components of the project management approach were still under development at the time of the audit.
The scope of the audit did not include the following:
- CHARS site selection process;
- Project management of the S&T program – scope was limited to key dependencies of the S&T program on the infrastructure design;
- Governance of the CHARS facility once completed, including reporting accountability, S&T program governance, and facility operations and maintenance; and,
- Project management processes that are the sole responsibility of PWGSC as the Real Property delivery agent – audit scope was limited to how this information was monitored and considered by AANDC project resources.
3. Approach and Methodology
The Audit of Project Management of CHARS was planned and conducted in accordance with the requirements of the Treasury Board Policy on Internal Audit and followed the Institute of Internal Auditors’ Standards for the Professional Practice of Internal Auditing.
The audit team examined sufficient, reliable and relevant evidence to provide a reasonable level of assurance in support of the audit conclusion. The principal audit techniques used included:
- Interviews with key individuals within AANDC’s Northern Affairs Organization and the Northern Policy and Science Integration Branch with responsibilities for the infrastructure project, S&T program, and oversight and accountability of the CHARS project (headquarters in the National Capital Region) and PWGSC resources with responsibility for project management, acquisitions, preparation of project approval documentation, and monitoring and oversight of the CHARS infrastructure project (Winnipeg and Edmonton Regional Offices);
- Review and testing of documentation related to the CHARS project in the areas of policy, project management, procurement and contract management, and monitoring and reporting; and,
- Site visits to PWGSC regional offices in Winnipeg and Edmonton to conduct interviews and review documentation.
The approach used to address the audit objective included the development of audit criteria, against which observations and conclusions were drawn. The audit criteria developed for this audit are included in Appendix A.
Additionally, relevant policies and directives referenced throughout the audit process are listed in Appendix B. Definitions for key terms referenced throughout the report, as well as the roles and responsibilities of key project stakeholders, are defined in Appendix C.
Selection of Audit Procedures
Through planning, it was observed that a significant portion of the project management responsibilities for the infrastructure project are within PWGSC’s Real Property Project Branch in the Western Region. In accordance with the performance standards outlined within Treasury Board’s Policy on the Management of Projects and the PWGSC’s National Project Management System Policy, PWGSC is responsible for, and plays a significant role in the areas of, project management, acquisitions, and contracting. Documentation and file review procedures were performed at the PWGSC Winnipeg and Edmonton regional offices due to physical location of the files.
4. Conclusion
The CHARS infrastructure project is being appropriately managed and a project management framework between AANDC and PWGSC is in place to help ensure that the CHARS facility is delivered on time and within budget, and to facilitate the alignment of the CHARS facilities with the strategic objectives of the S&T program planned for the facility. Some areas for improvement were noted to strengthen management practices in the areas of governance and oversight, and risk management.
5. Findings and Recommendations
Each audit criterion, which established the expectations against which the audit was conducted, was assessed by the audit team using a combination of documentation review, analysis and interview procedures. A conclusion for each audit criterion was determined and documented based on the results of applying these audit procedures. Where gaps were identified between expected and actual practices, the associated risk was evaluated to develop a conclusion and to document recommendations for improvement. Audit findings and related recommendations for improvement are categorized below by the key elements of project management: governance and oversight; project management; risk management; financial management; and, procurement and contract management.
5.1. Governance and Oversight
The audit examined whether an effective governance framework was established for the CHARS infrastructure project, including relevant oversight bodies, clear governance protocols and well-established accountabilities. In reviewing the governance and oversight structure in place over the CHARS infrastructure project, it was evident that a number of key formal and informal oversight functions are currently in place. Due to the unique nature of the CHARS project and the joint delivery of the facility by AANDC and PWGSC, inter-departmental oversight is an important component of the governance structure.
5.1.1 Formal CHARS Infrastructure Governance
There are formally established oversight bodies for the CHARS infrastructure project, including the Senior Project Advisory Committee, the CHARS Oversight Committee, and the 'CHARS Senior Management Committee'. There is also a project-level governance body, referred to as the 'CHARS Project Management Committee', that includes key project management resources from AANDC, PWGSC and third-party contractors.
Although the 'CHARS Senior Management' and 'CHARS Project Management Committees' are established and functioning as oversight bodies, they are not formally named committees within the current CHARS governance structure. These names have been created for audit documentation purposes to describe their respective roles and responsibilities. The audit team developed a diagram of the governance structure based on project documentation and interviews with key resources. This diagram, which was validated with AANDC key oversight stakeholders, is depicted below in Figure 2.
Senior Project Advisory Committee
The Senior Project Advisory Committee (SPAC) was formed from the pre-existing Assistant Deputy Minister (ADM) Committee on Arctic Science and Technology (S&T), which was formed in 2008 following the 2007 Speech from the Throne that announced the establishment of a Canadian High Arctic Research Station to support the federal Northern Strategy. The Committee's original objectives were three-pronged: to provide strategic guidance on the CHARS project; to advise on International Polar Year programming and legacy phases as initiatives began to wind down; and, to provide a forum for federal departments with Arctic S&T initiatives to promote better federal collaboration, facilitate advice on implementation of initiatives, and promote information exchange on Northern S&T and polar issues. Following the close of International Polar Year, the committee's objectives continue to be both serving as a federal forum for departments to discuss Arctic S&T initiatives and to play the role of the SPAC for the CHARS project. Membership of the SPAC is composed of a sub-group of members within the ADM Committee on Arctic S&T, and includes the following key federal departments and agencies with the common goal of furthering Northern and Aboriginal economic development and the development of Canada's science, research and technology field:
- Aboriginal Affairs and Northern Development Canada;
- Defence Research and Development Canada;
- Environment Canada;
- Department of Fisheries and Oceans Canada;
- Health Canada;
- Industry Canada;
- Natural Resources Canada;
- Public Health Agency of Canada; and,
- Public Works and Government Services Canada.
The role of the SPAC includes providing strategic advice to AANDC on the direction and development of the CHARS project as a whole, including both the infrastructure project and the S&T program. Upon commencement of project management for the CHARS infrastructure project, PWGSC's representation on the Committee, which included the ADM of Acquisitions, grew to include the ADM of Real Property.
The SPAC is chaired by AANDC's ADM of Northern Affairs Organization (NAO), and the secretariat role is played by the Director General (DG) and supporting resources from NAO's Northern Policy and Science Integration Branch. These resources all play key project and management roles on the CHARS project. The SPAC membership and high-level governance protocols are established in its Terms of Reference. In accordance with the Terms of Reference, the SPAC meets with ad-hoc frequency as quorum is called by the Committee Chair. Meeting agendas indicated that the SPAC met with a minimum of annual frequency between 2008 and 2013 to discuss agenda items, and typically on a quarterly basis – because the SPAC is also the ADM Committee on Arctic S&T, the SPAC function and CHARS update is one of multiple agenda items.
As previously mentioned, SPAC discussion on CHARS is concerned with the infrastructure project as it affects the S&T program and long-term sustainability of the facility once built. While issues or decisions related to CHARS can be taken to the SPAC for advice and recommendation by the participating members, approval by the SPAC is not required for CHARS related decisions, as AANDC is the responsible department for the success of the CHARS project. The ADM of NAO can escalate issues directly to AANDC's Deputy Minister as necessary. Through a review of SPAC meeting minutes, the audit found that records are not consistently documented for all SPAC meetings.
CHARS Oversight Committee
The CHARS Oversight Committee is an inter-departmental committee between AANDC and PWGSC that provides ADM-level oversight of the CHARS infrastructure project. The CHARS Oversight Committee was established in 2011 as a complementary oversight function to the SPAC, and is responsible for issues resolution and day-to-day decision-making on the project. Interviews with committee members and project management resources supported that the CHARS Oversight Committee plays an operational role and is focused specifically on the infrastructure project, whereas the SPAC is concerned with the CHARS project as a whole and primarily informs the S&T program development and implications of the facility design on the program. The CHARS Oversight Committee is chaired by AANDC's ADM of NAO and its membership includes the PWGSC's ADMs of Real Property and Acquisitions, as well as the PWGSC Regional DG of Real Property Projects for the Western Region.
The CHARS Oversight Committee's membership and high-level governance protocols are established is Terms of Reference. However, the Terms of Reference, written in 2011, does not reflect the current protocols in place regarding meeting frequency. Shortly after the creation of the Committee, it was determined that the Committee was only required to meet on an ad-hoc basis to deal with specific project issues, decisions, or risks as escalated, not on a bi-monthly basis as stated in the Terms of Reference. Similar to the SPAC protocol, AANDC's ADM of NAO is not required to bring all key issues to the CHARS Oversight Committee for decision or approval – issues brought before the Oversight Committee typically reflect a joint impact on both AANDC and PWGSC, or would benefit from the expertise of PWGSC as the leader in real property projects and acquisitions for the Government of Canada. Since its inception, the Oversight Committee has met twice; meeting minutes were only documented for one meeting at the time of this audit.
'CHARS Senior Management Committee'
The 'CHARS Senior Management Committee' is an inter-departmental oversight committee between AANDC and PWGSC that includes both Director-level management oversight and project / acquisition team-level resources. The 'CHARS Senior Management Committee' is chaired by PWGSC's Regional Director of Professional and Technical Services. Its membership includes AANDC's Director of Northern Policy and Science Integration, PWGSC's Regional Director of Professional and Technical Services, and AANDC and PWGSC designated Project / Acquisition Managers for the infrastructure project. In addition, as agenda items necessitate, other key resources, including responsible AANDC and PWGSC DGs and team-level resources, are invited to provide decisional support, information, and progress updates with respect to their project roles and responsibilities.
The 'CHARS Senior Management Committee' does not have a formally documented Terms of Reference outlining membership or high-level governance protocols. Its protocols are informed by the inter-departmental Communication Plan. Meeting minutes are consistently documented and circulated to key project resources by the Committee Chair.
'CHARS Project Management Committee'
The 'CHARS Project Management Committee' is a project-level oversight committee that is composed of the AANDC and PWGSC designated Project Managers and lead resources from both the Design Consortium and the Construction Manager. The 'CHARS Project Management Committee' provides oversight and management of issues as reported by participating members. Following the development of an inter-departmental Communication Plan in the summer of 2013, the 'CHARS Project Management Committee' was formed to meet the need for a centralized project management team that could efficiently resolve operational issues while at the same time ensure that both AANDC and PWGSC are aware and inform decisions that affect the third party contractors. This need was identified during the development of the Communication Plan.
The 'CHARS Project Management Committee' does not have a formally documented Terms of Reference outlining membership or high-level governance protocols. Its protocols are informed by the Terms of Reference for the Design Consortium. Records of decisions are consistently documented.
5.1.2 Governance Framework
The CHARS infrastructure project is a joint delivery effort between AANDC and PWGSC. With PWGSC assuming the role of the delivery agent for the infrastructure project, their National Project Management System (NPMS) and its associated processes, deliverables and approvals are being used to manage the project. Subsequently, the governance of the CHARS infrastructure project is defined, at a high-level within the NPMS-required Project Management Plan and the Project Charter (in addition to the individual Terms of Reference developed for oversight bodies). This includes high-level definitions for departmental roles, responsibilities, and issues management.
While the roles and responsibilities of the project resources are well understood at an operational level, an overall governance framework that describes the structure and the integration between the project team and various oversight bodies, including escalation procedures and reporting relationships between departmental team members, is not currently documented for the project as a whole. Governance protocols for how the project will continue to receive governance and oversight throughout the infrastructure project lifecycle up to and including project close out, are also not formally documented. Despite the lack of formal documentation of the governance structure, project management and team resources feel that the evolved governance structure in its current state is functioning appropriately. This finding is supported by interviews with project resources and a review of project documentation.
5.1.3 Informal Oversight
While there is DG and Director-level involvement in the formal oversight committees as described above, additional oversight at the DG and Director levels takes place on an informal basis via bi-lateral discussions between AANDC and PWGSC counterparts. These discussions are not governed by a Terms of Reference and key discussions are not formally documented.
Interviews with AANDC and PWGSC project management resources supported that the DG bi-lateral discussions occur on an as-needed basis; Director-level bi-lateral discussions occur on a monthly basis, with more frequent discussions occurring around key project milestones.
Finding:
A governance structure to oversee the CHARS infrastructure project exists and is operating, a formal governance structure that describes the integration between the oversight bodies, including escalation and reporting relationships and their roles and responsibilities pertaining to the other oversight bodies, is not documented. Terms of Reference were only available for two of the four key oversight bodies, and those found to be in existence were not up-to-date to reflect the current governance activities in practice. While records of decisions are kept for the oversight bodies, they are not consistently prepared for all bodies.
Recommendation:
1. The Assistant Deputy Minister of the Northern Affairs Organization should formalize the governance structure over the CHARS infrastructure project from the design-build phase, up to but not including the start of the operation phase, by ensuring that Terms of Reference, roles and responsibilities, and records of decisions are documented for key governance and oversight committees.
Finding:
As the CHARS infrastructure project moves from design to the implementation phase of facility construction and then further to commissioning and project close-out, the governance structure must change to adapt to the unique risks and requirements of the different project phases. A clearly defined process for evolving the governance structure to meet the changing needs of CHARS as it moves through the infrastructure project lifecycle does not currently exist.
Recommendation:
2. The Assistant Deputy Minister of the Northern Affairs Organization should develop a plan to address the evolving governance structure of the CHARS infrastructure project as it moves through to the design-build phase of the project to reflect the changing needs for governance, up to but not including the start of the operation phase.
5.2. Project Management
The audit examined whether project management processes and practices were in place, including applicable tools for financial tracking, monitoring, and reporting. Because the CHARS infrastructure project is a joint-delivery effort by AANDC and PWGSC, the audit also examined whether there is sufficient human resource capacity to support the effective and efficient achievement of project objectives, and whether activities between team resources are planned and coordinated in an effective manner to meet project objectives and ensure alignment with the S&T program needs.
5.2.1 Project Management Framework
In accordance with the TB Policy on the Management of Projects and the PWGSC National Project Management System (NPMS) Policy, PWGSC's NPMS framework and associated processes and approvals are used to manage the CHARS infrastructure project. This includes the usage of NPMS-prescribed tools, templates, deliverables and approval milestones, which are aligned with the TB project approval process. The NPMS framework and processes are rigorously followed in the management of the CHARS project. The project is currently at the end of the Design Phase and will progress to the Implementation Phase once Effective Project Approval is received.
In accordance with the NPMS framework, guidance, governance, accountability, and reporting protocols between AANDC and PWGSC have been established in key project governing documents, such as the Project Management Plan and the Project Charter. High-level departmental and key resource roles and responsibilities for governance, project management, risk management, financial management, and procurement and contract management are defined within these documents. Resource specific roles are also defined for the AANDC and PWGSC Project Managers and the AANDC Project Leader.
The active reporting requirements of both financial and non-financial information for PWGSC, as the day-to-day project manager of the infrastructure component, are outlined within these governing documents. Reporting requirements are accomplished by PWGSC's Project Manager through the Project Black Book, a tool specific to PWGSC's Western Region Real Property unit. The Project Black Book is an Excel-based workbook that tracks detailed financial and non-financial metrics of the project in the areas of project management, issues management, financial management, and contract management, and monitors project progress against the key performance indicators of project scope, budget, and schedule. Initially, project reporting by PWGSC to AANDC was limited in the provision of a summary of expenditures to date as well as invoices for cost recovery for PWGSC resources. At AANDC's request, the Project Black Book is now shared with the AANDC designated Project Manager on a monthly basis, which has facilitated a more effective departmental working relationship through the increase of knowledge and information-sharing.
AANDC's CHARS project reporting requirements are defined and aligned to established departmental reporting practices. The Northern Affairs Organization (NAO) provides progress reports on the CHARS infrastructure project through the Department's internal quarterly reports, which are submitted to the Deputy Minister, and annually within the publicly-available Departmental Performance Report. These reporting requirements were met on a consistent basis by AANDC, and the level of detail on project progress was appropriate for the audience of the report, with quarterly reports providing operational progress updates and the Departmental Performance Report, including project information at a strategic level.
5.2.2 Human Resources
There are approximately 10 FTEs devoted to the entire CHARS project at AANDC including both S&T program and infrastructure project resources, and 2.5 FTEs devoted specifically to the CHARS infrastructure project at PWGSC. In addition, PWGSC's Centre of Expertise (COE) on Architecture and Engineering provides technical guidance in the areas of architecture, engineering, northern construction, project costing and quantity surveying, and laboratory design, and also reviews key deliverables to the core project team.
Through audit procedures, it was observed that the core project team possesses a breadth of knowledge and project management experience from scientific, research, architectural, and engineering backgrounds. In situations where the core project team does not have the detailed technical knowledge or experience, the project team has access to PWGSC's COE on Architecture and Engineering and the specialized skills of their resources. Project processes requiring extensive technical expertise are contracted through merit-based competitions to technical consultants with subject matter expertise, as was the case for the CHARS Design Consortium and the Construction Manager. Consistent with the findings of the audit, project management capacity is perceived by the CHARS project team members and management to be sufficient in terms of both technical skills and resourcing levels.
5.2.3 S&T Program Dependencies on the Facility Design
Project documentation reflects that the need to include key dependencies from the S&T program into the facility's design is an integral component to the success of the CHARS project as a whole. High-level S&T program dependencies are known; for example, the CHARS facility must meet the needs of being a world-class research facility and laboratory, include conference and meeting spaces, and have traditional knowledge areas. However, the S&T program and its detailed definition are being developed in parallel with the design of the facility, limiting the opportunity of the infrastructure team to incorporate detailed S&T program requirements into the facility design.
The approach to the development of the CHARS project is unique since real property projects typically undertaken by PWGSC require the design and construction of infrastructure to meet the needs of an existing program or Statement of Requirements, whereas the CHARS facility and S&T program are being designed and approved in parallel out of necessity to meet the project timelines of a completed facility by 2017. While this poses a risk to the long-term success of the CHARS project, and indirectly impacts the infrastructure project team's ability to expediently proceed through the design phase, this risk is well-known to all project resources at AANDC and PWGSC. In addition, the design of the facility to-date has been flexible to allow for minor design and program changes as the S&T program is further developed. Procurement options were also reviewed to determine the best approach for the CHARS infrastructure project. Options reviewed included the design-build, design-bid-build, construction management, and public-private-partnership approaches. The construction management approach for procurement was chosen because it facilitates flexibility to both the design of the facility and the order of construction work undertaken by splitting the construction tenders into multiple packages and procuring multiple service providers. The risk related to the parallel development of the S&T program is also acknowledged and considered as part of the risk management responsibilities of AANDC, PWGSC, the Design Consortium, and the Construction Manager.
The project is currently preparing for Effective Project Approval for both the infrastructure project and the S&T program. Once the Effective Project Approval has been received, it will be difficult to make significant changes to the infrastructure design or the S&T program.
5.2.4 Communication Protocols
Communication protocols were examined from two perspectives: the level of communication between the AANDC infrastructure project team and the AANDC S&T program team; and, the communication between the AANDC and PWGSC infrastructure project resources.
AANDC Inter-team Communication
The necessity for regular and consistent communication between the AANDC infrastructure and S&T teams was identified as being important to the success of the overall CHARS project. The SPAC, a CHARS oversight body, serves to provide strategic direction to both the infrastructure project and the S&T program and provides high-level oversight on the alignment of the facility design with known S&T dependencies. AANDC project team resources, including the Lead Engineer, Project Manager, Project Leader and the ADM of the NAO, maintain dual responsibilities on both components of the CHARS project to provide additional oversight on the alignment between the facility design and S&T program needs.
Inter-departmental Communication
The Project Management Plan and Project Charter provide high-level communication protocols for the project teams at AANDC and PWGSC. There is evidence of regular inter-departmental communication between project resources, with day-to-day communication occurring on an as-needed, ad-hoc basis and not formally documented.
An inter-departmental Communication Plan was developed in September 2013 and later implemented to define necessary communication protocols and to facilitate the transparent, consistent, and sufficient communication between key project resources. The Communication Plan also resulted in the creation of the 'CHARS Project Management Committee' to ensure efficient communication and consistent involvement from both the PWGSC and AANDC Project Managers regarding the facility design. This was developed after project resources identified the need for a communication plan at the project team-level to formally define specific protocols for team communication, roles and responsibilities over decisions/approvals/communication, and the frequency of discussion.
Recommendation:
No recommendations were identified in this area.
5.3. Risk Management
5.3.1 Risk Management Methodology
The audit examined whether a risk management approach existed to ensure the regular capture, reporting, analysis and mitigations of risks to the project in meeting its objectives. It also examined whether lessons learned and best practices from related projects and audits were appropriately considered during the development of the approach to undertake the CHARS infrastructure project.
The Project Charter specifies that PWGSC is responsible for risk management associated with the day-to-day project management of the infrastructure project, while AANDC is responsible for risk oversight and risk management for the entire CHARS project. In executing audit procedures, it was noted that a risk management methodology exists and is functioning as part of the PWGSC National Project Management System (NPMS) framework to support risk management over the project management of the infrastructure component of CHARS. NPMS risk components that are being applied to the CHARS risk management are highlighted below:
- The NPMS provides project resources with a Risk Management Plan template that identifies common project management risks and assessment criteria, and is derived from the PWGSC departmental risk management approach and common methodology. It also provides typical consequences associated with common real property risks, and provides a high-level tool to facilitate the monitoring process.
- The NPMS requires that the project team develop risk management plans prior to issuing any Requests for Proposals, to consider the risks of the services being procured.
- In accordance with contract terms and conditions and standard processes for third party contractors supporting real property projects, both the Design Consortium and Construction Manager are responsible for assessing and reporting risks to PWGSC on a monthly basis via Risk Registers, which are then discussed at the 'CHARS Project Management Committee'. The Risk Registers consider risks specific to each of their technical roles and ongoing services on the project. The Risk Registers employed on the CHARS project were developed using contractor-specific or proprietary risk management methodology and assessment processes by both the Design Consortium and Construction Manager.
In addition to the above risk management processes, the various procurement options being considered for the construction services were subject to a risk assessment to determine the best option to address the project's objectives and provide the best opportunity for integration with the S&T program. This risk assessment supported the decision to adopt the construction management approach to procurement for the CHARS project.
While the NPMS Risk Management Plan template is used for the CHARS project, there is an inconsistency between risk management methodologies (including taxonomy, assessment criteria, and processes for mitigation and escalation) across the departments and third party contractors. For example, the risk assessment methodologies used by AANDC, PWGSC, the Design Consortium, and the Construction Manager to determine risk impact and likelihood each employ different point-scales and definitions. Under these methodologies, risk escalation is also occurring on an ad-hoc basis, with each responsible party using professional judgment to determine when escalation is appropriate. While project resources felt that risks, to date, have been escalated to the appropriate level of governance in a timely manner, the inconsistency of risk methodologies used by stakeholders impedes AANDC's ability to properly account for all risks across the CHARS project. Additionally, while the PWGSC and contractor risk management process is focused solely on the infrastructure project and does not focus on strategic risks to the overall CHARS project, risk management should holistically focus on the greater scope of the CHARS project.
In the interest of unifying and strengthening the risk management methodology across the entire CHARS project, AANDC contracted an external risk management professional to facilitate a risk management workshop and to develop an integrated Risk Management Plan that all parties would use and contribute to. In August 2013, a risk management workshop was held with key resources from AANDC, PWGSC, the Design Consortium and the Construction Manager. An integrated Risk Management Plan was developed shortly thereafter that includes specific risk rating criteria and detailed definitions, assessment considerations, and escalation procedures to facilitate the regular and consistent capture, reporting, analysis, and mitigation of risks across the entire CHARS project. It also defines the roles and responsibilities for key stakeholders in the CHARS project, including oversight at the ADM, DG, Director, and core project team levels, the S&T program team, and third party contractors.
As designed, the integrated Risk Management Plan will employ a Risk Register tool to be reported and shared amongst key stakeholders on a monthly basis. Risk information provided by the core project team, S&T program team, and third party contractors will be synthesized into an AANDC Risk Register by the Risk Management Quality Assurance Analyst. The integrated Risk Management Plan was finalized in November 2013, and has yet to be implemented. The Risk Management Quality Assurance Analyst position has also yet to be filled. The new risk management methodology is anticipated to be implemented prior to the commencement of the construction phase of the project.
Finding
Risk management activities to capture, assess, monitor and report the risks associated with the CHARS infrastructure project have been established; however, the risk management methodologies used by key stakeholders are not consistent. AANDC, PWGSC, the Design Consortium, and Construction Manager each have their own established processes and taxonomy for risk rating, monitoring, and reporting, which creates a challenge to synthesize risk information across the entire CHARS project. Risk management activities are limited to the capture, assessment, monitoring and reporting of risks related to the project management of the design and construction of the facility. Specific procedures for the escalation and mitigation of risks are not consistently documented.
An integrated Risk Management Plan and methodology was developed to facilitate the consistent and regular capture, reporting, analysis and mitigation of risks for the CHARS project. The integrated Risk Management Plan includes detailed risk management roles and responsibilities for project stakeholders, and a defined methodology for the consistent rating, monitoring, reporting, and escalation of risks for the entire CHARS project. At the time of this audit, the integrated Risk Management Plan has yet to be implemented. A key position identified in the Risk Management Plan, the AANDC Risk Management Quality Assurance Analyst, has yet to be staffed.
Recommendation:
3. The Assistant Deputy Minister of the Northern Affairs Organization should implement the integrated Risk Management Plan as soon as practicable, and maintain commitment to the risk management processes outlined therein.
5.3.2 Lessons Learned and Best Practices from Previous Projects
In executing the planned audit procedures, it was determined that lessons learned and best practices were being incorporated into and considered for the CHARS infrastructure project. A variety of experience possessed by the project team was leveraged in informing the CHARS project, including resource-specific professional experience in the areas of science, research, architecture and engineering, experience with other federal real property and crown construction and infrastructure projects in the high Arctic, and through consultations with teams and management that oversaw infrastructure projects that faced similar requirements or challenges.
In funding the Feasibility Report through the Arctic Research Infrastructure Fund (ARIF), the ARIF provided opportunities for the project resources responsible for the success of CHARS to gain experience from other Arctic research infrastructure projects and incorporate lessons learned, best practices, and other unique considerations into their project planning and design for CHARS. Through the ARIF, the AANDC CHARS team was able to gain first-hand knowledge and experience of Arctic research infrastructure functionality, including governance, building approach, in-house program development, and gaps in the current S&T research field in Canada. Special considerations learned through the ARIF projects, such as transportation of materials and weather variability (and the subsequent impact it can have on the project schedule), were highlighted in the Feasibility Report as "inspiring examples".
The PWGSC CHARS team, particularly the Winnipeg regional office, which is responsible for real property projects in the Canadian Arctic and territories, has extensive relevant experience with a combined 30+ years of experience in Arctic construction and commissioning between the project management resources. The responsible Regional DG at PWGSC has experience with working with AANDC's ADM of NAO on the Giant Mine Remediation and Faro Mine projects, which have allowed for more effective communication at the management level based on the previously established strong working relationship with the ADM of NAO.
Additional consultations were undertaken by the AANDC CHARS team with other government departments and project teams that had used the construction management approach to understand the unique challenges associated with its adoption. The AANDC CHARS team was also able to draw upon the experience of the Canadian Network of Northern Research Operators and the PWGSC Centre of Expertise in Architecture and Engineering, which also possess experience in northern construction and laboratory design.
5.4. Financial Management
The audit examined whether financial management processes are in place to help ensure the reasonableness of project estimates as well as ongoing tracking of progress against the project budget and contingency allowances. It also examined the processes in place for cost monitoring and reporting against financial metrics.
5.4.1 Costing Methodology
As per the Treasury Board (TB) Policy on the Management of Projects, project approval must be received at key milestones of project progress, specifically for Preliminary Project Approval and Effective Project Approval. To facilitate these approval processes, the National Project Management System's (NPMS) costing methodology and milestone approvals are aligned with the TB costing definitions for indicative and substantive cost estimates. TB approvals are received at the end of both the Definition and Design Phases as part of the NPMS. PWGSC is responsible for ensuring that cost estimates are produced as defined in the NPMS for Class D, C, B, and A estimates, with cost estimates evolving to be more accurate and specific as the project progresses.
As part of the design and construction management services contracted to third parties, the production of indicative and substantive cost estimates are the responsibility of the Design Consortium and the Construction Manager. Both contractors use their own methodologies to prepare initial cost estimates in line with the NPMS. Once estimates are prepared, AANDC and PWGSC project teams have the opportunity to review the cost estimates in detail, to ask questions, provide comments, and to ensure all design implications and features are considered. The PWGSC COE for Architecture and Engineering also reviews cost estimates to ensure that technical building standards are met within the cost estimate and supporting design. Following Preliminary Project Approval and the contracting of construction management advisory services, it is PWGSC practice to have both the Design Consortium and builder, in this case the Construction Manager, prepare Class C cost estimates using their distinct methodologies to ensure that estimates are accurate. The approach to dual estimates will be used for the remainder of the project to reduce the risk of inaccurate project costing.
5.4.2 Costing Contingencies
There are two types of contingencies related to the design and construction of a facility such as CHARS: design contingencies and construction contingencies. Both are defined to ensure that unknown costs are accounted for and are reasonably considered within the estimated budget. Cost contingencies will vary based on the statement of requirements, project complexity, and known risks.
Design contingencies, those covering minor changes to the design requirements of the facility or program requirements that subsequently affect design requirements, have been produced by third-party contractors throughout the CHARS project. As cost estimates are reviewed and become more detailed and accurate, the design contingency percentage allocation is reduced to reflect the increased accuracy. PWGSC continuously monitors contract spending for estimated costs and expenditures against the design contingency within the Project Black Book. While third-party contractors do not account for construction contingencies, their costing methodology must include an allocation for construction escalation costs over the life of the project.
Construction contingencies are accounted for as part of the Risk Management Contingency and Pre-Approved Amounts for Anticipated Amendments (PAAAs), a financial allowance of anticipated amendments related to the identified risks to the project. Projects that are not well-defined or have many known risks will have a higher PAAA allowance. While approval by the PWGSC Regional Project Manager is still required to access PAAA funds, this funding is set aside as part of the overall project budget. The CHARS infrastructure construction portion has approximately $14.1M (ie. 12%) allocated to PAAAs to account for known project risks and construction contingencies.
Because construction contingencies are highly unpredictable, there is still a risk that the PAAA allowance will be insufficient to cover project risks. Recognizing this, AANDC has developed a list of potential scope changes to the facility design that will allow for cost savings within the construction budget and regulations of the facility.
5.4.3 Cost Monitoring
PWGSC's Project Manager is responsible for monitoring and reporting the financial status of cash flow and draws against contract amounts and PAAAs within the Project Black Book. These financial status updates are provided to the AANDC Project Manager on a monthly basis for review and inter-departmental discussion. The financial status of the project is also discussed in detail at the 'CHARS Project Management Committee' between the AANDC and PWGSC Project Managers, the Design Consortium, and the Construction Manager. Discrepancies are escalated to the AANDC Project Director and to the 'CHARS Senior Management Committee' as necessary. The construction management approach to the implementation phase will also facilitate financial monitoring and control because of the iterative nature and relative flexibility of the approach.
Recommendation:
No recommendations were identified in this area.
5.5. Procurement Management
The audit examined whether procurement processes are in place to support a contracting process that is rigorous, yet sufficiently flexible to respond to project changes in a timely manner. This included the procurement and contracting practices in place as well as monitoring and reporting processes. The audit also examined the level of consideration for project objectives in the procurement processes undertaken thus far.
In accordance with the Treasury Board Policy on the Management of Projects, PWGSC is a common services organization that provides project management and acquisitions support to other government departments in implementing real property projects that pose a significant cost investment, project complexity, or risk. PWGSC's role and responsibility as the procurement and day-to-day manager of acquisitions and contracts is defined within CHARS project documentation such as the Project Charter and Project Management Plan.
5.5.1 Northern and Project Considerations
Because of the location and characteristics of the CHARS facility, it was important that resources with experience in Arctic construction and with an understanding of Aboriginal considerations were involved in the development of the project Requests-for-Proposal (RFP) and subsequent contracts. As the regional office responsible for Arctic and territorial construction, the PWGSC Winnipeg office has been assigned the responsibility of project management and contracting. PWGSC has procurement requirements outlined within Comprehensive Land Claim Agreements and AANDC has responsibilities to adhere to requirements within the Nunavut Land Claims Agreement. As a result, Aboriginal opportunity clauses that provided contractors with incentives for using Aboriginal professional services, labour, and materials were included in the RFPs for both the design consultant and construction management services. In compliance with the Nunavut Land Claims Agreement, PWGSC notified several Aboriginal businesses of the upcoming procurement opportunity prior to issuance of the formal RFP to encourage the participation of Aboriginal businesses in bidding on construction management services work for the CHARS infrastructure project.
As part of the NPMS procurement process requirements, risk assessments were prepared prior to the issuance of RFPs for the design consultant and for construction management services to identify project and service-specific risks that could potentially impact the CHARS infrastructure project. Within both of these procurement risk management plans (ie. the Design Consultant Services Risk Management Plan and the Construction Management Advisory Services Risk Management Plan), the risk of client approvals and client needs being insufficiently defined for the stage of the project were identified. Following the procurement of the Design Consortium, it was identified in the Design Consortium's Risk Register and decided by AANDC that it would be beneficial to procure and hire the contractor for construction management services as soon as practicable to inform the construction tender packages and contracts to accommodate the ongoing S&T program development and required approvals. While the successful Construction Manager was only procured for advisory services to reduce the impact of risks associated with TB approval of funding, the contract provides PWGSC with an option to exercise the remaining construction management service portion of the contract once Effective Project Approval is received, thereby streamlining any subsequent contracting processes.
5.5.2 Procurement Practices
As procurement and acquisitions are the responsibility of PWGSC, departmental policy and practices were used to procure the major services required for the CHARS infrastructure project. This included the procurement processes for the design consultant and for the construction management services.
Procurement of architectural and engineering design consultant services: The PWGSC Real Property standard approach to evaluating proposals for architectural and engineering design consultant services requires the use an evaluation methodology that focuses heavily on the technical merit of the proposal and the bidder, and considers the financial bid as a relatively small component of the overall proposal score. In accordance with this standard practice, the design consultant services were subject to a final scoring methodology of 90% weighting for the technical merit score, which was awarded based on bidder qualifications, the quality of the proposed design, and other considerations and performance in the design exercise, with the remaining 10% weighting for the financial bid score.
Usage of a design exercise and design jury with relevant perspectives: In order to ensure that architectural and engineering design consultant services are awarded based on appropriate technical qualifications, it is PWGSC standard practice to have a two-part proposal process, with a group of bidders being required to prepare a detailed design and project approach that is presented to a design jury. The design jury included members from both PWGSC and AANDC project teams, technical experts from the construction and engineering industry, experts with experience in Northern construction, and academia. The jury provided input and recommended a bidder based on the total cumulative rankings given by jury members.
Fairness monitoring: For projects with a high visibility and a high level of public interest, such as the CHARS project, it is the policy of the PWGSC Acquisitions unit to use fairness monitoring services to oversee the procurement processes for both the design consultant and construction management services. The use of fairness monitoring services helps to ensure that the bidding processes were conducted in a fair and transparent manner. The fairness monitors oversaw the entire process, including attending and reviewing proceedings from the bidder's conference; the RFP question and answers response process; the drafting and issuing of amendments; the issuance of award and regret letters to bidders; and, the conduct of debriefs with unsuccessful firms to provide a rationale for the final scoring.
AANDC and PWGSC project team members' communication and interaction regarding procurement and contracting is defined, at a high level, within the Project Charter and Project Management Plan. Similar to the design jury, members of AANDC and PWGSC were part of the evaluation committee responsible for scoring bids received during the construction management services procurement process to ensure that the client organization and the overall CHARS project strategic objectives were being enabled.
5.5.3 Contract Monitoring and Reporting
As part of the Project Black Book, both the Design Consortium and the Construction Manager contracts are being actively monitored and reported on a monthly basis to AANDC by the PWGSC Project Manager. Contracts are monitored by:
- Total billed costs, remaining contract limitations, and forecasted costs;
- Contract amendment authorities and the level of delegated financial authority for amendments and change orders;
- Contract amendment amounts, date issued and approval information, and draws of amended contract values against the PAAAs; and,
- Initiated or contemplated change orders requested by contractors or the project team for discussion and approval/rejection.
Similar to team communications, there was initially some ambiguity on the process for initiating change orders to accommodate AANDC's needs as the client organization. The recently developed Communication Plan specifically requires that all change orders must be communicated through the PWGSC and AANDC Project Managers for discussion with the affected third party. Once clarified, PWGSC issues and approves change orders in accordance with established departmental processes. Change orders or amendments are entered into the Project Black Book once issued to ensure timely and accurate contract monitoring.
Recommendation:
No recommendations were identified in this area.
6. Management Action Plan
Recommendations | Management Response / Actions | Responsible Manager (Title) | Planned Implementation Date |
---|---|---|---|
1. The Assistant Deputy Minister of the Northern Affairs Organization should formalize the governance structure over the CHARS infrastructure project from the design-build phase, up to but not including the start of the operation phase, by ensuring that Terms of Reference, roles and responsibilities, and records of decisions are documented for key governance and oversight committees. | Terms of Reference and documentation processes for key governance and oversight committees will be formalized. | Assistant Deputy Minister, NAO | February 2014 |
2. The Assistant Deputy Minister of the Northern Affairs Organization should develop a plan to address the evolving governance structure of the CHARS infrastructure project as it moves through to the design-build phase of the project to reflect the changing needs for governance, up to but not including the start of the operation phase. | A plan to address the evolving governance structure will be developed. | Assistant Deputy Minister, NAO | March 31, 2014 |
3. The Assistant Deputy Minister of the Northern Affairs Organization should implement the integrated Risk Management Plan as soon as practicable, and maintain commitment to the risk management processes outlined therein. | The project Risk Management Plan, Risk Register template, and Executive Summary Risk Dashboard have been developed, and will be used in ongoing project governance and management practices. The Risk Management / Quality Assurance analyst role will be filled by a member of the CHARS team to lead the implementation and ongoing management of the Integrated Risk Management Plan. | Assistant Deputy Minister, NAO | March 31, 2014 |
Appendix A: Audit Criteria
To ensure an appropriate level of assurance to meet the audit objectives, the following audit criteria were developed to address the objective. Audit criteria were developed in alignment with Treasury Board’s Audit Criteria related to the Management Accountability Framework: A Tool for Internal Auditors (March 2011).
Audit Criteria | Sub-criteria |
---|---|
Governance and Oversight | |
1. An effective governance framework is established and communicated, including an oversight body, clear governance protocols and well established accountabilities. | 1.1 An oversight body exists with a clear mandate, terms of reference, appropriate membership and governance protocols, each of which are appropriately communicated. |
1.2 Clear project governance protocols exist for the project which describes roles, responsibilities and reporting relationships for all those accountable. | |
1.3 Team members are aware of their roles and responsibilities regarding governance. |
Audit Criteria | Sub-criteria |
---|---|
Project Management | |
2a. Project management processes, practices, frameworks, financial tracking, monitoring tools, project reporting and HR capacity exist to support the effective and efficient achievement of project objectives. | 2.1 The project employs a rigorous and documented project management approach which includes associated monitoring tools, frameworks, templates and reporting (both financial and non-financial). |
2.2 There exists sufficient, qualified, HR capacity devoted to project management for a project of this size and complexity. | |
2b. Project activities between teams are planned and coordinated in an efficient and effective manner to meet project objectives and stakeholder needs. | 2.3 Dependencies between the S&T and Facility project are clearly identified on the project plan. |
2.4 Risk management identifies and mitigates the risks related to the dependencies. | |
2.5 Communication protocols exist between the two projects to assist in identifying dependencies and changes which could impact the other project on a timely basis. |
Audit Criteria | Sub-criteria |
---|---|
Risk Management | |
3. A risk management approach exists which ensures the regular capture, reporting, analysis and mitigation of risks which may impact the project meeting its objectives | 3.1 A formal risk management methodology is in place and well documented. |
3.2 The risk management methodology ensures the regular capture, reporting, analysis and mitigation of risks. | |
3.3 People with the appropriate skills and expertise are used in the identification and analysis of risks. | |
3.4 Results of previous projects, related audits and findings are appropriately considered in performing current projects and mitigation strategies. | |
3.5 Risks are escalated to the appropriate level within the governance structure. |
Audit Criteria | Sub-criteria |
---|---|
Financial Management | |
4a. A financial management process is in place that helped ensure both the reasonableness of the initial project estimates as well as ongoing tracking of progress against financial metrics. | 4.1 Initial financial estimates have been prepared by individuals with experience in construction north of 60. |
4.2 A rigorous costing methodology was used to develop the cost estimates, including the communication of those estimates and subsequent changes. | |
4.3 Cost estimates were created with contingencies. | |
4b. A financial management framework is in place to address ongoing budgeting, cost estimation, use of funds, monitoring and reporting of progress against financial metrics. | 4.4 Financial management hurdles are built into project stage gates. |
4.5 An estimated budget for the facility construction is developed at the appropriate level of detail and forecasts/variances are reported and monitored to support efficient and effective project management and monitoring. |
Audit Criteria | Sub-criteria |
---|---|
Procurement and Contract Management | |
5. That the procurement and contracting process is rigorous, yet sufficiently flexible to respond to project changes in a timely manner. | 5.1 That individuals with appropriate expertise in the construction of facilities north of 60 were involved in the development of project RFPs. |
5.2 Processes and service standards exist between AANDC and PWGSC to define and inform the procurement arrangement for CHARS. | |
5.3 There exist communication protocols between PWGSC and AANDC project staff to resolve contracting issues on a timely basis. | |
5.4 The RFP's and resulting contracts are written with sufficient flexibility to respond to changes in the project while still meeting contracting guidelines. |
Appendix B: Relevant Policies/Directives
The following authoritative sources (i.e. Policies/Directives) were examined and used as a basis for this audit:
- Treasury Board Policy on the Management of Projects
- Department of Public Works and Government Services Act
- Public Works and Government Services Canada National Project Management System Policy
Appendix C: Key Terms and Definitions
These key terms and definitions are used throughout the report – this appendix is intended to facilitate the common and consistent understanding of terminology as it applies to the CHARS project and this audit.
Aboriginal Affairs and Northern Development Canada (AANDC)
AANDC is the responsible department with accountability for the overall success of the CHARS project, and the client organization to PWGSC. Roles and responsibilities specific to the infrastructure project include management, review, and oversight of project documentation and key deliverables, as well as ensuring the strategic direction of the CHARS infrastructure project and being the lead department on overall project approvals.
Arctic Research Infrastructure Fund
The Arctic Research Infrastructure Fund was the result of the Federal Budget 2009 announcement of new funding for AANDC (then Indian and Northern Affairs Canada) to invest in maintaining and upgrading key existing Arctic research facilities, with funds allocated to projects on a competitive basis, and to be completed by March 31, 2011. The ARIF was implemented with the objective of providing near-term economic stimulus while building a strong foundation for Arctic research capacity that supports government priorities.
The ARIF announcement supported the commitment of the federal science and technology strategy to promote world class excellence in research and leverage Canada’s research strengths to achieve an economic and social advantage. In particular, it supported the commitment and plan to build a world-class High Arctic Research Station, which will leverage existing infrastructure by serving as a hub for scientific activities in Canada’s vast and diverse Arctic region. Within the $87M funding envelope of ARIF, $2M of funding was allocated to the CHARS project to undertake a Feasibility Study.
Centre of Expertise for Architecture and Engineering
The Centre of Expertise for Architecture and Engineering is an independent, skills-based group that exists within PWGSC’s Real Property Branch with the responsibility of providing technical review and advisory services for real property and major crown projects in the areas of architecture, engineering, northern construction, project costing and quantity surveying, and laboratory design.
Change Order
Contemplated or authorized Change Orders are any changes or deviations from the original contract documents, whether monetary or non-monetary, which require official authorization from the contracting department prior to any extra claims or work done.
Comprehensive Land Claim Agreement
Comprehensive Land Claim Agreements (CLCA) are negotiated in areas of the country where Aboriginal rights and title have not been addressed by treaty or through other legal means. These agreements are modern-day treaties between Aboriginal claimant groups, Canada and the relevant province or territory. While each one is unique, these agreements usually include such things as land ownership, money, wildlife harvesting rights, participation in land, resource, water, wildlife and environmental management as well as measures to promote economic development and protect Aboriginal culture. Many CLCAs also include provisions relating to Aboriginal self-government.
Construction Manager
The Construction Management firm joint-venture, referred to in the report as the Construction Manager, was hired to provide construction management support and advisory services on the CHARS infrastructure project, including advising on the construction planning process, issuing construction tenders, and fulfilling the role of the general contractor and site supervisor during the construction phase.
Design Consultant
The Design Consultant or Prime Consultant, in conjunction with its key sub-consultants, was hired to provide full service expertise on the CHARS infrastructure project in architecture, civil, structural, mechanical, and electrical engineering, along with specialist sub-consultants, such as cost consultants, landscape architects, energy analysts, wind modeling and engineering, environmental specialists, urban planners, economic planning consultants, sociologists, etc., as required to deliver the project.
Feasibility Report
A feasibility report is a document that assesses potential solutions to a business problem or opportunity, and determines which of these are viable for further analysis. The purpose of the feasibility report is to present the project parameters and define the potential solutions to the defined problem, need, or opportunity. Having brainstormed a variety of potential solutions, the project team expands on each of these potential solutions, providing sufficient detail, including very high-level costing information, to permit the project leader to recommend to the approving authority all of the viable potential solutions that should be further analyzed in the next phase (Business Case). Project constraints and limitations of expenditure are among the various factors that will determine viability.
National Project Management System
The National Project Management System (NPMS) refers to PWGSC’s methodology for project management, which prescribes the basic minimum requirements that must be met in the project life cycle. The NPMS defines distinct control points that are linked to PWGSC's project approval processes, and identifies critical deliverables required at each phase.
Nunavut Land Claims Agreement
The Nunavut Land Claims Agreement is an agreement between the Inuit of the Nunavut Settlement Area as represented by the Tungavik Federation of Nunavut and Her Majesty The Queen in Right of Canada. The objectives of the Nunavut Land Claims Agreement are:
- To provide for certainty and clarity of rights to ownership and use of lands and resources, and of rights for Inuit to participate in decision-making concerning the use, management and conservation of land, water and resources;
- To provide Inuit with wildlife Harvesting rights and rights to participate in decision-making concerning wildlife harvesting;
- To provide Inuit with financial compensation and means of participating in economic opportunities; and,
- To encourage self-reliance and the cultural and social well-being of Inuit.
Procurement Approaches to Real Property
Construction Management: The construction management approach is designed to be an efficient methodology for undertaking real property construction projects. Under this approach, the department retains full control of the project scope and budget, and the construction manager is hired early on in the design process to work with the design consultant and project team in an advisory capacity to inform design constructability, sequencing, budgeting and project scheduling as set by the Department. The construction manager is responsible for tendering individual construction trade packages and playing the role of the general contractor once contracts with service providers are in place. The final cost of the project is only known once the final construction tender is awarded as a contract.
The construction management approach offers the greatest flexibility to manage changes to the project with the least impact on time, and was determined to be the most suitable approach for the CHARS project. Within this audit report, the procurement process is referred to as construction management services, while the selected contractor is referred to as the Construction Manager.
Design-Bid-Build: The Design-Bid-Build approach to real property projects involves a two-part, linear methodology. First, a design consultant is retained through a competitive proposal process, to design the project and prepare design and construction tender documents. The department maintains maximum control of the final product (design, selection of materials) through the design process. Contractors subsequently submit competitive bids based on the scope of work outlined in those bid documents and the successful bidder is awarded a contract to construct the project. The final project cost is known once a contract is awarded for the construction portion. Relative to the delivery time of other approaches, this is generally the lengthiest methodology, as construction may only commence once the entire design is complete.
Design-Build:This methodology is another variation of the Design-Bid-Build and is considered a 'best-value' procurement method. The design consultant and the contractor are retained as a team. Competitive bids provide an innovative industry-driven design solution in response to the detailed client requirements/performance specifications that outline the owner's need. Design-Build poses a challenge to real property projects in being able to fully describe the final infrastructure product and needs generically and completely. However, the final project cost is known in advance with the least investment of capital in the project to date.
Public Private Partnerships: PPP Canada defines a P3 as a long-term contractual relationship between a public authority and the private sector that involves the following:
- Provision of goods or services to meet a defined output specification (i.e., defining what is required, rather than how it is to be done);
- Integration of multiple project phases (e.g., design, construction, operations);
- Transfer of risk to the private sector, which is anchored with private sector capital at risk; and,
- A performance-based payment mechanism.
P3s can take the form of a number of different models that vary according to the level of private sector involvement.
Project Black Book
Project Black Book is a tool specific to PWGSC’s Western Region Real Property unit. The Project Black Book is an Excel-based workbook that tracks detailed financial and non-financial metrics of the project in the areas of project management, issues management, financial management, and contract management, and monitors project progress against the key performance indicators of project scope, budget, and schedule.
Project Management
Project management is the systematic planning, organizing and controlling of allocated resources to accomplish identified project objectives and outcomes. Project management is normally reserved for focused, non-repetitive, time-limited activities with some degree of risk, and for activities beyond the usual scope of program (operational) activities.
Public Works and Government Services Canada (PWGSC)
PWGSC is the delivery agent responsible for day-to-day project management of the CHARS infrastructure project, providing services to AANDC in the areas of project management, acquisitions, and procurement.
Risk Register or Risk Log
This is a risk management tool commonly used as part of a project management approach. It acts as a central repository for all risks identified by the project or organization, and for each risk, it includes information such as impact, likelihood, mitigation efforts, owner, etc. Risk register and risk log terminology is often interchangeable.
Third Party Contractor
Specific to the CHARS project, procured contractors that provide PWGSC with professional and technical services for architecture and engineering design and consulting services (the Design Consultant) and construction management services (the Construction Manager).
Treasury Board
The central agency providing oversight and approval to the CHARS project (both the infrastructure project and S&T program) in accordance with the TB Policy on the Management of Projects.
Treasury Board Cost Estimate Definitions
PWGSC’s National Project Management System and associate deliverables are aligned to the Treasury Board cost estimate definition below to allow for effective and efficient project submission and approvals. The current definitions are as follows:
Indicative Estimate: is an estimate that provides a rough cost projection used for budget planning purposes in the early stages of concept development of a project. It is usually based on an operational statement of requirement (SOR), a market assessment of products and technological availability that would meet the requirement and other considerations such as implementation, life cycle costs and operational savings. Indicative Estimates are used to seek Preliminary Project Approval (PPA) and Lease Project Approval (LPA).
Substantive Estimate: an estimate of high quality and reliability. It is based on:
- detailed system and component design, design adaptation, work plans and drawings for components, construction or assembly, and installation. It includes site acquisition, preparation and any special requirements estimates. Contingency funding requirements must be justified based on line-by-line risk assessments, including market factors, industrial capability and labour considerations;
- all significant and identifiable deliverables, as well as the costs of the government's contribution to employee benefit plans (20 percent of all salaries charged to the project);
- all agreed upon objectives, including those resulting from procurement review; and,
- market assessment, where acquisition is through lease, lease purchase or capital lease. The provisional allowance for fit-up or special tailoring requirements will be subject to review and possible revision at the contract approval stage.
Categories of Estimates:
For complex and sizeable Real Property projects, there are five categories of estimates prepared throughout the project phases.
- Class 'D' (Indicative) Estimate: to be in unit cost analysis format (such as cost per m² or other measurement unit) based upon a comprehensive list of project requirements (i.e. scope) and assumptions. The Class D estimate is evolved throughout the phases of the Project Identification Stage, finally being incorporated into the cash flows in the Analysis Phase. For more complex projects such as laboratories, elemental cost analysis and the input of specific disciplines may be required. The Class D indicative estimates developed during the National Project Management System (NPMS) Feasibility Phase shall be revisited with cost planners in the Analysis Phase before finalizing.
- Class 'C' Estimate: to be in elemental cost analysis format latest edition issued by the Canadian Institute of Quantity Surveyors and based on a comprehensive list of requirements and assumptions, including a full description of the preferred schematic design option, construction/design experience, and market conditions. Class C estimates are developed during the NPMS Design Phase.
- Class 'B' (Substantive) Estimate: to be in elemental cost analysis format latest edition issued by the Canadian Institute of Quantity Surveyors and based on design development drawings and outline specifications, which include the design of all major systems and subsystems, as well as the results of all site/installation investigations. Class B estimates are developed during the NPMS Design Phase.
- Class 'A' (Pre-Tender) Estimate: to be in both elemental cost analysis format as well as trade divisional format latest edition issued by the Canadian Institute of Quantity Surveyors and based on completed construction drawings and specifications prepared prior to calling competitive tenders. The Class 'A' Estimate is generally expected to be within 5% to 10% of the actual contract award price for new construction. Tendering risks should be included in the project risk plan and costed accordingly. The accuracy of Class 'A' estimates can be influenced by many factors, including project complexity, market volatility, site remoteness, schedule rigidity, and the clarity of contract documents. Class 'A' estimates are prepared during the NPMS Implementation Phase and can be a more accurate Substantive Estimate, depending on the complexity of the project.