Audit of the Climate Change Adaptation Program

Established Governance Structures and Strong Collaboration and Documentation Related to Committees

The audit identified that there are robust governance structures established and implemented in the three Programs. The Program had clearly established governance structures with defined roles and responsibilities, documented in Terms of Reference. Through the review of meeting documentation such as records of decision, meeting agendas, and meeting minutes, it was evidenced that meetings were held regularly and consistently. In particular, advisory committees across the programs demonstrated consistent documentation and comprehensive meeting minutes regarding the vision of the Program, existing and renewed funding, financial status updates, key project updates, as well as key actions items for the Advisory Circle members and the respective Program team. The content discussed in these meetings was considered essential for supporting the Program's objectives. Additionally, collaboration among CCPN, FNA and ICBCM was outlined in interviews and exhibited through meeting documentation.

Detailed Documented Program Guidance Documentation

The audit noted that the Programs have developed detailed Program Operational Guides (POGs), as well as a Process Guide to Contribution Agreement to provide guidance and support to the Program staff. Each of the three Programs demonstrated a POG that included Program background, Program descriptions (i.e., overview, objectives, logic model, Program activities, funding authorities, Program participants, linkages with other Program, project funding eligibility criteria, project proposal process), and Program roles and responsibilities (i.e., headquarters Program staff, regional staff, Program partners/collaborators, and Program funding recipients). The POGs were sufficiently thorough to support the activities of the Program. The Process Guide to Contribution Agreement contained detailed information on the steps required to complete contribution agreements by the program. The Process Guide to Contribution Agreement, as well as two out of the three Program's POGs were updated and revised within the audit period (FY2021 to FY2024). However, the third POG had not been updated since 2019, and Program Management confirmed that there have not been any significant updates to the processes since 2019. Based on this confirmation, the audit team deemed the existing documentation to be sufficient.

Opportunity to Further Strengthen Training Consistency and Execution

The audit identified limited documentation around training within the CCACE. Through document review and discussions with Program Managers, it was observed that apart from these POGs, the Process Guide to Contribution Agreement, and orientation materials, no other evidence of comprehensive training materials was provided by the Program. Program-specific and role-specific training materials were not available across all Programs. The primary method to deliver training is informal, as such there is no formal process in place to deliver training, to track training attendance or follow up training that was not completed.

Interviews with Program Managers communicated that most of the training occurs on the job. However, this is not documented to evidence its completion. Additionally, Program Managers outlined that the Program have been under significant time and resource constraints due to the pandemic and emergencies over the past three fiscal years (2021-2024). As such, there is limited documentation for these areas.

Not having formally documented training poses several risks to the Program's effectiveness and continuity. Two key examples of these are that (1) the inability to document and track training limits the Program's strategic adaptability, making it difficult to respond effectively to emerging challenges and opportunities without a clear plan for knowledge transfer and staff development. Additionally, (2) the absence of documented training materials can result in operational delays and inefficiencies during turnover, as new or temporary staff may struggle to understand their roles and responsibilities without adequate training and guidance.

Established Process to Support Funding Recipients and Submitting a Request and Receiving Funding

The audit found that there is an established process to receive and assess project funding requests, including guidance on eligibility criteria in the POGs. Through interviews, it was noted that Programs follow an established process to review and approve funding requests. In interviews, it was also noted that the Programs provide funding recipients support in drafting and completing proposals, facilitating a smoother and expedited funding request process. In order to make the funding more accessible, the Programs accept project funding proposals year-round, allowing for continuous submission and review. During three interactions with recipients, including site visits and interviews, it was observed that they received adequate guidance and support from the Programs to complete their project proposals and accompanying documentation.

Opportunity to Standardize Tools and Processes in Project Approvals

The audit revealed that some Programs completed robust documentation in the funding approval process for projects sampled, while others did not or did not do so consistently. Maintaining additional records and/or standardizing the documentation processes for the three Programs in-scope, where appropriate, (i.e., checklists, scoring documents, approval feedback notes, etc.) could enhance accountability and transparency around the approval process, as well as improve tracking and reporting processes. For the projects sampled, evidence of project review and approval documentation was not consistently completed among the three Programs in-scope. The absence of detailed records, such as resource allocation guidelines, calculations, meeting minutes, and records of decisions, made it challenging to transparently outline the process for prioritizing projects and allocating funds.

Interviews with funding recipients and Program Managers indicated that discussions regarding project selection and funding may occur during meetings. However, these meetings are not consistently documented, and some are held through informal channels. Not having formal documentation hampers the ability to track and justify decisions made during these discussions, affecting the transparency and accountability of the funding allocation process. Some projects contained checklists directly aligned with the eligibility criteria, and/or meeting notes indicating discussions about project pros and cons, suggesting a review process. However, this was not consistently done to provide sufficient information discussing the funding decisions made.

While the different Programs strive to address the needs of various communities through their funding decisions, the documentation gaps present several risks. Two key examples of these are that (1) not having clear documentation of the rationale behind project prioritization and funding allocations makes it difficult to assess whether the most critical projects, in terms of need and impact, are being selected and funded, potentially affecting stakeholder trust and confidence in the process. Additionally, (2) the absence of a formal, documented process increases the risk of bias and inequity in funding allocations, as funding may not be distributed fairly and consistently without clear criteria and justification for decisions.

Opportunity to Enhance Tracking of Project Information and Implement Additional Review Controls over Internal Data and Reporting

The audit identified errors in the internal Project List/Quarterly Reporting of all three in-scope Programs (FNA, CCPN, and ICBCM). These informal internal communication reports reviewed for them included a Program summary, as well as summary of projects funded by region (i.e., total number of projects, and Program funding, per fiscal year), and a listing for each region, including the project recipient, project name, project description, and yearly funding. Mathematical errors were observed related to the accuracy and completeness of projects and project funding, leading to inaccuracies in total project numbers, individual project funding amounts, and total project funding.

Interviews with Program Managers highlighted that the inaccuracies stem from manual addition errors. While manual reviews of the tracking documents were outlined in interviews, there was no evidence to support that these reviews were actively in place and completed by a reviewer other than the preparer of the quarterly report. In interviews, it was confirmed that these reports are internal summaries that are used for communications within the Program only and that they are not relied on to compile external reporting or financial statements. While the information in these reports is not public facing, it is still suggested to ensure better oversight over these figures to maintain accurate records over the projects. Management also noted that projects are consistently tracked within Project Tracking Sheets for FNA, CCPN and ICBCM. However, formal review and reconciliation procedures over these tools were not formally documented and as such, could not be assessed.

Errors in tracking and reporting could lead to several issues. Two key examples of these are that (1) discrepancies in project tracking can result in the misallocation of resources, where funds may not be distributed effectively according to project needs, impacting the successful execution of projects and hindering Program objectives. Additionally, (2) inaccurate data can undermine the accuracy of program assessments and reporting, potentially leading to misaligned resource allocation, delayed identification of issues, and challenges in maintaining transparency with stakeholders. While the quarterly reports are not directly used for decision-making, maintaining accurate data remains essential for effective program oversight and accountability.

Flexible Reporting Through the Use of Data Collection Instruments (DCIs)

The audit revealed that the Program have implemented a flexible reporting system, enabling funding recipients to report on their projects upon completion in any format they prefer, such as a community engagement report or video. Additionally, recipients are required to complete DCI templates annually, which are designed to minimize effort. Reducing the annual reporting requirements for recipients reduces their administrative burden, allowing recipients to better allocate resources, and effectively manage project milestones. The Programs have consistently utilized DCI templates, which recipients generally completed and returned. Follow-ups were conducted for any missing or late submissions. Upon project completion, the flexible final reporting allowed project officers to gain a thorough understanding of the outcomes through creative presentations that were better tailored to the communities served by the projects.

Commitment to Continuous Improvement

The audit revealed that the Program is committed to continuous improvement. CCCED has established oversight and an ongoing performance evaluation process, which includes participation in departmental evaluations, OAG audits, reflections with governance committees, involvement in bilateral climate change discussions, and the development of the Indigenous Climate Leadership (ICL) Agenda. This was identified by the Program and further supported through the review of committee participation and resulting committee/meeting documentation. The Program outlined that this macro-level process has significantly influenced the direction and delivery of the Program. These processes have also supported the CCACE in the formulating of its policies, in building of capacity and collaboration and partnerships through facilitating communication between different stakeholders, including governments, non-governmental organizations, and the private sector.

Opportunity to Improve DCI Template Tracking for Greater Program Transparency and Accountability

Missing or incomplete DCI template submissions could pose several risks. Two key examples of these are that (1) without sufficient and timely updates, issues may become significant before being addressed, and opportunities may be missed. Program may not be able to adapt to changing circumstances and emerging trends, leading to less agile responses to new challenges and opportunities. Additionally, (2) not conducting periodic updates could lead to an erosion of trust and confidence among stakeholders, as they may not see progress. Without leveraging success stories, there is a risk of not being able to provide evidence of impact to Program stakeholders, including partners and Indigenous communities.

Opportunity to Develop a Structured and Documented Process for Integrating Indigenous Community Feedback into Program

The audit noted that there does not exist a documented feedback process for incorporating Indigenous community input into the three in-scope CCACE programs for the period 2021-2024. Although some lessons learned documentation was observed, there is not a regular and systematic approach to capturing and addressing feedback across all three of the in-scope Programs. While some evidence from meetings and reports outlined feedback from Indigenous communities and other stakeholders, there were no clear and consistent mechanisms in place to request, collect, analyze, and act on feedback. Document reviews showed that the CCACE does not utilize surveys and questionnaires, interview or focus groups, or feedback forms to directly gather and synthesize feedback from funding recipients. In interviews with Program Managers, it was noted that feedback is received verbally and informally through meetings with recipients. While documented feedback from Indigenous communities was available outside of the audit period, in the form of surveys, reports, and within the Evaluation of the Climate Change Adaptation and Clean Energy Program 2016-17 to 2020-21, no formal documentation of feedback from Indigenous communities was available for the audit period 2021-2024. The absence of formal feedback requests and limited documented communications, such as meeting minutes and records of decision, made it challenging to assess how stakeholder input was solicited, collected, processed, and acted on.

While informal and verbal feedback may be gathered, not having structured documentation and systematic feedback processes impedes the Program's ability to maintain a clear record of stakeholder input and its incorporation into Program adjustments. This gap in formal feedback and lessons learned processes introduces several risks and challenges for the Program. Two key examples of these are that (1) not having formal documentation of feedback processes compromises transparency and accountability within the Program, making it difficult to demonstrate how community feedback is considered and integrated, which could lead to an erosion of trust among stakeholders. Additionally, (2) this limits the Program's capacity to adapt and respond to Indigenous community needs, as not having documented evidence hinders the ability to make informed adjustments based on stakeholder input. Not making these adjustments could potentially affect the Program's ability to build and maintain collaborative relationships with communities.