Audit of Information Management/Information Technology Governance and Application Systems Integration

5.1.1 Management and Oversight Bodies

To determine whether effective management and oversight bodies for IM/IT initiatives were in place and functioning, the audit team assessed whether:

• IM/IT governance committees were in place;
• Terms of reference and membership were defined; and,
• Regular meetings were occurring and records of decision/ meeting minutes were retained.

We observed that an IM/IT Governance Committees framework was defined in 2011 which clearly sets out the relationship among the various IM/IT committees, as follows:

Key decision-making bodies defined in the framework were found to be in place, including Departmental Operations Committee (DOC), Director General Implementation and Operations Committee (DGIOC) and Information and Technology Stewardship Group (ITSG). For each, terms of reference were defined and approved, including membership from throughout the Department. It was noted that the membership of ITSG includes the regional Directors of Corporate Services, and DGIOC includes the Director General of the Regional Operations Sector, thus ensuring involvement of the regions in IM/IT governance. An examination of the six months from January to June 2013 indicated that regular meetings were occurring, attendance was tracked, and records of discussion prepared, including tracking of action items.

The Terms of Reference of ITSG sets out specific responsibilities for both IT and IM, including to provide consultation and recommendations on these areas to DGIOC and, if required, to DOC. A review of the records of discussion for both ITSG and DGIOC however indicate that the focus has been on IT while IM items are rarely discussed. DGIOC, at its July 2013 meeting, has recognized this as an area requiring improvement and is taking action to determine how to best advance and support IM priorities. This will be a key area as there are critical IM initiatives currently facing the Department, including recordkeeping compliance and the implementation of GCDocs.

Although the creation of Shared Services Canada does not change the overall IM/IT governance, the respective roles, responsibilities and processes are continuing to evolve between SSC and the Department, and the minutes examined reflected that SSC updates were regularly tabled.

An examination of a sample of IMB tactical committees, which included the Branch Management Team (BMT) and Architectural Standards Committee (ASC), confirmed that BMT meets on a regular basis and maintains records of decisions. For ASC, meetings have not been occurring regularly and the last record of discussion is from October 2012. The Director Enterprise Strategic Services, who is the Chair of ASC, has advised that ASC is being restructured to improve its effectiveness, including a revised Terms of Reference and membership. As noted in the IM/IT Governance Committee framework, ASC should play an important role in ensuring that "IM/IT investments comply with the approved departmental infrastructure design, hardware, software and information management standards, as well as Government of Canada policies and strategic investments". See further discussion regarding ASC in Section 5.2.2.

5.1.2 Strategic and Tactical Plans

The audit examined whether strategic and tactical plans for IM and IT are in place and are being implemented.

We observed that key planning activities have occurred. The IM/IT Strategy 2012-2015 has been developed and approved by DOC in February 2012. The Strategy outlines IMB's vision, strategic direction, and implementation strategy. An IM/IT Plan 2013 was then developed and sets out a number of initiatives to operationalize the strategy throughout 2012-13 to 2016-17, including initiatives relating to SSC transition planning. The Plan is refreshed on an annual basis. Additionally, a formal risk assessment for CFO Sector, which included IM/IT risks, was conducted in 2012, and has been incorporated into the CFO Sector 2013-2014 business planning exercise.

The IM/IT Strategy identifies as a core strategy the achievement of "IM/IT Plans approved through the corporate planning cycle and directly linked to government-wide and departmental objectives", which is consistent with TBS requirements as set out in the TBS Policy on the Management of Information Technology, which in turn has been adopted by the Department. To date however, there has been no structured process in place to ensure this alignment when developing the IM/IT Plan. This has been recognized as an area for improvement by IMB management and we have been advised by the CIO that TBS will also require this alignment in the IT Plan to be submitted to TBS by March 31, 2014. As a result, the CFO issued an integrated call in September 2013 to all sectors and regions for a five-year IM/IT, Procurement and Investment Plan, including templates for setting out specific initiatives with cost estimates developed with progressively less precision over each of the five years. Responses are due in late October 2013, and will be utilized as input to future IM/IT Plans and IM/IT resource planning, as this will provide information for planning purposes on all planned projects. These IM/IT Plans must also be developed within the context of the overall departmental Investment Plan, which is approved by DOC and which includes projects greater than $1 million.

Although the approved IM/IT Strategy was developed to integrate both IM and IT elements, a separate draft 'Enterprise Information Management Strategy 2010 to 2015' also exists, as does a draft IM Tactical Plan 2013-2014 to 2015-2016, which sets out 30 specific IM initiatives. We have been advised by the CIO that the IM/IT Strategy should supercede the draft Enterprise IM Strategy, and that a consolidated IM/IT Plan will be developed in the current fiscal year. Other than the IM toolset renewal, the current IM/IT Plan 2013 does not contain any IM initiatives, in contrast to the 30 set out in the draft IM Tactical Plan. At this time, it is unclear which IM priority initiatives have been approved, who has been assigned responsibility for each and whether progress is occurring.

Planning between IMB and the Sectors has been enhanced through the development of Memorandums of Understanding (MOU) in 2013-2014 between IMB and each sector to confirm funding for system development and maintenance activities. The MOUs address additional costs to be incurred throughout the year for non-core operational support and application development, including any related additional hardware/software and licensing. Costs charged under the MOUs relate only to incremental costs of hiring contractors and do not include any recovery of IMB employee time. This is the second year for the MOU process, and is enhanced from the prior year to establish MOUs by sector rather than on an individual application basis.

5.1.3 Policies and Directives

Policies and directives set out direction and guidance for the Department in the areas of IM and IT. In examining policies and directives, the audit team assessed whether:

• Policies and directives for key areas of IM and IT were developed and approved;
• Policies and directives were subject to regular review and updating; and,
• Policies and directives were effectively communicated to impacted parties.

Based on a review of the IM and IT policies and directives provided by IMB management, we observed that the suite policies and directives provides coverage of the expected areas, and, except for those that are still in draft, are consistent with the requirements set out in the related IM and IT policies and directives issued by Treasury Board Secretariat (TBS). We noted three AANDC directives still in draft as follows: Directive on Recordkeeping, Directive on Data Sharing and Directive on Data Stewardship. A comparison of the requirements of the draft AANDC Directive on Recordkeeping to those of the related TBS directive noted three areas that are required by TBS but are not included in the Department's draft Directive on Recordkeeping: conducting a risk assessment of resources with business value, establishing taxonomies or classification structures of resources with business value, and the communication of the risk of poor record keeping to employees. As departments must be compliant with the TBS Directive on Recordkeeping by March 2015, it is important that AANDC finalize and issue its Directive so that all related compliance activities can be defined and implementation appropriately planned.

The process for reviewing and updating IM and IT policies and directives is currently informal and inconsistent. We were advised by management that policies and directives are reviewed internally every three years and when the TBS issues new policies or directives or amends existing ones. Through our testing, we noted examples of policies that should have been reviewed according to these criteria but were not, including the AANDC Policy on Information Management (dated October 2008) and the INAC Security Management Framework (dated July 2008, and which contains the IT Security policy statement). Both examples have not been updated in five years, while the related TBS documents have both been updated since 2008. In addition, a recent IMB review of existing policies, directives and standards identified that a number of existing documents are improperly classified as policy versus directive versus standard.

IM/IT policies and directives are published on the IMB intranet page, to be referenced by impacted employees. We identified that some IM/IT directives that have not yet been finalized have also been posted, including the draft AANDC Directive on Recordkeeping and a draft AANDC Directive on Mail and Messenger Services. In addition, the AANDC Directive on IM/IT Project Portfolio Management which has been approved and was effective November 1, 2012 is posted on the Project Management page of the IMB intranet with 'track changes' active in the table of contents and a 'draft' watermark throughout.

The Department has recently issued two key documents related to IM/IT management – the AANDC Policy on Management of IT and the AANDC Directive on IM and IT Procurement Authorization – which set out requirements for CIO approval of all IM/IT-related architectures and procurement, and specifies financial coding for all such procurement. We were advised that the only communication of these items throughout the Department was through the records of discussion of the approving governance committees, with the committee members then responsible for further communication. This policy and directive have wide applicability across the Department, including to all program and project managers and to all responsibility center managers who provide financial coding for these items. A lack of appropriate and consistent communication of expectations to those impacted can have a negative impact on the rate of compliance.

5.2.1 Project Portfolio Management

A Project Portfolio Management Framework (PPMF) has been developed and documented, based on TBS guidance, to provide governance over IM/IT-enabled projects. The PPMF includes seven gates, or decision points, in a project life-cycle as follows:

• Gate 1 – Strategic Alignment
• Gate 2 – Project Approach
• Gate 3 – Business Case
• Gate 4 – Project Charter
• Gate 5 – Plan and Specifications
• Gate 6 – Solution Complete
• Gate 7 – Utilize and Retire

The PPMF includes requirements for approvals by governance committees at each gate and various project deliverables are specified at each gate to support the required approvals. Since its initial implementation in 2010, the PPMF was enhanced in 2011 to require an evaluation of opportunities based on their size, complexity and risk and to streamline the process for those of lower risk. The original PPMF and its enhancements were approved by the governance committees, including DOC. The PPMF documents and templates are published on the IMB intranet site. The AANDC Directive on IM/IT Project Portfolio Management was issued with an effective date of November 1, 2012 and sets out the governance to support adherence to the PPMF.

Our review of the project portfolio and detailed testing of a sample of three active projects indicated adherence to the requirements of the PPMF, except for the following areas:

• The requirements for integration of IM into PPMF processes are unclear. There is an 'IM Engagement in Project Management' document but it is in draft, even though it is dated 2011. Evidence of assessment and integration of IM requirements for all three projects tested was not available. For one project – Nunavut Map Selection – a Recordkeeping Agreement was prepared but there was no evidence of approval. For the others, although IM involvement may have occurred, it appears to be informal and no evidence was available.
• The Project Charter for FNCFS IMS was not signed off by the sponsor or Project Steering Committee. Instead of going through the standard departmental project approval gating process, the Branch submitted the Project for central agency approval. Although the Branch did not get the Project Charter approved by the sponsor or Project Steering Committee, the Project Charter was approved through the central agency approval process.
• Although gating approval by governance bodies is a key component of the PPMF, we were unable to locate current documentation that clearly specifies which governance bodies are responsible for approvals at each gate for each of the full and light PPMF process^{Footnote 1}. The current requirements for approvers by gate was only available by following changes approved by DOC in March 2011, with clarifications provided by the Project Management Office (PMO), who indicated that only the full and light processes have been implemented. A June 2013 presentation prepared by a customer relationship manager included the current requirements but the presentation was not generally available even within the PMO.
• Project close-out is not occurring, although it is a requirement of gate 7. Based on the listing of projects obtained from the PMO, there are 15 projects in gate 7 requiring close-out, some since 2010. Project close-out is an important phase in documenting lessons learned and benefits realization.

We were advised that some existing applications had been developed outside the PPMF, although no current projects were cited. The Department recently issued a Policy on Management of Information Technology which is effective January 1, 2013 and which prescribes that all projects with an IM/IT component be pre-approved by the CIO. Also, as noted previously, CFO sector has recently issued an integrated call to all sectors and regions for information on five-year investment plans. The combination of these two initiatives should prevent and detect circumvention of the PPMF in the future, although the effectiveness of this has yet to be proven.

5.2.2 Application Systems Integration

Application systems integration has been identified as an issue in the Department, with some existing applications having been built in silos, resulting in a lack of data sharing and potential for inconsistency of data across applications. Since implementation of the PPMF in 2010, application systems integration has been promoted by the requirements of gates 2 and 3 of the PPMF – Project Approach and Business Case respectively – where an assessment of the extent of reuse of systems, data, and business rules is required as part of the "Options Analysis" deliverable. Also contributing to integration, technology standards have been defined and a 'common data elements' database exists, although it was noted that this database is limited to only eight data elements, including items such as band, reserve and tribal council. Additionally initiatives such as enhancements to the data warehouse in 2013 to include education and child and family services contribute to addressing the departmental priority of improving information for decision-making.

To facilitate increased integration and reusability of information going forward, the IM/IT Strategy has identified the move towards enterprise architecture (EA) as a key enabler and "the foundation for core processes that facilitate the appropriate level of business integration in developing IM/IT solutions." Consequently, the IM/IT Plan for 2013 identifies a number of initiatives around EA, starting in 2012-2013 and extending over the 5-year planning cycle. However, the key EA initiatives identified for 2012-2013, which included the development of an EA strategy and integration of EA into the PPMF, did not occur due to resource issues. EA initiatives for 2013-2014 and subsequent years were to build on these 2012-2013 initiatives, so will now be delayed.

The Department engaged Gartner Consulting in 2012 to benchmark the application portfolio and assess applications based on business value, technical condition and support costs. This is valuable input to a potential rationalization of the application portfolio, with a focus on retiring applications that are of lower value and may have technical issues or relatively high support costs. The application portfolio information is now available in a tool for assessment purposes, and this can be used in support of the EA initiative.

In addition, as discussed in section 5.1.1, ASC is currently inactive, yet it is the body responsible for enforcing compliance with technology standards and will be expected to play a role in implementing EA.

5.3.1 Roles and Responsibilities

To assess accountability, we examined whether an organization structure is documented for IMB, with clear assignment of roles and responsibilities. We also examined whether roles and responsibilities as defined by TBS requirements had been clearly defined within IMB.

We found that the organization structure for IMB is set out in an organization chart dated April 2013, and this has been approved by the CIO and CFO. We were advised that the structure for the Enterprise Strategic Services Directorate as set out in the organization chart is in the process of being revised to more accurately reflect some of the Branch's priority areas, including implementing enterprise architecture, but that it is otherwise current. For IT, the roles and responsibilities per the organization chart were compared to the eight IT workstreams defined by TBS and were found to be clearly assigned to a directorate. For IM, the requirements of the TBS Directive on Information Management Roles and Responsibilities were examined and roles of IM specialists found to be consistent with the roles and responsibilities assigned within the IM directorate. The CIO has been designated as the senior official to represent the deputy head in discussions with TBS on IT and IM issues, as set out in the Department's Policy on Management of Information Technology and Policy on Information Management respectively.

5.3.2 Monitoring and Accountability

We examined whether key monitoring and accountability activities are performed in the following areas:

• Initiatives as set out in the IM/IT Plan;
• The portfolio of IM/IT-enabled projects; and,
• IM/IT spending in the regions and sectors.

Monitoring of compliance with the IM/IT strategy and the initiatives set out in the IM/IT Plan occur both informally through the CIO monthly meetings with each of his Directors and formally through the CFO Sector Quarterly Report. An examination of the CFO Sector quarterly report for the 2012-2013 fiscal year indicated that key IMB initiatives as set out in the IM/IT Plan were being tracked on a quarterly basis and results reported. Where quarterly deliverables had not been achieved, the reasons were noted as well as a plan to remediate, and this was tracked in the subsequent quarter. The exception to this is the key initiatives for enterprise architecture which, per the CIO, were parked for 2012-2013 since they had no resources to address these items at the time.

Although annual Management Accountability Framework (MAF) reporting is also a key monitoring tool, TBS has taken a modified, risk-based approach to MAF reporting, assessing Departments on a selection of high-risk / high priority Areas of Management. As a result, Departments have not been required to report on the MAF areas related to either IM or IT in the past three fiscal years.

IMB has also recently introduced project portfolio reporting to DGIOC and DOC, with the first presentation to DGIOC in May 2013 and DOC scheduled for late October. This has been implemented in response to a concern by governance bodies that individual projects get approved through the governance committees, but they are then managed through their development and implementation by the project Steering Committees, without further oversight by the senior governance committees. As a result, a process has been initiated to monitor the health of the portfolio of projects more closely, with portfolio reports to be prepared for DGIOC semi-annually, although expected to become quarterly, and the frequency to DOC yet to be confirmed. An examination of the May presentation to DGIOC noted it included a summary of the portfolio of projects by state (active, on hold, etc), by gate, by strategic outcome and by program area. The related Record of Discussion of DGIOC indicated a detailed discussion of the portfolio areas with requested enhancements to address risk areas on a go-forward basis.

The AANDC Policy on Management of IT, which was effective January 1, 2013, and the AANDC Directive on IM and IT Procurement Authorization, which was effective March 31, 2012, were issued to enhance control over IM and IT expenditures across all sectors and regions by requiring CIO pre-approval for all IM/IT expenditures as well as common financial coding to enable consistent tracking of IM/IT expenditures. A key activity being implemented to monitor compliance with these instruments is the IM/IT spend review currently underway, which was originally undertaken since required by Treasury Board, but will be used for additional internal monitoring purposes. This IM/IT spend review requires reporting from each sector and region of 2012-2013 IM/IT expenditures, including both salary and non-salary items, and classified into five categories as follows: distributed computing, application/ database development and maintenance, production and operations computing, telecommunications, and IT security. This data is expected to be gathered annually.

If this IM/IT expenditure exercise is to be an effective monitoring tool in determining whether all projects undertaken by regions and sectors were appropriately pre-approved, further detail will be required as to specific projects undertaken in each of these categories. The current reporting templates provide information within each category by type of expenditure, including salary, hardware, software, etc., but no information as to specific projects or initiatives that make up these amounts. As an example, our testing of BC region indicated approximately $159,000 was spent on a 'thin client' project, with the only evidence provided by the region to indicate approval by the Director of Enterprise Strategic Services is an e-mail in February 2013, which approved replacing desktops only and which was issued after the project was underway. Information on this project is not evident from the current IM/IT expenditure analysis. We note that this is the first year the related policy and directive have been in effect, and that the spend review is currently in process, with the immediate focus on submission of expenditure amounts to TBS for October 15, 2013. As a result, further analysis of the financial information and the follow-up by the CIO of any indications of unusual or potentially unauthorized spend activity has yet to be completed.