Archived - Evaluation of the Implementation of INAC's Gender-Based Analysis (GBA) Policy

Date: June 27, 2008

PDF Version   (419 kb, 55 pages)

Table of Contents

• 1. Introduction and Overview
  • 1.1 Introduction
  • 1.2 Federal directives and developments in Gender-Based Analysis
  • 1.3 INAC's approach to implementing Gender-Based Analysis
• 2. Evaluation Objectives and Methodology
  • 2.1 Evaluation objectives
  • 2.2 Evaluation methodology
  • 2.3 Research limitations
• 3. Relevance, Impacts and Effectiveness
  • 3.1 Relevance of INAC's Approach
  • 3.2 Success and Impacts
  • 3.3 Effectiveness of implementation
  • 3.4 Summary
  • 3.5 Accountability and Performance Measurement
• 4. Towards the Future: Issues Raised and Considerations
  • 4.1 Success factors and challenges
  • 4.2 Best Practices and Lessons Learned
  • 4.3 Potential modifications
• 5. Conclusions and Recommendations
  • 5.1 Conclusions
  • 5.2 Recommendations
• Appendices
  • Appendix A: List of Acronyms
  • Appendix B: Evaluation Matrix
  • Appendix C: Management Response / Action Plan

1. Introduction and Overview

1.1 Introduction

This report presents the findings of the Evaluation of the Implementation of INAC's Gender-Based Analysis (GBA) Policy conducted by the Audit and Evaluation Sector of Indian and Northern Affairs Canada. The Policy defines GBA '…as a lens of analysis that examines existing differences between women's and men's socio-economic realities as well as the differential impacts of proposed and existing policies, programs, legislative options, and agreements on women and men.'

The evaluation examines the relevance and impact of the Policy on INAC's efforts to apply gender-based analysis throughout the department (and thus meet federal requirements for the implementation of GBA across government); the effectiveness of the current implementation strategy, commonly known as the 'Repositioning Strategy,¹ and possible ways to better achieve and monitor intended outcomes.

The evaluation was conducted in response to the requirements of the 1999 Policy and coincides with the completion of the Repositioning Strategy which has guided implementation since 2003. It is expected that the results of this evaluation will inform the Department's future approach to implementing the Policy and integrating GBA into INAC's work.

The remainder of this chapter provides an overview of federal directives and developments in gender-based analysis and INAC's approach to integrating gender-based analysis into the Department's work. Chapter 2 reviews the evaluation objectives and methodology. Chapters 3 and 4 present the evaluation's findings with respect to relevance, impacts and effectiveness, while Chapter 4 highlights issues identified as facilitating or challenging the implementation of GBA, best practices and lessons learned, and possible areas for improvements. Conclusions and recommendations follow in Chapter 5. Appendices to the report contain a list of acronyms and the evaluation matrix which guided the enquiry.

1.2 Federal Directives and Developments in Gender-Based Analysis

In 1995, the Canadian government adopted a Federal Plan for Gender Equality.² The 1995 Plan called for the application of GBA in all federal policy development and analysis processes:

The federal government is committed through the Federal Plan to ensuring that all future legislation and policies include, where appropriate, an analysis of the potential for different impacts on women and men.

Individual departments will be responsible for determining which legislation or policies have the potential to affect women and men differently and are, therefore, appropriate for a consistent application of a gender lens [and] … will assume responsibility for undertaking gender-based analysis as appropriate within their operational spheres of activity (p.17-18).

As noted above, the Federal Plan outlined the responsibilities of departments and agencies for implementing gender-based analysis. The Plan emphasized shared responsibility, whereby Status of Women Canada (SWC) would work with 24 individual departments in applying the knowledge of GBA to policy and program development. SWC developed a GBA curriculum using a policy and program development framework, training materials and tools. SWC also developed and delivered customized training to departments, and developed a "train-the-trainer" program.

In May 2006, the House of Commons Standing Committee released its second report on the status of women in Canada.³ which made a number of recommendations for improving and sustaining the implementation of gender-based analysis across government. One recommendation was the integration and application of accountability mechanisms, and the suggestion that central agencies be responsible for their coordination.

The Government's response⁴ to this report noted that while Treasury Board Secretariat, Privy Council Office and Finance, as central agencies, each play a critical "challenge" role the responsibility of individual departments and agencies to ensure the completion of a thorough analysis of proposed policies and programs, including the application of GBA and the inclusion of gender considerations, nevertheless remained with individual departments.

Further, SWC aimed to continue to support both departments and central agencies to increase the sustainable application of, and accountability for, gender-based analysis. To this end, SWC would support departments in implementing organizational mechanisms and structures and by developing and delivering customized training and tools, case studies and public awareness materials. Additionally SWC would continue to work with departments on the creation of sustainable frameworks and accountability mechanisms and with the central agencies both to improve accountability and ensure a GBA lens is applied to all new proposals.

As noted by INAC's Gender-Based Analysis Champion in December 2007, GBA has been the subject of increasing interest and attention across government. Developments of note include:

• The revision of Treasury Board Submission Guidelines in late 2007 to remind departments of their responsibility to develop policies and programs compliant with overall federal policies. For instance initial analyses of GBAs should be implemented at the Memorandum to Cabinet (MC) stage to ensure their integration into programs and initiatives. and the findings respecting GBA compliance be reported in TB submissions; ⁵
• As part of its mandate to assess analytical capacities of policies and programs in relation to the Management Accountability Framework, Treasury Board has been tasked with assessing departmental use of GBA, determining whether it is appropriate to departmental needs;
• SWC is working with central agencies and departments to build their GBA capacity and to meet their accountabilities. This work has thus far included the creation (but not the implementation) of a set of indicators on trends in the situation of women in comparison to men;
• Each of the central agencies has appointed a senior official responsible for GBA who sits on a Steering Committee on Gender Equality with the overall goal of enhancing their organization's horizontal policy coordination and challenge functions in relation to GBA;⁶
• SWC announced that in 2008-09 it would provide support to selected departments in order to meet the specific needs of women, with a particular focus on Aboriginal women and women's economic prosperity; and
• SWC plans to work with other federal provincial and territorial departments in 2008-09 on evidence-based and culturally relevant policies (in the areas of Aboriginal women's human rights, economic prosperity and legal status), and with federal governments (in collaboration with Aboriginal women, provinces and territories) on action plans to enhance Aboriginal women's health, well-being, equality and security;⁷

1.3 INAC's Approach to Implementing Gender-Based Analysis

INAC created the Office of the Senior Advisor on Women's Issues and Gender Equality in 1998, which became the Women's Issues and Gender Equality Directorate in the following year (hereinafter cited as 'WIGE' or the 'Directorate'). WIGE is currently housed within the External Affairs and Gender Issues Branch (Policy and Strategic Direction Sector).

WIGE's mandate is to develop and implement a gender-based analysis policy in order to address gender equality issues across all departmental priorities, and as they relate to First Nations and Northern partners.

The Department's Gender-Based Analysis Policy

The development of INAC's GBA Policy of 1999 was informed by the Assembly of First Nations Women's Secretariat (now the Assembly of First Nations Women's Council) and the Native Women's Association of Canada, and developed in consultation with the departments of Justice, Human Resources and Development Canada (now Human Resources and Social Development Canada), Status of Women Canada, and the Canadian International Development Agency.

The Policy requires the application of gender-based analysis in all of the department's work, including all activities conducted and documents produced by the department to ensure they are developed to be as equitable and accessible as possible for all departmental clients.

The requirements, or objectives, of INAC's Gender-Based Analysis Policy are '…that:

• Gender-based analysis be integrated in all of INAC's work, including but not limited to:
  • The development and implementation of departmental policies, programs, communication plans, regulations and legislation;
  • Consultations and negotiations (including but not limited to self-government and land claims, treaty land entitlement and devolution);
  • Instructions and strategies on research, dispute-resolution, and litigation;
• Where gender equality issues arise, solutions be developed and implemented to prevent and remedy any inequality; and
• Where gender-equality issues cannot be addressed or fully addressed, the Women's Issues and Gender Equality Directorate be informed in a timely fashion and the issue be raised by the officer responsible for the policy, program or legislation with the Deputy Minister and, where appropriate, with the Minister.'⁸

Changes in INAC's Approach to Implementation

In 2002, INAC's Senior Policy Committee approved a Framework for Action which reaffirmed the Department's commitment to the Federal Plan for Gender Equality and identified a three pronged approach to integrating GBA into INAC's corporate priorities and strategic policy planning. This approach favoured shared responsibilities and training with sectors and regions, and identified specific activities where it was felt that Gender Equality Analysis (as GBA was then called) could be implemented.⁹

Subsequently, in 2003 INAC formally revised its service based approach to implementing the GBA policy to that of a capacity building approach. Among the issues contributing to this change was the tendency for requests for WIGE support (for reviewing MCs for example) to arrive at the last minute resulting in a reactive, troubleshooting, approach that was not believed to be sustainable.¹⁰

In order to better implement the Policy, WIGE proposed to develop and lead an approach which would seek to build capacity and competency throughout INAC with the goal of proactively supporting INAC's strategic priorities, goals and vision.

"The strategy places an emphasis on ensuring the responsibility to conduct GBA remains in the hands of the subject matter experts who design and implement policies, program, agreements and legislation as was originally intended by the GBA Policy".¹¹

INAC's Operations Committee adopted the GBA Repositioning Strategy and endorsed the following mechanisms to support the implementation of the GBA Policy:

• The WIGE Directorate coordinates INAC's GBA Policy by ensuring that it is reflected throughout the Department's business lines. For example, INAC's MC Guidelines explicitly state that all Memoranda to Cabinet reflect the application of GBA;
• A network of Gender-Based Analysis Representatives (GBARs¹²), to be nominated by Senior Management be established (and drawn from members of a former Advisory Committee) to horizontally integrate GBA across headquarters and regional offices, primarily focusing on the development of employees' capacity to conduct GBA.

Key objectives of the Repositioning Strategy include:

• Increase senior management commitment to GBA;
• Improve GBA training capacity within and beyond WIGE;
• Increase GBA competency of INAC employees;
• Motivate INAC employees to conduct GBA;
• Increase quantity and quality of GBA data, information and tools.

The anticipated outcomes for Year Five (2008) were:

• Overall quality of analysis is good. A substantial number of employees are regularly conducting GBA;
• Policy, program and legislative options without GBA are regularly filtered at the point of origin;
• High quality GBA is influencing most large-scale and potentially long-standing decisions;
• Senior management regularly looks for evidence of GBA; and
• The momentum to conduct GBA is quite high. Many people notice when GBA is absent or of particularly poor quality.

Key Activities

At present, the key activities at INAC with respect to the support and implementation of INAC's policy include the following:¹³

• Application of GBA. The implementation of the policy is expected to involve all sectors and units within the department. Departmental staff, with the support of WIGE and GBARs, are expected to apply GBA to the development and implementation of departmental policies, programs, communication plans, regulations and legislation; consultations and negotiations; and instructions and strategies on research, dispute-resolution, and litigation;
• Education and Outreach Activities. WIGE coordinates an annual training exercise for GBARs which provides background information on GBA, how it is applied at INAC, and cases for discussion and practice.¹⁴ GBA tools developed by WIGE are also available to Departmental employees, including a working guide, an on-line training course, and worksheets;
• The Directorate also conducts presentations throughout the department to increase the profile of GBA and the GBAR network, while the GBARs themselves may also conduct outreach and education activities in their sector, area or region. Up to recently WIGE distributed newsletters exclusively to the GBARs. For the first time, in February 2008, the newsletter was distributed to all INAC employees by the GBA Champion. INAC's GBA website has also been updated and revised, but was not yet functioning during the time in which the evaluation was conducted. WIGE also organizes events in honour of Women's History Month and International Women's Week/Day and via these events promotes GBA through quizzes and other promotional materials;'
• Support and Advice. Support and advice for GBA activities is provided through WIGE and the GBAR network. WIGE also undertakes environmental scanning of media articles, Standing Committee on the Status of Women (FEWO) websites and international sites.'

In addition to the above activities, WIGE also sits on interdepartmental GBA and gender equality committees led by Status of Women Canada, as well as a working group on gender equality indicators.

Roles and Responsibilities

As previously noted, WIGE was given a mandate to develop and implement the Department's Gender-Based Analysis Policy but. this Policy also directs WIGE, in consultation with the Audit and Evaluation Branch (now the Audit and Evaluation Sector), to develop measures to monitor the implementation of the Gender-Based Analysis Policy and evaluate its effectiveness, and to issue a yearly report on its activities.

As mandated by the 2003 Repositioning Strategy, INAC's Gender-Based Analysis Representative (GBARs) are expected to:

• provide support and linkages for their branch or region;
• act as a point of contact for GBA in their office or region;
• promote GBA;
• proactively seek tools, information, networking and capacity building activities;
• attend annual gender-based analysis training sessions (obligatory);
• remain informed of key issues and/or files in branch or region;
• advise colleagues to consider integration of GBA, suggesting resources and tools where possible;
• participate in GBAR network meetings and liaise with WIGE regularly to provide information on branch or regional GBA activities

GBARS, in fact, are expected to act as ambassadors of GBA, but they are not expected to '…become gender-based analysts for their branch or region, rather they will be advisors.'¹⁵

Furthermore GBARs are encouraged to develop a reporting mechanism with their managers and Director General or Regional Director General. They are also expected to provide annual input into an overall departmental roll-up regarding GBA progress, challenges, and activities.¹⁶

In response to the 2006 recommendations of the Standing Committee for the designation of Assistant or Associate Deputy Ministers responsible for gender-based analysis, INAC's Senior Assistant Deputy Minister agreed in 2008 to be the Department's GBA Champion.

Policy Implementation Resources

Resources for the implementation of the GBA Policy are not provided through the A-Base, but subject to annual review and approval. Prior to the development of the Repositioning Strategy, the Department had only one full-time analyst dedicated to this work.

At present, there are four WIGE staff members dedicated to GBA on a full time basis. Other WIGE personnel, including the Director of WIGE, also contribute to implementing the Policy. In addition, WIGE estimates that GBARs stationed at headquarters can spend up to ten percent of their time (and regionally-based GBARs slightly less time) on GBA-related activities.

In addition to the above dedicated resources, representation is currently being supplemented by two summer students, including one master's level student from the University of Toronto who is participating in a Pilot Internship Program with the Audit and Evaluation Sector.

2.0 Evaluation Objectives and Methodology

2.1 Evaluation Objectives

According to the evaluation's Terms of Reference its overall purpose is to examine the application by employees of INAC's GBA Policy to address differences between women's and men's socio-economic realities in the development of departmental policies, programs, and legislation to maximize results for all clients.¹⁷

The specific evaluation issues and questions are:

• The continued relevance of the GBA Policy in achieving departmental objectives:
  • Is INAC's approach to implementing GBA (Internal GBA Policy and Repositioning Strategy) still relevant and consistent with current federal requirements for GBA?
• The impact of the Policy in the development and implementation of departmental policies, programs, communication plans, regulations, legislation, consultations, negotiations, research, litigation and contracts – to ensure that they are supporting the equality of women and men:
  • To what degree have INAC's GBA Policy Objectives been achieved? (Questions 2-3)
  • To what degree have the objectives of the Repositioning Strategy been achieved?
  • Have there been any unexpected impacts with respect to the implementation or application of GBA? (either positive or negative)?
  • What internal/external factors have most helped / most challenged the achievement of results?
• The effectiveness of the current implementation strategy and possible ways to better achieve and monitor intended outcomes:
  • Were key activities identified in the Repositioning Strategy completed?
  • Are roles and responsibilities clear and adequate?
  • Is INAC's current approach to quality control adequate?
  • Is INACs current approach to measuring and reporting on results adequate?
  • What are some key success indicators, short, medium and longer term, for the implementation and impacts of GBA policy?
  • Are expenditures on GBA tracked in order to inform future planning and evaluative activities?
  • What best practices/lessons learned are there with respect to the implementation of GBA?
  • What priorities, alternative strategies or modifications could be made to strengthen the implementation, quality, and effectiveness of GBA and performance measurement and reporting on GBA?
  • What are the key challenges facing the implementation of GBA?

2.2 Evaluation Methodology

The evaluation research was conducted in part by external consultants and in part by INAC's Evaluation, Performance Measurement and Review Branch. The final report amalgamates the work of both internal and external researchers.

The evaluation methodology was designed to take into account:

• The horizontal and comprehensive nature of INAC's GBA Policy and its application;
• The absence of an overarching performance measurement framework and strategy; and
• Gaps in literature and external documents with respect to evaluations of the application of GBA in Canada or abroad.

As such, the evaluation research employs multiple lines of evidence, uses selection criteria favouring the inclusion of all sectors and/or regions across all lines of evidence, and draws upon key informants and survey respondents who play an important role in the Policy's application (e.g., GBARs, Senior Policy Analysts, Senior Managers) and others working in areas of importance in informing or integrating GBA into practice (e.g., Strategic Research and Analysis, Aboriginal Economic Development, Policy Development and Approvals, and Litigation Management and Resolution).

The evaluation matrix in Appendix B demonstrates how the evidence collected through the following methods were used to address the evaluation issues and questions:

Document, File and Literature Review

A document review was undertaken to obtain an understanding of the context and to collect evidence about the integration of GBA into the Department's work. This 'evidence' was defined in terms of the presence or absence of one or more of the following: a demonstrated use of gender-based analysis, statements asserting that a GBA had been conducted (and findings), gender sensitive language, and/or data or information disaggregated by sex.

The review focused on documents dating from 2002, the year before the Repositioning Strategy was implemented, to the most recently available versions of the reports or guides identified. Where the entire universe of a specific type of report was not examined, sampling techniques were employed to ensure representation from a cross-section of differing sectors over time:

• Memoranda to Cabinet (MCs), including Communication Plans (n = 36)
• Treasury Board Submissions (including Communication Plans where attached) (n = 23)
• Evaluation Reports (n = 39)
• Reports on Plans and Priorities (n = 8)
• Departmental Performance Reports (n = 6)
• INAC supported research reports (n = 8)
• Recipient Reporting Guides (n = 5 reports, about 325 reporting templates)

Documents and web-based materials on gender-based analysis practices of other federal departments were also reviewed. A search of available Canadian and international research and evaluation pertaining to GBA was conducted (for example, available evaluation reports from other federal government departments) to assess gender-based analysis practices and strategies to achieve gender equality goals. These documents are cited when referenced throughout the text.

WIGE program files were also reviewed to obtain contextual information as well as any evidence pertaining to activities, training, reporting and other activities related to the Department's integration of GBA into INAC processes and decision-making.

Key Informant Interviews and Focus Groups

To gather in-depth qualitative evidence, the evaluation plan called for the conduct of both key informant interviews and focus groups, The majority of the 39 contacts were conducted through interviews, as detailed below:

• INAC officials (n = 21)
  • Senior officials (13), WIGE representatives (2), and GBA Champion (1)
  • Including 1 focus group with 5 participants
• Gender-Based Analysis Representatives (GBARs) (n = 9)
  • Including 1 focus group with 4 participants
• Directors or Senior Policy Analysts with other federal departments (n = 9)
  • Status of Women Canada and Treasury Board of Canada Secretariat as well as Canadian Heritage, the Canadian International Development Agency, Citizen and Immigration Canada, Health Canada, and Human Resources and Social Development Canada.

Where appropriate, for example, to gain an understanding of the extent of agreement on specific points, the following guidelines were used for reporting findings related to the interviews, focus groups, survey and document review:

• Great Majority:  80% or more of respondents
• Majority (or Most):  51% - 79% of respondents
• Many:  From 40% to 50%
• Some:  From 20% to 39%
• Few:  < 20% of respondents or sources

Survey with INAC Directors, Senior Managers and Senior Analysts

A survey targeted at INAC Directors, Senior Managers and Senior Policy Analysts was conducted by the Evaluation, Performance Measurement and Review Branch to gather quantitative information to support the evaluation findings. The survey forms were distributed through the Department's Assistant Deputy Ministers and other unit heads (e.g., Chief Audit and Evaluation Executive, Chief Financial Officer, Regional Director Generals). A total of 62 INAC officials responded Classified by position, location, and other characteristics they included:

• 26 Directors, 17 Senior Managers, 11 Senior Analysts, 7 Other (e.g., ADM, RDG, Advisors) (n = 61)
• 37 work at HQ, 21 in the Regions, and 4 at Indian Oil and Gas Canada
• 51 managerial, supervisory or instructional staff (with 8 managing > 25 staff, 14 managing between 11-25 staff, and 29, 10 or under)
• 42 of the respondents worked at INAC for more than five years, 10 worked at INAC for less than 2 years, and the remaining 10, from 2 to 4 years
• 28 respondents were female, 33 male (n = 61)
• 25 provided contact information for further follow-up

The following charts provide information on the survey respondents' relations to INAC's Strategic Outcomes and Sectors:

Source: Survey Question 6a. Note: 21 respondents reported that their work supported more than one Strategic Outcome (13 reported supporting 2-4 Strategic Outcomes, and 8 reported supporting 5-6 Strategic Outcomes)

Source: Survey Question 6b. Note:1 respondent reported working in more than one sector.

Survey respondents were also asked about the type of activities they were engaged in as well as their involvement in these, that is, whether they actively developed or contributed to proposals (including MCs and Treasury Board Submissions), delivered, implemented or conducted activities, and/or analyzed, reviewed or commented on these activities.

Source: Survey Question 7. Note: The table above excludes two 'Other' responses.

As depicted above, the respondents are thus actively engaged in the range of activities as specifically cited in the objectives of INAC's GBA Policy:

• The development (e.g., MCs and TB Subs) and implementation of departmental policies, programs, communication plans, regulations and legislation;
• Consultations and negotiations (including but not limited to self-government and land claims, treaty land entitlement and devolution);
• Instructions (e.g. contracts and TORs) and strategies on research, dispute-resolution, and litigation.

In addition, the survey also asked respondents about their involvement with performance measurement, reporting, audits, evaluations and/or reviews – areas of interest in terms of evolving government priorities on accountability.

2.3 Research Limitations

While significant efforts were made to ensure even representation of regions and sectors in the survey and interview samples, participation proved to be uneven. For example, despite the support of Senior Management in distributing the evaluation survey, 62 employees completed and returned the questionnaire, whereas there were approximately 340 potential survey respondents when the survey was distributed (based on an analysis of INAC's telephone directory and organizational chart),. Moreover, two of four planned focus groups (one with HQ GBARs and the other with Senior Managers) were cancelled due to the unavailability of participants.

These developments were partially compensated for through the conducting of additional one-on-one interviews, and partially through the methodological design which favoured the development of several lines of evidence and sampling techniques for all lines of evidence thereby enabling participants and evidence from a broad range of activities, sectors, locations and strategic outcomes.

3.0 Relevance, Impacts and Effectiveness

This Chapter presents the evaluation's findings pertaining to relevance, impacts and effectiveness.

3.1 Relevance of INAC's Approach with Federal Requirements for GBA

According to the Second Report of the Standing Committee, 'since 1995, various departments have implemented a variety of mechanisms for, and approaches to, integrating GBA.' By 2005-06, departmental approaches covered the spectrum, from the integration of GBA into departmental strategic frameworks and business lines, to establishing networks of GBA specialists who offered training, and developed tools and resources.

INAC's most recent implementation strategy includes a significant range of the above approaches (with the exception of the inclusion of GBA into the Department's strategic frameworks, and the fact that the Department's GBARs are not specialists, but play an advisory role). The Department's approach also includes many of the elements identified by Status of Women Canada as 'building blocks to success.'¹⁸ These include the development of a tailored GBA policy with requirements for performance indicators and evaluation, a repositioning strategy with identified outcomes and timeframes, a focus on capacity building, training, pilot projects, dissemination activities and the development of tools including guides and manuals.

The above factors were highlighted by senior management key informants as showing the positive relation between INAC's approach (i.e. INAC's Policy and Repositioning Strategy) and federal requirements for integrating GBA into the work of departments.

At the same time, and as noted in the previous chapter, the Government has recently highlighted the importance of increasing accountabilities for implementing gender-based analysis across Government. Analysis of the available evidence, described in greater detail further in this chapter, suggests that INAC's approach has not placed an emphasis on accountability and that performance measurement activities associated with the implementation of GBA at INAC are weak and require updating.

It should be noted however that this situation is not unique to INAC within the context of GBA. Firstly, the Government's emphasis on this issue implies a de facto recognition that the issue needs to be addressed across government. Secondly the situation is not unique within the context of emerging developments in other horizontal policy areas at INAC. For example, INAC and other federal departments are currently working to strengthen accountability, the identification of measurable results, performance measurement and reporting in order to better track and demonstrate the results and achievements of their implementation of the Official Languages Policy and Sustainable Development Strategy.

With respect to Official Languages, INAC has recently developed an Official Languages Accountability Framework in response to the findings of its first Performance Report Card (in 2006-07) from the Commissioner of Official Languages (This performance assessment exercise, launched in 2004, involves all departments and was intended to provide Senior Management with information on the performance of their institutions regarding official languages).

Moreover the lead office on INAC's Sustainable Development Strategy is now reviewing aspects of its governance structure, and in consultation with the Audit and Evaluation Sector is revising its performance measurement strategy and framework. This development follows upon the October 2007 Report of the Commissioner of the Environment and Sustainable Development which found that there was a need for a federal strategy to guide the efforts of individual departments and set clear expectations and oversight by parliamentary committees to hold departments accountable for appropriate integration of environmental issues when making management decisions.

In terms of the continuing relevance of INAC's approach to implementation, the comments made by some interlocutors and analysis of the Department's approach imply that the examination should go beyond looking at the profile of the approach. The Department's initial appreciation of the risks, level of effort, accountabilities and timeframe required to fully integrate and sustain the horizontal policy into all business lines should also be revisited.

These issues are explored further throughout this document and in the evaluation's findings related to impacts, effectiveness, challenges and best practices.

3.2 Success and Impacts

3.2.1 Achievement towards INAC's GBA Policy Objectives

Impacts on the Integration of GBA into INAC's Work

As outlined below, the evaluation's examination of impacts focused on areas specifically mentioned in the GBA Policy objectives. The research also looked at other areas, including performance measurement and evaluation. These key indicators were not explicitly mentioned, but are now identified by the federal government as priorities towards the systematic integration of GBA across government.

The evaluation evidence identified some key advances, many of them recent, but in general concluded that advances in integrating GBA into the Department's work have been limited. The following paragraphs document areas in which the integration of GBA was the most evident or promising:

Examples of areas in which there was clear evidence of the application of GBA:

On-Reserve Matrimonial Real Property
Key informants knowledgeable about On-Reserve Matrimonial Real Property Reserve (MRP) reported that gender-based analysis has been utilized to examine the issue of on-reserve matrimonial interests or rights. This issue has implications for men, women, children and families. The development of the proposed Family Homes on Reserves and Matrimonial Interests or Rights Act followed a comprehensive, national collaborative consultation process with INAC, the Native Women's Association of Canada (NWAC), the Assembly of First Nations, as well as provincial and territorial groups.¹⁹ The gender-based impacts of the proposed Act were identified through a gender-based analysis. For example, the Provisional Federal Rules contained in the Act allows for courts to grant emergency protection orders granting one spouse or common-law partner temporary exclusive occupation of the family home. This will provide victimized spouses or common-law partners in abusive relationships a safe place to reside, albeit temporarily.²⁰

Research
INAC's Strategic Research and Analysis Directorate indicated that it routinely looks at gender issues it its research and regularly uses data disaggregated by sex, including data from Statistics Canada. One highlighted example involved the Directorate's application of the United Nation's Human Development Index to First Nation communities and registered Indians in which scores were disaggregated by sex.²¹

Physical evidence of the integration of GBA into INAC supported research was clear. Six of the eight INAC supported research reports reviewed for the purposes of the evaluation included research or information that contained gender considerations (e.g., inclusion of disaggregated data, consultations with representative groups, presentation of secondary or primary research on gender issues). The reports containing gender considerations dealt with employment, education, language, mobility and the impacts of amendments to the Indian Act. The research reports without gender considerations dealt with an economic development program and social and demographic changes.

Aboriginal Economic Development
Economic Development considers gender issues and examines gender-based opportunities for business and economic development in Aboriginal and northern communities. For example, as a result of a GBA conducted by this sector, INAC has implemented a micro-financing program that provides support to organizations that help women in business.

Education
Key Informants indicated that the Education Branch is currently working to incorporate gender-based analysis into their policy and program development activities, including a planned program review. Information on the student population is being analyzed to understand how men and women currently use the program and what changes could be made to improve access and success. The Branch's upcoming work plan is expected to carry forward on the gender issues raised through the gender-based analysis.

The Education Branch is also developing and/or improving tools to enhance the integration of GBA into policy and program development. For example,it is developing a research plan in collaboration with the Strategic Research and Analysis Directorate in which. At least two of the themes in the current draft outline plans to investigate gender issues. Efforts are also underway to improve and integrate education program databases to better support program and policy information needs, including that of more comprehensive gender-based data.

While the above examples provide support that GBA is being applied at INAC, the evidence strongly indicates that one cannot use these as the basis of generalizations for the entire Department. For instance the document review findings indicate that perceptions of GBA usage are higher than practises (see below): some respondents believed that approximately 50 percent of INAC's policy documents contain a GBA, and Key Informants thought that about 85 to 90 percent of MCs include a GBA but in fact survey evidence indicates that only 12 of 54 respondents reported seeing regular evidence of GBA in INAC proposals.

With some exceptions, including research sponsored by the Strategic Research and Analysis Directorate described above, only a minority of documents reviewed contained a GBA. Furthermore there were several areas where senior INAC Key Informants indicated that the integration of GBA was particularly limited. These included negotiation, some areas of education, litigation, communications, infrastructure, and program implementation and delivery. Some of these areas and others are discussed in the following paragraphs.

Examples of areas in which evidence of the application of GBA was more limited:

Memoranda to Cabinet and Treasury Board Submissions

Approximately one-third of the first 30 Memoranda to Cabinet reviewed contained evidence of GBA, a low rate in view of INAC's specific requirement for the inclusion of GBA in its MCs. However this incidence rate is much higher than that found in TB submissions.

The evaluation reviewed 23 Treasury Board Submissions dated from 2002 to 2007 and originating from all sectors of INAC. The findings were:

• Three of the 23 documents raised gender issues (e.g. identification of a gender-related issue, definition of expected outcomes);
• None of the Treasury Board Submissions indicated that a GBA had been conducted or provided a summary of the GBA results in the document; and
• In those instances where data was presented in the Submissions, the data was not disaggregated by sex.

In order to test whether our sampling technique may have contributed to this uneven picture, the Evaluation Branch selected 10 of the Treasury Board Submissions in which there was no evidence of GBA and tracked down their originating MC. Eliminating those MCs which dated from before the adoption of the GBA policy, it was found that 6 of the submissions had actually originated from MCs in which there was evidence of GBA.

Negotiation and Consultation
The evidence and findings in these areas of work are more limited and more mixed than others highlighted in this report. Given the role assigned to consultation in the GBA Policy and the importance ascribed to consultation and negotiations to the Department's work and outcomes, further future research may be required to assess strengths and weaknesses of GBA in these areas (and/or awareness of current practices).

Some key informants indicated that consultations with representative groups could be enhanced and that GBA could be further integrated into negotiation processes. There were comments suggesting a need to ensure the cultural relevance of gender-based analyses, while some respondents expressed concerns about imposing views of gender equality on INAC's clients.

The survey evidence gave a particularly mixed picture. For example, about two- thirds (21 of 30) survey respondents indicated that they regularly (6) or sometimes (15) saw evidence of GBA in consultations and negotiations.²² A related question asked respondents about how GBAs were conducted in their areas.²³ Nineteen of 58 respondents stated that GBAs regularly (3) or sometimes (16) involved consultations with beneficiaries or representative organizations. Note that, along with the 'review of external documentation,' this area ranked lowest of those sources identified by respondents as 'regularly' informing GBAs, however, it was also the most commonly cited method respondents reported as using 'sometimes' (see also issues raised with respect to consultations around on-reserve marital property, section 3.2.2).

Program Activity Architecture
INAC has been involved in an exercise to review and strengthening its Program Activity Architecture and associated performance measurement frameworks. The evaluation's review of a preliminary version of the sectoral frameworks being developed in this regard found no evidence of the application of GBA. Strategic outcomes and performance indicators did not use gender sensitive language and the draft performance indicators did not demonstrate an intent to gather sex disaggregated performance data.

Final Evaluation Reports²⁴
Thirty-one of 39 evaluation reports reviewed for the years 2002 to 2008 contained no evidence of GBA. The remaining eight dealt largely with economic development programs. For example, the Evaluation of INAC's Proposal-Driven Economic Development Programs (2005) summarized key labour force and employment data by gender and included consultation with First Nations and Inuit women's organizations.

Other gender-sensitive evaluations included the recent Evaluation of INAC's Income Assistance Program (2007). This evaluation concluded that gaps in information on the gender of clients, among others, challenged evaluative work. Lastly, an analysis of students requiring high cost special education services conducted as part of the Formative Evaluation of INAC's Special Education Program (2007) resulted in the assessment that approximately two-thirds of the students reported on INAC's Nominal Roll as in receipt of support from the program were male (and concluded that more information was required about high-cost special education students and their needs in general).

The evidence collected from the file reviews, interviews, focus group and survey supports findings regarding the limited integration of GBA at INAC. For example, the great majority of key informants and focus group respondents indicated that there was modest or no integration in their own area of work. GBA was generally viewed as an add-on by key informants and focus group respondents rather than an integral or routine part of work. Most of the nine GBAR respondents indicated that they had not conducted GBAs in the course of their own work. However, analysis of two progress report summaries²⁵ contained on file found that one-third to over one- half of GBARs reported using GBA in their own work fairly frequently (more than once a month).

The great majority of GBAR focus group respondents and key informants indicated that they rarely or occasionally received requests for information about GBA. Similarly, a progress report summary contained on file (undated) found that the majority of GBARs rarely or occasionally (less than once a month) responded to requests from colleagues. According to interview and focus group evidence, the primary nature of requests from colleagues related to clarification of GBA concepts.

A review of WIGE program files showed that past departmental staff surveys, conducted in 2003 and 2004, found that the majority of survey respondents did not use GBA on a regular basis. More specifically 79% of staff/managers (n=136) reported not using the GBA policy and 3% reported using GBA regularly (2003). In 2004, 63% of managers and staff (n=43) reported never using GBA and 12% reported using GBA regularly. Additional surveys were not conducted to enable assessment of trends with respect to awareness and use of GBA.

During this evaluation survey respondents were asked a series of questions about their level of involvement, and that of their staff, with gender-based analysis. Few respondents reported having had much experience with GBA. According to the findings, 12 of the 62 respondents are involved in developing responses or addressing gender issues, 6 conduct, manage and /or commission GBAs, 3 review GBAs, and 8 track and/or report on sex disaggregated performance measurement data.

Survey respondents were also asked whether they regularly looked for evidence of GBA, in line with the anticipated five year outcomes of the Repositioning Strategy. They were also asked whether they saw evidence of GBA, as a further measure of the degree to which GBA has been integrated into the Department's work.

As shown below in Table 4, at the high end, 20 (of 57) respondents reported regularly looking for evidence in GBA in proposals, including MCs and Treasury Board Submissions. At the other end of the scale, one of 17 respondents reported looking for GBA in litigation and no respondents (0 of 15) reported looking for evidence of GBA in the area of dispute resolution. Less than one-half of the respondents answering this series of questions reported regularly looking for evidence of GBA across all areas. In several areas, less than one-quarter of respondents reported regularly looking for evidence of GBA.

Source: Survey Questions 11 and 12.

In general, far fewer respondents reported that they regularly saw evidence of GBA, no matter what their work area. Amongst the areas with the least regular sightings of GBA are performance measurement, audits and evaluations. These responses correspond with key informant views on gaps in accountabilities and performance measurement.

According to the 2005/2006 [GBA] Annual Report, GBAs were being conducted in an intermittent manner and often late in the development of new initiatives. While anecdotal evidence was heard that this remains a common practice, the issue was not explicitly examined as an evaluation issue or question. However, it should be noted that Treasury Board has recently emphasized that analyses should begin as early as possible if they are to effectively inform the policy proposal process.

3.2.2 Extent to which Gender Issues are Raised and Addressed

The evidence suggests that activities are and have been instigated by WIGE to raise and discuss gender issues within the department, and externally with other departments and representative organizations. INAC, for example, supports funding for conferences (e.g., the National Aboriginal Women's Summit). In addition, WIGE consults with women's groups on a variety of issues. INAC and Status of Women Canada are engaged in collaborative work on issues impacting Aboriginal women.²⁶ For example, a working group called the Sisters in Spirit Initiative, involving Status of Women Canada, INAC and other federal government departments, is working to identify how policies and programs in federal departments can be adapted or improved to better serve the needs of Aboriginal women.

Respondents consistently mentioned two examples of the issues currently being addressed by INAC, highlighting that the GBA's conducted by the Department's can inform significant issues, such as fundamental legal challenges and high-risk situations:

On-Reserve Matrimonial Real Property Issues
In 2006, the Native Women's Association of Canada (NWAC), Assembly of First Nations, and INAC jointly announced the beginning of their separate Canada-wide consultation processes.²⁷ Each organization was responsible for working with their constituents to identify solutions to address on-reserve matrimonial real property issues. Following the completion of these consultations, the Ministerial Representative submitted her report, which included recommendations for further action, to the Minister on March 9, 2007. The report recommended legislation be passed in which individual First Nations would develop a solution compatible with their system of housing and land distribution while promoting gender equality., The report also recommended that the legislation contain emergency provisions that would immediately offer protection to women, children and families, especially those in a situation of family violence. The report also made several non-legislative recommendations which are being taken into consideration in the development of an implementation plan for draft legislation.²⁸ NWAC expressed dissatisfaction with the consultation process following the report²⁹ and indicated that they are not supportive of the proposed legislation without any companion non-legislative solutions and resources required to implement legislation in order to fully protect women.³⁰

Violence and Aboriginal Women. Aboriginal women are three times as likely as non-Aboriginal women to report an experience of some form of violence perpetrated by their spouse. The anticipated result of INAC's Family Violence Prevention Program is a reduction in family violence and a more secure family environment for children on-reserve, by providing abuse prevention and protection services for Aboriginal children and their families on reserve.

The evaluation's examination of INAC's web-site, conducted as part of the document review, indicates that a number of gender related issues are commonly raised or highlighted by the Department, particularly with respect to on-reserve matrimonial real property and family violence. Other issues raised include:³¹

Equity issues around Bill C-31 Status and Membership. This 1985 Bill amended the Indian status and band membership provisions of the Indian Act with the goal of bringing the Indian Act into conformity with the right to gender equality under the Canadian Charter of Rights and Freedoms. A series of research studies, some sponsored by SWC, show that gender issues continue to surface around the Indian Act and Bill C-31.

Justice and Human Rights. Section 67 of the Canadian Human Rights Act provides that nothing in that Act affects any provision of the Indian Act, leaving status Indian women without similar protection that other Canadian women have.

Power and Decision-Making. Aboriginal women seek greater political participation and input into decision making for First Nations, Métis, and Inuit women.

On a daily working basis, however, the findings appear less positive. The majority of key informants and focus group participants reported that gender issues were raised infrequently in their area of work and few could cite examples of gender issues being routinely addressed. Areas of concerns included the situation of boys and education,; ensuring cultural relevance in INAC's identification of gender issues, and a wider appreciation of gender incorporating. This view is not new: INAC's 2005/06 Annual Report stated that the "… GBA that has been conducted tends to be light in nature, wherein gender equality issues may be acknowledged, but not necessarily addressed."³²

The results of the survey support the above finding. Depending on the level, only about one-third or fewer respondents reported being satisfied that gender issues were being effectively addressed in their area, sector and/or department. However, close to one-half of respondents reported not knowing whether gender issues were being effectively addressed:

Source: Survey Question 22.

As few as nine survey respondents ( about 20 %) indicated that the GBAs in their area had resulted in the identification of gender issues while 39 of the 62 reported that they did not know whether issues had been identified. Seven comments were received on this topic, in which issues related to marital property, family violence, education and employment, and the impacts of budget cuts were raised, in some cases with respect to greater challenges faced by males, or lack of knowledge concerning the situation of males.

Only two respondents indicated that they knew that solutions or strategies had been implemented to address the gender issues which had been raised. In line with the Policy objectives in cases where gender issues cannot be resolved at the working level, three other survey respondents reported that they were aware of issues which had been directed to WIGE, and two where either the Deputy Minister or Minister had been advised.

3.2.3 Achievement of Repositioning Strategy Objectives

This section summarizes the evaluation's findings with respect to INAC's expectations for strengthening management commitment, training, competencies and motivation and increases in the quantity and quality of GBA data information and tools.

Degree to which training capacity has extended beyond WIGE
While program documentation indicates that WIGE once contemplated developing a continuous learning GBA program, training now is largely limited to an annual two-day exercise. Most of the nine GBARs interviewed reported receiving the annual two day training. The training was viewed as useful for building awareness and knowledge of GBA and for providing networking opportunities with other GBARs. Even so, many of the respondents indicated that they did not feel sufficiently competent to conduct a GBA or to assess the quality of a GBA.

GBARs who took the annual GBA INAC training last year (Fall 2007) noted improvements in the training as it had included more examples of how to apply GBA. Two respondents reported that they missed the session because there was inadequate advance notification (i.e. two weeks).

Regional GBAR respondents suggested that GBA training should be customized to regional needs. The need for GBA staff specific training to ensure full application of GBA in a respective branch or area of work was suggested by a few key informants and was a frequently cited suggestion in progress report summaries (contained in WIGE's files). The importance or relevance of tailored training is instanced when the Privy Council Office (PCO), after having received initial training in GBA from SWC, undertook jointly with the SWC to modify their training approach to better reflect the PCO's specific needs.³³

Program files indicated that in addition to the annual two day GBAR training sessions, WIGE has also offered information sessions, GBAR Forums, and half day workshops on GBA since the implementation of the Repositioning Strategy. However, data from progress reports indicate that GBAR uptake on these other training activities is quite low.

Increased quantity and quality of GBA data, information and tools

Data and information
When asked whether they had adequate information to effectively develop GBAs or track performance, eleven of 60 respondents indicated that either they had access to beneficiaries and/or representative organizations always or most times, compared with 17 of 61 who had the same access to data disaggregated by sex, and 17 of 61 to issue related GBA research:

Source: Survey question 25.

Specific information gaps mentioned by survey respondents included community level statistical data and recipient data disaggregated by sex (with one respondent expressing concern that recent reductions in reporting requirements would reduce their capacity to assess differential gender impacts and emerging issues).³⁴

Corroborating evidence pertaining to gaps in performance measurement data comes from the evaluation's review of various editions of INAC's Recipient Reporting Guides. These Guides contain reporting templates to assist recipients in complying with the reporting requirements of their funding agreements, the primary means by which INAC gathers information about program performance is through recipient reporting. Much attention has been focussed, government wide, on reporting requirements recently both to ensure that these do not overly burden recipients and to ensure that the government is collecting only that data essential for performance measurement purposes. INAC initiatives related to Smart (or Quarterly Reporting) are emblematic of the efforts which are being undertaken to address these concerns.

The Guides were first reviewed to identify the number of reports which collected information on participants or the reach of the programs or initiatives (e.g., direct or indirect beneficiaries, clients, audiences, among others). Then these report templates were examined to assess the degree to which sex disaggregated data was being collected. The results are shown below in Table 7. It is clear that the proportion of reports requiring disaggregated data has increased over time (from about one-third over 1999-2000, to just under two-thirds for 2008-09). However, by 2007-2008, the last year for which data would have been available to respondents, only one-half of the reports that requested information on participants or beneficiaries required that data be disaggregated by sex.

Source: Document review.

It is beyond the scope of this evaluation to assess the quality and quantity of policy and program related research available across all INAC's priorities and/or sectors. A review of INAC's website did show that sex disaggregated statistics and data are being published and disseminated, and some of these on a regular basis. These include surveys pertaining to First Nations governance (the "Communities First" survey) and annually released statistics on the registered First Nations population by gender and residence.³⁵

One particular study, commissioned by WIGE and supported by the Strategic Research and Analysis Directorate, should be highlighted for its scope and potential to inform policy making, performance measurement and evaluations.³⁶ The study, based on the 2001 Census, and serving as an update to a 1996 census based report, provides a detailed overview of the socio-economic characteristics and circumstances of Aboriginal women (as compared to Aboriginal men and to non-Aboriginal male and females). According to WIGE personnel, preliminary discussions are now underway for a revision based on the 2006 census.

Tools and supports
As many of the GBAR respondents were relatively new staff, they were not able to provide perspectives as to whether there was an increase in quantity and quality of GBA data information and tools. Only a few GBAR respondents reported using existing tools and information. Two GBAR respondents reported that they had used the online collaboration tool and found it useful. One GBAR respondent reported that the online collaboration tool was under-utilized, particularly the chat room and discussion forum and that it was difficult to post information. Some other INAC key informants were generally not aware of GBA-related information and tools.

As shown below, survey respondents reported using a variety of supports and sources to inform gender-based analyses. Also demonstrated, however, is that close to one-third of respondents in many cases were not aware of the potential to draw upon GBARs, WIGE, or the GBA website.

Source: Survey Question 29.

Fewer respondents contributed information of their appreciation of the utility of the supports available (16 to 32 depending on the type of resource), with most typical response indicating that the source was 'somewhat useful', rather than not or very useful (Survey Question 30).

WIGE has been taking steps to improve the quality, reach and 'user friendliness' of its range of GBA tools, supports and resources. These steps include updating INAC's GBA webpage,³⁷ and the development of a streamlined Working Guide which WIGE officials said had proven useful following the announcement of Treasury Board's revised guidelines which highlight federal government requirements for GBA.³⁸ Moreover, a newsletter, originally designed as a means to reach and inform GBARS, was circulated by WIGE through INAC's GBA Champion to all Departmental employees for the first time in February 2008 (The name of the newsletter was also changed recently, at the suggestion of one of the GBARs, from Wheel and Sprocket, to Raising the GBAR: Gender Matters).

Quality of GBAs
According to the social researcher Jeremy Hull in his 2001 census- based profile of Aboriginal women [referred to on p.24 above]:

'… the relative positions of different segments of the population change depending on the issue being considered. These findings suggest that those involved in policy and program development need to define both the populations and the issues that they are concerned with as precisely as possible in order to best achieve their policy goals.³⁹

Since INAC's GBA Policy was created in 1999, the complexity of issues INAC deals with has grown immensely, spurring a greater need for informed analyses and research. However, it proved difficult, if not impossible to assess the quality of most of the GBAs associated with the Memoranda to Cabinet (and Treasury Board Submissions) from the documents alone. For example, 3 of the 13 indicate that the initiatives would equally benefit men and women, but did not provide evidence to support this claim. Two of the MCs indicated that a GBA was conducted and no gender issues were found. Again no supporting evidence was provided. None contained data disaggregated by sex. At the same time, eight of the MCs provided substantially more depth and detail including the identification of gender issues and strategies to enhance benefits.

The survey results indicated that less than one-third of respondents were satisfied with the quality of GBAs in their area of work, sector or at INAC. More than one half of respondents indicated that they were unable (did not know) to assess the quality of GBAs

Source:  Survey Question 21.

The evaluation also sought to find out more about how GBAs were developed at INAC or if they were published (as an indicator of dissemination). However, about 25 respondents reported that they did not know whether the GBAs conducted in their area were conducted in-house or externally, or whether the results were published. Twenty-four respondents reported that GBAs were conducted in-house regularly or sometimes; nine that they were conducted externally. Eleven respondents indicated that GBAs were regularly or sometimes published (Survey Question 16, n = 62).

The survey also attempted to see whether GBAs at INAC involve review of internal and/or external sources, research or consultations with experts or stakeholders. Twelve respondents reported regularly or sometimes using just one source of information, 9 between 2 to 4 sources, and 10 between 5-6 differing sources or types of information to develop GBAs (Survey Question 17).

However, depending on the sub-question, between 24 to 30 respondents stated they did not know how GBAs were developed, while 13 to 25 respondents reported that GBAs in their area did not involve one or more of the types of information sources identified in the survey. Due to the design of the survey questionnaire, it is unclear whether respondents did not know how the GBAs were developed, or whether the high number of 'Don't Know' and 'Never' responses arose because GBAs are not conducted.As one respondent commented, the questionnaire may have erred in assuming that GBAs were regularly being completed.

Increase in staff competencies

INAC senior manager key informants and policy focus group participants indicated that staff in their areas of work do not have sufficient competency to conduct GBAs. Many GBAR and policy respondents reported being unsure as to how GBA applied to their areas of work (e.g. program development and delivery, negotiations, infrastructure, etc.).

These findings are corroborated by the survey findings, below, where the majority of respondents indicated that they, or the employees they manage, supervise or instruct, do not have sufficient knowledge to work with or use GBA:

Source:  Survey questions 23 and 24. * (n = 62)

Changes in attitudes and practice: Managerial commitment and staff motivation
Following the recommendations of the House of Commons Standing Committee that departments designate an Assistant or Associate Deputy Minister responsible for gender-based analysis, the Senior Assistant Deputy Minister at INAC agreed to be the Department's Gender-Based Analysis Champion. This development was perceived by knowledgeable Key Informants as a positive step towards strengthening managerial commitment to GBA at INAC.

However, it was generally held by interviewees, focus group and survey respondents that gender-based analysis was not on the agenda of INAC's senior management. In a few instances, key informants reported resistance to GBA by senior management (e.g. insufficient support for attendance at GBAR training, unwillingness to support application of GBA).

The survey results provide corroborating evidence. Thirty-three respondents were not certain whether their managers looked for evidence of GBA in their work, only about one in six respondents stated that their own superiors regularly looked for such evidence of GBA (Survey Question 13, n = 62). As shown below in Table 11, roughly the same number of respondents reported having managerial support for applying gender-based analysis as those reporting that they did not know whether they had managerial support.

Source: Survey question 27.

The great majority of interviewees and focus group participants indicated that, in general, GBA is viewed in their area as an add-on and that other priorities often take precedence. According to respondents, this appears to stem from lack of awareness and a lack of understanding on how to apply GBA or how it is relevant to their area of work. The great majority of key informants indicated that the key challenge in motivating staff arose from lack of levers/pressure to conduct GBAs. Policy Analysts interviewed reported that success stories (i.e. how GBA has led to positive impact on community) and concrete examples of GBA integration would be motivational.

3.3 Effectiveness of implementation

This subsection presents findings in relation to the extent to which the Repositioning Strategy has been implemented as planned, in terms of capacity building activities, tool development, work with senior management, and the establishment of the GBAR network.

Prioritized Departmental Capacity Building Activities and Facilitated Resource/ Tool Development
WIGE's training curricula has recently been revised to include more examples of GBA application. A streamlined working guide has also been produced in an effort to improve dissemination of this resource. The revised newsletter is currently being circulated to all INAC employees. While WIGE offers departmental capacity building opportunities to all branches, the Directorate noted that it has taken advantage of organizational changes to reach incoming senior managers.

Working with Senior Management to Build GBA Commitment
WIGE liaises with senior management to raise awareness of GBA (e.g. outreach, information-raising campaign, providing information, presentations) and has worked closely with some sectors and branches, including the Strategic Research and Analysis Directorate and the Aboriginal Economic Development Branch, providing support and information with respect to GBA. Taking the example of Economic Development, WIGE recently reviewed the sector's research plan, and they are informing the development of a new federal government wide Aboriginal economic development framework (work on this Initiative is being led by INAC). WIGE also provides support, resources and tools to encourage GBARs to engage with senior management (and many GBAR respondents reported working with senior management to engender awareness and support for GBA).

GBA Representatives are in Place and Provide Support
The GBAR network has been established with representatives from most branches and regions, but is not yet be in a strong position to offer support. The Repositioning Strategy anticipated a network of 40 GBARs. A review of the program files indicates up to 51 possible placements, an increase attributed by WIGE to organizational changes since 2003. In this regard it should be noted that since the strategy was implemented, INAC has assumed responsibilities for a wider range of Aboriginal sectors and issues (see also section 4.1). As of the end of January 2008, there were 34 GBARs. Of these, 27 were officially nominated to their position, and another seven, who were not officially nominated, are working as seconds or temporary replacements.

Sixteen, or more than one-half, of these official 27 GBARS had a year or less experience as GBARs. Five have remained in their positions since 2003, and the other 6 were nominated sometime between 2004 and 2006. In February 2008, WIGE indicated through its newsletter that they were currently canvassing for new Gender-Based Analysis Representatives.

Some senior INAC key informants and GBAR respondents reported that people do not generally volunteer to be GBARs and it is viewed as an onerous task. In addition, interlocutors noted that the high turnover of staff in GBAR positions (and of staff in general) makes it challenging to establish a stable network. Respondents also noted that the network is not adequate to reach all staff, particularly new staff, and that GBA training should be part of staff orientation.

In this regard, about one-third of survey respondents (20 of 62 ) reported that they did not know whether their area,be it Branch, Sector or Region, had a Gender-Based Analysis Representative, while just over one-half (33) indicated they did have a GBAR.

Gender-Based Analysis Representatives are expected to be the branch/regional point of contact for gender-based analysis questions or concerns from colleagues and managers. GBAR respondents confirmed their understanding of this objective, indicating that their role was to provide advice and information, rather than to guide someone through a GBA. In the event that someone required assistance with GBA, most GBARs reported that they would direct the person back to WIGE.

Ultimately GBAR respondents reported that they only rarely or occasionally provide information to colleagues (i.e. clarification about GBA concepts), but that they most commonly engage in awareness raising activities directed to senior management and staff. Requests for information were noted to be sparse. GBAR respondents reported that they did not network with other GBARs via teleconference or online tools.

Pilot Project to Support Full Integration of GBA in the Education Authority Renewal
Subsequent to the launch of the Repositioning Strategy, a Memorandum of Understanding between WIGE and the Education Branch was approved. The pilot project began with a series of regular meetings between WIGE and the Education Branch, a series of staff training sessions, the establishment of a newsletter (i.e., the GBA newsletter now known as Raising the GBAR: Gender Matters, reportedly initially focused on Education), and an examination of work plans to assess how GBA could be better integrated into the activities of the Branch. The pilot project did not continue. Informed respondents concluded it was unsustainable, and it was affected by a period of successive staff turnovers.

However WIGE has continued to work towards developing new initiatives, including recent work with Aboriginal Economic Development, cited at various points in this report, which is also of strategic interest, given the Government's priority on aboriginal economic development.

3.4 Summary

Survey respondents appreciation of the degree of change they have seen in their work area over the past couple of years provides corroborating evidence of the extent to which the Strategy's five year expectations were achieved and the degree to which outputs were either achieved - or perceived to have been - achieved.

Source: Survey Question 31.
Note: Responses collated only from those respondents who identified working at INAC for more than two years.

No unexpected impacts from the implementation of INAC's GBA policy or implementation strategy were identified. by key informants and focus group respondents (survey respondents were not asked for their perspective on this issue).

3.5 Accountability and Performance Measurement

Clarity of Roles and Responsibilities
Many key informants were not aware of the roles and responsibilities for implementation, performance measurement and reporting. Of those who were knowledgeable, some key informants reported that the role of the GBAR was not clear. For example, a few GBAR key informants were vague as to what types of support they should provide or the level of effort expected. Regional GBAR respondents noted that it was particularly difficult to define their roles with respect to regional operations (i.e. program implementation, delivery).

Adequacy of INAC's Approach to Quality Control
There are no specific responsibilities assigned to overseeing the quality of GBA in policy or program documents within INAC. INAC's Policy Development and Approvals Directorate, does however, play a challenge function in the development of policies. The Directorate ensures that the three corporate lenses are applied – Sustainable Development, GBA and Official Languages. Prior to MCs being submitted to the DG Policy Review Committee, analysts are expected to review the MCs to ensure they contain a GBA. If a GBA is missing then they are usually sent back to the originator who is referred to an area GBAR.

Senior INAC key informants were not aware of a system for quality control with respect to GBA within their area of work. GBAR respondents noted that they did not feel sufficiently competent to assess the quality of a GBA. They also pointed out that it was not part of their role as GBARs to conduct quality analysis or control. An effective quality control system would also have to include clear criteria for assessment according to key informants.

Key informants also noted that central agencies also need to play a strong challenge function to provide for appropriate external checks for quality control. Recent confirmation and clarification of central agencies' roles may strengthen the external challenge function in the future.

Adequacy of INAC's Approach to Performance Measurement and Reporting
The evaluation evidence indicates that INAC's current approach to performance measurement is not in line with current management practices or evolving practices for GBA across government, as the following findings indicate:

• While INAC commissioned the development of a performance monitoring and evaluation framework in 2002, the evidence suggests that this framework was never fully implemented and current staff were unaware of its development. In addition, reporting requirements are conflicting and WIGE and GBARs do not appear able to meet their reporting responsibilities as outlined in program documentation;⁴⁰
• Although INAC has taken steps to instruct employees that GBAs are to be included in memoranda to cabinet, there is no specific role assigned to overseeing the quality of GBA in policy or program documents;
• Many GBAR respondents reported that they complete the annual progress report template, discuss GBA activities with senior management and send the report to WIGE. WIGE reported inconsistent reporting from GBARs and difficulties in obtaining reports. A review of available progress report summaries (undated) on file for one year suggests that about half of GBARs had provided progress reports to WIGE;
• A review of the Departmental Performance Reports (2001/02 to 2006/07) found that the achievement of GBA is not reported against the achievement of specified indicators, milestones or objectives; and
• A program file review found one annual report for the fiscal year 2005/2006 and one GBA summary (undated) containing raw data from GBAR progress reports. Staff surveys were conducted in 2003 (n=136) and 2004 (n=51) to measure awareness and use of GBA.

Perspectives on Performance Measurement and Key Success Indicators
During the evaluation, some key informants provided a number of insights and issues which should be considered for the future implementation of the GBA Policy. These include concerns that the objectives and milestones of the repositioning strategy were not realistic given the number of other competing priorities and few levers available to motivate people to conduct GBAs.

Respondents highlighted concern for improved monitoring of the quality of GBAs or taking steps to strengthen their quality and consistency for program and policy approval processes (e.g. checklists or templates).

Current developments also underline the importance of working with SWC, federal and provincial governments, as well as with representative organizations, to inform the development of performance indicators and data collection. Priority areas for consideration include ensuring gender sensitive Program Activity Architecture indicators, comparative measures across Aboriginal groups and between Aboriginal and non-Aboriginal groups (e.g., such as Human Development and Community Well-Being indices, and other efforts which contribute to measuring the Department's success in delivering services and support which meet or exceed provincial norms).

Many key informants suggested that in the short and medium term it is important to track the extent that GBA is being used. Suggested short and medium term indicators include:

• Level of incorporation of GBA in planning of activities (research, program evaluations and reviews, etc.);
• Level of awareness and knowledge with respect to GBA;
• Extent GBA is part of the policy development approval process;
• Level of competency to apply GBA;
• The presence of branch GBA work plans; and
• The number of MCs and TB Submissions reflecting a GBA process.

With respect to long-term indicators some key informants indicated the importance of assessing the impact of GBA with respect to gender equality and its impact on First Nations, Métis, Inuit and northern peoples. Suggested long-term indicators included:

• Policies, programs and legislation that have incorporated GBA;
• Extent to which GBA is part of work/routine;
• Evidence that GBA has influenced decisions, adjustments to policies and programs, and the development of new policies and programs;
• Evidence of sex disaggregated data and gender sensitive indicators;
• Extent of use of gender sensitive indicators in INAC's performance measurement, monitoring and evaluation processes; and
• Reduced inequalities between men and women.

Tracking of Expenditures on GBA Activities
The evaluation found no evidence that expenditures on GBA were being tracked systematically through the Department. For example, while WIGE notes that in HQ, GBARs can spend up to 10 percent of their time on GBA-related activities, the GBAR reporting template does not track time spent on these activities.⁴¹

4.0 Towards the Future:Issues Raised and Considerations

This chapter examines issues which have been identified as either facilitating success or presenting challenges to the systematic integration of gender-based analysis at INAC, highlights best practices and lessons learned, internally and externally, and concludes with a discussion of potential alternatives or modifications.

4.1 Success Factors and Challenges

Factors facilitating success
In those instances where GBA is being applied, key informants primarily attributed this to strong senior management support and interest in GBA. Senior managers more actively involved in GBA noted that accessibility of information and support from WIGE was useful in building awareness and knowledge of GBA.

Interview and focus group evidence indicated that the mandatory nature of GBAs in relation to MCs contributes to the inclusion of GBA in these documents. However, some key informants questioned the quality of GBA contained in these documents.

Those sectors/areas having a strong GBAR (with good interpersonal skills and knowledge about GBA) reported this as an important enabler. A few key informants reported that an accessible internal resource familiar with the area of work was preferable to hiring an external GBA consultant.

Challenges
Key informants and focus group participants cited a number of challenges affecting the application of GBA within INAC. The most frequently cited challenges are noted below:

• Insufficient Accountability Mechanisms. The absence of external pressure to implement GBA was cited as a key factor affecting application and integration of GBA within INAC. Key informants reported that central agencies should perform a strong challenge function to ensure application. However, it is too early to assess the impact of recent change requiring GBA in Treasury Board Submissions and the Government's response and clarification of the central agencies' challenge function. Lack of monitoring and quality control within INAC also contributes to low levels of integration according to many INAC key informants. The integration of the GBA into strategic outcomes for the department would also provide levers for GBA integration.
• Lack of Knowledge about the GBA Policy and its objectives. While many key informants and focus group participants perceived that general awareness of GBA has increased at INAC, they noted that there remains a considerable lack of knowledge and misconceptions about GBA. For example some are confused regarding the distinction between GBA and employment equity. GBAR and Senior Policy Analysts respondents indicated that in many cases they do not understand how GBA could be applied to their area of work and require examples and assistance to show them how GBA is applied. Many key informants see GBA as most relevant to the policy development stage and to social issues that are perceived to negatively impact women. A few key informants wondered whether gender equality views could be "imposed" on the community. The majority of GBAR respondents reported that while they are knowledgeable about GBA, they themselves lack competency to conduct GBAs or to provide assistance in conducting them.⁴²
  
  While approximately the same number of survey respondents reported being very familiar or familiar with the term GBA (25) and with the GBA Policy (23), a greater number of respondents reported being not at all, or not very familiar (18), with the Policy than they were with the term gender-based analysis (10).
  
  The conception that gender-based analysis dealt with "women's issues" was felt to be a barrier to examining this issue more thoroughly through a gender lens. This view was heard from various interlocutors and in the survey throughout the evaluation research, often in connection with concerns about lower educational outcomes of males, about advantages of looking at families as a whole, or for the need to ensure a thorough analysis of policy impacts, one which encompasses a wider range of demographic and socio-economic factors amongst others.
  
  Confusion around this issue at the federal level was also noted in the 2006 Government Response and reference was made to various changes in nomenclature since 1995. In this respect it should be remembered that the office charged with implementing INAC's GBA Policy is named the 'Women's Issues and Gender Equality Directorate', A name change has recently been approved for the Directorate which will bring it more in line with the Branch's name (External Affairs and Gender Issues Branch), which was also recently changed.
• Staff Turnover and Competing Priorities. Many key informants reported that the regular staff turnovers, including turnovers in GBARs, made building GBA capacity difficult and required continuous efforts to build GBA knowledge. In addition, WIGE does not receive A-base funding, making it a challenge to retain staff dedicated to the implementation of the GBA policy. Many key informants and focus group participants reported GBA appears to be in competition with (rather than an integral part of) other priorities relating to culture, language rights and sustainable development.
• Availability of data disaggregated by sex. The limited availability of such data at the program level (as well as other program level data) is seen as a challenge by key informants (see, for example WIGE, 2007, and section 3.5 above).⁴³
• Cultural relevance, complexity and engagement. Both interview and documentary evidence show that there are unique challenges and issues in addressing gender equality for Aboriginal peoples.⁴⁴ A few INAC key informants expressed uncertainty as to how to apply gender-based analysis in First Nation communities, others expressed concern that there was a need for a greater understanding of issues, beneficiaries and representative organizations in light of the expanding mandate of INAC.
  
  INAC's growth is exemplified by the transfer from the PCO of the Office of the Federal Interlocutor for Métis and Non-Status Indians in 2004. This office provides funding to Métis, Non-Status Indian and off-reserve Aboriginal organizations to build capacity and electoral and financial accountability, In that same year the Prime Minister announced the creation of an Inuit Secretariat at INAC. In 2006 Aboriginal Business Canada was transferred from Industry Canada to INAC, and most recently in 2008 responsibility for the Indian Residential Schools has also been transferred to INAC.

4.2 Best Practices and Lessons Learned

Research shows that there are a number of common challenges experienced by governments attempting to integrate gender-based analysis throughout their organizations (note this practice is typically referred to as 'mainstreaming'). These include: insufficient political will; inadequate accountability mechanisms; inadequate knowledge; inadequate resources (information, budget, tools and expertise); and the need for more strategic planning and implementation.⁴⁵ Addressing such challenges requires moving beyond a training approach to structural and procedural changes.⁴⁶

The following summary is based on interviews with INAC and other departmental representatives closely involved with the delivery of GBA, and literature on best practices and lessons learned. The literature review includes a review of evidence-based research, both Canadian and international, primarily with respect to the adoption of mainstream GBA approaches within governments. However, there are limited evidence-based studies, (i.e. needs assessments, research and evaluations) available on the effectiveness of GBA, particularly within a Canadian government context. Of the federal departments interviewed for this study, for example, only one indicated that they were currently involved in an evaluation exercise (which should be ready later this year), and one other stated that their planned evaluation activities have been postponed.

Strengthened Accountability
The need for clear accountability in the implementation and monitoring of GBA was cited by all departmental key informants as critical to the success of GBA. The literature review also supports this view. Setting up a clear accountability structure is made more challenging by the fact that a gender mainstreaming approach requires individuals and work units across the organization to implement it. Accountability is needed at both the individual and at the higher levels (e.g. annual reporting to key decision-makers/executive).⁴⁷ A clear accountability structure should include the following:⁴⁸

• Clear and explicit goals and objectives. In the international literature a noted successful strategy for implementation involves the inclusion of gender equality objectives in overall program/project objectives where possible;⁴⁹
• A mechanism to incorporate GBA early in the work plan of any new legislation, litigation, policy or program;
• Holding managers accountable for implementing GBA in their areas of responsibility (e.g. through annual progress reports or performance appraisals); and
• Clear identification of responsibilities with respect to application of GBA throughout all stages of the policy/program cycle, by including these responsibilities in job descriptions, work or branch plans, programming guidelines, performance evaluations, Terms of Reference and regular reporting mechanisms.⁵⁰

One interviewee from Health Canada noted that having a formal GBA implementation plan had facilitated uptake in the department. This plan included details on the departmental infrastructure for the implementation of GBA. A designated group was responsible for reporting on GBA implementation to senior management. The plan also included details as GBA capacity building through facilitated workshops at different levels and pilot projects that allowed employees to apply GBA to current issues. The third part of the implementation plan was the development of tools and resources such as a GBA checklist and case studies. Progress on the implementation of. GBA in work-plans helped to facilitate application in the initial phases.

To monitor the quality of GBA one department considers the following:

• Whether quality data were available for GBA;
• Whether an appropriate GBA methodology was used;
• Peer review; and
• Involvement of target population in GBA.

Monitoring approaches of other government departments were found to differ, for instnace CIDA's monitoring framework focuses on the impact of GBA in terms of gender equality. They do not monitor the extent to which GBA is applied by the department. According to other departmental key informants and the literature review it is critical to monitor the impact of GBA on gender equality. ⁵¹

Other interview and survey respondents raised issues related to area management responsibilities and the potential to apply non-compliance penalties. A few also noted that the importance of monitoring the quality of GBA, but as one cautioned: "You want to avoid having GBA being treated as something to be done "as a checkmark or a compliance issue."

In this respect it was also noted that clear reporting requirements to senior management had also helped to keep GBA implementation on track. Key informants at INAC and other departments reported that reporting to senior management on progress in GBA implementation is important for strengthening accountability, and also builds awareness and knowledge among senior managers.

Systematic and Strategic Planning and Implementation
Both interviews and literature highlight the effective use of pilot projects in testing the application of GBA in specific areas and in building capacity of staff to conduct GBAs. According to the literature, successful implementation of GBA is associated with systematic planning and implementation of GBA. The use of action plans, work plans or operational frameworks can be effective tools for implementation.⁵² A Spanish meta-evaluation study of eleven evaluations of gender equality plans concluded that the existence of clear and specific structured plans (clear objectives, actions, timeframe and assigned resources) influenced the evaluations positively.⁵³ In addition, plans should include "…realistic strategies…based on concrete, incremental steps that are selected in light of long-term goals."⁵⁴ The integration of GBA also requires the commitment and mobilization of staff, adequate financial and human resources, appropriate timeframes and achievable targets.⁵⁵

According to international experience, GBA is more effective when phased in gradually, in a select number of policy areas. In addition, pilot projects are considered a good method to test the process and the outcome of the implementation of GBA in a specific area.⁵⁶ A strategic approach to GBA was also cited where GBA is focused (at least initially) on areas where it is most likely to make a difference, or in areas aligned with strategic priorities.⁵⁷

Dedicated Resources to GBA Activities
Some key informants in other departments and at INAC noted that dedicated resources, both human resources and funding, are needed to implement GBA. In a summary of key lessons with program-level application of GBA at CIDA, it was noted that commitment to gender equality would be enhanced by committing resources to gender analyst experts, ensuring adequate resources for building GBA capacity, and monitoring and reporting of gender equality results.⁵⁸

Access to High Quality Training, Tools, and Information
Key informants in other departments reported that high quality training and tools were essential in building capacity. Case studies showing how GBA is applied and GBA checklists detailing steps for applying it were viewed as especially helpful in building people's capacity to apply GBA. Access to gender-disaggregated data and gender sensitive evidence was viewed as facilitating policy decisions.

In a review of international experience with mainstreaming approaches to gender-based analysis, practices to develop knowledge and competency include: ongoing training and awareness raising activities, developing sector specific case studies and analysis.⁵⁹ British Columbia and Sweden have also used "flying experts" who are temporarily hired in a unit or department to provide intensive help. If the purpose of training is to enable employee competency to conduct GBAs, there is a need for specialized and routine training.⁶⁰

Strong Senior Management/Corporate Commitment to GBA
It is important to have strong and visible senior management support for GBA. Other departmental respondents noted that an active and highly visible GBA champion helped to contribute to the integration and use of GBA in their departments.

Constructive Collaboration with Representative Groups
Ongoing collaborations with representatives outside government based on the representative's role as equality-seeking organization, and a process for consultation which facilitates exchanges about issues and priorities are viewed as an important part of gender-based analysis.⁶¹

4.3 Potential Modifications

The following suggestions were derived from key informants and focus group participants, survey, literature and document review, as well as through analysis of best practices, lessons learned and analysis of previously discussed evaluation findings in general.

Improve Accountability and Quality Control
Improved accountability and quality control were viewed as critical ingredients to the integration of GBA into the Department's work and how to raise the profile of the Policy and its results (at present the GBA Policy implementation/achievements are not reflected in INAC's Program Activity Architecture nor Departmental Performance Reports).

A strong central agency challenge function, in addition to departmental mechanisms to monitor the use and quality of GBA were also highlighted as important in ensuring GBA application. The importance of building quality control into the policy approval process was also mentioned. Responsibilities for monitoring and quality control should be clearly delineated. It was noted that GBA should be included in each sector's priority areas; otherwise it will not get done. One suggestion was to include a section in the annual GBA progress report – on the Branch's GBA goals for the following year –to help to measure subsequent annual progress.

Referring to the example of Official Languages, other informant suggestions included developing GBA reporting templates or review guides to assist with the completion of GBAs for Memoranda to Cabinet and Treasury Board Submissions.

Build Awareness and Motivate Employees through Communications, Success Stories and Senior Management Commitment
Many GBAR and policy respondents indicated that employees would be motivated to apply GBA if they understood how GBA applied to their work, and how GBA could lead to positive outcomes. Suggestions were made to distribute success stories.which should reveal the impacts GBA can have and form part of general orientation of employees.

Visible senior management commitment and support were also viewed as important to motivating employees. A suggestion was made to require each sector to deliver a presentation about GBA to staff.

INAC key informants also noted that messaging and communications are critical. For example, communications about GBA are more effective if they come from more senior levels, such as Director Generals. Communications need to provide a clear explanation as to why GBA is important – for example, the relevance of GBA should be related to potential impacts on communities.

Build Competency to Apply GBA through Specific Examples, Training, Tools and Increased Access to GBA Expertise
GBAR and policy respondents noted that they lack adequate understanding of how to conduct GBAs and how GBAs can be applied to specific areas or sectors (e.g. program implementation and delivery, regional level, negotiations, infrastructure). GBARs and policy respondents suggested that staff competency could be increased through greater access to specific examples showing how GBA is applied to specific situations and access to experts within the branch/region that could provide direct guidance for using and applying GBA. This is corroborated by progress report summaries. Previous feedback from GBARs indicated that specific examples of GBA application in relation to different sectors and programs would be useful learning tools. Some key informants reported that policy documents should clarify how GBA is to be incorporated into departmental work.

Many GBARs and Policy respondents felt that there should be some GBA training directed to staff (perhaps as part of orientation training or a presentation delivered by senior management of each branch).

Identify Areas of Priorities/Phased or Risk Based Approach for the Application of GBA
Key informant evidence highlights the challenges of applying GBA to all aspects of Departmental work – particularly when there is a lack of understanding as to how GBA is applied across sector/program areas.

The literature review indicates that a best practice is to phase in GBA gradually to test the process and the outcome of the implementation of GBA in a specific area.⁶² Pilot projects could possibly serve as the basis of success stories or GBA application examples that could be used to build awareness and competence in the department.

This appreciation is corroborated by the findings of the Expert Panel on Accountability Mechanisms for Gender Equality.⁶³ The panel, established by the Government of Canada in the fall of 2005, reviewed the process by which gender-based analysis and gender equality issues are reported. Their findings were based on an analysis of the input of stakeholders and organizations with expertise in reporting on similar issues, existing models and best practices.

The Panel argued against undertaking gender-based analysis of every policy and every program, because '... the simple reality is that gender-based analysis is still little understood by either public servants or the public at large.' Instead it recommended focusing on a few "big ticket" items and counselling the Government to '...consider its key leverage points and spend energy on making the most of the results flowing from those points.' Further recommendations focused on the potential role to be played by the central agencies of Finance and Treasury Board, the value of engaging non-governmental organizations, and the demonstration of leadership within the government.

Additional issues
Survey respondents were asked whether they were aware of any issues or factors which could challenge the sustainability or further implementation of GBA in the Department (Survey Question 34). Twelve comments were received. These included concerns about a lack of awareness of the Policy, its application or its potential to improve outcomes, as well as perceptions about uneven commitment to implementation (4), workload demands (3), and the advantage of increasing the role of central agencies in monitoring GBA across departments.

Survey Respondents were asked whether they would identify specific issues which should be addressed (Survey Question 35). Seven respondents replied. The issues raised included calls for a broader definition of 'gender' analysis (to include analysis on sexual orientation), as well as references to employment equity concerns. Other issues raised by survey respondents pertained to INAC's relations and consultations with Aboriginal clients in terms of awareness raising about GBA and capacities for GBA. Specific issues raised included a call for greater attention to gender in economic data, issues identified as particularly affecting males (in terms of the National Child Benefit, adult education and upgrading), as well as greater information about women's leadership roles in Bands, and family violence.

Survey respondents were asked how important they felt the following range of supports were to sustaining or strengthening the outcomes of GBA at INAC:

Source: Survey Question 30.

Fourteen respondents also provided insights into the type of incentives which could be used (one respondent noted that no incentives should be used as it should be considered a regular part of one's job). These comments were similar in nature to another series of comments (16) received regarding suggestions for sustaining or strengthening the quality and outcomes of gender-based analysis (Survey Question 34) and thus are presented together below:

• Greater and or sustained efforts at raising the profile of GBA, awareness building and/or outreach, training, communications and dissemination;
• Develop measures and mechanisms to ensure that GBA is informing critical processes (highlighted were Policy Committee oversight, contracting, management oversight and GBA templates or review guides). Three touched upon the need for tools to facilitate inputs into decision-making documents, with one indicating a role for Policy and Strategic Direction, and another noting that a specific template had been developed at INAC (in Economic Development) to support the application of gender-based analyses;
• Develop a risk-based approach so as to formally identify key program and policy areas which would benefit from GBA (rather than continuing to attempt to mainstream GBA homogenously across the Department);
• Greater access to gender related information, data and research, incorporation into performance measurement and program architecture;
• Recognition of the need for support from Senior Management and suggestions on how to achieve it (e.g., linking GBA to performance agreements, performance pay or penalties);
• Greater clarity of Policy requirements, or access to fora where the effects of gender-based discrimination can be understood, or that employees and intended beneficiaries should have greater knowledge of expectations in this area; and
• A mention that audits and evaluations should also include a mandatory gender analysis, and greater monitoring and identification of instances where GBA had not been conducted.

5.0 Conclusions and Recommendations

5.1 Conclusions

Many elements of INAC's Gender-Based Analysis Policy were found to be consistent with federal requirements for the application of GBA. INAC's focus on capacity building, education, and support via the GBAR network are an appropriate means for integrating GBA.

At the same time the Government has recently placed priority on the importance of increasing accountabilities for implementing gender-based analysis across Government and for sustaining the conduct of GBA. Analysis of the available evidence indicates that INAC's approach to accountability in the implementation of its GBA Policy requires strengthening and updating.

Good and promising practices of the application of GBA and in addressing gender issues raised through GBA are clearly evident. However, following more than nine years of implementation, the level of integration into the departments work remains uneven and very limited in many key areas, including proposal development and performance measurement.

Overall, INAC lacks sufficient structure or accountability mechanisms, and capacity for a comprehensive and sustained implementation of GBA across all the Department's work. The evaluation found few levers that motivate or provide incentives for application of GBA (e.g. creation of gender sensitive performance targets, incentives, templates, and consequences).

Lastly, since the GBA Policy was first announced, INAC's mandate has expanded significantly and the Department is working with an increasingly complex and diverse profile of critical issues, clients and representative organizations. These dynamics pose an even greater demand for risk assessment as well as for quality and culturally relevant policy analyses and performance measurement frameworks and strategies.

The challenges faced for improving the effective application of GBA are not unique to INAC within the context of GBA. For one, the Government's emphasis on this issue indicates recognition that the issue need be addressed across government. Moreover, the situation is not unique within the context of emerging developments in other horizontal policy areas.

At present, for example, INAC and other federal departments are working to strengthen accountability, the identification of measurable results, performance measurement and reporting in order to better track and demonstrate the results and achievements of their implementation of the Official Languages Policy and Sustainable Development Strategy.

It would be beneficial for the Department to both learn from and inform these efforts as it moves forward in strengthening the application and impacts of gender-based analysis.

5.2 Recommendations

In order to further the implementation, measurement and impacts of INAC's Gender-Based Analysis Policy, it is recommended that INAC should:

1. Revisit expected achievements, activities and timeframes, introducing more targeted elements (benchmarks) and a risk-based approach to implementation. A particular focus should be placed on addressing issues and areas of high risk to the achievement of INAC's strategic outcomes;
2. Update and strengthen INAC's approach to accountability, performance measurement and reporting in line with evolving federal practice and directives respecting GBA, in particular, and horizontal policies, in general;
3. Review and revise INAC's approach to training, outreach and support (as currently provided through WIGE and GBARs) so as to enhance the outcomes of a more risk-based and targeted implementation approach in the short to medium term, and continue towards improving awareness and increased and sustainable capacities at the departmental level over the longer term;
4. Introduce mechanisms, tools and measures to better facilitate management support and oversight in the short term, and to strengthen the quality of GBAs and their outcomes over the longer term (e.g., GBA review guides and/or templates for conducting GBA, best practices, examples, establishing checkpoints at various points through INAC's internal approval processes, among others);
5. Continue to work within the Department, and with Status of Women Canada, central agencies, representative organizations, beneficiaries and other stakeholders in order to strengthen the implementation and impacts of GBA, in particular, and horizontal policies, in general;
6. Review and revise the Policy's governance structure, as well as the roles and responsibilities of WIGE, GBARs, senior management and employees, particularly with respect to performance measurement, quality control and reporting; and
7. Review financing arrangements for implementing the Policy to provide a more stable base for supporting and sustaining the application of GBA at INAC, taking into account preceding recommendations.

Appendices

Appendix A: List of Acronyms

CIC: Citizenship and Immigration Canada
CIDA: Canadian International Development Agency
DAGE: Diversity and Gender Equality
DOJ: Department of Justice
DPR: Departmental Performance Report
GBA: Gender-based Analysis
GBAR: Gender-based Analysis Representative
HC: Health Canada
HRSDC: Human Resource and Social Development Canada
INAC: Indian and Northern Affairs Canada
IRPA: Immigration and Refugee Protection Act
MC: Memorandum to Cabinet
MRP: Matrimonial Real Property
NWAC: Native Women's Association of Canada
PCO: Privy Council Office
RPP: Report on Plans and Priorities
SWC: Status of Women Canada
TBS: Treasury Board Secretariat
WIGE: Women's Issues and Gender Equality Directorate

Appendix B: Evaluation Matrix (Planned)

Appendix C: Management Response / Action Plan

Evaluation Project Title:  Evaluation of the Implementation of INAC's Gender-Based Analysis (GBA) Policy
Evaluation Project #:   07/12
Region or Sector:   Policy and Strategic Direction

1. INAC, Women's Issues and Gender Equality Directorate, (2003), Gender Equality Analysis Repositioning Strategy, Operations Committee, June 18, 2003. (return to source paragraph)

2. SWC, (1995), Setting the stage for the next century: The Federal plan for gender equality. Ottawa: Status of Women Canada. (return to source paragraph)

3. Second Report of the House of Commons Standing Committee on the Status of Women (Gender-Based Analysis: Building Blocks for Success), (April 2005). (return to source paragraph)

4. Government Response to the Second Report of the Standing Committee on the Status of Women Gender-Based Analysis: Building Blocks for Success (September 2006) (return to source paragraph)

5. Treasury Board of Canada, (2008), Guide to Preparing Treasury Board Submissions, Section 9.7.3.   (return to source paragraph)

6. See Footnote 4 for source (return to source paragraph)

7. SWC, Report on Plans and Priorities, 2008-09 (return to source paragraph)

8. INAC, (1999 and 2006), Gender-Based Analysis Policy. (return to source paragraph)

9. INAC, WIGE, (2003), Gender Equality Analysis Repositioning Strategy (Operations Committee), Annex B (return to source paragraph)

10. INAC, WIGE, (2003), Gender Equality Analysis Repositioning Strategy (Operations Committee) (return to source paragraph)

11. WIGE, (2005/2006), Annual Report on Gender Equality Analysis, Draft, (from program files) (return to source paragraph)

12. Note that these Representatives were originally titled Gender Equality Analysis Representatives (return to source paragraph)

13. Goss Gilroy Inc., (2002), Gender Equality Analysis Policy Evaluation Framework; and INAC, (2007), Welcome to the GBAR Network, (memo) (return to source paragraph)

14. INAC, (circa 2006-2007), Nomination of a Gender-Based Analysis Representative for your Branch / Region, see also WIGE, (2007), Introduction to the Working Guide on GBAR (Draft). (return to source paragraph)

15. INAC, (undated), Welcome to the GBAR Network (What are your Reporting Requirements?), (memo) (return to source paragraph)

16. Ibid. (return to source paragraph)

17. Note the evaluation issues appear in the Terms of Reference for the Evaluation of INAC's Gender-Based Analysis Policy (approved December 2007); the evaluation questions were subsequently developed to assist in addressing these issues. (return to source paragraph)

18. Status of Women Canada, (2004), An Integrated Approach to Gender-based Analysis, 2004 Edition (return to source paragraph)

19. INAC, (2006), Departmental Performance Report, (2005-2006). This consultation was also reported in NWAC, (2007), Reclaiming our Way of Being: Matrimonial Real Property Solutions. (return to source paragraph)

20. INAC, Gender-Based Analysis. Issue Paper No. 10. Backgrounder - Family Homes on Reserves and Matrimonial Interests or Rights Act. (return to source paragraph)

21. INAC, (2004), Measuring the Well-Being of Aboriginal People: an Application of the United Nations' Human Development Index to Registered Indians in Canada, 1981-2001. (return to source paragraph)

22. Survey Question 12. (return to source paragraph)

23. Survey Question 17 (a-g). (return to source paragraph)

24. INAC's Evaluation Reports (return to source paragraph)

25. The files contained one annual progress report dated 2005/06 and progress summaries for another undated year. (return to source paragraph)

26. WIGE, (2007), SWC and INAC – Areas for Collaboration. (return to source paragraph)

27. As reported in Wendy Grant John, (2007), Report of the Ministerial Representative Matrimonial Real Property Issue on Reserve, some criticisms were raised with respect to the consultation process. These included short timelines for participant consultations and dissemination practices which favoured the internet over a more community based approach. (return to source paragraph)

28. Ibid. (return to source paragraph)

29. NWAC, (2007), Matrimonial Real Property Issues Paper. (return to source paragraph)

30. NWAC, (October 18, 2007), Progress Release. Aboriginal Women Will Hold Government Accountable to Its Words of Action. (return to source paragraph)

31. SWC, (August 2005), Gender Equality Consultation, Aboriginal Women: An Issues Backgrounder. (return to source paragraph)

32. See Footnote 13 for source. (return to source paragraph)

33. See Footnote 4 for source. (return to source paragraph)

34. Comments were received from 14 respondents. Some respondents offered comments not directly related to the survey question, identifying gaps of a differing nature, including lack of access to training, GBA examples, communication and publication challenges, the challenges of applying GBA in a regional context, and limited understanding of GBA. (return to source paragraph)

35. See, for example, INAC, Communities First: Questionnaire Results, and First Nations Governance and First Nations and Northern Statistics Section, INAC. Registered Indian Population by Sex and Residence, (annual publications). (return to source paragraph)

36. Jeremy Hull, (2006), Aboriginal Women: A Profile from the 2001 Census, prepared for the WIGE Directorate (INAC). (return to source paragraph)

37. The updated website is not yet available. (return to source paragraph)

38. INAC, (2007), Introduction to the Gender-Based Analysis Working Guide. (return to source paragraph)

39. See Footnote 34 for source. In addition to his study of Aboriginal women in the 2001 census, Jeremy Hull (Prologica Research, Winnipeg) has conducted several census based studies for INAC, including work on post-secondary education and labour market outcomes, Inuit social trends and Aboriginal single mothers. In 1995 Hull, along with Steward Clatsworthy and Neil Loughran, examined issues related to patterns of employment, unemployment and poverty for the Royal Commission on Aboriginal Peoples. (return to source paragraph)

40. Goss Gilroy Inc., (2002), Gender Equality Analysis Policy Evaluation Framework. (return to source paragraph)

41. WIGE, Letter to Senior Manager, Nomination of a Gender-Based Analysis Representative for your sector, (circa 2007), and Gender-Based Analysis Annual Progress Report, 2006-2007 (return to source paragraph)

42. As previously mentioned, GBAR respondents indicated that it was not within their role/responsibilities to provide assistance for conducting GBAs. (return to source paragraph)

43. Themes in Domestic and International Literature Respecting Experience with Gender Analysis and Gender Mainstreaming as Strategies to Achieve Gender Equality Goals, August 2007, (Author, location unknown, Report produced by WIGE). (return to source paragraph)

44. Ibid. (return to source paragraph)

45. The Scottish Government, (2003), Learning From Experience: Lessons in Mainstreaming Equal Opportunities. (return to source paragraph)

46. See footnote 41 for source. (return to source paragraph)

47. UNDP, (2005), Evaluation of Gender Mainstreaming in the UNDP. (return to source paragraph)

48. Department of Justice, (2002), Needs Assessment; C. Saulnier, Bentley, S., Gregor, F., MacNeil, G., Rathwell, T., and Skinner, E., (1999). Gender Planning: Developing an Operational Framework for En-Gendering Healthy Public Policy. Halifax: Maritime Centre of Excellence for Women's Health. (return to source paragraph)

49. A review of gender mainstreaming by the Swedish International Development Cooperation Agency found that projects with the most explicit gender equality objectives also had the most positive impacts on gender equality (cited in J. Hunt, (2004), Successful Strategies for Addressing Gender Equality Issues in Programs and Projects: What Works?, Development Bulletin, no. 64, pp. 53-57). (return to source paragraph)

50. Some of the other department representatives indicated that it is important to assign responsibilities in branch plans/ work plans. A clear implementation plan was also cited by one external respondent as critical for successful implementation. See also footnote 46, above (Saulnier et al.). (return to source paragraph)

51. Ibid. (return to source paragraph)

52. Ibid. See also: Bustelo, M, (2003). Evaluation of Gender Mainstreaming: Ideas from a Meta-Evaluation Study. (return to source paragraph)

53. Bustelo, M, (2003). Evaluation of Gender Mainstreaming: Ideas from a Meta-Evaluation Study. (return to source paragraph)

54. Schalkwyk cited in Saulnier et al., (1999: 5), (See footnote 46 above for full citation). (return to source paragraph)

55. See footnote 41 for source. (return to source paragraph)

56. SWC, (2005), Looking Ahead: Sustaining Gender-Based Analysis (GBA) in the 21st Century, GBA Conference, January 2005. (return to source paragraph)

57. J. Hunt, (2004), Successful Strategies for Addressing Gender Equality Issues in Programs and Projects: What Works?, Development Bulletin, no. 64, pp. 53-57. (return to source paragraph)

58. CIDA, (2007), A Report on the 2007 CIDA Gender Equality Forum. Gatineau: Canadian International Development Agency. (return to source paragraph)

59. The Scottish Government, (2003), Learning From Experience: Lessons in Mainstreaming Equal Opportunities. (return to source paragraph)

60. See footnote 46 above for source (Saulnier et al.). (return to source paragraph)

61. Ibid. and SWC, (2001), Canadian Experience in Gender Mainstreaming. (return to source paragraph)

62. SWC, (2005), Looking Ahead: Sustaining Gender-Based Analysis (GBA) in the 21st Century, GBA Conference, January 2005. (return to source paragraph)

63. Georgina Steinsky-Schwartz, Dorienne Rowan-Campbell, Louise Langevin, (2005), Equality for Women: Beyond the Illusion, Expert Panel on Accountability Mechanisms for Gender Equality, Final Report – December 2005. (return to source paragraph)

64. Senior officials: Research, Information Management, Policy Development and Approvals Directorate, Strategic Outcomes/ Performance Measurement, etc. (return to source paragraph)

65. That is, continuing relevance with respect to first objective of the 1995 Federal Plan for Gender Equity; Relevance with recent recommendations of the Second Report on the Standing Committee on the Status of Women, (May 6, 2006), particularly recommendation 9, which deals with departmental responsibilities. (return to source paragraph)

66. GEA: Gender Equality Analysts, now 'GBARs' or Gender-Based Analysis Representatives. (return to source paragraph)

(Return to source)