1. Introduction
Trócaire[1] is an Irish development agency that was established in 1973 by the Catholic Bishops of Ireland. Trócaire works in over 20 countries in partnership with local civil society organizations implementing both long-term development and emergency response programmes. Trócaire has been working in Kenya for over 30 years and established its Nairobi office in 1994. Working with over twenty five local partner organisations, Trócaire focuses on Secure Livelihoods and Environmental Justice (SLEJ); Governance and human rights (GHR), Integrated Gender and HIV (IGH). Trócaire contributes towards ‘a just Kenya where communities have agency and are realising their fundamental rights through critical engagement within a context of accountable and equitable governance.’ (Country Strategic Plan-CSP: 2015-20). Our country programme is built on an ecological model that integrates a change process based on micro, meso and macro level engagements with strong partnerships, a commitment to capacity building, clear advocacy priorities, high levels of financial oversight and holistic programme design to bring about social change.
Trócaire works at 3 levels across all programmes: Micro: Communities (especially women and youth within those communities) are empowered to be more ‘resilient’ to hazards, shocks and conflict i.e. are able to identify problems, come up with solutions, plan to address their problems and carry out those plans so that the rights of all are protected; Meso: Civil society is vibrant and promotes citizens’ concerns and rights while county level government is more accountable and managing an equitable devolved governance system – responsive to citizen’s needs; and Macro: Government is influenced to be more accountable, equitable and responsive to citizen’s needs.
2. Brief Overview of the Programme
Trócaire ASALs resilience programme (ARP), which is an integrated livelihoods and humanitarian programme, focuses on the Arid and Semi-Arid Lands (ASALs) of Kenya which cover 80% of the country and experience low, infrequent and unreliable rainfall, thus recurrent droughts. The programme seeks to contribute to poverty reduction and resilient livelihoods in the targeted ASAL areas in Kenya. It builds and consolidates on the work done in the previous Sustainable Livelihoods and DRR programme (KEN08-01) and the humanitarian response programme (KEN1101) programme. It therefore has three complementing cycles that explain the resilience and Development Livestock Research for Rural (LRRD) cycle: i) responding to disaster, ii) supporting recovery and rehabilitation and iii) supporting development.
The ARP programme expected impact is to contribute to poverty reduction and resilient livelihoods in the target ASAL areas in Kenya. The ARP has four main objectives namely:
1. To improve the sustainable livelihoods’ security of targeted vulnerable HHs through increased income, food and diversified livelihoods strategies.
2. To ensure that vulnerable pastoralist, agro pastoralist and marginal mixed farmers have secure access to natural resources, particularly water and land, which are managed sustainably.
3. To enable 52,360 households and communities in the targeted Arid and Semi-Arid Lands districts of Kenya to better mitigate risk, prepare for and respond to, humanitarian crisis and shocks.
4. To ensure target communities benefit from the achievement of the programme objectives through improved programme management.
The programme is being implemented by the partners:
The programme is implemented with financial support to partners while Trócaire has the responsibility for grant management, supporting the capacity development of partners and donor coordination.
Our Project Approach to Resilience and Sustainable Livelihoods:
Micro- improving and diversifying crops production through training on improved ecologically sensitive agricultural practices, improved seed varieties, alternative crops and promoting alternative livelihood strategies; livestock breed improvement, improved access to animal health and livestock marketing; improved irrigation techniques; improved post-harvest and pest management practices; strengthening the role and capacities of women and youth on entrepreneurship and micro finance; building capacity for engagement in policy at local levels. Conflict mitigation and resolution in contested areas.
Meso- Natural Resource Management (NRM) including pasture and rangeland management; county level policy engagement; development of Early Warning Systems (EWS) and Community Managed Disaster Risk Reduction (CMDRR) and knowledge and skills in disaster prevention and mitigation; Conflict transformation, especially in cross-border situations.
Macro- creating an enabling environment for resilient livelihoods through lobbying and engaging government on climate change policy, natural resources policies, disaster risk reduction and governance of the ASALs, including responses to potential rights abuses related to the extraction of natural resources.
Target Groups
The evaluation will be conducted within the communities in the areas where the partners have been implementing the programme, specifically targeting programme beneficiaries. There are approximately 23,000 HHs that have directly benefitted from programme interventions, and they are categorised into four main groups based on their livelihoods:
i. Pastoralists – Focus on rearing camels, cattle, sheep and goats.
ii. Agro-pastoralists – these combine extensive livestock rearing and rain-fed cereal production for household consumption.
iii. Marginal Mixed/ Sedentary farmers – practice mixed farming, cultivating food crops (sorghum, maize, green grams and other cereals) along with modest sheep and goat herding.
iv. Urban poor – these are households who have lost their livestock and now depend largely on human labour.
3. Purpose of the End line and Programme Evaluation
The main purpose of the End Line Study and Programme Evaluation is to establish the changes experienced by this target population with regard to the indicators included in the Results Framework. As baseline study was conducted prior to the implementation of this five-year programme (January 2012 – February 2017), so anticipate being able comparison of the results. This will form a key element of our commitment to accountability both to the donors and our beneficiaries. The secondary aim of the evaluation is to gain insight into key elements of the implementation process which either explain why reported changes occurred and shed light on howsuch changes were achieved. We expect that an assessment of impact and understanding of key processes will support the generation of programme recommendations that could be considered for the design of future programmes.
We intend to contract an independent consultant to carry out this work and expect our partners to play a key role in designing and carrying out the evaluation such as confirming the suitability of tools, introducing the consultant to the communities and assisting the consultant in data collection, as well as commenting on the analysis of all findings.
4. Scope of Work
This is an external evaluation and should comply with the international evaluation standards (OECD), Trócaire guidelines and provide a space for learning around best practices for supporting target communities. Trócaire is seeking to identify a consultancy team who will undertake this end line evaluation within the agreed terms of reference.
5. Proposed Methodology
The evaluation exercise will be consultative and participatory, entailing a combination of desk review, interviews, household surveys, FGDs and data analysis. While interviews are a key instrument, a range of data sources will be triangulated to ensure that the evaluation is sound and objective. A random sample of the required sample size will be extracted from a beneficiary database of 23,351 households so far reached. On the basis of the foregoing, the consultants will further elaborate on the method and approach in a manner commensurate with the assignment at hand and reflect this in the inception report, which will subsequently be approved by the programme team in consultation with key stakeholders.
The evaluation report will be shared with Trócaire, donors, government agencies, partner organisations, INGOs and NGOs working on resilience particularly in the ASALs of Kenya.
The key inputs to the evaluation should be as follows:
• Interviews with key staff such Directors, Project Managers, Project Coordinators, Project Officers, M&E Officers, Field Facilitators/ Monitors and others deemed necessary by the review team.
• Interviews with stakeholders including respective county government departments, other NGOs working in the various counties and community leaders
• Field visits, household surveys and FGDs in project sites in the 7 counties – Tharaka Nithi, Kitui, Embu, Turkana, Baringo, Nakuru and Nairobi.
• Substantive documentation:
a) At Trócaire level: KEN1102 – ARP Programme Proposal Document (PPD) and budgets; KEN11-02 Results Framework (RF) and baseline – including logic model and risk register; all substantive field reports; Internal review reports (including Gender & HIV mainstreaming, microfinance review and water inspection) and others as deemed necessary by the evaluation team.
b) At Partners level: Respective partners project proposals, partner RFs, monitoring reports, annual and bi-annual financial and narrative reports and project review reports.
The Specific Objectives of the end line study/ evaluation are:
i. To collect data against the programme outcomes and indicators in order to assess progress. There is a total of 20 indicators in the Results Framework (see annex). Data collected should inform the findings of the evaluation;
ii. Assess the contribution of regional (based at Counties) and policy partners in influencing legislative and policy change at County and national level;
iii. Examine progress in terms of the following programme parameters:
• Impact: To assess documentary evidence of impact of the programme with reference to approaches and methodologies used to reduce poverty and builds resilience – knowledge, attitude and practices. This includes intended, unintended, positive or negative changes. It involves the main impacts and effects resulting from activities on the local social, economic, environmental and other development indicators. A few questions to consider during the process include:
• Relevance: Analyse the appropriateness of the project design, activities, strategies and approaches in the light of the operational context, the timeliness of the response and its adaptation to the livelihoods security situations. It involves assessing the extent to which the activities are suited to the priorities and policies of the target groups, target HH and donors.
• Effectiveness: Analyse the extent to which activities and outputs have attained its objectives. This should include an analysis of programme implementation strategies employed to reach desired results.
• Impact: To assess documentary evidence of impact of the programme with reference to approaches and methodologies used to reduce poverty and builds resilience – knowledge, attitude and practices. This includes intended, unintended, positive or negative changes. It involves the main impacts and effects resulting from activities on the local social, economic, environmental and other development indicators.
• Sustainability: Assess the extent to which the project interventions took into consideration longer term needs of the target population and to what extent programme results or benefits will be sustainable after programme closure. The programme has to be environmentally as well as financially sustainable.
• Innovation and technology: To identify, review and recommend promotion and scale up of innovations and appropriate technology that enhance livestock and agricultural productivity and build resilience to drought and other shocks.
The consultant should follow the design of the baseline study and adopt its tools, while recognising that these may need some adaptation considering that some of the indicators were borrowed from the HEA study in Turkana and Tharaka Nithi respectively.
6. Proposed Stages of Review & Reporting
6.1. Develop an inception report with a review framework. This will be done after discussion of the TOR and will include: detailed plan on the evaluation process, plan of respondents with the inquiry method, timelines and budget.
6.2. Collection and review of partner information and identifying information gaps through a desk review. This should include review of necessary government policy documents.
6.3. Develop and agree on set tools to use and field data collection process, based on tools used in Baseline and Mid Term Review process; data collection must be based specifically on the output and outcome indicators contained in the RF
6.4. Train enumerators and data management clerks on the use of tools – digital data would be preferred. This should include pre testing of tools.
6.5. Conduct fieldwork to gather information and fill identified gaps from the desk review.
6.6. Prepare and share statistical tables and coding lists (as appropriate). This should be disaggregated by gender and location/ or partner for comparison with the baseline.
6.7. Preliminary analysis, draft report and feedback to wider group (including decision makers at partner level)
6.8. Final analysis and report of findings and recommendations (25 – 30 pages excluding annexes). This should contain the following:
• Executive summary (5 pages). This should cover: background to the programme, brief overview of aims of the evaluation, brief summary of the methodology, key findings, conclusions, recommendations and summary of management response;
• Introduction (1 page).
• Description of evaluation methodology (3 – 4 pages)
• Situation analysis with regards to the outcomes, outputs and partnership strategies (4-5 pages)
• Key findings, including best practices and lessons learnt (3 – 4 pages); This section of the report should be clearly structured to show levels of achievement against each indicator contained in the Results Framework, providing quantitative and qualitative evidence for achievement of impact for each indicator
• Analysis of opportunities to provide guidance for future programming (3 – 4 pages)
• Conclusions and recommendations (4-5 pages)
• Appendices: charts, TOR, field visits, people interviewed, documents reviewed etc.
6.9. Dissemination of findings. A part from a validation and stakeholder workshops, the final report will be submit to Trócaire headquarters for final review and endorsement by the Strategic Impact Unit. This process will declare the report to the donors and the wider public.
7. Evaluation Team Composition and Competencies
The suggestive team for this assignment may include:
i. MEAL Expert having working experience in Resilience, Disaster Management, Disaster Risk Reduction and Policy Advocacy;
ii. A gender and inclusion expert with Women Empowerment and Disaster Risk Reduction
iii. A community mobilisation and participation expert
iv. Finance expert – able to make deductions on Cost-Benefit Analysis (CBA) and or Value for Money (VfM)
v. Enumerators will be recruited from respective sites and should have a post-secondary level of education
vi. At least 3 partner staff will be fully involved in the facilitation and mobilisation where necessary
Note: The consulting firm has the flexibility to suggest its own teams as deemed fit
8. Evaluation Quality and Ethical Standards
The evaluators should take all reasonable steps to ensure that the evaluation is designed and conducted to respect and protect the rights and welfare of the people and communities involved and to ensure that the evaluation is technically accurate and reliable, is conducted in a transparent and impartial manner, and contributes to organizational learning and accountability. The evaluation team will also commit to adhering to Trócaire’s Safeguarding Programme Participant Policy and Code of Conduct.
9. Duration and Schedule
The End of Programme End line and Evaluation will take place from 10th October to 20th of November.
10. Anticipated Outputs/ Deliverables
The consultant will be responsible for the following deliverables:
10.1. Inception report. The inception report will be a scoping exercise for the evaluation and will include a detailed description of the proposed methodologies, data collection and reporting plans with draft data collection tools such as interview guides and survey questionnaires, the allocation of roles and responsibilities within the team, a timeframe with firm dates for deliverables, and the travel and logistical arrangements for the team. The inception report shall be submitted after 5 days of commencing the consultancy.
10.2. Debriefings/feedback to management at all levels: The team will report its preliminary findings to Trócaire 2 days after carrying out fieldwork, in Nairobi. The team leader will incorporate any comments relating to factual inaccuracies etc., and present the full draft report to the evaluation management team within one week of the debriefing meeting.
10.3. Draft report: A draft report, identifying key findings, conclusions, recommendations and lessons for the current and future projects, and taking into consideration the outputs of the debriefing session. The report should be clearly structured to provide specific evidence of achievement and effectiveness against each indicator in the RF.
10.4. Statistical tables and coding frameworks: This should be shared with the draft report
10.5. Final End Line and Evaluation Report: A final evaluation report of a maximum of 30 pagesexcluding appendices, clearly setting out recommendations arising from the evaluation will be submitted 5 days after receiving comments from the evaluation management team. The content and structure of the final analytical report with findings (evidence of achievement and effectiveness of programme against each indicator providing both quantitative and qualitative data), recommendations and lessons learnt covering the scope of the evaluation should meet the requirements of the Trócaire Field Programme Monitoring and Evaluation Policy.
10.6. Updated Programme Results Framework: This gives a summary of performance of the programme against all indicators and outputs.
10.7. PowerPoint presentation of key findings: A summary of context, finding and recommendation for high level/ management discussion of the programme performance.
11. Intellectual Property
All the materials, information and reports, the output of the evaluation exercise shall be the property of Trócaire and the consultant is bound by Trócaire’s confidentiality requirements. All data sets and transcripts must be provided to Trócaire Kenya office and Head Office in electronic copies and on a CD (2). The evaluators will not be allowed, without prior authorization in writing, to present any of the analytical results as his or her own or to make use of the evaluation results for private publication purposes.
HOW TO APPLY:
1. Submission of Proposals
Based on the above, Trócaire is inviting interested parties to submit expressions of interest. Individuals or firms applying shall detail the following:
• Consultants’ profile and Capability Statement describing the technical capacity and experience of the firm or group of individuals
• Names and resume of individuals or team members proposed and their roles in the achievement of the assignment
• 3 professional referees (preferably previous clients) and sample reports of similar assignments taken in the recent past
• Interpretation of the Terms of Reference (TORs)
• Detailed evaluation design with implementation plan and time frames. The assignment will commence by 10thOctober 2016 and must end by 20th of November 2016
• Interested candidates should submit their application to the following email address: hrkenya@trocaire.org with“KEN1102 – ASALS Resilience Programme 2012 – 2017 External Evaluation Consultancy” as the subject of the e-mail. For detailed ToR and Results Framework, send a request through the above email.
• Deadline for submission is 5th of October 2016