ADSE is calling for eligible bidders to apply for the following call for consultancy services
Evaluation Terms of Reference
Evaluation title: Integrated Initiatives to Combat Food Insecurity End of Project Evaluation
Partner: Anglican Development Services-Eastern (ADS-Eastern)
Project Name: Integrated Initiatives to Combat Food Insecurity
Locations: Kithungo in Makueni County; Itoleka, Mutulu and Kiomo in Kitui County
Period reviewed: Phase 1: 1 July 2011 – 30 June 2014
Phase 2: 1 July 2015 – 30 June 2018
1.1 Target group; structure and working methods
In the project areas, organized groups have been brought together by ADSE to form a Community Based Organization (CBO) for ease of operation and to have a structure that will continue coordinating the project activities at the community level following phase over. The CBOs are composed of between 10 and 15 organized groups varying from youth groups, self-help groups to women’s groups all involved in different activities based on their common interest. The table below illustrates the project areas, the names of the CBOs and the number of groups. Each CBO is registered with the Government ministry of Culture and Social Services for regulation and is managed by an elected Project Management Committee. When the groups were brought together to form the CBOs, they were trained by ADSE in collaboration with the Government Ministry of Culture and Social Services on constitution development which led to the election of officials and registration and it is the constitution document that guides the implementation of the projects at the community level. The Project Management Committee is composed of representatives from the groups who take feedback to the group about any decision made at the management level. With this there is proper coordination of activities at the CBO level. The Project Management Committee meets once every month to deliberate on issues pertaining the implementation of the projects and reflection meetings are conducted on a quarterly basis. Capacity building of the CBOs in governance and leadership aspects has been done in order to ensure that they are in a position to continue with development and manage the projects well to ensure that the benefits to the communities are not jeopardized.
The CBOs are used as a means to reach out to the larger community members since most of the project activities, although coordinated and safeguarded by the CBOs, benefit the whole of community members, e.g. water structures, demonstration of best farming methodologies etc. In case of water structures’ construction, all community members participate in the construction but are led by the CBO who have to institute a site committee for operation and maintenance of such a project, to ensure continuous supply of benefits.
Table 1: Project areas, names of the CBOs and the number of groups:
|Project area/Location||CBO Name||No. of groups||County|
1.2 Participation of the target group
The target groups were involved in the project design process. This was done through an in-depth consultation and involved in the key milestones of the project design as detailed below.
They were involved during the baseline survey that was conducted during the first phase of the IFSP in 2011 and during the PVCA (Participatory Vulnerability Capacity Assessments) in order to bring out the hazards and potentials existing in the areas that formed the basis of the interventions.
Community members have also been involved in the mid-term evaluation conducted in 2014 that provided insights into the programming of this phase. The CBOs have been trained on PAMM (Participatory Activity Monitoring Matrix) which is a tool that is used to plan for the year and is also used to do monitoring.
The PAMM is an annual plan that is created by the CBO at the beginning of every year and it is the same document that is used during reflection meetings to check how implementation of the planned activities has taken place during the period and whether there are areas that need improvement. Implementation follows the agreed plan and the community members are involved in the same. In the construction of water structures, the community members provide all the unskilled labor and local materials while the organization provides the hardware materials (cement, iron bars, biding wire) and skilled labour. During demonstrations, they participate in the preparation, learning and even providing the farm where the training takes place. These are strategies geared towards increasing the ownership of the projects and sustainability of the same.
1.3 Expected Project Outcome and Impact
Development goal: Improved/enhanced food and livelihood security of the rural population in Kitui and Makueni counties of Eastern Region, Kenya by the end of the project period, June 2018.
- Increased animal and crop productivity.
- Increased access to potable water.
- Increased vegetation cover.
- Enhanced effective governance of the CBOs to undertake effective planning and monitoring of interventions in their communities.
- Empowered communities, including children that are able to claim and defend their human rights through documenting, reporting and referral of cases of abuses and violations.
- Enhanced ADSE Capacity to effectively deliver the programme by June 2018.
This is an end of project evaluation which will focus on impact to date, lessons learnt and analysis about whether the project has realised its intended outcomes. The evaluation’s learnings and recommendations will also be important to assist ADS-Eastern in its future implementation of similar projects in other locations.
The evaluation will consider three objectives:
- Assessment of the success of ADS-E in meeting its goals, objectives and targets as outlined in the project proposals.
- Review effectiveness of current strategies for sustainability and recommend how sustainability can be strengthened in future ADS-E projects. Sustainability of development activities should be a key consideration in all program planning, design and implementation. To this extent, the evaluation should examine the ability of the program to encourage participation and ownership of the development activities by program stakeholders and the wider community.
- Identify learnings and program adaptations for future implementation of projects by ADS-E. This should also include recommendations for local stakeholders and the community to support them in managing activities after the project exits.
The OECD DAC Criteria for Evaluating Development Assistance should provide this evaluation with a guiding framework. See Key Evaluation Questions below for further information.
This evaluation will be conducted for the benefit of both ADSE and the CBOs as the implementers. The results/findings and recommendations of the evaluation will be shared with Transform Aid International (TAI), CBOs as well as any other key stakeholders. The Consultant’s findings and recommendations will be used as learnings to adapt and design future projects.
3. Subject and Focus
The Consultant will review the activities of the Integrated Initiatives to Combat Food Insecurity project from the beginning of the project on 1 July 2011 to the end of the project on 30 June 2018, using Key Evaluation Questions (KEQs).
The KEQs for this evaluation are tied to the criteria developed by the Development Assistance Committee (DAC) of the Organisation for Economic Cooperation and Development (OECD).
The DAC criteria identifies five ‘pillars’ that should be taken into account in an evaluation of a development intervention:
- Relevance: The extent to which the aid activity is suited to the priorities, needs and policies of the target group or donor.
- Effectiveness: The extent to which the project has achieved its intended objectives.
- Efficiency: Efficiency measures the outputs – qualitative and quantitative – in relation to the inputs. It is an economic term which signifies that the intervention uses the least costly resources possible in order to achieve the desired results. This generally requires comparing alternative approaches to achieving the same outputs, to see whether the most efficient process has been adopted.
- Impact / Outcomes: The positive and negative changes produced by a development intervention, directly or indirectly, intended or unintended. This involves the main impacts and effects resulting from the activity on the local, social, economic, environmental and other development indicators. The examination should be concerned with both intended and unintended results and must also include the positive and negative impact of external factors, such as changes in terms of trade and financial conditions.
- Sustainability: Sustainability is concerned with measuring whether the benefits of an activity are likely to continue after donor funding has been withdrawn. Projects need to be environmentally as well as financially sustainable.
For this particular evaluation, we would like to focus on effectiveness, impact / outcomes and sustainability, with focus to a lesser extent on relevance and efficiency. Cross-cutting issues of participation, child protection and inclusion should also be considered.
4. Key Evaluation Questions (KEQs)
The main questions this evaluation seeks to answer are:
- To what extent were the project’s intended objectives achieved?
- Investigate achievement of overall outcomes, including changes in attitudes, behaviours, practices and relationships.
- Consider to what extent has the program been able to facilitate the community in the implementation of their Action Plans (CAPs)?
- Consider the technical capacity of the Community-Owned Resource Persons (CORPS) as community service providers. Are they operating as intended and are they contributing to changes in the communities?
- Consider also the CBOs and whether the organizational structure of the target groups forming CBOs was helpful, adequate, or added value. This might be looked at from different perspectives – beneficiaries, government, ADS-E and other stakeholders.
- What were the supportive factors and obstacles encountered during the implementation of the project which influenced the achievement or non-achievement of the objectives?
- What has changed for community members, especially the most marginalised / vulnerable, due to their involvement in the project?
- Were the beneficiaries satisfied with the quality and delivery of services? If not, in what way did the services not meet with beneficiary expectations and why?
- Did staff have the experience and expertise to carry-out the activities envisaged?
- Which strategies should be continued in future projects, and which should be adjusted and/or stopped?
Impact / Outcomes
- What real difference have the program components/sectors/activities made to the beneficiaries?
- What long-term changes have been produced in the community through the program?
- What were the direct and indirect, intended and unintended, positive and negative impacts of the program?
- Has the program empowered stakeholders to ensure access to better services in the community? Has the program empowered community to seek better services from stakeholders?
- To what extent have the benefits of the project continued after the project ended?
- What is the effectiveness of the mechanisms / approaches for sustainability adopted by the project?
- Consider particularly CBO development as a strategy for sustainability.
- What are the major factors that influenced the achievement or non-achievement of sustainable program outcomes?
- What was the level of community ownership during and after project implementation?
- What actions have been taken to ensure the ownership by the beneficiaries, local communities, local governments and other stakeholders of the program?
- To what extent have target community members been able to link with, and access resources from other agencies and actors in their areas?
- What lessons can be learned, and are there any recommendations to strengthen sustainability in future projects?
- Were project participants and stakeholders actively and meaningfully involved in project design, implementation, re-design and monitoring?
- How have children and youth been actively involved in the project?
- How can participation be strengthened in the design and implementation in future projects?
- Did this project reach, address and include the most vulnerable and marginalised in target communities (e.g. members of marginalised groups, women, girls, people with disability, people living with or affected by HIV/AIDS etc)?
- To what extent has the project focused on, addressed, and resulted in gains in gender equity and disability inclusion.
- How have community groups been strengthened and equipped to identify and address gender concerns and disability inclusion?
- How can inclusion be enhanced in the design and implementation of future projects?
- How effective was the project in improving welfare of children and their families in the project areas.
- To what extent has the project focused on, addressed, and resulted in gains in issues affecting children in the society.
- Has the project empowered agencies, systems, and communities to effectively prevent and mitigate the effects of child maltreatment, to protect children, and to strengthen families?
- How can child protection be enhanced in the design and implementation of future projects?
Other questions this evaluation seeks to answer are:
- How relevant was the project design to the priorities and needs of the target group?
- Were the activities and outputs of the programme consistent with the overall goal and the attainment of its objectives, as well as with the intended impacts and effects?
- How can relevance be ensured in the design and implementation of future projects?
- Were the most efficient processes adopted?
- Were activities cost-efficient?
- Were objectives achieved on time?
- Was the programme or project implemented in the most efficient way compared to available alternatives?
- Are there any recommendations about how to ensure efficiency in future projects?
5. Evaluation Approaches and Methods
Stakeholder participation is fundamental. The Consultant is expected to conduct a participatory evaluation providing for active and meaningful involvement by beneficiaries, investments partners, and other relevant parties.
The groups that should be involved in the evaluation include:
- Project Management Committees (PMCs)
- Self-help Group (SHGs)
- Community-Owned Resource Persons (CORPS)
- School Child clubs
- Public schools/teachers/school management committees
- Child Protection Committees (CPCs)
- Youth clubs
- Government Line Ministries and other Agencies operating in the area.
- DRR Committee
- Peace committees
6. Documents and Reports
Evaluation Work plan/Inception Report
The Consultant will prepare an evaluation work plan/inception report to operationalize and direct the evaluation. The work plan/inception report will describe how the evaluation will be carried out, bringing refinements, specificity and elaboration to the Terms of Reference. It will be approved by TAI’s International Programs Coordinator and ADS-Eastern’s Program Manager, and act as the agreement between parties for how the evaluation will be conducted.
The evaluation work plan/Inception report will address the following reporting elements:Overview of Investment
- Expectations of Evaluation (including restrictions/refinements to ToR)
- Roles and Responsibilities
- Evaluation Methodology
- Evaluation Framework
- Information Collection and Analysis
- Work Scheduling
The evaluation will include a site visit to Makueni County and Kitui Counties to consult with ADS-Eastern’s field personnel and project stakeholders; and to collect information in accordance with the requirements stipulated in the evaluation workplan. ADS-Eastern’s Program Manager is to be briefed on arrival and before departure from the field.
The Consultant will prepare an evaluation report that describes the evaluation and puts forward the Consultant’s findings, recommendations and lessons learned. The presentation of results is to be intrinsically linked to the evaluation issues, establishing a flow of logic development derived from the information collected.
All Key Evaluation Questions (KEQs) in this ToR must be addressed and answered by the Consultant to the extent the evaluation process has allowed them to do so, except where the Consultant has previously expressed constraints or an inability to address any particular KEQ. KEQs can be revised according to further discussion prior to signing the contract. Any other KEQ not addressed should be explained in the limitations section of the report.
The final report should be a maximum 50 pages, excluding annexes, and should be written in English. It should contain an executive summary of a maximum 2-3 pages. The report should follow the following format:
- Title page
- Short description of Consultant
- Acronym list
- Executive Summary
- Project description and overview
- Constraints / Limitations
- Findings (i.e answer to all the questions posed in the ToR)
The report may include quotes, photos, graphs, case studies etc.
The evaluation is planned from September to November/December 2018.
Draft Workplan/Inception Report
A draft workplan outlining the proposed methods to be used, target respondents, as well as method of engagement with project staff will be submitted within one (1) week of the signing of the contract.
Final Workplan/Inception Report
Within one (1) week of receiving TAI/ADS-Eastern’s comments on the draft workplan, the Consultant will produce a final evaluation workplan.
The field mission is expected to be not longer than two weeks in duration. This is subject to reviewing coverage.
Draft Evaluation Report
The Consultant will submit a draft evaluation report outlining findings and recommendations for review by TAI/ADS-Eastern within two (2) weeks of returning from mission.
Final Evaluation Report
Within 5 days of receiving TAI and ADS-Eastern’s comments on draft report, the Consultant will submit a final evaluation report, including an executive summary.
The final report should be submitted to TAI and ADS-Eastern by 1 December 2018.
A feedback session with staff to discuss and develop recommendations will occur with ADS-Eastern and staff.
8. Evaluation Team and Independence
The evaluation team will consist of at least 1 external Consultant and 1 translator. The Consultant may make recommendations regarding how many members the team requires.
ADS-Eastern staff members will also be attached to the evaluation team to observe the process and to support the Consultant where necessary, including in data collection.
To create safe spaces for community members to share openly and to avoid operator bias, an independent translator will be secured for this evaluation. The Consultant may secure an independent translator that speaks the community’s local languages, however, if this is not possible, ADS-Eastern will assist in engaging an independent translator (i.e. someone who has not been involved in the project previously in any form).
TAI’s International Programs Coordinator and ADS-Eastern’s Program Manager is responsible for:
- Overall responsibility and accountability for the evaluation.
- Guidance throughout all phases of execution.
- Approval of all deliverables.
The Consultant is responsible for:
- Guidance and directions to achieve the purpose and objectives of the evaluation.
- Conducting the evaluation.
- The day–to–day management of operations.
- Regular progress reporting to TAI’s International Programs Coordinator, particularly if any unforeseen issues arise.
- The development of results.
- Consolidation and presentation of evaluation findings and writing the final evaluation report.
- The production of deliverables in accordance with contractual requirements.
ADS-Eastern will be responsible for:
- Providing relevant documentation for data information collection.
- ADS-Eastern will arrange meetings with community people and stakeholders during field data collection.
- Providing other support to the evaluation team as required to meet key stakeholders during the entire evaluation period.
The Consultant will report to both TAI’s International Programs Coordinator and ADSE’s Executive Director.
Other country logistics required to carry out the evaluation needs to be included and identified in the evaluation work plan.
- Consultant Qualifications
The Consultant will need to:
- Be a reliable and effective project manager with extensive experience in conducting evaluations and a proven record in delivering professional results.
- Fluent in English and Kiswahili.
- Note: The local language spoken is Based in or experienced in the region.
- Follow and agree to abide by TAI and ADS-Eastern’s processes, rules and policies, including Child Protection Policy, Child Protection Code of Conduct, anti-corruption and fraud policy etc.
- Follow participatory, strengths-based approaches to the evaluation.
- Use a mixed methods approach to data collection.
- Focus on program learning and improvement when presenting findings and recommendations.
- Engage openly with the local community, listen to and synthesise varied perspectives.
- Ensure children, youth, women, girls and people with disability participate in the evaluation process.
- Have advanced skills and knowledge in monitoring, evaluation and learning methods and approaches; conducting community-based evaluations; and project sustainability mechanisms and processes.
The Consultant should have:
- Experience and skills working with long-term projects.
- Experience and skills working with local staff to develop and conduct evaluations.
- Cross-cultural sensitivity.
- Knowledge and experience in relevant subject matters, e.g. livelihoods, agriculture, WASH, health, leadership.
- Experience and skills in learning-oriented data processing, information analysis, and report writing.
- Experience and skills in participatory processes, rural and social development, and cross-cutting such as gender and disability inclusion.
- Commitment to accomplish work by given deadlines.
11. Ownership and Confidentiality
The report will become the property of ADS-Eastern and its key donor, Transform Aid International. The Consultant agrees that the information obtained remains confidential and any publication or citing needs prior written approval from ADS-Eastern and TAI.
12. Child Protection
TAI and ADS-Eastern are both dedicated to ensuring the safety of children involved in and connected with our projects. Both organisations consider child abuse to be unacceptable in all circumstances and are therefore committed to ensuring that all steps are taken to ensure the safety of children that we work with. For that reason, the Consultant and translator engaged by TAI and ADS-Eastern for this evaluation will be required to complete Child Protection documentation.
Costs to be covered by TAI/ADS-Eastern:
- All reasonable aspects of the evaluation according to the budget agreed on with the Consultant and outlined in the Terms of Engagement.
- In-country logistic related expenses.
- Expenses of ADS-Eastern staff involved in evaluation activities.
- Field visits, lodging, food and travel of external Consultant and independent translator.
- Consultancy fees.
- Translator fees.
Costs to be covered by Consultant and included in expression of interest:
- The consultancy fees.
- Stationaries if any required.
The budget should include agreed daily rate for evaluation team members.
The Consultant is to provide an invoice at the conclusion of the evaluation to cover accommodation, transport, meals and facilities for the evaluation team. If pre-payment is required, the Consultant is to provide an invoice with estimated costs. This amount will be reconciled with the overall budget at the conclusion of the evaluation with the final payment.
|Consultant Activities||Tentative timeline|
|Literature review||3 days|
|Workplan draft and final||1 day|
|Field visit for data collection||Maximum 10 days
(to be discussed further depending on coverage)
|Draft report preparation and sharing to TAI & ADS-Eastern||5 days|
|Feedback and finalise draft report||1 day|
|Revise report (as per comments) and provide final report||1 day|
|Presentation / findings workshop to ADS-Eastern||0.5 days|
The Consultant is to provide:
- Cover letter / Expression of Interest based on the requirements of this ToR.
- Curriculum Vitae with 3 references.
- Sample work of the Consultant – previous evaluation report (sensitive data may be redacted).
- Applications should be submitted by email by 15th Oct 2018.
- To submit applications, please email Tara Buddhipala at firstname.lastname@example.org and John Mutua at email@example.com.
- For more information, contact Tara Buddhipala at firstname.lastname@example.org