Introduction

A program is an organized set of activities, delivered by people with resources, intended to change something for a defined population. A school vaccination clinic, a provincial quit-smoking line and a community kitchen are all programs, and each rests on claims that someone has to check: that the program is needed, that it is delivered as intended, that it reaches the people it was designed for, that it changes their health or circumstances, and that its benefits justify its costs. Program evaluation checks such claims systematically and turns the evidence into judgements that people can act on.

This section defines evaluation, separates it from research, introduces the reasoning that turns evidence into a judgement of value, and sets out the main types of evaluation across a program’s life. Sections 2 to 4 then survey approaches to evaluation, the Canadian context, and the evaluation cycle that organizes the course.

1.1 The Running Case: The Cedar Valley Connector Program

Each lesson uses one invented program as its main worked example, so that concepts from different weeks can be seen working together on the same case.

The case raises most of the questions this course addresses. Managers want to know whether referred adults come to a first meeting. The steering committee wants to know whether people feel less lonely, and for whom. The First Nations partners want to know whether the program is culturally safe. The executive must decide whether the second wave proceeds, and the staggered launch may let an evaluator compare clinics with and without the program, a possibility that Lessons 6 and 7 examine.

1.2 Defining Evaluation

The most widely used definition of evaluation comes from the philosopher and evaluation theorist Michael Scriven, for whom evaluation is the systematic determination of the merit, worth or significance of something (Scriven, 1991; Davidson, 2005). The definition places judgement at the centre of the work. An evaluation that describes what a program did, without reaching any conclusion about how good, valuable or important it was, has stopped short of evaluation in Scriven’s sense. The three terms in the definition name different kinds of value.

Other definitions emphasize different features of the same activity. Carol Weiss (1998) defined evaluation as the systematic assessment of the operation or outcomes of a program, compared with explicit or implicit standards, as a means of contributing to its improvement. Rossi, Lipsey and Henry (2019) define it as the use of social research methods to investigate the effectiveness of programs in ways adapted to their political and organizational settings. The definitions share three elements: evaluation is systematic, it compares findings with a standard of what ought to be, and it is meant to inform decisions about a particular program.

Formative and summative purposes

Scriven (1967) also introduced the distinction between formative and summative evaluation. Formative evaluation is conducted to improve a program while it is being developed or delivered, and its main users are the people who run the program. Summative evaluation is conducted to reach an overall judgement about a program, usually to inform a decision about whether to continue, expand, change or end it, and its main users are funders and decision-makers. Robert Stake’s analogy, which Scriven (1991) reports, compares formative evaluation to the cook tasting the soup and summative evaluation to the guests tasting it. The same method can serve either purpose: a survey of Cedar Valley participants is formative if connectors use it to adjust their practice, and summative if the executive uses it to decide whether the second wave proceeds.

1.3 Evaluation and Research

Evaluation and research use the same methods: surveys, interviews, administrative data, experiments and statistical models. Students who come to evaluation from research training often treat the two as the same activity. Mathison (2008) argues that the two share methods and differ most clearly in purpose, and the difference in purpose gives each a different primary audience. Research aims to produce knowledge that holds beyond the setting in which it was produced, and its primary audience is other researchers. Evaluation aims to inform judgements and decisions about a particular program, and its primary audience is the people who make or are affected by those decisions.

The distinction is a matter of emphasis, and many studies sit between the two. An impact evaluation that compares first-wave and second-wave Cedar Valley clinics would inform the health authority’s decision and could also add to the international evidence on social prescribing. The difference still matters in practice: it affects who frames the questions, how quickly findings are needed, and, as Section 3 explains, whether research ethics board review is required.

1.4 Evaluative Reasoning: Criteria, Standards and Rubrics

If evaluation ends in a judgement, an evaluator needs a defensible way of getting from evidence to that judgement. Scriven described a general logic of evaluation, which Fournier (1995) set out as four steps. The evaluator establishes the criteria of merit, which are the dimensions on which the program will be judged. The evaluator then constructs standards, which specify how well the program must perform on each criterion to be judged excellent, adequate or poor. The evaluator measures performance and compares it with the standards. Finally, the evaluator synthesizes the results across criteria into an overall judgement of merit or worth.

Where criteria come from

A good evaluation states where its criteria came from. The program’s objectives are the obvious source, although Davidson (2005) notes that objectives can omit important effects or be set at levels that are easy to reach, and Scriven’s proposal of goal-free evaluation asked evaluators to judge a program’s actual effects, intended or unintended, against the needs of the people it serves. Other sources include assessed needs, ethical and legal requirements, professional standards of practice, what comparable programs achieve, and the values of interest holders, including participants and communities. For Cedar Valley, a criterion of cultural safety comes from the Indigenous health partnership, a criterion of reach from the concern that referrals be acted on, and a criterion of cost from the executive’s need to compare the program with other uses of the money.

Rubrics

An evaluative rubric sets out the criteria in rows and describes, for each criterion, what performance at each level looks like. Rubrics make the evaluator’s reasoning visible, and they allow interest holders to agree on standards before the evidence arrives, which protects the evaluation from the temptation to move the standards once results are known (Davidson, 2005; King et al., 2013). The rubric below is the one the Cedar Valley steering committee agreed for a six-month review of the first wave. Lesson 10 returns to rubrics and to the synthesis of mixed evidence in more depth.

The committee also agreed a minimum requirement for synthesis. The program could not be rated better than adequate overall if cultural safety was rated poor, whatever its performance on the other criteria. Such a requirement is sometimes called a bar or a hurdle, and it records the committee’s view that a program which harms some participants cannot be rescued by good results for others.

1.5 Types of Evaluation Across the Program Life Cycle

Programs pass through a life cycle. A need is identified, a program is designed, it is implemented and adjusted, it settles into stable delivery, and eventually it is reviewed, scaled up, changed or ended. Different evaluation questions make sense at different stages, and the most common types of evaluation can be placed on that cycle.

Rossi, Lipsey and Henry (2019) arrange these types as a hierarchy in which each level depends on those beneath it: an assessment of need, then of program design and theory, then of implementation, then of outcomes and impact, and finally of cost and efficiency. The ordering has a practical lesson. An impact evaluation of a program that was never delivered as designed tells decision-makers little, because a null result cannot distinguish a poor idea from a poorly implemented one. Section 2 returns to this point through Weiss’s distinction between implementation failure and theory failure.

Daniel Stufflebeam’s CIPP model divides the cycle in a similar way, organizing evaluation around a program’s context, input, process and product. Several neighbouring activities are often confused with evaluation.

An evaluation plan usually combines several of these types. A plan for Cedar Valley might combine a process evaluation of the first wave, an impact evaluation comparing first-wave and second-wave clinics, and an economic evaluation that draws on both. Which questions take priority depends partly on the evaluator’s approach, the subject of Section 2.

Introduction

Two competent evaluators given the same program can design very different evaluations. One may begin by asking the health authority’s executive what decision the evaluation must inform. Another may begin by mapping how the program is supposed to produce change. A third may begin by asking the communities served what a good program would mean to them. Each is following an evaluation approach, a set of prescriptions about whose questions an evaluation should answer, what counts as credible evidence, how judgements of value should be made, and what role the evaluator should play. Evaluation theorists use the word theory for these prescriptive models (Alkin & Christie, 2004). This section surveys the main approaches and organizes them on the evaluation theory tree.

2.1 The Evaluation Theory Tree

Marvin Alkin and Christina Christie (2004) drew the history of evaluation theory as a tree. Its roots are the two traditions from which evaluation grew: a demand for accountability, meaning that those who spend public or charitable money should show what it achieved, and systematic social inquiry, which supplied the methods for doing so. The trunk divides into three branches according to the main concern of each theorist. The methods branch, which grew from Donald Campbell’s work on experimental and quasi-experimental designs, is concerned above all with producing credible knowledge about program effects. The use branch is concerned with ensuring that evaluations are used by the people who make decisions. The valuing branch, which grows from Michael Scriven’s insistence that evaluation must reach judgements of value, is concerned with how those judgements are made and whose values count.

The tree has been revised several times. Later editions of Evaluation Roots moved theorists along and between branches, and the third edition (Alkin & Christie, 2023) categorizes approaches in place of individual theorists, placing culturally responsive, transformative and Indigenous approaches on the valuing branch. Mertens and Wilson (2012) proposed a fourth branch for approaches centred on social justice. The placements describe each approach’s primary emphasis. Every serious evaluation attends to methods, use and values to some degree, and the tree is most useful as a reminder of which of the three an approach is prepared to trade away when they conflict.

2.2 The Use Branch

The approaches on the use branch begin from the observation that many evaluations are commissioned, completed and then ignored. They respond by building the evaluation around the people expected to act on it.

Use-branch approaches draw attention to the different ways in which evaluations are used. Instrumental use occurs when findings directly inform a decision, such as changing the connector training. Conceptual use occurs when findings change how people think about a problem, for example by showing managers that transport is the main barrier to participation. Process use, a term Patton introduced, refers to the learning that occurs among people who take part in an evaluation, regardless of its findings. Symbolic use occurs when an evaluation is commissioned or cited to legitimize a decision already made. Lesson 10 examines the conditions under which evaluations are used.

2.3 The Methods Branch

The methods branch descends from Campbell’s work on experimental and quasi-experimental designs and from the view that an evaluation’s first duty is to produce credible evidence about whether a program causes the changes it claims. Lessons 6 to 8 of this course belong to this tradition. Two approaches on the branch focus on explaining how programs work.

Theory-driven evaluation

Huey-Tsyh Chen (1990, 2015) argued that evaluations which test only whether outcomes changed treat the program as a black box. Theory-driven evaluation makes the program’s theory explicit and tests its links: what the program does, what it is expected to change first, and how those early changes are expected to lead to the final outcomes. Chen separates the change model, which states the causal process the program relies on, from the action model, which states the arrangements needed to deliver it. Lesson 3 teaches both.

Program theory helps an evaluator interpret disappointing results. Weiss (1998) distinguished implementation failure, in which a program is not delivered as planned, from theory failure, in which the program is delivered as planned but the causal process it relies on does not occur. Suppose loneliness scores at Cedar Valley do not fall. If connectors held few meetings and made few linkages, the problem is implementation. If meetings and linkages happened as planned, but people who joined groups felt no less lonely, the problem lies in the theory, and the program itself needs rethinking.

Realist evaluation

Ray Pawson and Nick Tilley (1997) argued that programs do not work the same way for everyone, so the useful question is what works, for whom, in what circumstances, and why. Realist evaluation holds that programs produce outcomes by triggering mechanisms, which are the ways participants respond to the resources a program offers, and that mechanisms operate only in certain contexts. Its analytic unit is the context-mechanism-outcome configuration. Reporting standards for realist evaluations, known as RAMESES II, were published by Wong and colleagues (2016). This lesson places realist evaluation on the methods branch beside theory-driven evaluation because its main contribution is an account of causal explanation.

2.4 The Valuing Branch

The valuing branch starts from Scriven’s argument, introduced in Section 1, that evaluation must reach judgements of value and that the evaluator is responsible for making them defensible. Later theorists on the branch asked whose values should define merit. Robert Stake’s responsive evaluation organized evaluations around the concerns of the people involved in a program, and Ernest House and Jennifer Greene argued that evaluations should give fair representation to the interests of less powerful groups. Two families of approaches on this branch are central to health program evaluation in Canada.

Culturally responsive evaluation

Culturally responsive evaluation places the culture and context of the community served at the centre of every stage of an evaluation, from framing the questions to sharing the findings (Hood, Hopson & Kirkhart, 2015). Hood and his colleagues trace its roots to African American evaluators and scholars of education in the United States, whose work showed that evaluations designed without attention to culture could misread programs serving their communities. It asks evaluators to examine their own cultural position, to choose methods and measures that make sense to participants, and to involve community members in interpreting results. Kirkhart (1995) argued that culture is a matter of validity, since conclusions drawn through measures or relationships that participants do not recognize can be wrong. The American Evaluation Association’s Public Statement on Cultural Competence in Evaluation (2011) took a similar position. Mertens’s transformative approach extends this reasoning by orienting the whole evaluation toward human rights and social justice.

Indigenous evaluation

Indigenous evaluation is evaluation grounded in Indigenous knowledge systems, values and governance, and led by or conducted in partnership with Indigenous communities. Its principles include relationships built on respect and trust, accountability to the community, self-determination over questions, data and interpretation, and the recognition of Indigenous ways of knowing as legitimate evidence. Kirkness and Barnhardt’s (1991) four Rs of respect, relevance, reciprocity and responsibility, first written about First Nations students in higher education, are often cited as a starting point. LaFrance and Nichols (2009) developed an Indigenous Evaluation Framework with tribal colleges in the United States, and Chouinard and Cousins (2007) reviewed evaluation practice with Indigenous communities in Canada and elsewhere. Cram and Chouinard (2023) describe culturally responsive Indigenous evaluation as an approach in its own right on the valuing branch.

2.5 Choosing and Combining Approaches

Approaches are tools, and a single evaluation often combines several. The choice depends on the purpose of the evaluation, the stage of the program, the decisions to be made, the relationships among those involved, and the resources available. The table compares the approaches in this section.

The approaches surveyed here have grown up in many countries, but evaluation in Canada also takes shape within particular policies, professional bodies and ethical rules. Section 3 turns to that context.

Introduction

Evaluation in Canada has a particular history. The federal government has required departments to evaluate their programs since the 1970s, Canada has a national evaluation society with its own competencies and a professional designation, and Canadian research ethics policy draws an explicit line between program evaluation and research. A graduate student planning an evaluation of a British Columbia health program will meet all three. This section covers federal evaluation policy, the evaluation profession, the standards by which evaluations are judged, and the ethics of evaluation.

3.1 Federal Evaluation Policy and the Policy on Results

The Treasury Board issued its first formal program evaluation policy in 1977, and an Office of the Comptroller General created in 1978 housed a program evaluation branch. Successive policies followed, including the 2009 Policy on Evaluation. The current instrument is the Policy on Results, which took effect on July 1, 2016 and replaced the Policy on Evaluation and its related instruments. The policy is supported by a Directive on Results, which sets out the detailed requirements.

The Policy on Results joins performance measurement and evaluation in one framework. Its aim is to improve the achievement of results across government and to improve understanding of the results government seeks, the results it achieves and the resources it uses to achieve them. Each department organizes its work in a Departmental Results Framework, which sets out its core responsibilities, the departmental results it intends to achieve and the indicators that will track them, and in a Program Inventory, which lists its programs. Each program has a Performance Information Profile that states its expected results, indicators and data sources. Deputy heads designate a Head of Performance Measurement and a Head of Evaluation and establish a Performance Measurement and Evaluation Committee. Each year they approve a five-year departmental evaluation plan.

The policy gives departments more discretion than earlier policies over which programs to evaluate, so that evaluation can follow risk, need and priority. One requirement remains fixed. Under section 42.1 of the Financial Administration Act, ongoing programs of grants and contributions must be reviewed for relevance and effectiveness every five years, and the policy requires departments to evaluate such programs with five-year average actual expenditures of $5 million or more per year at least once in that period. Federal evaluations typically address questions of relevance, effectiveness and efficiency, and departments publish their evaluation reports with management responses. The evaluation reports of the Public Health Agency of Canada and Health Canada are useful models of the format.

3.2 The Evaluation Profession in Canada

The Canadian Evaluation Society (CES) was formed in 1981. It is a bilingual national association with regional chapters, and it publishes the Canadian Journal of Program Evaluation. Evaluation is not a regulated profession in Canada, so anyone may call themselves an evaluator. The CES has addressed this through a statement of competencies and a professional designation.

Competencies for Canadian evaluation practice

The CES competencies were developed as part of its credentialing program and revised in 2018. They are grouped into five domains, which together describe what a competent evaluator knows and does.

The Credentialed Evaluator designation

The CES introduced its Professional Designations Program in 2009, and through it awards the Credentialed Evaluator (CE) designation. Applicants must hold a graduate degree, or a graduate certificate or diploma in program evaluation, must have the equivalent of two years of full-time evaluation-related work experience within the previous ten years, and must document, with examples from their work, how they demonstrate the competencies in each domain. Credentialed evaluators maintain the designation through continuing professional learning. The designation is voluntary. Some employers and funders ask for it, and it gives graduate students a clear map of the competencies they need to build.

Ethical guidance for evaluators

The CES publishes ethical guidelines for its members, and many Canadian evaluators also use the American Evaluation Association’s Guiding Principles for Evaluators, revised in 2018, which set out five principles: systematic inquiry, competence, integrity, respect for people, and common good and equity. These guidelines apply whether or not a project falls under research ethics review, a point that becomes important in Section 3.4.

3.3 The Program Evaluation Standards

The Program Evaluation Standards are the most widely used statement of what makes an evaluation good. They are developed by the Joint Committee on Standards for Educational Evaluation, a body of professional associations from the United States and Canada, and the current third edition was written by Yarbrough, Shulha, Hopson and Caruthers (2011). Lyn Shulha, one of its authors, worked at Queen’s University. The edition contains 30 standards organized under five attributes. The first four attributes appeared in earlier editions, and the 1999 CDC framework presented in Section 4 adopted them. The third edition added evaluation accountability.

The standards guide planning as well as review. During planning, an evaluator can use them as a checklist of risks: a design that answers the executive’s question too late fails on utility, a survey that doubles the length of each connector meeting fails on feasibility, and an agreement that leaves data ownership unstated fails on propriety. After an evaluation, the standards support metaevaluation, the evaluation of an evaluation. The attributes can conflict. A larger sample improves accuracy and reduces feasibility, and full disclosure of findings for small groups improves transparency while risking the identification of participants. The standards leave such conflicts to the evaluator’s judgement, and the evaluator’s task is to make the trade-off explicit.

3.4 The Ethics of Evaluation

When evaluation requires research ethics review

The Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans (TCPS 2), whose current edition dates from 2022, is the joint research ethics policy of the Canadian Institutes of Health Research, the Natural Sciences and Engineering Research Council and the Social Sciences and Humanities Research Council. Institutions eligible for funding from these agencies, which include universities and many hospitals and health authorities, must apply it to research conducted under their auspices. Article 2.1 defines research as an undertaking intended to extend knowledge through a disciplined inquiry or systematic investigation. Article 2.5 states that quality assurance and quality improvement studies, program evaluation activities, and performance reviews do not constitute research under the policy, and do not fall within the scope of research ethics board (REB) review, when they are “used exclusively for assessment, management or improvement purposes.”

Three further points in the policy shape how Article 2.5 is applied. If data collected for evaluation are later proposed for use in research, that secondary use may require REB review. The application of Article 2.1 states that the choice of methodology and the intent or ability to publish findings are not factors that determine whether an activity is research requiring review, so neither a randomized design nor a plan to publish settles the question by itself. Finally, the policy notes that activities outside REB review can still raise ethical issues that would benefit from careful consideration by an individual or body able to give independent guidance. An evaluation exempt from REB review is still subject to ethical scrutiny.

When research involves First Nations, Inuit or Métis Peoples, Chapter 9 of TCPS 2 applies, and researchers must seek engagement with the relevant community where the research is likely to affect its welfare. Evaluations that fall under Article 2.5 are outside the formal scope of that chapter, but its principles of community engagement, respect for Indigenous governance and collaboration remain the appropriate standard for any evaluation involving Indigenous communities.

Screening evaluation and quality improvement projects

Because evaluation projects outside REB review still need ethical scrutiny, several Canadian organizations use structured screening tools. The best known is the ARECCI (A pRoject Ethics Community Consensus Initiative) set of decision support tools, developed in Alberta and offered by Alberta Innovates. The ARECCI Ethics Screening Tool takes a project lead through four steps: preliminary questions, the project’s primary purpose, a set of risk filters, and the screening results. It produces a summary score that indicates the level of risk to participants and suggests the appropriate type of review, which may be a research ethics board, an ARECCI second opinion review, or an internal organizational review. A companion ARECCI Ethics Guideline Tool asks project teams to work through six ethics considerations for their project’s objectives, methods and outcomes. In British Columbia, Interior Health’s project ethics policy asks quality improvement and evaluation teams to complete the ARECCI guidelines while developing a project and to use the screening tool to determine its primary purpose and level of risk, with the level of review scaled to that risk.

Ethical issues in evaluation practice

The policies, competencies, standards and ethical rules in this section describe the conditions under which an evaluation is carried out. Section 4 turns to the sequence of tasks that make up an evaluation and to the evaluation plan that sets them out.

Introduction

Evaluators need a map of the tasks an evaluation involves and the order in which they are usually done. This course uses the framework published by the United States Centers for Disease Control and Prevention (CDC) for that purpose. The framework is a United States federal document, but its steps are general enough to organize evaluation work in Canadian settings, and it is widely taught in public health. This section presents the original 1999 framework and its 2024 update, applies the cycle to the Cedar Valley Connector program, and shows how to write the program description on which an evaluation plan rests.

4.1 The CDC Framework for Program Evaluation (1999)

The Framework for Program Evaluation in Public Health was published in 1999 in the CDC’s Morbidity and Mortality Weekly Report series of recommendations and reports (Centers for Disease Control and Prevention, 1999). A CDC working group wrote it to summarize the essential elements of program evaluation in a form that public health practitioners could use. It describes six steps arranged in a cycle and four groups of standards for judging the quality of an evaluation.

The four standards, utility, feasibility, propriety and accuracy, were taken from the second edition of the Joint Committee’s Program Evaluation Standards, introduced in Section 3. The steps are presented as a cycle because each depends on the ones before it and because evaluation of a program usually continues through several rounds. A program description written in Step 2 shapes the questions chosen in Step 3, and the findings shared in Step 6 feed the next round of engagement and description. In practice, evaluators move back and forth among the steps.

4.2 The 2024 Update

In 2024 the CDC published the CDC Program Evaluation Framework, 2024 (Kidder et al., 2024), also in its recommendations and reports series. The update keeps the six-step structure and the practical, nonprescriptive character of the original, and it makes four main changes.

First, the framework begins with a new step, Assess context, which asks evaluators to consider four factors before planning: readiness for evaluation, the interest holders involved, the place in which the program operates, and the evaluation capacity available. Second, several steps were renamed and revised. Step 2 remains Describe the program, covering need, inputs, activities, outcomes, contextual factors and stage of development, often summarized in a logic model. Step 3 becomes Focus the evaluation questions and design, and its products are a purpose statement, a statement of the type of evaluation, a list of intended users and uses, the evaluation questions and a description of the overall design. Step 4 remains Gather credible evidence. Step 5 becomes Generate and support conclusions. Step 6 becomes Act on findings, whose key elements are planning, preparing findings for use, and facilitating the move from insights to action.

Third, the update adds three cross-cutting actions that apply at every step: engage collaboratively, advance equity, and learn from and use insights. The authors describe these actions as the largest change from the 1999 framework. Engagement, which was the first step in 1999, now runs through the whole cycle. Fourth, the four standards are replaced by the five federal evaluation standards adopted in the United States under the Foundations for Evidence-Based Policymaking Act of 2018 and set out by the Office of Management and Budget: relevance and utility, rigour, independence and objectivity, transparency, and ethics. The update also replaces an older term, built on the word stake, with interest holder, explaining that the older term can imply a power differential and has a violent connotation for some American Indian and Alaska Native tribes and tribal members. This course uses the term interest holder for similar reasons, and it uses the newer term when describing the 1999 framework and other sources that use the older one.

For Canadian evaluators, the CDC framework is a useful organizing cycle, while the Program Evaluation Standards, the CES competencies and TCPS 2 from Section 3 remain the main Canadian references for quality and ethics. The two sets of standards are compatible, and an evaluation plan for a British Columbia program can cite both.

4.3 Applying the Cycle to the Cedar Valley Connector Program

Each step of the cycle corresponds to work that the Cedar Valley evaluation team must do, and to one or more lessons of this course. The tabs use the 2024 step labels.

The cross-cutting actions apply to every tab. Engaging collaboratively means that the steering committee and the First Nations partners shape each step. Advancing equity means asking at each step whether the program and the evaluation serve older adults who face the greatest barriers, such as those without transportation or those whose first language is neither English nor French. Learning from and using insights means that the six-month review feeds back into program design.

4.4 From the Cycle to an Evaluation Plan

An evaluation plan sets out, before any data are collected, how an evaluation will be carried out. Its parts follow the cycle in the order an evaluator works, and a plan prepared for a health authority usually ends in a proposal with costing and an executive summary written for decision-makers. The table shows where this course teaches each part, using the Cedar Valley Connector program as the worked example throughout.

4.5 Writing a Program Description

A program description is the shared account of what a program is, on which every later part of an evaluation depends. If interest holders disagree about what the program does or is meant to achieve, the evaluation questions will be contested and the findings disputed. A good description is specific, using numbers, names of activities and eligibility rules. It is written in a neutral descriptive voice and separates what the program intends from what it has been shown to achieve. It covers six elements: purpose, population, activities, resources, setting and history. These map closely onto the elements of the CDC’s second step.

Bringing It All Together

This lesson set out the foundations on which the rest of the course builds. Evaluation is the systematic determination of a program’s merit, worth or significance, and it differs from research mainly in its purpose and audience. Its reasoning moves from criteria to standards to evidence to a synthesized judgement, and a rubric agreed in advance makes that reasoning visible. The types of evaluation, from needs assessment to economic evaluation, answer different questions at different stages of a program’s life, and an impact claim requires a credible counterfactual.

Evaluators approach this work from different positions. The evaluation theory tree organizes approaches by whether they emphasize credible methods, the use of findings, or the values that define merit, and real evaluations often combine approaches. In Canada, evaluation takes place within the Treasury Board Policy on Results, the competencies and designation of the Canadian Evaluation Society, the Program Evaluation Standards, and the ethics rules of TCPS 2, including Article 2.5 and the screening tools that support it.

The CDC framework, in its 1999 and 2024 versions, organizes the evaluator’s tasks into a cycle, and the 2024 cross-cutting actions of engaging collaboratively, advancing equity, and learning from and using insights apply throughout. The fictional Cedar Valley Connector program, introduced in this lesson, will carry these ideas through every lesson that follows, beginning with the program description in Section 4.5.

The final reflection asks you to apply ideas from all four sections to a manager’s plan for evaluating the Cedar Valley Connector program.

Final Knowledge Assessment

This glossary defines the terms, frameworks and people introduced in Lesson 1, and you can search it at any point in the lesson.