1.1 What Program Theory Is

Every health program carries an argument about how change happens. The fictional Cedar Valley Connector program, which runs through this course, pays seven community connectors to meet older adults who screen as lonely and to link them with groups, volunteer roles, transportation help and services. In doing so, the Cedar Valley Health Authority acts on several beliefs: that loneliness among older adults partly reflects too few opportunities for social contact, that a trusted person can help someone take up opportunities they would not take up alone, that taking part in groups produces relationships people value, and that less loneliness will in time mean better health and fewer emergency department visits. These beliefs may be well supported, partly supported or mistaken, and they are rarely written down in one place. Evaluators use the term program theory for this argument once it has been made explicit.

Leonard Bickman (1987) defined program theory as the construction of a plausible and sensible model of how a program is supposed to work. Rossi, Lipsey and Henry (2019) describe it as the set of assumptions about how a program's activities relate to the social benefits it is expected to produce, together with the strategy and tactics the program has adopted to deliver those activities. Both definitions contain two parts. The first is a causal account, which explains why the program's activities should produce the intended changes. The second is an operational account, which explains what has to be organized, by whom and for whom, so that the activities actually take place. Section 1.2 shows that Huey-Tsyh Chen built his framework around exactly this division.

Three meanings of "theory"

Students meet the word "theory" in several senses in evaluation, and confusing them causes avoidable errors. Program theory is specific to one program or one type of program: it is the account of how the Cedar Valley Connector program is meant to reduce loneliness. Social science theory is a general explanation of a phenomenon that holds across many settings, such as the work of Louise Hawkley and John Cacioppo (2010) on how loneliness shapes attention, expectations and behaviour in social situations. A good program theory often draws on social science theory, and a program theory that contradicts well-established social science theory deserves scrutiny. Evaluation theory is a theory about how to evaluate, such as the approaches placed on Alkin and Christie's evaluation theory tree in Lesson 1. This lesson is concerned with program theory, and it uses social science theory as one of its sources.

Chen (1990) originally distinguished normative theory, which describes what the program should be, from causative theory, which describes how the program is expected to work. In his later work (Chen, 2005, 2015) these became the action model and the change model, the terms used in this course.

Espoused theory and theory-in-use

Chris Argyris and Donald Schön (1974) distinguished the espoused theory that people give when asked to explain their actions from the theory-in-use that can be inferred from what they actually do. The Cedar Valley program describes its practice as person-directed, with connectors finding resources that match each participant's interests. An evaluator who observes meetings might find that some connectors steer participants toward the two or three groups they know best, because those referrals are quick and reliable. The espoused theory holds that a match with a person's interests sustains participation; the theory-in-use holds that familiar, reliable groups do. The two theories predict different patterns in the data, and an evaluation that measures only the espoused theory will misread what the program is doing.

1.2 Chen's Change Model and Action Model

Huey-Tsyh Chen's conceptual framework of program theory (Chen, 2005, 2015) is a widely used way of dividing a program's theory into parts that an evaluator can examine separately. The framework has two components. The change model states the causal process the program relies on. The action model states the arrangements the program must make so that the causal process can be set in motion.

The change model

The change model has three elements. The first is the goals and outcomes, the changes the program ultimately seeks. The second is the determinants, the factors the program attempts to change because it believes they lead to the outcomes. Determinants are sometimes called intervening variables or mediators. The third is the intervention or treatment, the set of program activities that act on the determinants. For the Cedar Valley program, the goals are lower loneliness, greater social participation and better self-rated health. The determinants are the participant's opportunities for social contact, their confidence about joining a group, their practical access to activities (in particular transportation), and the quality of the new relationships they form. The intervention is the combination of connector meetings, a co-developed connection plan, linkage and accompaniment, transportation help and small grants to community partners.

Naming the determinants is the step that most program documents omit, and it matters most to an evaluator. If the program believes that confidence is the main determinant, the evaluation should measure confidence before and after the meetings; if transportation is the main barrier, it should record whether transport was arranged and used. Without named determinants, the evaluator can measure only the final outcomes, which is the black-box situation described in Section 1.3.

The action model

The action model describes what must be organized so that the intervention reaches the intended people in the intended form. Chen identifies six elements: the implementing organization, the program implementers who deliver the intervention, the associate organizations and community partners whose cooperation the program needs, the ecological context that supports or hinders delivery, the intervention and service delivery protocols, and the target population together with the procedures for reaching and retaining it.

The table applies the action model to Cedar Valley. Each question it raises concerns implementation, which is why process evaluation (Lesson 5) is organized largely around the action model.

Chen's framework makes a practical point about program design. A sound change model with a weak action model produces a good idea poorly delivered, while a strong action model with a weak change model produces a well-run program that does not change the outcome. The action model is evaluated mostly with process data, such as referral counts, meeting logs and staff interviews, and the change model mostly with data on the determinants and outcomes.

Three vocabularies for the same distinction

Other authors draw the same line with different terms, and the tabs set three common vocabularies side by side.

1.3 Why Evaluations Need Program Theory

Many outcome evaluations compare outcomes between people who did and did not receive a program. Such a design can show whether outcomes changed, but on its own it treats the program as a black box, and nothing is learned about what happened in between. Carol Weiss argued throughout her career that this kind of evaluation leaves decision-makers unable to act on its findings (Weiss, 1995, 1997, 1998). Her argument rests on a distinction that goes back to Edward Suchman (1967), between a program that fails because it was never properly delivered and a program that fails because the causal process it relies on does not work.

Implementation failure and theory failure

Implementation failure occurs when the program is not delivered as planned, so the causal process is never set in motion. Theory failure occurs when the program is delivered as planned, but the expected causal process does not occur, or it occurs without producing the outcome. The two failures call for different responses. Implementation failure calls for better delivery: more staff, better training, clearer protocols or stronger partnerships. Theory failure calls for a different program, because delivering the same activities more faithfully will not help. An evaluation that measures only final outcomes cannot tell the two apart, because both produce the same result in the outcome data.

Weiss made the further point that positive findings are equally hard to use without program theory. Suppose the Cedar Valley evaluation finds that loneliness fell more among participants than among comparable older adults in second-wave clinics. The health authority must then decide what to keep when the program expands, whether the transport fund is worth its $40,000, and whether the partner grants could be cut. An evaluation that tracked only the final outcome cannot say which components carried the effect. A theory-based evaluation measures the intermediate links and so can show, for example, whether people who were accompanied to a first activity were more likely to keep attending than people who received only information.

What a theory-based evaluation adds

Weiss (1995, 1997) set out several benefits of evaluations that trace a program's theory. They locate the point at which a chain of expected changes breaks, provide early evidence from short-term links, help interpret mixed results, and support generalization by showing what must be present for the program to work elsewhere, such as in the second-wave clinics. They also strengthen causal claims when randomization is not possible, because a program that produces each predicted intermediate change in the predicted order is a more plausible cause of the final outcome. Weiss (1995) made this argument for comprehensive community initiatives, and her paper is widely credited with bringing the term "theory of change" into evaluation. Section 3 develops theories of change, and Lessons 6 to 8 return to stronger causal designs.

1.4 Making Program Theory Explicit

Program theory is seldom found ready-made. The evaluator assembles it from program documents (for Cedar Valley, the first-wave business case, the report of the eighteen-month pilot and the connector training manual), from interviews and workshops with designers, managers, frontline staff and participants, from observation of practice, which reveals the theory-in-use, and from research and social science theory, which indicate whether the program's assumptions are supported elsewhere.

A deductive approach starts from research and formal theory, an inductive approach builds the theory from observing the program, and a participatory approach develops it with the people who run and use the program. Most practical work combines all three. At Cedar Valley, the evaluation team would draft a theory from the pilot report and the literature on social prescribing, test it in a workshop with connectors and the four older adults with lived experience on the steering committee, and then check the revised version against what connectors do in their meetings.

Whose theory counts is a substantive question. The First Nations partners may understand connection in relational terms that include family, community, land and culture, and the land-based pathway they are co-designing rests on that understanding. Lesson 4 discusses the Indigenous evaluation principles and governance arrangements under which partners shape how their programs are described and evaluated.

In Canada, the Treasury Board of Canada Secretariat has published guidance for federal evaluators on theory-based approaches to evaluation, and federal program evaluations under the Policy on Results (Lesson 1) commonly begin with a logic model or theory of change, the subject of Sections 2 and 3.

Limits of theory-driven evaluation

Theory-driven evaluation has well-documented limits. A systematic review by Coryn and colleagues (2011) found wide variation in how closely evaluations described as theory-driven followed the approach, with many using a theory to describe the program without using it to design data collection or analysis.

2.1 What a Logic Model Is

A logic model is a diagram, usually fitting on a single page, that shows the sequence from the resources a program uses, through the activities it carries out and the products of those activities, to the changes it expects to produce. The W. K. Kellogg Foundation's Logic Model Development Guide (2004) and the University of Wisconsin-Extension's training materials (Taylor-Powell and Henert, 2008) established the format most evaluators now use. McLaughlin and Jordan (1999) described the logic model as a way of telling a program's performance story: a short, structured account of what the program does, for whom, and with what intended results.

A logic model is a summary of a program theory. It is strong on sequence and weak on explanation. It shows that connector meetings are expected to lead to participation and that participation is expected to lead to lower loneliness, but it does not say why, under what conditions, or for whom. The theory of change in Section 3 supplies that explanation. In practice, evaluators often draft the logic model first, because its familiar structure helps interest holders agree on what the program does, and then develop the theory of change to explain the links that matter most.

Logic models serve several purposes. In planning, they show whether the planned activities are plausibly sufficient for the intended outcomes and whether the resources match the activities. In evaluation, they show what can be measured at each stage, so that evaluation questions (Lesson 4) and indicators (Lesson 5) can be attached to specific boxes and arrows. In communication, they give funders, staff, partners and participants a shared picture of the program. In management and accountability, they underpin performance measurement frameworks in health authorities and federal departments.

2.2 The Components of a Logic Model

The standard logic model has four main components, with outcomes divided into three time frames, and two supporting elements. The University of Wisconsin-Extension model groups activities and participation together as outputs, while the Kellogg guide lists activities and outputs separately and calls long-term changes at the community or system level "impact". This course uses the six-column version shown in the table, which keeps activities and outputs apart because the distinction matters for measurement.

Separating outputs from outcomes

The most important distinction in a logic model is the one between outputs and outcomes. An output describes what the program delivered; an outcome describes a change in someone or something as a result. Outputs are counted in units of service (meetings held, plans written, trips funded, people reached), and the program can increase them by working harder or spending more. Outcomes depend on how participants and partners respond, and the program can influence them without controlling them. A useful test is to ask whether the program could produce the result through its own effort alone. If it could, the result is an output. The number of older adults who attend a first meeting is an output, because it counts people reached by an activity. Whether those people later attend a community group they chose is an outcome, because it depends on their own decisions, on the group and on transport.

Outcome statements are clearest when they name who changes, what changes, in which direction and by when. "Participants report lower loneliness on the three-item UCLA Loneliness Scale at twelve weeks" is a better outcome statement than "reduced isolation", because it names the population, the measure and the time point. The time frames themselves are relative to the program. A twelve-week program can reasonably treat changes at twelve weeks as short-term, while a five-year community initiative might treat changes in the first year as short-term. The logic model should state the time frames it uses.

Reading a logic model as a chain of if-then statements

The Wisconsin model adds a further way of reading the outcome columns. Short-term outcomes are usually changes in learning (awareness, knowledge, attitudes, skills, confidence or motivation), intermediate outcomes are changes in action (behaviour, practice or decisions), and long-term outcomes are changes in conditions (health, social or economic circumstances). This sequence is a useful check: an outcome that appears early in the model but describes a change in health status, or that appears late but describes a change in knowledge, may be misplaced.

2.3 The Cedar Valley Connector Logic Model

Figure 2.1 shows a full logic model for the first wave of the Cedar Valley Connector program. It uses the facts established in Lesson 1, and it is drawn from top to bottom so that each component has room for specific statements. Most published logic models run from left to right, and either direction works as long as the components appear in order and the statements are specific.

Several features of the model reflect deliberate choices. The outputs include the actual counts from the first six months (312 referrals and 241 first meetings), which makes clear that outputs are counts of delivery and reach. The ratio between them is already informative: 241 of 312 referred older adults, or 77.2 percent, attended a first meeting, and Lesson 5 shows how such ratios become process indicators. The short-term outcomes include one change in a system as well as two changes in participants, because the partner grants are meant to change how community groups receive newcomers. Lower loneliness appears as an intermediate outcome, since the program expects it to follow from sustained participation over several months. Emergency department visits and primary care visits appear only as long-term outcomes. Lesson 1 noted that the planners' expectations about health service use have not been tested, and placing them at the end of the model, with intermediate steps in between, keeps the model honest about how far they are from the program's activities.

The model also has limits that a reader should notice. Its arrows run between rows, so it does not show which activities are expected to produce which outcomes; the transport fund, for example, is meant to act mainly on attendance, while the partner grants act on how groups receive newcomers. The assumptions are listed but not attached to particular links. The land-based connection pathway appears only as a co-design activity, because its own theory is still being developed with the First Nations partner. The theory of change in Section 3 addresses the first two limits, and a nested sub-model would address the third once the pathway is designed.

2.4 Formats: Linear, Nested and Outcome-Chain Models

Logic models come in several formats. The choice depends on the program's structure, the audience and the evaluation's purpose. Figure 2.2 sketches the three formats this course uses.

The three formats are complementary. A common practice is to present a linear model in the main body of a report, a nested set of models in an appendix for program managers, and an outcome chain for the links the evaluation tests. Programs with complex features, such as outcomes that emerge from interactions among many organizations, may need models that are revised as the program develops; Rogers (2008) suggested drawing such models with explicit uncertainty and revisiting them at agreed intervals.

2.5 Common Errors in Logic Models

Logic models are easy to draw and easy to draw badly. The errors below appear often in program documents and in student work, and each has a straightforward correction.

The draft contains at least five errors. "Home visits" is an activity listed among the inputs. "Fewer falls" is an outcome listed among the outputs. "400 home visits per year" is an output listed as a short-term outcome. "Improved quality of life in the province" is a population outcome beyond the reach of two paramedics, and it is vague. The model has no intermediate outcomes, so it leaps from visits to quality of life, and it states no assumptions, such as that older adults will act on medication advice or that home care has capacity to accept referrals. A corrected model would move each statement to its proper column, add short-term outcomes such as "medication problems identified and resolved" and intermediate outcomes such as "fewer falls and fewer readmissions among visited patients within six months", restrict the long-term outcome to the program's patients, and list its assumptions.

2.6 Interactive: Build the Cedar Valley Logic Model

The builder below contains fifteen statements about the Cedar Valley program. Sort each one into the correct logic model column, then check your model. Feedback explains each misplaced statement. The statements differ from those in Figure 2.1, so the exercise tests the distinctions themselves.

2.7 From Logic Model to Evaluation

A completed logic model tells the evaluator where to look. Each output suggests a process indicator, such as the proportion of referred adults who attend a first meeting or the mean number of meetings per participant. Each outcome suggests an outcome indicator, such as the proportion of participants still attending a chosen group at six months or the mean change in the loneliness score. Each assumption suggests a question the evaluation should answer, and each external factor suggests a rival explanation the evaluation should rule out. Lesson 4 turns these into prioritized evaluation questions with the program's primary intended users, and Lesson 5 specifies the indicators, data sources and targets.

The logic model also guides the scope of the evaluation. A program in its first year, like the Cedar Valley first wave, has data on inputs, activities, outputs and some short-term outcomes, and only preliminary data on intermediate outcomes. A sensible first-year evaluation concentrates on the upper part of the model and on the earliest outcome links, while the design for estimating effects on loneliness and health service use is prepared for the second wave (Lessons 6 to 8). The logic model makes this sequencing visible to the health authority, which reduces the risk that a premature outcome evaluation is used to judge a program that is still being established.

3.1 From Logic Model to Theory of Change

The term "theory of change" entered evaluation through the work of the Aspen Institute Roundtable on Comprehensive Community Initiatives in the 1990s. Carol Weiss (1995) argued that community initiatives spanning housing, employment, health and education could not be evaluated well with designs borrowed from controlled trials, and that evaluators should ask the initiatives to spell out how and why their activities were expected to lead to their long-term goals. James Connell and Anne Kubisch (1998) developed the idea into a method and proposed that a good theory of change should be plausible, doable and testable: plausible in that the links make sense to the people involved and are consistent with evidence, doable in that the program has the resources to carry it out, and testable in that its links can be measured. Andrea Anderson's (2005) practical guide set out the backward-mapping method that most practitioners now use.

Theories of change spread to international development (Vogel, 2012) and to health research, where De Silva and colleagues (2014) showed how they support the development and evaluation of complex interventions and Breuer and colleagues (2016) reviewed their growing use in public health. The current Medical Research Council and National Institute for Health and Care Research framework for complex interventions (Skivington et al., 2021) treats an explicit program theory as a core element of intervention development and evaluation.

How a theory of change differs from a logic model

Some organizations use the two terms interchangeably. This course follows the distinction in the table, which reflects common practice (Funnell and Rogers, 2011).

3.2 Backward Mapping

Backward mapping builds a theory of change by starting at the end. The planners first agree on the long-term outcome, then ask what must be in place for that outcome to occur, then ask the same question of each answer, and continue until they reach conditions that the program's interventions can produce directly. Each condition identified in this way is a precondition: an outcome that must be achieved before the outcome above it can be achieved. Anderson (2005) describes the method in the following sequence of steps, which this course adopts.

The first step is to identify the long-term outcome. It should be specific enough to measure, important to the people the program serves, and realistic for the program's scale. The second step is to map the preconditions backward. For each outcome, the planners ask what has to be true immediately before it, and they test each proposed precondition by asking whether the higher outcome could occur without it. If it could, the precondition is not necessary and may not belong on the map. When the set of preconditions under an outcome is complete, the planners ask whether those preconditions, together with the stated assumptions, would be sufficient. The third step is to identify the interventions: the program activities that produce the earliest preconditions. Gaps at this stage reveal preconditions that no activity addresses, which point to missing activities or partners. The fourth step is to state the assumptions and rationales for each link. The fifth step is to specify indicators for each precondition, naming the population, the change expected and the time frame; Lesson 5 develops this step. The sixth step is to write a narrative that explains the pathway in prose.

After the map is drawn backward, it is read forward as a test, with each link read as a sentence of the form "this happens, so that that happens". If a sentence sounds implausible, or an obvious step is missing, the map needs revision. Skipping this forward reading produces the same long leaps that Section 2 identified in logic models.

3.3 Assumptions and Rationales

Two related terms describe the reasoning behind each link. A rationale explains why one outcome is expected to lead to the next, by citing research evidence, practice experience, pilot data or formal theory. An assumption is a condition that must hold for the link to work, but that the program does not control or has not tested. Some writers, including Anderson (2005), use "assumptions" to cover both. The practical point is that each link on the map should carry a short statement of why the planners expect it to hold and what would have to be true for it to fail.

Assumptions take several forms. Some concern the causal link itself, for example that new contacts reduce the kind of loneliness that participants experience. Some concern the context, for example that suitable groups exist within reach of rural participants. Some concern participants, for example that older adults referred by a clinician will be willing to meet a connector. Some concern implementation, for example that clinicians will screen consistently during busy appointments. Assumptions can be rated on two dimensions: how important the assumption is to the theory, and how strong the evidence for it is. Assumptions that are important and weakly supported should receive the most attention in the evaluation.

3.4 The Cedar Valley Theory of Change

Figure 3.1 shows a theory of change for the first wave of the Cedar Valley Connector program, built by backward mapping. It uses the same program facts as the logic model in Figure 2.1, but it is organized as a pathway of preconditions, and it attaches assumptions A1 to A5 from the table above to specific links.

The map was built in the order backward mapping prescribes. The planners began with the long-term outcome: better self-rated health and fewer emergency department and social-reason primary care visits among older adults in the region's participating clinics. They asked what must come first and identified the intermediate outcome of lower loneliness and higher social participation among participants. They placed the accountability ceiling between these two levels, because health and service use depend on many factors beyond the program, and because the evidence that reducing loneliness improves health (assumption A5) is weaker than the evidence for the earlier links. Below the intermediate outcome, they identified two preconditions that must both be present: participants must keep taking part after the twelve weeks end, and they must form contacts they experience as meaningful. Taking part without meaningful contact, or a single meaningful conversation without continued participation, would be unlikely to reduce loneliness. Both of these depend on a middle precondition, attendance at a first activity or role that the participant chose, and that in turn depends on four early preconditions, each produced by one of the four interventions.

The forward reading of the map yields a testable narrative. Clinicians screen and refer, so that eligible older adults are identified; connectors meet participants and agree a plan, so that participants have a reason and a plan to take part; accompaniment and transport help make attendance feasible, and partner grants make groups welcoming, so that participants attend a first activity they chose; participants keep attending and form meaningful contacts, so that loneliness falls and participation rises. The map makes two points visible that the logic model did not. It shows that first attendance is a bottleneck through which every pathway passes, which makes it an early indicator of whether the theory is working. It also shows that the weakest assumptions (A3 and A4) sit on the links just below the accountability ceiling, which tells the evaluation team where to concentrate its qualitative work.

3.5 Contribution Analysis

Lessons 6 to 8 teach designs that estimate how much of an observed change a program caused, by comparing outcomes with a counterfactual: an estimate of what would have happened without the program. This is attribution. Such designs are not always feasible. A program may serve everyone eligible, leaving no comparison group; it may be too early in its development for an outcome evaluation; or its outcomes may depend on so many other actors that a single effect estimate would mean little. John Mayne, then at the Office of the Auditor General of Canada, proposed contribution analysis for these situations (Mayne, 2001). It asks a different question from attribution: whether it is reasonable to conclude that the program made an important contribution to the observed outcomes, given the evidence on its theory of change and on other influencing factors.

Contribution analysis aims to reduce uncertainty about a program's contribution, and its product is a contribution story: an evidence-based account of how the program contributed to the outcomes, which links in the theory of change are well supported, and which rival explanations have been examined. Mayne (2001) first presented the approach for use with performance measurement data in the Canadian Journal of Program Evaluation, and he later described it in six steps (Mayne, 2008) and reviewed its development (Mayne, 2012). The tabs apply the six steps to the first six months of the Cedar Valley program.

Strengths and limits of contribution analysis

Contribution analysis suits programs in settings where many factors influence outcomes and an experimental counterfactual is unavailable. It makes the reasoning behind a causal claim explicit, uses evidence the program already collects, and directs new data collection to the weakest links. It does not estimate an effect size, so it cannot say how much of the change the program caused, and its conclusions depend on the evaluator's judgement about rival explanations, which can lean toward confirming the program's theory. Mayne (2012) encouraged combining it with other methods; Befani and Mayne (2014), for example, combined it with process tracing, which applies formal tests to the evidence for each causal link. Contribution analysis is best seen as complementary to the designs in Lessons 6 to 8, and at Cedar Valley the second-wave rollout offers a comparison group that can turn the contribution story into a stronger causal claim.

4.1 The Realist Account of Causation

The program theories in Sections 1 to 3 describe a single pathway that is expected to apply to everyone the program serves. Ray Pawson and Nick Tilley (1997) argued that this expectation is usually wrong. In Realistic Evaluation, they observed that the same program produces different results for different people in different settings, and that evaluations reporting only an average effect conceal this variation. They proposed that the useful question is what works, for whom, in what circumstances, and why. Later realist writing extends the question to ask in what respects and to what extent a program works (Pawson, 2013).

The approach rests on scientific realism. Its central claim for evaluation is that programs do not produce outcomes directly: they offer resources, such as information, support, opportunities or material help, and outcomes depend on how people respond. Pawson and Tilley called the response a mechanism and argued that mechanisms operate only in certain contexts. This generative account of causation explains an outcome by identifying the mechanism that produced it and the conditions that allowed it to operate. It differs from the successionist account behind experimental designs, in which a cause is inferred from a regular association between program and outcome under controlled conditions. Realist evaluators accept that experiments can estimate average effects, and they argue that an average effect tells decision-makers little about where and for whom to deliver a program.

Lesson 1 introduced realist evaluation as one of the approaches on the methods branch of Alkin and Christie's evaluation theory tree, and it presented two Cedar Valley configurations. This section teaches how to write such configurations and how to assemble them into a program theory that an evaluation can test.

4.2 Context-Mechanism-Outcome Configurations

The analytic unit of realist evaluation is the context-mechanism-outcome configuration, usually abbreviated in writing as CMOC. It states that in a given context, a given mechanism is triggered, which produces a given outcome. Pawson and Tilley (1997) expressed the relationship in a short formula, and Dalkin and colleagues (2015) refined it by dividing the mechanism into two parts.

Each element has a precise meaning. The context is the set of conditions that determines whether a mechanism is triggered. It includes characteristics of participants, such as their history, relationships, beliefs and circumstances, and features of the setting, such as available services, organizational culture and community norms. A demographic label such as "rural" is a context only if the evaluator can say why it matters; "rural participants who no longer drive and live beyond walking distance of any group" states the condition that makes transportation help important. The mechanism is the participant's response to what the program offers. It is usually hidden, it operates at the level of reasoning or feeling, and it must be inferred from evidence. The outcome is the result, intended or unintended, and it may be an early outcome such as attendance or a later one such as lower loneliness.

The table below sets out four configurations for Cedar Valley. The first two develop the examples given in Lesson 1. The third describes a configuration in which the program may fail or cause harm, which realist evaluation treats as equally informative. The fourth illustrates a mechanism that operates through a volunteer role and a sense of purpose.

The third configuration has direct implications for the program. If the evaluation finds evidence for it, Cedar Valley might change its protocol for participants with signs of low mood, for example by pacing the move toward group activity more slowly or by linking them with mental health support first. This kind of finding is what realist evaluation is designed to produce: specific guidance about how to adapt a program to the circumstances of different participants.

Writing configurations well

Three errors are common in configurations written by evaluators new to the approach. The first is to write a program activity as the mechanism, as in "the mechanism is accompaniment"; accompaniment is the resource, and the mechanism is the change in the participant's reasoning that accompaniment produces. The second is to write a context that does not explain anything, such as "women" or "people in the city", without stating what about that group makes the mechanism more or less likely to fire. The third is to state an outcome that cannot be observed, such as "empowerment", without saying how it would be recognized. A useful drafting device is the "if-then-because" statement: if the program offers this resource to people in this context, then this outcome will follow, because people will respond in this way. The "because" clause contains the mechanism.

4.3 Developing an Initial Program Theory

A realist evaluation begins with an initial program theory: a set of tentative configurations that state how the program is expected to work, for whom and in what circumstances. The evaluation then tests and refines this theory, and its main product is a refined program theory. Initial program theories are built from several sources. Program documents and interviews with designers and staff reveal the reasoning behind the program. Realist interviews, in which the evaluator presents candidate theories to an interviewee and asks them to confirm, refute or refine them, are a distinctive tool for this purpose; Pawson (1996) called this the teacher-learner relationship, and Manzano (2016) gave practical guidance. The research literature, including realist reviews of social prescribing (Husk et al., 2020; Tierney et al., 2020), offers configurations that other researchers have proposed and tested. Formal social science theories, which realists call middle-range theories after Robert Merton, offer general explanations of mechanisms, such as theories of how loneliness affects attention to social threat (Hawkley and Cacioppo, 2010). People with lived experience, such as the older adults on the Cedar Valley steering committee, can say which mechanisms ring true and which do not.

The configurations in an initial program theory should be developed with the people whose programs they describe. For the land-based connection pathway being co-designed with one First Nation, the theory of how land-based activities support connection belongs first to that Nation and its knowledge holders. An evaluator's role would be to help articulate and test configurations in ways the partners consider appropriate, under the governance arrangements discussed in Lesson 4.

4.4 What Works, for Whom, in What Circumstances and Why

Realist evaluation changes the practical design of an evaluation in several ways. Sampling for interviews is purposive across contexts, so that the evaluation hears from participants in the circumstances where different mechanisms are expected, such as bereaved and non-bereaved participants, urban and rural participants, and those who stopped attending as well as those who continued. Quantitative analysis is planned around the configurations, for example by comparing outcomes across the contexts they specify. Findings take the form of refined configurations, which tell decision-makers where and for whom the program works and what to adapt.

The approach is especially useful when an average result conceals opposing effects. Suppose the Cedar Valley evaluation finds little average change in loneliness. A realist analysis might show that bereaved participants who were accompanied improved substantially, while participants with low mood who were pushed toward groups early withdrew. The average would mislead the health authority, which might abandon a program that works well for a large group of participants. The realist finding instead points to a targeted change in protocol.

Realist evaluation also has limits. Context and mechanism are hard to separate in practice, and different evaluators may classify the same factor differently. The approach demands time and skill, particularly for realist interviewing and retroductive analysis, and its findings are harder to summarize for decision-makers who want a single answer. Configurations developed after the data are collected can become stories that fit the evidence too easily, a risk reduced by stating the initial program theory in advance and reporting evidence that contradicts it.

Choosing among the three forms of program theory

The three forms are complementary. Most evaluation plans include a logic model and a theory of change, and many add realist configurations for the links where variation across participants is expected. The Cedar Valley worked example in Section 4.5 shows the first two.

4.5 Worked Example: The Cedar Valley Logic Model and Theory of Change Narrative

The worked example below sets out a logic model and a theory of change for the Cedar Valley Connector program. The first part is the logic model, which for Cedar Valley is Figure 2.1 in Section 2. The second part is a one-page theory of change narrative with stated assumptions, which follows. The narrative refers to the pathway in Figure 3.1, and a narrative of this kind is normally accompanied by a diagram like it.

Bringing It All Together

This lesson has treated program theory as the argument a program makes about how change happens, and it has shown four ways of making that argument explicit. Chen's change and action models divide the theory into the causal process the program relies on and the arrangements needed to deliver it, and Weiss's distinction between implementation failure and theory failure shows why an evaluation that measures only final outcomes cannot explain what it finds. Logic models summarize the theory as a sequence from inputs to outcomes, and their value depends on precise definitions, especially the line between outputs and outcomes.

Theories of change explain the sequence. They are built backward from the long-term outcome, read forward as a test, and annotated with the assumptions and rationales that make each link testable, with an accountability ceiling that separates what a program is judged on from what it contributes to. Contribution analysis uses a theory of change to build a credible account of a program's contribution when attribution is not possible, and the Cedar Valley example showed how regression to the mean and loss to follow-up limit what a pre-post change can show. Realist program theory adds the question of for whom and in what circumstances a program works, expressed as context-mechanism-outcome configurations.

For the fictional Cedar Valley Connector program, these tools converge on the same priorities: first attendance at a chosen activity as an early indicator, and the assumptions about continued participation and meaningful connection as the links most in need of evidence.

The final reflection asks you to apply the lesson's tools to a program you have not seen before.

Final Knowledge Assessment

This glossary defines the terms, frameworks and people introduced in Lesson 3; use the search box to filter entries.