Environmental Scans
Finding & Synthesizing Health Evidence
Learning objectives for this lesson:
- Define an environmental scan, describe its origins in organizational planning, and explain how health services and public health organizations use scans.
- Distinguish passive from active scanning, and distinguish an environmental scan from a literature review, a needs assessment, an asset map and a program evaluation.
- Plan a document and website review and build a program inventory that records every candidate program and the decision made about it.
- Design a short survey of organizations and a key informant interview guide, and plan contact, consent and data governance for both.
- Define the unit of a scan, write eligibility criteria for it, build a sampling frame from several sources and judge the frame's completeness.
- Choose between a census and a purposive sample for each scan method, and plan the sampling of jurisdictions for a comparison across provinces.
- Design and pilot a charting template with a data dictionary, a source for each entry and a date of verification.
- Combine a scan with a literature review through shared categories and a joint display, and report a scan with a flow of programs, stated denominators, limitations and a date of currency.
- Build a sampling frame, inventory and charting table for a scan of five to ten organizations, and draft a joint display that links a scan with a review.
This course was developed by Dr. Kiffer G. Card, Faculty of Health Sciences, Simon Fraser University, drawing on the Cochrane Handbook for Systematic Reviews of Interventions and the JBI Manual for Evidence Synthesis.
What an Environmental Scan Is: Origins and Uses in Health
Learning Objectives for this section
- Define an environmental scan and describe the features that published definitions of scans share.
- Trace the origins of environmental scanning in organizational planning, including Aguilar's four modes of scanning.
- Describe how environmental scans are used in health services and public health, with published examples.
- Distinguish passive from active scanning, internal from external sources, and codified from tacit knowledge.
- Distinguish an environmental scan from a literature review, a needs assessment, an asset map and a program evaluation.
- Write a scan question with sub-questions, using the Cedar Valley scan as a model.
Introduction
Lesson 1 introduced the environmental scan as a close relative of the review that studies current programs and services through documents, websites, surveys and interviews. In Lesson 2 the Cedar Valley team wrote a scan question beside its review question, and Lessons 5 and 7 noted that many grey-literature documents describe community programs in detail with little or no evaluation, which makes them useful to a scan even when a review cannot include them as studies. This lesson teaches scan methods in full.
Section 1 defines the scan and describes its origins and uses. Section 2 covers four methods: document and website review, program inventories, surveys of organizations and key informant interviews. Section 3 covers the sampling of organizations and jurisdictions and the charting template, and Section 4 shows how to combine a scan with a literature review and how to report it.
The fictional Cedar Valley Health Authority in British Columbia serves about 210,000 residents, of whom about 46,000 are aged 65 and older, across Cedar City, several smaller communities and 24 primary care clinics. Before launching a community connector (social prescribing) program for older adults, its planning team asked a small evidence team, made up of a health authority evidence officer, a university librarian and a student intern, for a rapid scoping review and an environmental scan within twelve weeks.
The review, which asks which community-based interventions have been evaluated for reducing loneliness or social isolation among adults aged 65 and older, includes 42 studies and 26 grey-literature documents. The scan asks which such programs currently operate in British Columbia and elsewhere in Canada, who runs them, and how they are funded and evaluated. It ran in weeks 3 to 10 of the project, alongside the review, and identified 18 programs operating in British Columbia, of which 11 responded to a short survey and 7 took part in key informant interviews. Other numbers in this lesson are illustrative and consistent within it.
1.1 What an Environmental Scan Is
An environmental scan collects and interprets information about current programs, services, policies, organizations and trends in a defined setting, so that an organization can plan. In this phrase the environment is the organizational setting around the people who are planning. For a health authority, that setting includes community organizations, municipal and provincial governments, funders, professional groups and the residents it serves.
Published definitions vary. Following a protocol published in BMJ Open (Charlton et al., 2019), Charlton and colleagues (2021) reviewed 96 studies that used environmental scans in health services delivery research and reported that the field lacks a standard definition and uses its terms inconsistently. Their thematic analysis of how these studies defined or described scans produced several themes, including the four shown in the cards below, and the authors proposed a working definition for health services delivery. In a later survey of 47 people, mostly in Canada, who had conducted or used scans, Charlton and colleagues (2024) found that respondents often described a scan as a snapshot of a defined setting and found scans hard to distinguish from needs assessments.
Select each card to read what the theme means and how it applies to the Cedar Valley scan.
Taken together, these descriptions point to five features of a well-designed scan: it covers a defined setting at a stated time, it serves a stated decision, it draws on several kinds of sources, it follows a documented procedure that someone else could repeat, and it produces a structured account of what exists.
1.2 Origins in Organizational Planning
Environmental scanning began in business. In his 1967 book Scanning the Business Environment, Francis Aguilar described scanning as the acquisition of information about events and relationships outside a company that would help its senior managers plan the company's future course of action. Aguilar distinguished four modes of scanning, which differ in how purposeful and structured the search for information is. Chun Wei Choo (2001) later linked these modes to research on information seeking and organizational learning, and argued that organizations use all four, moving from broad viewing to formal search as their questions become specific.
| Mode (Aguilar, 1967) | What it involves | Example in a health authority |
|---|---|---|
| Undirected viewing | General exposure to information with no specific purpose in mind | A manager reads a seniors' newsletter and notices a story about a new friendly-visiting program. |
| Conditioned viewing | Directed exposure to selected sources or topics, while staying alert to what they show | Planners subscribe to updates from the provincial ministry and from a social prescribing network. |
| Informal search | A limited, unstructured effort to find specific information | A planner phones two colleagues to ask which local groups run programs for isolated older adults. |
| Formal search | A deliberate, planned effort that follows a stated procedure | The evidence team searches defined directories, surveys every eligible program and interviews selected leads. |
An environmental scan as this course teaches it is a formal search in Aguilar's sense. The other three modes still matter, because they often prompt a scan and supply leads that the formal search then checks.
Scanning became part of strategic planning, where its results feed tools such as a SWOT analysis, which sets an organization's internal strengths and weaknesses beside the external opportunities and threats it faces. Many scans also group external information under political, economic, social and technological headings (PEST). Graham, Evitts and Thomas-MacLean (2008) note that scanning spread from business to the non-profit sector, where the United Way of America used comprehensive scanning in the early 1980s, and that health researchers have since used scans in areas such as mental health, nutrition, women's health, Indigenous health, housing and knowledge transfer. The timeline below shows the publications this lesson draws on.
1.3 Uses in Health Services and Public Health
Charlton and colleagues (2021) found that health services researchers conducted scans for many purposes, of which the most common was to examine the current state of programs, services or policies. Scans informed recommendations in 20 percent of the 96 studies they reviewed. Health organizations use scans to plan new programs, compare policies across provinces or countries, map the services available to a population, find gaps and partners, and prepare for a needs assessment or an evaluation.
Rowel and colleagues (2005) described an environmental scan conducted in 2003 as part of the needs assessment for a project to increase cancer screening among African Americans in Baltimore, Maryland. The scan combined a review of community organizations and interest holders, local cancer incidence data, and community assets and liabilities with focus groups and key informant interviews with service providers. The authors reported that the scan gave the project team quick access to information about local events, trends and relationships, and they described the scan as flexible, responsive and cost-effective, with the added benefit of mobilizing the community. They also observed that the environmental scan was poorly defined, rarely evaluated and seldom used in public health, and that little guidance existed on how to design, conduct or evaluate one.
Wilburn, Vanderpool and Knight (2016) described a scan for a statewide project on human papillomavirus (HPV) vaccination in Kentucky. Their scan combined interviews, a survey of health care providers, reviews of the literature and policy, a scan of television coverage and registry data. From that experience they proposed a seven-step process, which the table below sets beside the Cedar Valley scan.
| Step (after Wilburn, Vanderpool and Knight, 2016) | How the Cedar Valley team applied it |
|---|---|
| 1. Determine leadership and capacity for the project | The evidence officer led the scan; the intern built the inventory and charted programs; the librarian advised on directories and websites. |
| 2. Establish the focal area and purpose of the scan | The scan question from Lesson 2, tied to the design of a community connector program. |
| 3. Create and keep to a timeline with incremental goals | Weeks 3 to 10 of the twelve-week project, with the sampling frame closed by the end of week 7. |
| 4. Determine the information to be collected | A charting template whose fields answer each sub-question (Section 3). |
| 5. Identify and engage interest holders | The planning team, program leads and the health authority's Indigenous health team (Lesson 2). |
| 6. Analyze and synthesize the results into a concise summary report | Counts with stated denominators, a framework summary of interview notes and a joint display with the review (Section 4). |
| 7. Disseminate the results to interest holders | A scan report for the planning team, a summary for participating programs and a section of the evidence brief (Lesson 12). |
In Canada, CADTH, now Canada's Drug Agency, has published environmental scans as one of its report types. Its 2015 process document describes scans of health care practices, processes and protocols inside and outside Canada, based on a limited literature search, a survey of Canadian health care decision-makers, or both. These scans give a snapshot of current practice and do not make recommendations for or against a technology (CADTH, 2015).
1.4 Passive and Active Scanning, and the Sources a Scan Uses
Graham, Evitts and Thomas-MacLean (2008) distinguish two modes of scanning. Passive scanning collects knowledge that already exists, such as published literature, reports and information from established contacts. Active scanning creates new knowledge with data collection tools such as surveys and interviews, and it includes a willingness to revise what the team thought it knew. The same authors distinguish internal sources, such as an organization's own memos, meeting minutes and reports, from external sources about the political, social, economic and technological context. They also distinguish codified knowledge, which is written down in reports, statistics and policy documents, from tacit knowledge, which lives in people's experience and comes out in interviews, focus groups and conversations.
Most scans in health use both modes. In the review by Charlton and colleagues (2021), the most common methods were literature reviews (71 percent of studies), key informant or semi-structured interviews (46 percent) and surveys (35 percent), and 53 percent of studies combined passive and active approaches.
1.5 How a Scan Differs from Related Products
A scan overlaps with several other products that health organizations commission, and the differences lie mainly in the question asked and the sources used. The table compares them.
| Product | Main question | Main sources | Typical output |
|---|---|---|---|
| Literature review (Lessons 1 to 10) | What has research found? | Published and grey research reports | A synthesis of studies |
| Environmental scan | What exists now in a defined setting, and what is changing? | Websites, documents, directories, organizations and people | An inventory and structured description of programs, policies and trends |
| Needs assessment | What does a population need, and how do needs compare with services? | Population data, resident surveys and consultations | A ranked list of needs and service gaps |
| Asset mapping | What strengths and resources does a community have? | Residents, community groups and local knowledge | A map or list of community assets |
| Program evaluation | Does this program work, and how? | Program records, participants and staff | A judgement about one program |
| Jurisdictional scan | How do other provinces or countries handle this issue? | Legislation, policies, reports and officials | A comparison table across jurisdictions |
The boundaries are porous. A scan often supplies the service side of a needs assessment, and a jurisdictional scan is a scan whose unit is a province or country (Section 3).
1.6 The Cedar Valley Scan Question
In Lesson 2 the team wrote its scan question with the same framework as its review question, treating organizations as the population: which organizations currently deliver community-based programs for socially isolated older adults, how those programs are designed, governed, funded and evaluated, in British Columbia and elsewhere in Canada. The team then wrote sub-questions, each of which points to the sources most likely to answer it.
| Sub-question | Main sources |
|---|---|
| 1. Which programs operate in British Columbia, where, and who runs them? | Directories, funder lists, websites and referrals |
| 2. What do the programs offer, to whom, and how do participants reach them, including referral from primary care? | Websites, documents and the survey |
| 3. How are the programs staffed and funded, and how secure is their funding? | The survey, annual reports and funder documents |
| 4. Are the programs evaluated, and with which measures? | The survey, evaluation reports and interviews |
| 5. What do program leads see as barriers and facilitators, and what would they advise a health authority starting a connector program? | Key informant interviews |
| 6. Which models in other provinces might inform the Cedar Valley program? | Public documents and websites from selected provinces |
The scan's methods were set out in a short appendix to the review protocol (Lesson 2). Section 2 describes each method and the ethical questions it raises.
For each request a health authority might receive, decide whether an environmental scan, a literature review, a needs assessment or a program evaluation is the best primary product. (1) "Which organizations in our region offer transportation for older adults to medical appointments, and what do they charge?" (2) "Do exercise programs reduce falls among adults over 75?" (3) "Did our two-year pilot of a walking group reduce loneliness among participants?" (4) "How do the other provinces fund community paramedicine, and who governs it?" (5) "What are the unmet health and social needs of older adults living alone in our rural communities?"
(1) An environmental scan, because the question asks what exists now in a defined region and who provides it. (2) A literature review, because the question asks what research has found about an intervention's effect. (3) A program evaluation, because the question concerns the effect of one local program. (4) A jurisdictional scan, which is a scan whose units are provinces; its sources are policies, reports and officials. (5) A needs assessment, because the question asks about the needs of a population, although a scan of available services would usually form one part of it.
Reflection
A public health unit in a mid-sized Canadian city plans to launch a peer-support program for the mental health of international post-secondary students, and asks you for an environmental scan. An environmental scan collects and interprets information about current programs, services, policies, organizations and trends in a defined setting, to inform planning. Passive scanning gathers information that already exists, such as documents, websites and directories; active scanning generates new information, for example through surveys of organizations and key informant interviews. (a) Write a scan question and three sub-questions. (b) Name two passive and two active sources you would use and say what each would add. (c) In two or three sentences, explain how your scan differs from a literature review (which asks what research has found) and from a needs assessment (which asks what a population needs and how its needs compare with services).
(a) Scan question: which organizations in the city currently offer mental health supports for international post-secondary students, what do those supports involve, and how are they staffed, funded and evaluated? Sub-questions: which supports exist on and off campus, and who runs them; how do students find and reach them, including in languages other than English; and which supports use peers, and how are peer supporters trained and supervised?
(b) Passive sources: the websites and annual reports of the city's colleges and universities, which list campus counselling and international student services; and the regional community services directory, which lists settlement agencies and community mental health services. Active sources: a short survey of service managers, which adds facts that websites omit, such as staffing, hours, languages offered and whether any outcomes are measured; and key informant interviews with three or four managers and a peer-support coordinator, which explain why students do or do not use services and what a new program should avoid duplicating.
(c) A literature review would tell the unit what studies of peer support for international students have found, wherever they were done, while the scan describes what exists in this city now. A needs assessment would ask students what they need; the scan describes the services side, and its results could form one part of a later needs assessment.
Minimum 20 characters required.
Question 1: Which finding did Charlton and colleagues (2021) report from their scoping review of 96 studies that used environmental scans in health services delivery research?
Question 2: A health authority planner phones two colleagues to ask which local groups run programs for isolated older adults. Which of Aguilar's four modes of scanning does this illustrate?
Question 3: Graham, Evitts and Thomas-MacLean (2008) distinguish passive from active scanning. Which activity is an example of active scanning?
Question 4: Which request is best answered by an environmental scan as the primary product?
Scan Methods: Documents, Program Inventories, Surveys and Interviews
Learning Objectives for this section
- Plan a document and website review for a scan, with a log that records each source, the date it was searched and a saved copy of what was found.
- Build a program inventory that lists every candidate program once and records the decision made about it.
- Design a short survey of organizations with clear items, suitable response options and a contact plan, and report its response rate.
- Write a key informant interview guide with main questions and probes, and plan how interview notes will be kept and checked.
- Identify the ethical questions a scan raises, including research ethics review, consent, confidentiality and Indigenous data governance.
Introduction
Section 1 showed that most scans combine passive methods, which gather information that already exists, with active methods, which generate new information from organizations and people. This section works through the four methods the Cedar Valley team used, in the order in which it used them. Each method filled gaps left by the one before it, as Figure 11.3 shows.
2.1 Document and Website Review
A document and website review searches the public record for programs and reads what it says about them. For a scan of community programs, the most useful sources are organization websites, annual reports, reports to funders, program brochures, municipal and health authority lists of community resources, provincial directories and the reports of funding programs. The Cedar Valley team began with the sources it had already found for the review. Lesson 5 described the targeted website searches, and Lesson 7 showed that the review excluded 12 documents from other sources because they described a program without evaluating it. Those 12 documents, together with many of the 26 grey-literature documents that the review included, became the first documents of the scan.
The team then searched sources chosen for the scan. These included the provincial 211 information and referral directory (bc211), which lists community and social services; the list of local programs funded through Better at Home, a provincially funded program that supports older adults to live independently; the community resource lists of British Columbia's regional health authorities; and the websites of municipalities and seniors' centres. Section 3 explains how these sources were combined into a sampling frame.
The documentation rules from Lesson 5 apply here. For each search the intern recorded the source, the date, the search terms or browsing path, the number of listings found and the person who searched. For each document kept, the intern saved a PDF or archived copy of the web page, because program websites change and disappear, so that a reader can later see exactly what the source said when the scan used it.
Reading program documents critically
Program websites and brochures are written to attract participants and funders, rarely report funding, reach or evaluation, and may describe a program that has since closed or changed. The Cedar Valley team treated documents as the starting point for each program's record, confirmed key facts through the survey or interviews where it could, and coded information that a document did not mention as "not reported". A field coded "not reported" describes the source and says nothing about whether the program has the feature.
2.2 The Program Inventory
A program inventory is a running list of every candidate program the scan identifies, with the source or sources where each was found and the decision made about it. It plays the part that the reference library and screening record play in a review (Lesson 7), and as in a review the same item turns up in several sources. The team therefore distinguished a listing, meaning one mention of a program in one source, from a program, meaning the service itself. De-duplicating listings into programs comes first, and screening programs against the eligibility criteria (Section 3) comes second.
The inventory template below shows the fields the intern used, with an example entry for each of three kinds of decision. Candidate programs carry inventory codes, eligible programs later receive charting codes P01 to P18, and the descriptions are fictional.
| Field | Entry for INV-07 | Entry for INV-12 | Entry for INV-41 |
|---|---|---|---|
| Inventory ID | INV-07 | INV-12 | INV-41 |
| Program and operator | Telephone befriending service run by a volunteer seniors' society | Home support service (housekeeping and yard work) run by a non-profit | Weekly lunch club run by a church volunteer group |
| Listings and sources | 3: provincial directory, municipal website, grey-literature document G-14 | 2: provincial directory, funder list | 1: referral from a survey respondent |
| Web address or contact | Saved copy W-102 | Saved copy W-117 | Coordinator's public email address |
| Date first found | Week 3 | Week 3 | Week 6 |
| Location | Elsewhere in British Columbia, rural | Cedar Valley region | Cedar Valley region |
| Decision | Include (charted as P05) | Exclude | Unclear, then include after a phone call confirmed it was active (charted as P16) |
| Reason (first that applies) | Not applicable | 3. No social connection component | Not applicable |
| Decided by and checked by | Intern; checked by evidence officer | Intern; checked by evidence officer | Intern and evidence officer together |
Exclusion reasons were recorded from an ordered list, so that each excluded program has one reason and the counts add up, as in full-text screening (Lesson 7). The order was: (1) not currently operating; (2) based in a facility such as a long-term care home or hospital; (3) no component that addresses social connection; (4) not mainly for adults aged 65 and older; and (5) a one-time event. Because the scan was rapid, the intern made each decision and the evidence officer checked every exclusion and a sample of inclusions, and the two decided every unclear case together. Section 4 reports the counts.
2.3 Surveying Organizations
Documents rarely say how a program is funded, how many people it reaches or whether it is evaluated, and those were among the questions the planning team cared about most. A short survey of the organizations that run eligible programs can fill these gaps. An organizational survey asks one informed person, usually the program coordinator or manager, to report facts about the program on its behalf.
Several design principles apply. The survey should be short; the Cedar Valley survey took about ten minutes. It should ask only what documents cannot answer, and it can show what the team has already charted and ask the respondent to confirm or correct it. Closed items should have response options that cover every possibility, including "don't know", and questions about money can offer ranges and a "prefer not to say" option. Each item should ask about one thing and state its reference period, such as "in the past 12 months". The survey should be piloted with one or two people similar to the respondents, which the team did with the coordinators of two programs that fell outside the scan's eligibility criteria.
Contact procedures also affect response. Dillman, Smyth and Christian (2014) recommend a planned sequence of personalized contacts, each with a different purpose. Surveys of organizations tend to have lower response rates than surveys of individuals (Baruch and Holtom, 2008), so a scan should plan its follow-up from the start. The Cedar Valley team sent an advance email from the health authority's planning lead explaining the purpose, an invitation from the evidence officer with a link to the survey, two reminders a week apart and a final telephone call, and kept the survey open for three weeks (weeks 5 to 7).
| Item | Cedar Valley survey wording (abridged) | Response options |
|---|---|---|
| 1 | We have recorded the program's name, operator and contact as shown. Please correct anything that is wrong. | Pre-filled fields with space for corrections |
| 2 | Is the program currently operating? | Yes; paused (please give the date it paused); ended (please give the date) |
| 3 | Which one of the following best describes the program's main activity? | Group activity; one-to-one befriending or telephone contact; community connector or social prescribing; intergenerational; technology-based; other (please describe) |
| 4 | How do participants usually come to the program? Select all that apply. | Self-referral; family or friends; a primary care clinician; a community connector or another agency; other |
| 5 | How is the program mainly delivered? | In person; by telephone; online; a mix |
| 6 | In the past 12 months, about how many older adults took part? | Fewer than 25; 25 to 49; 50 to 99; 100 to 249; 250 or more; don't know |
| 7 | Who delivers the program? | Paid staff only; volunteers only; paid staff and volunteers |
| 8 | Which sources fund the program this year? Select all that apply, then mark the largest. | Provincial government; health authority; municipality; charitable foundation or United Way; donations or fees; other; don't know |
| 9 | When does the program's current funding end? | Within 12 months; in one to three years; ongoing with no end date; don't know |
| 10 | In the past three years, which of these has the program done? Select all that apply. | Commissioned an external evaluation; measured participant outcomes itself; counted activities or participants only; none of these; don't know |
| 11 | If the program measures participant outcomes, which questionnaires or measures does it use? | Open text |
| 12 | Could you share any evaluation report from the past three years? | Yes (upload or email); no; none exists |
| 13 | Are there other programs in British Columbia that we should include? | Open text |
| 14 | Would you be willing to take part in a 30- to 45-minute interview? | Yes; no |
Item 3 uses the intervention categories from the review's charting form (Lesson 8), so that the scan and the review can later be compared category by category (Section 4). Item 13 is a form of snowball sampling, discussed in Section 3. Of the 18 eligible programs, 11 completed the survey, a response rate of 61 percent (11 divided by 18). The team reported both numbers and compared respondents with non-respondents on what the documents showed, and found that 5 of the 7 non-responding programs were run entirely by volunteers, compared with 3 of the 11 that responded. Survey results therefore probably describe paid-staff programs better than volunteer-run ones, a likely case of non-response bias, and the report says so.
A colleague drafted these items for a survey of organizations that run programs for isolated older adults. Identify the problem with each and rewrite it. (a) "How many participants does your program serve and how many volunteers does it have?" (b) "Given the strong evidence that befriending reduces loneliness, does your program offer befriending?" (c) "How many people have taken part in your program?"
(a) The item is double-barrelled: it asks two questions with one answer space. Split it into two items, each with ranges and a "don't know" option. (b) The item is leading, because its preamble signals the answer the team hopes for and makes a claim that the review has not yet established. Ask simply, "Does your program offer one-to-one befriending, either in person or by telephone?" (c) The item has no reference period, so a program that started last month and one that started in 1998 will answer different questions. Ask, "In the past 12 months, about how many older adults took part in the program?" and offer ranges.
2.4 Key Informant Interviews
A key informant is a person who knows about a topic because of the position they hold, such as a program coordinator, a funder or a health authority manager. A key informant interview asks such a person about the program or system they know, more than about their own experience as an individual. Key informant interviews suit the how and why questions that a survey handles poorly, such as why referrals from clinicians are rare and what a newcomer should avoid. HSCI 207 Lesson 10 teaches how to write an interview guide, how to probe and how to pilot a guide; this section applies those skills to a scan.
The Cedar Valley interviews lasted 30 to 45 minutes and took place by video call or telephone in weeks 6 to 8. The evidence officer asked the questions and the intern took notes. With consent, calls were recorded so that notes could be checked, and within a day the intern completed a summary organized by the guide's topics. Each informant received the summary of their interview and was invited to correct it, a form of member checking. The main questions and probes of the guide appear below.
Main question: "Could you tell me about the program and how it started?" Probes: "Who started it, and why?" "How has it changed since then?" This opening question is easy to answer and lets the informant set the scene.
Main question: "How do older adults usually find their way to the program?" Probes: "Do clinicians or other agencies refer people to you? How does that work in practice?" "Which older adults do you think the program does not reach?"
Main question: "Which organizations does the program work with most closely?" Probes: "What does the program's relationship with primary care look like?" "Have you worked with a community connector or social prescribing service? What happened?"
Main question: "How secure is the program's funding over the next few years?" Probes: "What happens to the program when a grant ends?" "What would help the program plan further ahead?"
Main question: "How does the program know whether it is helping people?" Probes: "What, if anything, do you measure?" "What would you measure if you had the time or support?"
Main question: "Our health authority is planning a community connector program for older adults. What advice would you give us?" Probes: "What should we avoid?" "How could a new program work with yours without competing for the same participants or volunteers?"
The team invited 10 programs to interview and 7 took part, one informant from each. Section 3 explains how the 10 were chosen, and Section 4 how the notes were analyzed.
2.5 Ethics, Consent and Data Governance
Surveys and interviews collect information from people, so a scan must settle early whether it needs research ethics review. In Canada, the Tri-Council Policy Statement (TCPS 2, 2022) sets out which activities count as research. Its Article 2.5 states that quality assurance and quality improvement studies, program evaluation activities and similar work, when used exclusively for assessment, management or improvement purposes, do not constitute research under the policy and do not fall within the scope of research ethics board review. Its Article 2.2 exempts research that relies exclusively on information that is publicly available, under stated conditions. A scan done by a health authority to plan its own services often falls under Article 2.5, but the judgement belongs to the organization's research ethics or quality improvement office, and plans to publish the scan as research can change the answer. HSCI 207 Lesson 5 covers research ethics review in Canada.
The Cedar Valley team asked the health authority's research ethics office, which judged the scan to be a planning activity outside the scope of research ethics board review. The team still followed ethical practice. Each invitation explained the purpose of the scan and how information would be reported, and taking part was voluntary. Programs were named in the report only with their permission, quotations were attributed by role, recordings were deleted once the summaries had been checked, and sensitive information about funding and staffing was reported in aggregate. As Lesson 6 described, no scan data were entered into any artificial intelligence tool.
Programs led by Indigenous organizations require particular care. Chapter 9 of TCPS 2 sets out expectations for research involving First Nations, Inuit and Métis Peoples, and the First Nations principles of OCAP (ownership, control, access and possession), developed by the First Nations Information Governance Centre, assert First Nations' control over data about their communities. As decided in Lesson 2, the Cedar Valley team planned its outreach to Indigenous-led programs with the health authority's Indigenous health team and agreed with each organization what information about its program would appear in the report.
Reflection
You are planning the active methods of a scan of 22 community programs that offer free or low-cost exercise classes for older adults in one British Columbia health region. Public documents already give each program's name, operator, location and activities. The planning team also wants to know how many people each program reaches, how it is funded, whether it is evaluated, and why some programs struggle to attract men. Under the Tri-Council Policy Statement (TCPS 2), Article 2.5, quality improvement and program evaluation activities used exclusively for assessment, management or improvement do not fall within the scope of research ethics board review. (a) Write four survey items, with response options and a reference period where relevant, covering reach, funding, evaluation and one other topic. (b) Write two main interview questions, each with one or two probes. (c) Describe your contact plan for the survey. (d) Say how you would decide whether the scan needs research ethics review, and name two ethical practices you would follow whatever the decision.
(a) Reach: "In the past 12 months, about how many adults aged 65 and older took part?" (fewer than 25; 25 to 49; 50 to 99; 100 or more; don't know). Funding: "Which sources fund the classes this year? Select all that apply, then mark the largest" (provincial government; health authority; municipality; foundation; fees or donations; other; don't know). Evaluation: "In the past three years, which of these has the program done?" (external evaluation; measured participant outcomes itself; counted participants only; none; don't know). Other: "About what proportion of participants in the past 12 months were men?" (under one quarter; one quarter to one half; over one half; don't know).
(b) "How do older adults usually find out about the classes?" Probes: "Which groups do you think you are not reaching?" "What have you tried?" Second: "Why do you think fewer men take part?" Probe: "Has anything worked to change that?"
(c) An advance email from the health authority explaining the purpose, an invitation with the link, two reminders a week apart and a final phone call, with the survey open for three weeks.
(d) I would ask the health authority's research ethics or quality improvement office whether Article 2.5 applies, noting that plans to publish could change the answer. Whatever it decides, I would make participation voluntary with a clear information letter, and name programs or quote informants only with permission.
Minimum 20 characters required.
Question 1: In the Cedar Valley program inventory, how does a listing differ from a program?
Question 2: A draft survey item reads: "How many participants does your program serve and how many volunteers does it have?" What is the main problem with this item?
Question 3: Of the 18 eligible Cedar Valley programs, 11 answered the survey. Five of the 7 non-responding programs were run entirely by volunteers, compared with 3 of the 11 that responded. What should the team conclude?
Question 4: What does Article 2.5 of the Tri-Council Policy Statement (TCPS 2) state?
Sampling Organizations and Jurisdictions, and Building a Charting Template
Learning Objectives for this section
- Define the unit of a scan (a program, an organization or a jurisdiction) and write eligibility criteria for it.
- Build a sampling frame from several sources and judge its completeness from the overlap among those sources.
- Choose between a census and a purposive sample for each method, and select key informants for maximum variation.
- Plan the sampling of jurisdictions for a comparison across provinces or countries.
- Design and pilot a charting template with a data dictionary, a source for each entry and a date of verification.
Introduction
A review defines its population of studies with eligibility criteria and finds them with a documented search. A scan faces the same two tasks for programs, organizations or jurisdictions, with one added difficulty: no database indexes community programs, so the scan must assemble its own list before it can decide which entries belong.
3.1 Choosing the Unit of the Scan
The unit is the thing the scan counts and describes. Three units are common. A program is an organized service with its own activities and participants, such as a weekly walking group. An organization is the body that runs one or more programs, such as a seniors' society or a municipality. A jurisdiction is a government territory, such as a province, that sets policy and funding. The choice follows from the question. A question about what services exist for older adults calls for programs as units; a question about which organizations could partner with a health authority calls for organizations; a question about how provinces fund social prescribing calls for jurisdictions.
The Cedar Valley scan used the program as its unit, because the planning team wanted to know what kinds of programs existed and how each was funded and evaluated. The template also recorded the operating organization and any partners, because one organization may run several programs and one program may be delivered by several partners. The 18 eligible programs were run by 16 organizations: two organizations each ran two eligible programs. The Cedar Valley report states the unit at the top of every table, because counts of organizations and of programs differ.
3.2 Eligibility Criteria for Programs
Eligibility criteria for a scan work like those for a review (Lesson 2): each criterion should be clear enough that two people applying it to the same listing reach the same decision. The Cedar Valley criteria parallel the review's Population, Concept and Context, so that the scan and the review describe the same kinds of programs.
| Element | Include | Exclude | Reason |
|---|---|---|---|
| Population | Programs whose participants are mainly adults aged 65 and older living in the community, or that run a distinct stream for them | Programs for all ages with no older-adult stream | Matches the review's population |
| Concept | Organized, recurring programs with an aim or component that addresses loneliness, social isolation or social connection | Practical services with no social component, such as housekeeping or transportation alone; one-time events | Matches the review's intervention categories |
| Context | Community-based programs operating in British Columbia at any time in the 12 months before the scan | Programs in long-term care homes or hospitals; programs that had closed more than 12 months earlier | The planning team needs current community programs |
The 12-month rule for "currently operating" allows for programs that pause over the summer or between grants.
3.3 Building a Sampling Frame
A sampling frame is the list from which a study identifies or selects its units. An ideal frame contains every eligible unit exactly once and nothing else. A survey of physicians can often use a licensing register as its frame, but no single list of community programs for older adults exists in British Columbia, so the Cedar Valley team built its frame from six kinds of source. The table shows how many listings each source contributed.
| Source | Listings | Notes |
|---|---|---|
| Provincial 211 information and referral directory | 19 | Searched with the categories for seniors and for social and recreational programs |
| List of local Better at Home programs | 14 | Several offered only practical help and were later excluded under reason 3 |
| Regional health authority community resource lists | 11 | Lists varied in format and date of last update |
| Municipal and seniors' centre websites | 9 | Searched for the larger municipalities in each region |
| Grey-literature documents from the review (Lesson 5) | 7 | Program descriptions and reports that named a British Columbia program |
| Referrals from survey respondents and interviewees | 4 | Added until the frame closed at the end of week 7 |
| Total | 64 | Listings, before duplicates were removed |
Removing 21 duplicate listings left 43 unique candidate programs, and screening against the criteria left 18 eligible programs (Section 4 reports the reasons for exclusion). The frame closed at the end of week 7 so that the team had time to chart and verify every program.
Judging the completeness of the frame
A team can never be certain that a frame is complete, but the overlap among sources gives a useful signal. If most eligible programs appear in several independent sources, the sources are probably covering the same population well. If many appear in only one source, other programs probably appear in none. Figure 11.4 shows the overlap for the 18 eligible Cedar Valley programs. Thirteen appeared in two or more sources and 5 in only one, and 2 of those 5 were found only through referrals. Both were small volunteer-run programs, which suggests that the frame under-represents programs of that kind. The report states this limitation. Formal capture-recapture methods, which estimate a population's size from overlapping lists, assume independent lists, which directories that copy from one another rarely are, so the team used the overlap descriptively.
3.4 A Census or a Sample?
When the number of eligible units is small, a scan can include all of them, which is called a census. When it is large, or when a method is too costly to apply to every unit, the scan selects a sample. The choice can differ by method within the same scan. The Cedar Valley team charted all 18 programs from documents and sent the survey to all 18, because both steps were cheap. Interviews were expensive in time, so it selected programs for interview with purposive (judgement) sampling, which chooses units deliberately for what they can contribute. The cards describe four strategies that scans use.
Every eligible unit is included. A census suits small populations and cheap methods, although it is only as complete as the frame. Cedar Valley used a census for documents and the survey.
Units are chosen to differ as much as possible on characteristics that matter to the question, so that findings cover the range of situations. Patton (2015) describes it as one of several purposive sampling strategies. Cedar Valley used it for interviews.
Participants are asked to suggest other units, which helps to find programs that directories miss. It tends to find units connected to those already found, so it adds to a frame without guaranteeing completeness.
Units are included because they are easy to reach, such as programs whose staff the team already knows. It tends to over-represent larger, better-connected organizations and should be reported as a limitation when used.
For the interviews, the team listed the characteristics on which programs varied: main activity, location in the Cedar Valley region or elsewhere in British Columbia, urban or rural setting, type of operating organization, whether the program was Indigenous-led and whether it evaluated outcomes. It also decided to include programs that had not answered the survey, so that the interviews could hear from them. It invited 10 programs, 8 that had responded to the survey and 2 that had not, and 7 took part: 6 survey respondents and 1 non-respondent. Counting both methods, 12 of the 18 programs (11 by survey and 1 by interview only) provided information directly, and the other 6 were charted from documents alone.
| Program | Main activity | Location | Setting | Operator | Survey |
|---|---|---|---|---|---|
| P02 | Group activity | Cedar Valley region | Urban | Non-profit seniors' centre | Responded |
| P05 | One-to-one befriending or telephone | Elsewhere in British Columbia | Rural | Volunteer seniors' society | Responded |
| P08 | Community connector | Elsewhere in British Columbia | Urban | Non-profit in partnership with a health authority | Responded |
| P11 | Community connector | Cedar Valley region | Rural | Non-profit community services society | Responded |
| P13 | Intergenerational | Elsewhere in British Columbia | Urban | Municipality | Responded |
| P15 | Group activity | Elsewhere in British Columbia | Rural | Indigenous-led organization | Did not respond |
| P17 | Technology-based | Elsewhere in British Columbia | Urban | Non-profit | Responded |
The seven interviewed programs cover five of the six activity categories, both locations, both settings and four kinds of operator. No interviewed program was a one-to-one program in the Cedar Valley region, which the report notes as a gap in the interview sample.
3.5 Sampling Jurisdictions
A jurisdictional scan compares how provinces, territories or countries handle an issue, for example how each funds or governs a type of program. Canada has 10 provinces and 3 territories, so a national jurisdictional scan can often take a census of all 13; CADTH's process for environmental scans, for example, includes surveys of health care decision-makers across Canada (CADTH, 2015). When time is short, a scan selects jurisdictions purposively, for example those with a similar health system or a well-known model, and the report states which were included and why.
The Cedar Valley scan question also asked about programs elsewhere in Canada. With twelve weeks, the team chose a lighter method for this part. It selected four other provinces, chosen to vary by region and population size and to include provinces with publicly documented programs, and reviewed public documents and websites only, with no survey or interviews. It charted 9 programs from those provinces with a shortened version of the template. The report keeps these 9 separate from the 18 British Columbia programs, because they were found and described with different methods and cannot be counted together.
3.6 Building the Charting Template
A charting template is the form on which the scan records the same set of fields for every unit. As with the review's charting form (Lesson 8), each field answers part of the question, each has a definition and allowed values in a data dictionary, and the form is piloted before full use. Two features are particular to scans. Because information comes from several sources that may disagree, the template records the source of each entry. And because programs change, it records the date on which each program's information was last verified.
The Cedar Valley template appears below in abridged form. Its activity categories are those of the review's charting form (Lesson 8), which made the integration in Section 4 possible.
| Field | Definition and allowed values | Preferred source |
|---|---|---|
| Program ID and name | P01 to P18; name as the operator uses it | Inventory |
| Operator and partners | Operator type: non-profit; municipality; health authority; Indigenous-led organization; other. Partners named | Website, survey |
| Location and setting | Cedar Valley region or elsewhere in British Columbia; urban, or rural and small community | Website |
| Main activity | Group activity; one-to-one befriending or telephone; community connector or social prescribing; intergenerational; technology-based; other. Main activity is the one with the most contact hours; record a secondary activity if present | Survey, website |
| Delivery mode and frequency | In person, telephone, online or mixed; weekly, every two weeks, monthly or other | Survey, website |
| Eligibility and referral routes | Age and other criteria; self-referral, family, primary care clinician, connector or agency, other | Survey, interview |
| Staffing | Paid staff only; volunteers only; both | Survey |
| Reach | Older adults taking part in the past 12 months, in the survey's ranges | Survey, annual report |
| Funding | All sources this year and the largest; end date of the largest source | Survey, annual report |
| Evaluation status | External evaluation; internal outcome measurement; activity or satisfaction monitoring only; none reported | Survey, evaluation report |
| Outcome measures used | Names of questionnaires or measures as reported | Survey, evaluation report |
| Links with primary care | Formal referral pathway; informal contact; none reported | Interview, survey |
| Barriers and facilitators | Brief summary in the informant's terms | Interview |
| Sources and verification | Saved copy codes, survey and interview dates; date the entry was last verified; conflicts between sources and how each was resolved | All |
Three coding rules prevent common errors. First, the template distinguishes "not reported" (no source mentions it), "don't know" (the respondent did not know) and "not applicable" (the field does not apply), because the three mean different things when the team later counts programs. Second, when sources disagree, the team records the most recent direct report, usually the survey or the interview, and notes the conflict. Third, every entry carries its source.
Before charting all 18 programs, the intern and the evidence officer charted three programs independently and compared their entries. They agreed on 45 of 54 entries (3 programs with 18 entries each), or 83 percent. Of the 9 disagreements, 4 concerned the main activity of a seniors' centre that ran both drop-in groups and outreach visits, 3 concerned whether participant satisfaction surveys counted as outcome measurement, and 2 concerned which end date to record when a program had several funding sources. They added three rules to the data dictionary: the main activity is the one with the most contact hours, which is the rule the review used in Lesson 8; satisfaction surveys count as monitoring and not as outcome measurement; and the funding end date is that of the largest source.
Operator and partners: a non-profit, with a partnership agreement with a regional health authority. Location and setting: elsewhere in British Columbia; urban. Main activity: community connector or social prescribing; secondary activity, group activity. Delivery: in person and by telephone, about every two weeks. Eligibility and referral: adults aged 65 and older living in the community; referrals mainly from primary care clinicians, with some self-referral. Staffing: paid staff and volunteers. Reach: 100 to 249 older adults in the past 12 months (survey). Funding: health authority (largest) and a charitable foundation; largest source ends in one to three years (survey). Evaluation status: external evaluation; report shared with the team (document D-03). Outcome measures: a three-item loneliness scale and a well-being scale, as named in the evaluation report. Links with primary care: formal referral pathway (interview). Barriers and facilitators: connectors attend clinic team meetings, which helped referrals grow; winter transport limits group activities (interview). Sources: saved web page W-088 (week 4), survey (week 6), interview (week 7), document D-03. Conflict: the website described weekly meetings, and the survey reported meetings every two weeks since a staffing change; the survey value was recorded. Last verified: week 9, when the program confirmed its interview summary.
Program P12's website, last updated about two years before the scan, says: "Our Tuesday Tea Club welcomes all seniors aged 60 and over for conversation, games and a light lunch. Generously funded by the City." The program's survey response reports that its main activity is a group activity; that 25 to 49 older adults took part in the past 12 months, most of them over 70; that it is run by volunteers only; that its funding comes from the municipality (largest) and donations, with the municipal grant ending within 12 months; that it counted participants only; and, for outcome measures, "don't know". Chart the main activity, eligibility, staffing, funding, evaluation status and outcome measures, and note any conflict between sources.
Main activity: group activity (survey and website agree). Eligibility: adults aged 60 and older, with participants mainly over 70; the program meets the criterion of serving mainly adults aged 65 and older, and the age difference should be noted in case the planning team asks. Staffing: volunteers only (survey). Funding: municipality (largest) and donations; largest source ends within 12 months (survey). The survey adds donations to the website's account and is recorded as the more recent direct report, with the difference noted. Evaluation status: activity or satisfaction monitoring only (survey). Outcome measures: "don't know" (survey), which differs from "not reported" and from "not applicable" and should be coded as such. The entry's sources are the saved web page and the survey, with the survey date as the date of verification.
Reflection
A scan of youth mental health walk-in services in British Columbia uses the service as its unit. Its sources produced these listings: a provincial directory, 30; health authority resource lists, 18; school district websites, 12; and referrals from service managers, 5. The team removed 23 duplicate listings and screened the remaining candidates against its criteria, excluding 16. Of the eligible services, 15 appeared in two or more sources and 11 in only one, including 6 found only by referral. The budget allows 8 interviews; the survey is cheap to send. (a) Calculate the number of listings, candidate services and eligible services, and check that the numbers reconcile. (b) Interpret the overlap among sources for the completeness of the frame. (c) Decide whether to use a census or a sample for the survey and for the interviews, and name three characteristics you would vary when selecting services for interview. (d) List five fields for the charting template, each with allowed values.
(a) Listings: 30 + 18 + 12 + 5 = 65. Candidates: 65 − 23 = 42. Eligible: 42 − 16 = 26. The overlap figures reconcile, because 15 + 11 = 26.
(b) Eleven of 26 services (42 percent) appeared in only one source, and 6 were found only by referral, which suggests that the directories miss many services and that other services were probably found by no source at all. I would add sources, for example community health centre and youth agency websites, keep asking for referrals until the frame closes, and report the likely under-coverage as a limitation.
(c) A census for the survey, since there are only 26 services and the survey is cheap. For interviews, a purposive maximum variation sample of 8, varying by health authority region, urban or rural setting and operator type (health authority, non-profit or school district), and including one or two services that did not answer the survey.
(d) Operator type (health authority; non-profit; school district; Indigenous-led organization; other). Setting (urban; rural or small community). Hours (weekdays only; includes evenings; includes weekends). Ages served (under 19; 12 to 24; other). Evaluation status (external evaluation; internal outcome measurement; activity monitoring only; none reported). Each entry would also record its source and the date it was verified.
Minimum 20 characters required.
Question 1: In the Cedar Valley sampling frame, 13 of the 18 eligible programs appeared in two or more sources and 5 in only one, including 2 found only through referrals. What does this pattern most reasonably suggest?
Question 2: Why did the Cedar Valley team use a census for the survey but a purposive maximum variation sample for the interviews?
Question 3: Which distinction does the Cedar Valley charting template make when information is missing?
Question 4: The Cedar Valley team reviewed public documents for 9 programs in four other provinces, without a survey or interviews. How should the scan report treat these programs?
Combining a Scan with a Literature Review, and Reporting a Scan
Learning Objectives for this section
- Explain why a scan and a literature review answer different parts of a planning question, and how each can inform the other.
- Describe the levels at which a scan and a review can be integrated: design, methods, and interpretation and reporting.
- Build a joint display that sets review findings beside scan findings using shared categories.
- Analyze scan data with counts that state their denominators and sources, and summarize interview notes with a framework approach.
- Report a scan with a flow of programs from identification to participation, a description of methods, a statement of limitations and a date of currency.
Introduction
The Cedar Valley planning team asked two questions: which community-based interventions have been evaluated for reducing loneliness or social isolation among older adults, and which programs of this kind operate in British Columbia now. The rapid scoping review answers the first and the environmental scan answers the second. A decision about what kind of community connector program to launch, and with which partners, needs both answers read together. This section shows how the team combined the two products, how it analyzed the scan data, and how it reported the scan so that readers could judge it and update it.
4.1 Why Combine a Scan with a Review
A review and a scan look at the same field from two sides. The review describes what research has found about programs, wherever they ran; most of the 42 studies in the Cedar Valley review were conducted outside Canada. The scan describes what is available locally, whether or not anyone has studied it; most of the 18 British Columbia programs have never been evaluated in a published study. Read together, the two products show where local practice and research evidence coincide, where local programs operate with little evidence behind them, and where the evidence describes models that no local program offers.
Combining the two is common practice: in the review by Charlton and colleagues (2021), a literature review was the most frequent method used within scans. In the Cedar Valley project the literature component is a full rapid scoping review, so the integration needs to be planned explicitly.
4.2 Levels of Integration
Combining a scan and a review is a form of mixed-methods work, and the mixed-methods literature offers a useful vocabulary. Fetters, Curry and Creswell (2013) describe integration at three levels. At the design level, the components may run in sequence, with one informing the next, or in parallel, with their results brought together at the end. At the methods level, one component can build on another, for example by using its categories, or connect to it, for example by drawing its sample from the other's results. At the level of interpretation and reporting, the findings can be woven together in narrative or set side by side in a joint display. HSCI 207 Lesson 6, Section 2.4, and HSCI 207 Lesson 12, Section 3.6, treat integration levels and joint displays more fully; the definitions here are enough to follow this lesson without that course. The tabs show where each level appeared in the Cedar Valley project.
The scan ran in parallel with the review (weeks 3 to 10), and the two were brought together in the synthesis that began in week 9 (Lesson 1), which suited the twelve-week deadline.
The scan's main-activity field and survey item 3 used the intervention categories of the review's charting form (Lesson 8). The 12 program descriptions that the review excluded because they reported no evaluation went to the scan as documents, and 7 grey-literature documents supplied listings for the sampling frame.
Programs found by the scan were checked for evaluation reports. The 3 external evaluation reports that programs shared during the scan were passed to the review team and are among the 7 documents received from organizations in the review's flow diagram (Lesson 7).
The team built a joint display that sets the review's findings for each intervention category beside the scan's findings for the same category (Section 4.3). The evidence brief in Lesson 12 draws on both.
4.3 A Joint Display for Cedar Valley
A joint display is a table or figure that places the results of two components side by side so that readers can see how they relate and what new interpretation their combination supports. Guetterman, Fetters and Creswell (2015) described joint displays as a way of integrating quantitative and qualitative results in health sciences research, and the same idea works for a review and a scan. Shared categories make a joint display possible. Because the Cedar Valley scan charted programs with the review's intervention categories, each row of the display compares like with like.
| Main activity | Review: studies in the certainty ratings (Lesson 8) | Review: certainty for loneliness | Scan: British Columbia programs (n = 18) | Scan: programs measuring outcomes | Integrated interpretation |
|---|---|---|---|---|---|
| Group activity | 8 randomized trials | Low; may reduce loneliness slightly | 6 | 2 | The most common local model has the most trial evidence, and few local programs measure outcomes. |
| One-to-one befriending or telephone | 6 randomized trials | Low; may reduce loneliness | 4 | 2 | Evidence and local supply both exist, and telephone programs reach rural participants. |
| Community connector or social prescribing | 5 non-randomized controlled studies | Very low; very uncertain | 4 | 2 | The model Cedar Valley plans has the least certain evidence; the two local programs that measure outcomes could share their tools. |
| Intergenerational and technology-based | 5 non-randomized controlled studies across the two categories | Not rated; too few studies in each category | 2 (1 in each) | 1 | Little evidence and few local programs. |
| Other | Not part of the ratings | Not rated | 2 | 0 | Multi-service seniors' centres that do not fit one category. |
The display supports three interpretations that neither product supports alone. First, the local supply of programs is concentrated in group activities, where trial evidence is most plentiful, although its certainty is low. Second, the community connector model that Cedar Valley plans to launch rests on very uncertain evidence and has only four counterparts in the province, two of which measure outcomes; this strengthens the case, made in Lesson 8, for launching the program with an evaluation built in, and suggests that the team could adopt a short outcome measure already used by those programs. Third, a connector program depends on having programs to refer people to, and the scan shows which kinds of programs exist and where. All results in this section belong to the fictional case and are illustrative.
4.4 Analyzing Scan Data
Counts with denominators and sources
Most scan results are counts and proportions, and the most frequent reporting error is a proportion whose denominator is unclear. Because the survey and interviews reached only some programs, each count must say which programs it describes and where the information came from. "Seven of the 18 programs measured at least one participant outcome (3 through an external evaluation), according to surveys, evaluation reports and program documents" is a complete statement. "Of the 12 programs that provided information directly, 8 reported that their largest source of funding ends within three years, and 3 reported a formal referral pathway from primary care" is another. The table shows the largest funding source for all 18 programs, drawn from the survey where available and from annual reports and websites otherwise.
| Largest funding source | Programs (n = 18) |
|---|---|
| Provincial government | 5 |
| Health authority | 4 |
| Charitable foundation or United Way | 4 |
| Municipality | 2 |
| Donations or fees | 1 |
| Not reported | 2 |
Summarizing interview notes
The interview summaries were analyzed with framework analysis, also called the framework method (Gale et al., 2013), which suits structured questions and short timelines. The team built a matrix with one row for each of the 7 programs and one column for each topic of the guide, wrote a short summary in each cell with the informant's key phrases, and then read down the columns to find patterns and differences. HSCI 207 Lesson 11 teaches framework analysis in more depth. In the illustrative Cedar Valley matrix, most informants said that participants arrived by word of mouth and that links with clinics depended on individual clinicians. Informants with short grants described a cycle of hiring, training and losing staff, several said they would measure outcomes if given a short shared tool, rural informants named transport as the main barrier, and most advised a new connector program to refer people into existing programs and bring resources such as volunteer training.
The team then compared what different sources said about the same topic, a step called triangulation. The survey showed that 3 of 12 programs had a formal referral pathway from primary care; the interviews explained why the number was low and what had made the pathway work in program P08. Where sources disagreed, the report says so.
4.5 Reporting a Scan
Charlton and colleagues (2024) noted that the EQUATOR Network, which catalogues reporting guidelines for health research, listed no reporting guideline for environmental scans, and that many published scans describe their methods incompletely. Until one exists, a scan can borrow from the guidelines for each of its components: PRISMA-S for the searches of websites and directories (Lessons 4 and 5), PRISMA-ScR for charting and summarizing (Lesson 10), the Checklist for Reporting Results of Internet E-Surveys (CHERRIES; Eysenbach, 2004) for an online survey, and the Consolidated Criteria for Reporting Qualitative Research (COREQ; Tong, Sainsbury and Craig, 2007) for interviews. The checklist below gathers the items the Cedar Valley team reported.
| Item | What to report |
|---|---|
| Question and purpose | The scan question, sub-questions and the decision the scan serves |
| Unit and eligibility | The unit (program, organization or jurisdiction) and the criteria with reasons |
| Sampling frame | Every source, the date searched, the search terms or path, and the date the frame closed |
| Selection | A flow of units from listings to eligible programs, with ordered exclusion reasons |
| Survey | The instrument, mode, contact sequence, dates, pilot, response rate with numerator and denominator, and a comparison of respondents with non-respondents |
| Interviews | How informants were selected, the guide, who interviewed, recording and notes, analysis and member checking |
| Charting | The template and data dictionary, the pilot and its agreement, coding rules and the handling of conflicting sources |
| Ethics and governance | The decision about research ethics review, consent procedures, confidentiality and agreements with Indigenous organizations |
| Results | Counts with denominators and sources, and summaries of interview findings |
| Limitations and currency | Known gaps in the frame and samples, and the date to which the information is current |
Lesson 7 noted that the scan is reported separately from the review's PRISMA flow diagram, because its counts describe programs and people. Figure 11.5 shows the flow of programs from identification to participation. Its structure follows the PRISMA diagram, with boxes that must reconcile: 64 listings minus 21 duplicates gives 43 candidate programs, and 43 minus 25 exclusions gives 18 eligible programs.
Limitations and currency
Every scan describes a moment in time, so its report should state the date to which its information is current, and a reader in two years should treat it as a historical record until it is updated. The Cedar Valley report states that program information was current at the end of week 10 and lists the scan's known limitations: the frame probably under-represents small volunteer-run programs (Section 3.3); volunteer-run programs were also less likely to answer the survey (Section 2.3); survey and interview information is self-reported and may present programs favourably; the interview sample included no one-to-one program in the Cedar Valley region; and programs in other provinces were described from documents only. The team kept the inventory, saved copies and template so that the scan could be updated.
4.6 Worked Example: The Cedar Valley Scan Report
The Cedar Valley scan report runs to about twelve pages plus appendices. Open each part to see what it contains.
One page giving the question, the methods, the main findings with denominators, the joint display's interpretations and the date of currency.
The scan question and six sub-questions; the unit and eligibility criteria; the six sources of the sampling frame with dates; the survey, its contact sequence and its response (11 of 18 programs, 61 percent); the selection of 10 programs for interview and the 7 interviews completed; the charting template and its pilot (83 percent agreement before the rules were revised); and the ethics decision and agreements with Indigenous organizations.
Figure 11.5, the flow of programs; a characteristics table of the 18 programs by main activity, location, setting, operator, reach, funding and evaluation status; and a narrative of interview findings by topic. A sample sentence reads: "Of the 18 programs, 6 offered mainly group activities, 4 one-to-one befriending or telephone contact, 4 community connector services, 1 an intergenerational program, 1 a technology-based program and 2 a mix of services. Seven measured at least one participant outcome, 3 of them through an external evaluation."
The joint display (Section 4.3), with a paragraph on each interpretation and a cross-reference to the review report.
The limitations and date of currency above, and implications for the connector program, such as referring participants into existing programs and adopting a shared outcome measure.
The inventory, the template and data dictionary, the survey, the interview guide, the search log and the 9 programs from other provinces.
A colleague’s draft scan report, on a different question, contains this paragraph: "Most programs (64%) are evaluated. Funding is a major concern for programs. Clinicians rarely refer patients to programs." Using what you know about reporting scans, identify at least three problems and rewrite the paragraph for the colleague’s scan, in which 9 of 14 survey respondents reported measuring participant outcomes, 8 of 14 reported that their largest funding source ends within three years, and 5 of 8 interviewed program leads described referrals from clinicians as rare.
The paragraph gives no denominators, so readers cannot tell that 64 percent means 9 of 14 survey respondents. It does not say which programs the figures describe, so readers may assume they cover all programs in the scan, although non-respondents may differ. "Evaluated" is undefined: measuring outcomes, counting participants and commissioning an external evaluation are different things. "A major concern" and "rarely" are not tied to a source or a count. A revision: "Of the 14 programs that answered the survey, 9 reported measuring at least one participant outcome, and 8 reported that their largest funding source ends within three years. Five of the 8 program leads interviewed described referrals from clinicians as rare." The report should also compare survey respondents with non-respondents, so that readers can judge whether the figures describe the programs that did not answer.
The Cedar Valley team now has a charted review of 42 studies and 26 grey-literature documents, a scan of 18 British Columbia programs and a joint display that links them. Lesson 12 turns these into a report written to PRISMA 2020 and its extensions and an evidence brief for the planning team.
Reflection
A rapid review of school-based physical activity interventions rated certainty for daily physical activity as follows: classroom activity breaks, 12 randomized trials, low certainty (may increase activity); after-school programs, 9 randomized trials, moderate certainty (probably increase activity); and active travel programs, 4 non-randomized studies, very low certainty (very uncertain). A parallel environmental scan of 20 British Columbia school districts, charted with the same categories, found activity breaks in 14 districts (3 of which measure outcomes), after-school programs in 4 districts (2 of which measure outcomes) and active travel programs in 9 districts (1 of which measures outcomes). A district can offer more than one kind of program. A joint display is a table that sets the results of two components side by side so that readers can draw interpretations that neither supports alone. (a) Describe the rows and columns of a joint display for these results. (b) Write three integrated interpretations. (c) Write one results sentence for the scan report that states its denominator and source.
(a) One row for each category (activity breaks, after-school programs, active travel). Columns: review studies and designs; review certainty and statement; districts offering the program (of 20); districts measuring outcomes; and an integrated interpretation.
(b) First, after-school programs have the most certain evidence (moderate, 9 trials) yet operate in only 4 of 20 districts, so they are the clearest candidate for expansion if cost and feasibility allow. Second, activity breaks are the most widespread local program (14 districts) and have low-certainty evidence, and only 3 of the 14 districts measure outcomes, so districts could adopt a shared measure to learn whether local breaks increase activity. Third, active travel programs operate in 9 districts on very uncertain evidence, and only 1 measures outcomes, which makes them the strongest case for building evaluation into any expansion.
(c) "Of the 20 school districts in the scan, 14 offered classroom activity breaks, and 3 of those 14 reported measuring students' physical activity, according to the district survey and program documents."
An alternative strong answer might add a column for gaps, noting any category with evidence but no local programs, which in these data does not occur.
Minimum 20 characters required.
Question 1: What is a joint display?
Question 2: What made the Cedar Valley joint display possible?
Question 3: Which sentence reports a scan result in the way Section 4 recommends?
Question 4: Which statement about reporting guidance for environmental scans is accurate?
Final Assessment
Bringing It All Together
This lesson followed the fictional Cedar Valley evidence team through the environmental scan that it ran beside its rapid scoping review. An environmental scan collects and interprets information about current programs, services, policies, organizations and trends in a defined setting, to inform planning. The method began in business planning, where Aguilar described four modes of scanning in 1967, and health services and public health organizations now use it to plan programs, compare jurisdictions and find gaps and partners. Published definitions still vary, and Charlton and colleagues found that most scans combine passive methods, which gather existing information, with active methods, which generate new information from organizations and people.
The Cedar Valley team combined four methods. It searched directories, websites and documents and saved a copy of each source; it built a program inventory that turned 64 listings into 43 candidate programs and 18 eligible ones, each with a recorded decision; it surveyed all 18 programs, of which 11 responded; and it interviewed leads from 7 programs chosen for maximum variation. It judged the completeness of its sampling frame from the overlap among sources, kept 9 programs from other provinces separate because they were found with different methods, and charted every program with a piloted template that recorded the source and date of each entry.
Finally, the team integrated the scan with the review through shared intervention categories and a joint display, which showed that the connector model it plans rests on very uncertain evidence and has few local counterparts. It reported the scan with a flow of programs, counts that state their denominators and sources, a statement of limitations and a date of currency.
Key Takeaways from this lesson
- An environmental scan describes what currently exists in a defined setting, such as programs, services, policies and organizations, so that an organization can plan.
- Published definitions of scans vary, and recurring themes describe the scan as an instrument of discovery, a form of knowledge synthesis, forward-looking and a support for decisions.
- Scanning began in business planning, and Aguilar's modes range from undirected viewing to formal search; a planned environmental scan is a formal search.
- Most scans combine passive methods, such as documents, websites and directories, with active methods, such as surveys of organizations and key informant interviews.
- A program inventory lists every candidate program once, separates listings from programs, and records a decision with one ordered reason for every exclusion.
- Because no database indexes community programs, a scan builds its sampling frame from several sources and uses the overlap among them as a signal of completeness.
- A census suits cheap methods and small populations, while costly methods such as interviews use purposive samples chosen for maximum variation.
- A charting template with a data dictionary, a source for every entry and a date of verification makes scan data comparable, checkable and updatable.
- Shared categories let a scan and a review be integrated in a joint display that shows where local practice and research evidence coincide or diverge.
- A scan report states every denominator and source, shows a flow of programs from identification to participation, names its limitations and gives a date of currency.
Core Concepts Reviewed
Section 1: definitions of the environmental scan, Aguilar's modes of scanning, uses in health services and public health, passive and active scanning, codified and tacit knowledge, and how scans differ from reviews, needs assessments, asset maps and evaluations.
Section 2: document and website review with saved copies, the program inventory with listings and ordered exclusion reasons, organizational surveys with contact plans and response rates, key informant interview guides, and ethics, consent and Indigenous data governance.
Section 3: the unit of a scan, eligibility criteria for programs, sampling frames built from several sources, census and purposive sampling, jurisdictional scans, and charting templates with data dictionaries and piloting.
Section 4: integration of a scan with a review at the design, methods and interpretation levels, joint displays, counts with denominators, framework analysis of interview notes, triangulation, and reporting a scan with a flow of programs and a date of currency.
The final reflection asks you to design a complete environmental scan for a new planning question, drawing on all four sections of the lesson.
Reflection
A health authority plans a community connector program for adults living with chronic pain and asks for a rapid scoping review and an environmental scan within ten weeks. The review asks which community-based self-management and social programs have been evaluated for adults with chronic pain, and with what outcomes. Design the scan. Address: (a) the scan question and the unit of the scan; (b) two eligibility criteria that parallel the review's criteria; (c) at least three sources for the sampling frame and how you would judge its completeness; (d) which methods you would use, and whether each uses a census or a sample; (e) five charting fields, including how you would record the source and date of each entry; (f) how you would integrate the scan with the review; and (g) three items your report must include, with one likely limitation.
(a) Which organizations currently deliver community-based self-management, peer-support or social programs for adults living with chronic pain in the province, and how are those programs designed, funded and evaluated? The unit is the program, with the operating organization recorded.
(b) Population: programs whose participants are mainly adults living with chronic pain. Concept: organized, recurring programs with a self-management, peer-support or social component; clinical treatment alone is excluded, matching the review.
(c) The provincial community services directory, chronic pain society and peer-support network lists, and health authority resource lists, with referrals from survey respondents. I would count how many eligible programs appear in two or more sources; many single-source programs would signal missing ones.
(d) Documents and websites for every eligible program (census); a short survey of every program (census); and six to eight key informant interviews chosen for maximum variation by program type, region and operator.
(e) Main activity, using the review's categories; referral routes; funding and its end date; evaluation status; and outcome measures. Each entry carries its source code and the date it was last verified, with conflicts noted.
(f) Shared categories allow a joint display setting the review's evidence beside the number of local programs and their evaluation status.
(g) A flow of programs from listings to participants, a response rate with numerator and denominator, and a date of currency. A likely limitation is that small volunteer-run peer groups are under-listed and less likely to respond.
Minimum 30 characters required.
Final Knowledge Assessment
Question 1: Which feature best distinguishes an environmental scan from a literature review?
Question 2: Which description fits the environmental scan reported by Rowel and colleagues (2005)?
Question 3: According to Graham, Evitts and Thomas-MacLean (2008), tacit knowledge lives in people's experience. Which source is most likely to yield tacit knowledge about a program?
Question 4: A scan finds a home support service that offers only housekeeping and yard work to older adults in the community, and it is currently operating. Using the Cedar Valley ordered exclusion reasons, which reason applies?
Question 5: A program's website describes weekly meetings, and its more recent survey response reports meetings every two weeks. Under the Cedar Valley coding rules, what does the team record?
Question 6: A scan invited 25 organizations to complete a survey, and 15 did so. Which sentence reports the response in the way this lesson recommends?
Question 7: In the terms used by Fetters, Curry and Creswell (2013), using the review's intervention categories to write a scan survey item is integration at which level?
Question 8: In the Cedar Valley joint display, the community connector model had very low certainty evidence for loneliness and four counterparts in British Columbia, two of which measured outcomes. Which implication does the display support?
Question 9: A health authority plans a scan with surveys and interviews to design one of its own programs. Which approach to research ethics is appropriate?
Question 10: What do the First Nations principles of OCAP assert?
Question 11: Which description best fits a sampling frame?
Question 12: A scan completed in 2026 states that its information is current to March of that year. How should a planner reading it in 2028 treat its findings?
Question 13: In the review by Charlton and colleagues (2021), what did just over half (53 percent) of the included studies do?
Question 14: A scan asks how each Canadian province and territory funds social prescribing. What is the most suitable unit for this scan?
Question 15: Before charting all programs, two team members chart three programs independently, each with 18 entries, and agree on 45 entries. What is their percent agreement?
Glossary: Key Terms, People & Frameworks
📚 Reference page, available throughout the lesson
Definitions of the terms, tools and people introduced in this lesson on environmental scans.