The NCCHPP presents a few examples of use and adaptation of its Framework for Analyzing Public Policies by different public health actors in Canada. Please find below six examples collected during a call for stories, launched in December 2017.
These examples aim to facilitate network-building among public health actors through the discovery of common interests and/or challenges encountered. Through these stories, you can observe how some public health actors have made the NCCHPP’s analytical framework their own by adapting it to suit their needs; perhaps it might inspire other public health colleagues to use it to carry out their own projects.
- the context in which the NCCHPP’s analytical framework was used;
- the adaptations made;
- if appropriate, the data method collection used;
- the lessons learned;
- the impact on public policies, programs and/or interventions;
- if appropriate, the next steps.
Also, if any resources have been published relating to the projects, we provide links so that you may access them (in their language of publication, in French or in English).
Example 1: Presenting the Results of an Intervention for Controlling Ragweed to Municipal Policy Makers
Translation of an example presented by Elisabeth Masson
Centre Intégré de Santé et de Services Sociaux de la Montérégie-Centre, Direction de santé publique [Integrated health and social services centre for Montérégie-Centre, public health authority]
Context of use
Possessed of these conclusive results, we used the NCCHPP’s analytical framework to translate the results into a more suitable format for more effectively reaching the target audience, namely municipal policy makers in Québec. Several of the NCCHPP’s dimensions for analyzing public policies match up with those of municipal administrations.
Adaptations made
Ultimately, five of the framework’s dimensions remained unchanged while another was adapted (Unintended effects was replaced by Unexpected effects) in favour of a term that would be more evocative for the target audience.
Data collection methods
The NCCHPP’s analytical framework was not used for our data collection, but rather for the presentation of the results. For details on the data collection methods used in the context of our project, please see the scientific summary referenced under the section heading To learn more.
Lessons learned
Our study lent itself well to the use of the NCCHPP’s analytical framework. Synthesizing the data on the effects produced by the project (soon to result in a municipal policy!) proved relatively simple given their scientific robustness.
Impact on public policies, programs and/or interventions
To learn more:
- Brochure intended for municipal policy makers which relies on the NCCHPP’s analytical framework:Direction de santé publique de la Montérégie, Agriculture et agroalimentaire Canada, & Ministère de la santé et des services sociaux du Québec. (2011). Réduire le pollen de l’herbe à poux : mission réaliste — Le succès d’une communauté mobilisée. Longueuil, Canada: Agence de la santé et des services sociaux de la Montérégie. Retrieved from: http://publications.msss.gouv.qc.ca/msss/fichiers/2011/11-244-02.pdf
- Scientific summary of the project: Direction de santé publique de la Montérégie, Agriculture et agroalimentaire Canada, & Ministère de la santé et des services sociaux du Québec. (2012). Projet herbe à poux 2007-2010 – Résumé scientifique – Phase 1 – Évaluation de l’efficacité d’un projet de mobilisation pour la lutte contre l’herbe à poux sur la qualité de vie des personnes allergiques. Retrieved from: http://publications.msss.gouv.qc.ca/msss/document-000549/
Agriculture and Agri-Food Canada. (2011). The effects of cutting on pollen and seed production in ragweed. Retrieved from: http://www5.agr.gc.ca/resources/prod/doc/pmc/pdf/fs_herbe-a-poux_eng.pdf
Example 2: Producing a Knowledge Synthesis on the Effectiveness of Health Warnings on Sugary Drink Containers
Adaptations made
The NCCHPP’s framework was adapted, not only to take into account the type of evidence available in the scientific literature, but also, most importantly, to take into account the perspective adopted for this project, in which health warnings were viewed as an intervention strategy aimed at modifying the socio-cultural environment. Political and economic dimensions were not the central focus of this study; so, even though they were taken into account, they were not subjected to a thorough analysis.
Together with the NCCHPP’s analytic framework, the main resources used were:
- A statement of criteria for evaluating the effectiveness of warning labels (article by Argo and Main, Meta-Analyses of the Effectiveness of Warning Labels);
- The theoretical and conceptual guidelines proposed by Baril and Paquette for the study of social norms related to diet;
- reference framework from the Ministère de la Santé et des Services sociaux du Québec: Pour une vision commune des environnements favorables à la saine alimentation, au mode de vie physiquement actif et à la prévention des problèmes reliés au poids.
Example 3: Reviewing and Assessing Nutrition-Related Chronic Disease Prevention Interventions
Sheila Tyminski, Population Public Health Strategy Director, Alberta Health Services
Context of use
Adaptations made
The recommendations we provided for each intervention were largely based on the analytical dimensions of effectiveness, equity, and unintended effects. As the dimensions of cost, feasibility and acceptability of an intervention will depend on implementation strategies and the local context, we advised readers of our report to undertake a situational analysis to include these dimensions prior to making decisions about whether to implement an intervention to ensure it fits their local context and target audience. These dimensions were not weighted as part of our analysis.
We also compared the NCCHPP’s framework to the GRADE approach for reviewing evidence and found alignment, as they looked at similar dimensions.
Data collection methods
We collected data from systematic reviews. We primarily looked for evidence on effectiveness, but we pulled out themes about equity, unintended effects and other considerations where relevant. The data extracted and conclusion statements were shared with content experts to help validate our findings.
Lessons learned
Our advice for public health colleagues who are considering using this framework: when planning interventions, it will help if the framework’s dimensions are considered at the beginning of the project.
Impact on public policies, programs and/or interventions
Our evidence review will help inform chronic disease prevention (CDP) stakeholders in Alberta, primarily policy and program planners and decision makers, of the current evidence on nutrition-related CDP interventions. The information can help support them with priority setting and planning.
Next steps
The report is available at https://www.albertahealthservices.ca/info/Page15343.aspx.
Please email publichealthnutrition@ahs.ca if you have any questions about this work.
Example 4: Producing a Comparative Analysis of Federal Regulatory Options for E-Cigarettes in Canada
An example presented by Suraj Chavda
Student, Master of Public Health, University of Alberta (degree completed in 2017)
Intern, Canadian Cancer Society (Summer 2015)
Context of use
The NCCHPP’s framework was used for my capping project for my Master of Public Health in Health Policy and Management at the University of Alberta. The project’s goal was to analyze four potential policy solutions for the Canadian federal government’s e-cigarette regulation framework and to make an evidence-based recommendation.
The NCCHPP’s framework (the Practical Guide) was provided to me by my political science professor, Dr. Kevin Wipf. This project was completed during my internship at the Canadian Cancer Society, under the guidance of Angeline Webb, Senior Policy Advisor at the Canadian Cancer Society, Alberta & Northwest Territory Division.
This analysis was done prior to the 2015 federal election. Up to that moment, little had been done yet to regulate e-cigarettes in Canada, thus there was a political need for this analysis. There were municipal and provincial movements to regulate e-cigarettes, but a federal stance had not been taken. Thus, there was a policy gap, as there were specific legal domains that fell within federal jurisdiction that needed to be addressed.
The primary source of the four e-cigarette policy options that I had analyzed was the 2015 House of Common’s Standing Committee on Health report on e-cigarettes in Canada entitled, Vaping: Towards a Regulatory Framework for E-Cigarettes.
I applied the NCCHPP’s framework to these four policy options and then I compared the analytical dimensions between the four options to determine which was the best recommendation. When comparing the outcomes I used a simple scoring system (+,++, +++) to rank the options. Translating the analytical dimensions to ascoring system was based on the pros and cons and I applied the score based on what I thought the major benefits and drawbacks were. I did not use any rigid translation criteria. Below is the results table of my analysis using the NCCHPP’s framework from my capping project.
Assessment of Regulatory Frameworks using the NCCHPP’s Framework
Effectiveness | Unintended Effects | Cost | Feasibility | Acceptability | |
Tobacco Act | + + | + + | + + | + + | + + |
Medical Devices Regulations | + + + | + | + + + | + | + + |
Canada Consumer Product Safety Act | + | + + + | + | – – – | – – |
New E-cigarette Policy (Recommended) | + + + | – | + + + | + + + | + + + |
Adaptations made
I chose not to use the Equity analytical dimension in this analysis as the impact of e-cigarettes on vulnerable populations had not been discussed extensively in the literature thus it would not have contributed substantially to the analysis.
Data collection methods
A literature review was completed to serve as the scientific basis for many of the conclusions from the analysis. These papers were found by searching the literature from databases such as MEDLINE and PubMed. I used key terms such as “e-cigarettes”, “policy” and “tobacco renormalization” among many others. I also used grey literature, legislation in Canada (provincial and municipal) and position statements from other organizations to inform my analysis.
Lessons learned
The NCCHPP’s framework is a practical tool that guided this analysis for real-world applications. It was not limited to specific policy theories and allowed the conclusions to be made logically rather than through subjective interpretation. The framework was multidimensional which fostered a well-rounded analysis. I find this framework gave my conclusions credibility. My analytical experience was limited before I used this framework. After this project, I feel I can confidently assess policies and their potential consequences.
My advice for public health colleagues who are considering using the NCCHPP’s analytical framework would be to ensure you are using peer-reviewed evidence when doing so. It is acceptable in my opinion to use grey literature or other sources; however, peer-reviewed evidence will provide a stronger foundation for your arguments when applying this framework. Although I did not include equity in my analysis, I did justify it based on the current evidence.
Impact on public policies, programs and/or interventions
I presented my capping project at the National Conference on Tobacco or Health in Ottawa in 2016. It was very well received by the audience. My paper was submitted to a representative from Health Canada who was working formulating e-cigarette policies at the time. Currently legislation is in development that is in concordance with the main recommendations of my analysis. It should be noted, however, that I have not yet heard back from Health Canada as to whether they have used my analysis. My analysis was also shared with a policy representative at the Canadian Dental Hygienists Association.
To learn more:
Chavda, S. (2016). An analysis of federal regulatory options for e-cigarettes. Presentation made on March 1st, 2016 at the National Conference on Tobacco or Health [PowerPoint slides].
Example 5: Facilitating a Deliberative Process Focused on Interventions for Promoting Healthy Lifestyles and Preventing Obesity
Yann Le Bodo, Research Officer, Plateforme d’évaluation en prévention de l’obésité [PEPO—platform for the evaluation of obesity prevention], Centre de recherche de l’Institut universitaire de cardiologie et de pneumologie de Québec – Université Laval [CRIUCPQ – UL—research centre, university institute of cardiology and pneumology of Québec City at Laval University]
Chantal Blouin, Researcher, Direction du développement des individus et des communautés, Institut national de santé publique du Québec [INSPQ—Québec’s public health institute]
Nathalie Dumas, Sociologist, research and communications, contractual research officer
Philippe De Wals, Scientific director, PEPO, CRIUCPQ – UL
Johanne Laguë, Assistant director of scientific programming and quality, Direction du développement des individus et des communautés, INSPQ
Context of use
The NCCHPP’s analytical framework was used in the context of a deliberative process integral to the “Comment faire mieux?” [CFM—how can we do better?] project. This project was coordinated jointly by the Plateforme d’évaluation en prévention de l’obésité [the platform for the evaluation of obesity prevention] at the Institut universitaire de cardiologie et de pneumologie de Québec – Université Laval [the university institute of cardiology and pneumology of Québec City at Laval University] and the Institut national de santé publique du Québec [Québec’s public health institute]. The project, which was carried out in 2014-2015, had two main objectives:
- Establish a portrait of the Québec experience in promoting healthy eating habits and physical activity and in preventing obesity since 2006: this objective was fulfilled through a documentary search and a synthesis report spotlighting 166 public interventions in Québec and 120 international recommendations.
- Identify avenues for improving the implementation of current or future interventions: this objective was fulfilled through a deliberative process. It was at this point that the NCCHPP’s analytical framework was used.
Subsequently, the CFM project team formulated 50 avenues for improvement, taking into account both the synthesis report and the results of the deliberative process.
Adaptations made
In the context of the CFM project, in addition to the NCCHPP’s six analytical criteria associated with the effects (effectiveness, unintended effects, equity) and the applicability of interventions (cost, feasibility, acceptability), we considered two cross-cutting criteria: the sustainability of interventions and their evaluability.
These eight criteria were used in a flexible manner during the deliberative process: they were not systematically assessed when a proposal was made, but participants were invited to refer to these criteria when expressing their views and, afterward, when prioritizing the proposals that had been formulated.
Data collection methods
The CFM deliberative process brought together a group of 25 participants representing a diversity of stakeholders (experts, representatives of different government departments and of civil society organizations), who were invited based on criteria such as experience in health promotion, scientific excellence and experience with decision-making processes. The process involved two steps:
- A forum held over two days (December 2014) during which the content of the synthesis report prepared by the CFM project team structured exchanges. This report had been shared with the participants prior to the forum, along with an abridged version, as recommended by the NCCHPP’s Method for Synthesizing Knowledge About Public Policies. The participants appreciated this tool as it fostered reflection. At the forum, the six criteria of the NCCHPP’s framework were presented. Participants were able to refer to these criteria when expressing their views, as were the facilitators when summarizing the discussions. Throughout the design and implementation of the deliberative process, the CFM project team also benefited from the advice of an expert from the McMaster Health Forum.
- At the forum’s conclusion, the participants and facilitators agreed on the need to prioritize the proposals that had been formulated. Therefore, an online survey was conducted among the 25 participants in the forum so that they could express their views regarding a list of proposals generated by the CFM project team based on the suggestions made at the meeting. Respondents were invited to refer to the criteria in the NCCHPP’s framework along with the cross-cutting criteria of sustainability and evaluability; definitions of these various criteria were available for reference throughout the survey.
Lessons learned
Given the very flexible manner in which the NCCHPP’s framework was used in the context of this project, its use posed no real difficulties. However a more systematic use of the framework would probably have required some reflection concerning the relevance and feasibility of using tools for collecting or analyzing data that would allow for a criterion-based analysis of each proposal.
Impact on public policies, programs and/or interventions
To learn more
Complete work and summary:
- Le Bodo, Y., Blouin, C., Dumas, N., Laguë, J., & De Wals, P. (2016). L’Expérience québécoise en promotion des saines habitudes de vie et en prévention de l’obésité: comment faire mieux? Québec, Canada: Presses de l’Université Laval. Retrieved from: http://iucpq.qc.ca/fr/comment-faire-mieux
Articles:
- Blouin, C., Robitaille, É., Le Bodo, Y., Dumas, N., De Wals, P., & Laguë, J. (2017). Aménagement du territoire et politiques favorables à un mode de vie physiquement actif et à une saine alimentation au Québec. Lien social et Politiques, 2017(78), 19-48. https://www.erudit.org/fr/revues/lsp/2017-n78-lsp03015/1039337ar.pdf
. - Le Bodo, Y., Blouin, C., Dumas, N., De Wals, P., & Laguë, J. (2017). The Quebec experience in promoting healthy lifestyles and preventing obesity: How can we do better? Obesity Reviews, 18(9), 967-986. http://dx.doi.org/10.1111/obr.12559
Example 6: Embedding the Framework Within an Organizational Policy Development and Decision-Making Tool
An example presented by:
Dr. Julie Kryzanowski, Deputy Medical Health Officer, Population and Public Health, former Saskatoon Health Region
Jill Aussant, Public Health Nutritionist, Population and Public Health, former Saskatoon Health Region
Carolyn Chu, Senior Public Health Nutritionist, Population and Public Health, former Saskatoon Health Region
Tanya Dunn-Pierce, Health Promotion Department Manager, Population and Public Health, former Saskatoon Health Region
Adaptations made
We did not adapt the NCCHPP’s analytical framework. However, we use it in conjunction with other resources and tools:
1. As our staff work with the HPD Policy Development and Decision-Making Tool they are encouraged to apply the PPH Policy Framework and consider the following factors:
- How the policy being considered aligns with the department’s priorities and approaches;
- How the policy connects to PPH’s health status reports and Chief MHO Calls to Action;
- How the policy aligns with the Truth and Reconciliation Commission of Canada’s Calls to Action;
2. Other suggested tools for practitioners such as:
- How to Identify Policy Options (drawn from the European Commission’s Better Regulation Toolbox)
- Policy Approaches to Reducing Health Inequalities (NCCHPP)
- A Framework for Analyzing Public Policies: A Practical Guide (NCCHPP)
- Critical Appraisal Tool AGREE II (Appraisal of Guidelines for Research and Evaluation)
- Selected Tools to Facilitate the Integration of Health in All Policies (NCCHPP)
- Policy Readiness Tool (PLACE Research Lab, School of Public Health, University of Alberta)
Data collection methods
We have not used the NCCHPP’s framework in our “day to day” work yet. It does look like a thorough framework and we anticipate that we will collect data to analyze the policy directions and options when working through our vetting tool. We anticipate that we will use group brainstorming, consultation with experts and deliberative processes most often.
Lessons learned
We familiarized ourselves with the NCCHPP’s framework by reading the Practical Guide and completing the Online Course. It is a helpful tool to conceptualize the dimensions of policy development. We really liked the combination of analysis related to the effectiveness of the policy as well as the considerations for implementation and strategy development (for instance, feasibility).
Our recommendation for public health colleagues would be: use the NCCHPP’s framework in a way that fits your organization. We integrated it into our vetting tool to add another layer of analysis.
Impact on public policies, programs and/or interventions
We are at the early stages of implementing the HPD Policy Development and Decision-Making Tool. A workshop for HPD staff was conducted to orient the staff to the tool and have them work through using the tool with policies related to their areas of work. Feedback was collected from staff, and the tool, along with the standard work document, is now ready for implementation.
Next steps
- Trialing a few policies using the policy vetting tool (e.g., Complete Streets and Respectful Workplace policy);
- Feedback from Managers;
- Once approved, guiding staff to know about the tools contained within the HPD Policy Development and Decision-Making Tool such as the NCCHPP analytical framework.
To learn more
- The Population and Public Health Policy Framework developed at Saskatoon Health Region and brought to the Saskatchewan Health Authority.