Healthcare systems

Healthcare Quality Excellence Diploma

1
2
 Public policy
Health policy Health public policy
Public health policy Health sector policy HiAP

3
Please use the following citation when referencing this document:

Region of Peel – Public Health. Structure of public health: A literature review. Mississauga, ON: Region of Peel – Public Health; 2019.

4
1 Issue & Context
The 2019 provincial budget proposed substantive changes to the public health sector in Ontario. 1 These include:

 Establishing 10 regional public health entities and 10 new regional boards of health (BOHs) with one common governance
model by 2020-2021.
 Adjusting the provincial-municipal cost-sharing of public health programs.
 Streamlining the Ontario Agency for Health Protection and Promotion (Public Health Ontario) to enable greater flexibility with
respect to non-critical standards based on community priorities.1

Currently there are 35 local public health units in Ontario operating under different governance models (e.g., autonomous,
regional/municipal, semi-autonomous). The Ontario Public Health Standards (2018) outline the requirements with respect to the core
public health functions of:

 assessment and surveillance;

 health promotion and policy development;
 health protection;
 disease prevention; and
 emergency management.2

The provincial government has proposed geographic boundaries for the 10 regional public health entities (RPHEs), including:

 six large, urban RPHEs with a population over one million;

 one Toronto RPHE; and
 three rural/northern RPHEs with a population less than one million.

The intent is to have autonomous BOHs for the 10 new RPHEs in place by April 1, 2020.

As part of a Council of Medical Officers of Health (COMOH) working group, Peel Public Health (PPH) conducted this literature review to
prepare for upcoming consultations with the public health sector.

2 Literature Review Question

What components of public health organization or structure contribute to public health performance (e.g. achievement of core public
health functions and/or impact on population health status)?

3 Literature Search
An iterative search of published literature was conducted to scope the evidence and refine search terms. Frequently used subject headings
(e.g., MeSH terms) to index public health services and systems research were identified. Due to time constraints these terms were used to
focus the search.

The final search was conducted on June 14, 2019. Databases searched were: Cochrane Database of Systematic Reviews, Evidence-Based
Practice and Health Technology Assessment, National Health Service (NHS) Economic Evaluation Database, Global Health, Ovid
Healthstar, MEDLINE, MEDLINE In- Process, Cumulative Index of Nursing and Allied Health Literature (CINAHL), and Health Business

5
Elite.

6
The search was limited to English-language and articles published in the past 10 years. A filter for synthesized literature was
applied to all databases.

A search of unpublished (grey) literature was conducted on June 20, 2019. Resources searched were: McMaster Health Forum, Health
Systems Evidence, Public Health Services and Systems Research, the National Institute for Health and Care Excellence (NICE) Evidence
Search, Turning Research into Practice (TRIP) database, Australian National University – Research School of Population Health
publications, Sax Institute, Ontario Public Health Libraries Association (OPHLA) Customized Google Search Engine, Google and
DuckDuckGo.

Key informants with expertise in systematic reviews, public health, and/or health systems research were contacted. Authors of relevant
documents were also contacted to request more information about their review methodology and to suggest additional evidence.
Reference lists of relevant documents were scanned (Appendix A).

4 Relevance Assessment
Search results were assessed using the following relevance criteria:

 Inclusion criteria
o public health context
o describes organizational, structural or institutional components of public health organizations, agencies or
systems (e.g. jurisdiction, governance, infrastructure, workforce, leadership)
o performance outcome (e.g. achievement of public health functions, delivery of effective public health interventions,
efficient use of resources, equitable access to services and/or population health status)
o setting similar to Canada (e.g. United States, United Kingdom, New Zealand, Australia)
o synthesized literature (e.g. reviews or “review of reviews”)

 Exclusion criteria
o discussion paper or commentary

Two reviewers (JM and RS) independently screened the titles and abstracts of the published literature. One reviewer (RS) screened the titles
of grey literature hits while conducting the search. Any potentially relevant results were then screened by two independent reviewers (JM and
RS). Disagreements were discussed until consensus was reached or in consultation with a third reviewer (BB).

5 Results of the Search

The searches yielded 1105 results. After removal of duplicates, the titles and abstracts of 1102 documents were assessed for relevance, of
which 1090 were excluded. Twelve articles were reviewed in full by two independent reviewers (JM and BB). Two review of reviews, one
by Brownson and collegues3 and one by Hyde and Shortell4, were relevant. One systematic review by Dilley and colleagues 5 was also
relevant. Two literature reviews that incorporate expert consultation were also identified. 6,7 Despite a lack of detailed methods, the review
team determined that these documents by Carlson et al.6 and

7
Naylor and Buck7 provided valuable information relevant to the review question and they were included (Appendix B).

Overlap of the studies/reviews included in these relevant documents was assessed. One review of reviews by Brownson and colleagues3
included two of the other relevant articles by Hyde and Shortell4 and Dilley and collagues.5 The review team decided to extract data on all
three documents3,4,5 since they provide additional detail not available in the review of reviews by Brownson et al.3

6 Critical Appraisal
Two reviewers (JM and RS) independently appraised the quality of two review of reviews and the one systematic review using the Health
EvidenceTM Quality Assessment tool.8 Disagreements were discussed until consensus was reached. Both review of reviews by Brownson et
al.3 and Hyde and Shortell4 rated moderate (5/10). The systematic review by Dilley et al.5 received a weak rating (4/10). Limitations of all
three documents were: a lack of quality assessment of included studies/reviews, unclear weighting and failure to consistently provide the
data from included studies/reviews when describing the findings.

There was not a suitable tool to appraise the two literature reviews. 6,7 These documents lacked: a transparent description of the search
strategy and selection of included papers, quality assessment, and consistent description of all included evidence. Some information about
the search strategy was provided by the author of one literature review upon request.7 Overall both documents should be considered of
weaker methodological quality.

7 Description of Included Documents

The most synthesized documents are presented first.

Brownson, R et al. (2012): Fostering more-effective public health by identifying administrative evidence-based
practices: A review of the literature3

The objective of this moderate quality review of reviews was to identify administrative evidence-based practices (A-EBPs) associated with
local public health performance. A-EBPs were defined as agency level structures and practices positively associated with public health
performance. Outcomes included performance of the local health department or public health system in the National Public Health
Performance Standards Program in the United States (US), implementation of evidence-based practices, workforce capability, achieving
service objectives for specific program areas (e.g., maternal/child health, immunization etc.) and population health outcomes. Details were
not provided about how outcomes were defined or measured. Thirty reviews and 65 single studies were included. Almost all studies were
conducted in the US. The designs of included studies were not described. The quality of included reviews and studies was not assessed.

The included evidence was synthesized narratively into macro (system)-level A-EBPs and micro (local)- level A-EBPs. Macro-level referred
to the infrastructure for local public health practice. Micro-level A- EBPs were described as administrative or management practices that
are modifiable in any local public health system. Micro-level A-EBPs were the focus of this article and these were further categorized into
high and moderate priority. High priority A-EBPs were associated with an outcome of interest in numerous studies or at least one review
article; focused on local-level administrative or management change; and deemed modifiable in a short (<1 year) or medium (1-3 years)
time frame for a reasonable

8
cost. Moderate priority A-EBPs were associated with an outcome of interest in at least one study but no reviews; or were deemed to require
a longer time frame to modify (Appendix C).

Hyde, J & Shortell, S. (2012): The structure and organization of local and state public health agencies in the US: A systematic
review4

The objective of this moderate quality review of reviews was to describe the organization and structure of local and state public health
agencies in the US and determine the influence of organizational and structural characteristics on public health performance and/or health
outcomes. Health outcomes were assessed using population health status data. Performance outcomes were defined as the capacity to
provide the 10 essential public health services that were measured through national performance standard surveys or investigator-
developed surveys. There was variation across studies in the type and number of indicators used for each essential service. Seven reviews
and 54 single studies were included. All included studies were conducted in the US. Most studies were cross-sectional (n=36) in addition to
some with longitudinal (n=8) or case study designs (n=10). The quality of included reviews and studies was not assessed.

The included evidence was synthesized narratively into three categories: 1) descriptions of the structure and organization of governmental
public health agencies; 2) associations between public health structure, organization, and performance; and 3) relationship between public
health organization and health outcomes. Only evidence from the second and third category was relevant to this report (Appendix C).

Dilley, J et al. (2012): Quality improvement interventions in public health systems: A systematic review5

The objective of this weak quality systematic review was to identify quality improvement (QI) initiatives implemented in the US public
health system. Performance was assessed through practice improvements or population health outcomes. Details were not provided about
how these outcomes were defined or measured. Fifteen single studies about 18 separate QI interventions were included. All studies were
conducted in the US at state-level or large public health units. The designs of included studies were not described. The quality of included
studies was not assessed.

The included evidence was synthesized narratively into three categories: 1) organization wide QI interventions (Big QI) that used a systems
approach to influence numerous programs and services; 2) QI targeting specific program or services (small QI); and 3) QI of administrative
or management practices (mix of Big and small QI). (Appendix C)

Carlson, V et al. (2015): Defining the functions of public health governance.6

The objective of this literature review was to determine if accepted governance functions continue to reflect the role of public health
governing entities in the US. The desired outcome was a list of governance functions that describes how governing entities support and
guide public health service. This could be used use alongside public health core functions and services to provide insight into how
governing entities participate in the public health system. Two categories of literature were reviewed: foundational works (n = 3) and
additional works that address board of health functions (n = 44, including the three reviewed under foundational works). Eighteen
orientation manuals from public health governing entities were also reviewed to support data triangulation. The quality of included
evidence was not assessed.

9
Six themes were identified that concerned the roles and responsibilities of governing boards. These were sent to individuals with expertise
in public health governance or health department operations for review. Feedback was received through facilitated discussion and
consensus was built to create the final list of public health governance functions and their definitions (Appendix C).

Naylor, C. and Buck, D. (2018): The role of cities in improving population health: International insights.7

The objective of this report was to explore how England’s cities can govern more effectively to influence population health. This report
drew on the experience of several international cities and included specific examples of how these cities have approached complex health
issues such as: obesity, HIV, air quality and mental health. The findings of this report were derived from: 50 in-depth qualitative interviews
(25 focused on London and 25 focusing on other international cities); a literature review of relevant evidence and data; and roundtable
discussion with experts to validate the findings.

Findings were synthesized into two components necessary for cities to influence population health: 1) governance arrangements and 2)
functions that cities or their partners can perform (Appendix C).

10
8 Synthesis of Findings
A conceptual framework by Handler and colleagues9 was adapted to organize the findings of this literature review. The review
team also considered two articles that built on this framework10,11

Figure 1 Adapted from Handler (2001),9 Meyer (2012)10 and Guyon (2016)11

11
Macro-level structural capacity

Governance

Public health governance functions and structures vary. Public health governing bodies can support and guide public health performance
through six functions of public health governance:

1. leading or contributing to policy development;

2. ensuring adequate resources are available;
3. monitoring improvements;
4. engaging with partners;
5. exercising legal authority; and
6. assuming responsibility for public health performance.6

Governance structures were identified as an important factor influencing a city’s ability to have an impact on population health.7
Centralized governance models have the potential to be more efficient, coordinated and allow for costs to be shared and resources to be
redistributed.7 In contrast, this could decrease engagement with community members and responsiveness to local needs.7

Consistent with some of the functions of public health governance described above, effective partnerships between local, city and
regional tiers of government are required for cities to have an impact on population health.7 In addition, city leaders must understand
and use the regulatory powers within their control to impact population health.7

The relationship between governance and public health performance is inconclusive

A governing board of health with a policy making role was positively associated with performance of essential public health services.4
The review authors note this finding may be limited to larger
jurisdictions with a population ≥100,000 people.4 Organizational control in public health organizations can be centralized, decentralized or
mixed.4 There were varied findings on the impact of organizational control on public health performance. In some studies, centralization of
authority was associated with better public health performance compared to decentralized or mixed structures.4 In other studies shared
state and local authority was associated with improved public health performance. 4 There were also studies that found no relationship
between organizational control and public health performance.4

Boundaries

Jurisdictional size predicts public health performance

In many studies, the size of the jurisdiction served by a public health organization had a strong association with public health
performance.3,4 Overall, public health organizations with larger jurisdictions performed better than smaller ones.4 When public health
organizations served a larger population they had increased capacity to provide essential services.4 Most included studies did not report an
optimal jurisdictional size, however in studies that did, population size ranged from ≥50,000 to 100,000 people.4 In one included study,
population size was positively associated with performance up to 500,000 people but beyond that public health performance declined.4
There was not enough evidence to determine whether regionalization of public health services is associated with improved public health
capacity and performance.4

12
Finance

Per capita funding and public health expenditures predict public health performance

Public health finances have a strong association with public health performance. 3 Financial considerations occur both at the macro and
micro levels of public health systems.3 At the macro-level, both expenditures per capita and expenditures per staff full-time equivalents
(FTEs) are associated with improved public health performance.3,4 At the micro-level, funding allocation and fiscal priorities can influence
public health performance.3 These can include allocation of resources to quality improvement, innovation, information access, and
training.3

Workforce size and composition

Workforce size and composition are associated with public health performance

Public health organizations with more staff perform better than organizations with fewer staff. 3,4 Specifically, a public health workforce
with a high proportion of staff relative to the size of the population served is associated with better performance for most essential
services.3,4 A workforce with a mix of disciplines and diverse experience and training is also positively associated with performance. 3 The
distribution of public health expertise at various levels of the public health system is another important consideration when trying to
influence population health.7

Micro-level structural capacity

Workforce Development

Workforce development can support public health performance

At the micro-level, a skilled and competent workforce is essential to the performance of public health organizations.3 Providing staff with
opportunities for professional development and access to technical assistance are considered high priorities.3 Workforce training is often
on-the-job and competency- based.3 The educational background(s) and competencies of the public health workforce must be
considered.3

Leadership

The relationship between leadership and public health performance is inconclusive

Leadership is frequently cited as having an important influence on public health performance however findings are mixed.4 The association
between the educational training of public health leaders and public health performance is inconclusive.4 Several leadership characteristics
were identified as high priority.3 This included public health leaders’ skills and background; values and expectations; and use of participative
decision-making.3

At the city level, leaders can have significant impact on the social determinants of health through formal and informal powers.7 Outside of
formal governance structures and powers, leaders can influence population health by networking, partnering with others, and creating a
culture of learning, innovation and continuous improvement.7 Leadership style and skills should fit with the model of governance (e.g.
consensus-based decision-making).7

13
Organizational culture

Organizational culture can support public health performance

Organizational culture was identified as a high priority element of structural capacity that can influence public health performance.3
Leaders help to shape organizational culture through free flow of information, support for innovation and creating a learning
organization.3

Interorganizational relations

Interorganizational relationships can support public health performance

Relationships and partnerships are essential to a strong public health system. 3,4 Involving outside organizations in the planning and
provision of public health services is associated with improved public health performance.3,4 This could include schools, hospitals, social
services, community organizations, businesses, law enforcement and academic organizations.3

Cities need to be able to link parts of the system internally as well as connect to other cities and external partners to impact population
health.7 These connections are useful to exchange lessons learned or to address shared problems.7

Infrastructure

Physical infrastructure (e.g. facilities and equipment) is essential to public health capacity. However, the evidence reviewed did not include
research on the relationship between infrastructure and public health performance.

Data and information sources

Access to information for evidence-informed decision-making is an element of structural capacity that influences public health
performance.3 The are other important information sources and technologies that could impact the structural capacity of public health,
however the evidence reviewed did not include research on the effect of these on public health performance.

Quality improvement

Quality improvement (QI) processes are essential to assessing public health performance.5 QI
interventions can be used to “improve the efficiency or effectiveness of a program, process or
organization.”5 QI processes require careful consideration of the measures of public health performance being examined.5 Most included
studies did not link public health performance directly to population health outcomes. 5 There is a need to establish whether achievement
of a public health performance measure will improve population health.5

9 Limitations
This literature review was conducted within a short time frame. The search of unpublished (grey) literature was not exhaustive. The
search strategy for published literature was narrowed using subject headings and further refined with keywords. This could have missed
potentially relevant articles that were not indexed using these headings.

14
The search was limited to synthesized evidence and most of the included reviews were published in 2012. Newer single studies relevant
to this topic (published since 2012) would not have been identified through this literature review.

The findings of this review are predominantly based on cross-sectional studies which cannot establish causal relationships.
Additionally, the quality of the studies included in each article was not assessed.

The included articles did not consistently provide: detail about how public health performance was defined and measured; or data
from the studies/reviews that they included.

Most single studies included in this evidence were conducted in the US. The generalizability of this research to the Canadian context
is unclear. There was also a lack of research available about public health structure for small or rural public health departments.4

10 Conclusions
The macro-level elements of structural capacity in a public health system must be considered when restructuring public health in
Ontario. These include:

 Governance function and structure

 Jurisdictional size and boundaries
 Finance
 Workforce size and composition

In addition, micro-level elements of structural capacity will need to be considered when forming regional and local public health
entities. These include:

 Workforce development
 Leadership
 Organizational culture
 Interorganizational relations
 Infrastructure
 Data and information systems

15
11 Acknowledgements

This literature review was conducted and written by staff at Peel Public Health

Authors

Jackie Muresan, Advisor-Knowledge Broker Rebecca

Spark, Advisor

Bev Bryant, Manager

Dr. Monica Hau, Associate Medical Officer of Health

Technical Support

Shant Alajajian, Librarian Specialist

Other Contributors

Kiana Torshizi, Health Promotor Inga

Pedra, Advisor

Kayla Carneletto, Coordinator

16
References

1. Government of Ontario. 2019 Ontario Budget: Protecting what matters most [Internet]. Queens’s Printer for Ontario; 2019 [cited
2019 June 28]. Available from: http://budget.ontario.ca/pdf/2019/2019-ontario-budget-en.pdf
2. Ministry of Health and Long-Term Care. Ontario Public Health Standards: Requirements for Programs, Services, and Accountability
[Internet]. Ontario; 2018. [cited 2018 June 28]. Available from:
http://www.health.gov.on.ca/en/pro/programs/publichealth/oph_standards/docs/protocols_guidel
ines/Ontario_Public_Health_Standards_2018_en.pdf
3. Brownson RC, Allen P, Duggan K, Stamatakis KA, Erwin PC. Fostering more-effective public health by identifying administrative
evidence-based practices. Am Prev Med 2012; 43(3):309-319.
4. Hyde JK, Shortell SM. The structure and organization of local state public health agencies in the U.S.
Am J Prev Med 2012; 42(5S1):S29-S41.
5. Dilley JA, Bekemeier B, Harris JR. Quality improvement interventions in public health systems. Am J Prev Med 2012; 42(5S1):S58-
S71.
6. Carlson V, Chilton MJ, Corso LC, Beitsch LM. Defining the functions of public health governance. Am J Public Health 2015;
105(S2):S159-S166.
7. Naylor C, Buck D. The roles of cities in improving population health: international insights [Internet]. TheKingsFund; 2018 June 15
[cited 2019 June 26] Available from:
https://www.kingsfund.org.uk/sites/default/files/201806/Role_cities_population_health_Kings_Fun d_June_2018_0.pdf
8. Health Evidence. Quality assessment tool – review articles [Internet]. [Updated 2018 October 26; cited June 24, 2019]. Available
from: https://www.healthevidence.org/documents/our-appraisal- tools/quality-assessment-tool-dictionary-en.pdf
9. Handler A, Issel M, Turnock B. A conceptual framework to measure performance of the public health system. Am J Public Health 2001;
91(8):1235-1239.
10. Meyer AM, Davis M, Mays GP. Defining organizational capacity for public health services and systems research. J Public
Health Manag Pract 2012; 18(6):535-544.
11. Guyon A, Perreault R. Public health systems under attack in Canada: evidence on public health system performance
challenges arbitrary reform. Can J Public Health 2016; 107(3):e326-e329.

17
Appendices

Appendix A: Search Strategy

Appendix B: Literature Search Flowchart Appendix

C: Data Extraction Tables

18
Appendix A: Search Strategy
Database: EBM Reviews - Cochrane Database of Systematic Reviews <2005 to June 5, 2019>, EBM Reviews - Health Technology Assessment <4th

Quarter 2016>, EBM Reviews - NHS Economic Evaluation Database <1st Quarter 2016>, Global Health <1973 to 2019 Week 23>, Ovid Healthstar

<1966 to April 2019>, Ovid MEDLINE(R) <1946 to June Week 2 2019>, Ovid MEDLINE(R) In-Process &

Other Non-Indexed Citations <1946 to June 12, 2019> Search Strategy:

1 exp Public Health Administration/ (29186)

2 exp Public Health/og [Organization & Administration] (40582)

3 Community Health Services/og [Organization & Administration] (17085)

4 ("organiz*" or "organisation*" or "system*" or "structur*" or "config*").ti,ab. (8295105)

5 ("review*" or "meta-analys*" or "synth*" or "overview*" or "guideline*").ti,ab,pt. (8414547)

6 ("performance*" or "effective*" or "efficien*" or "equit*").ti,ab. (5378336) 7 1 or 2 or 3

(85195)

8 4 and 6 and 7 (7727)

9 5 and 8 (2588)

10 remove duplicates from 9 (1291)

11 limit 10 to english language [Limit not valid in CDSR; records were retained] (1177)

12 limit 11 to yr="2009 -Current" (615)

***************************

19
Appendix A: Search Strategy (continued)
Search of CINAHL and Health Business Elite using EBSCO platform

20
Appendix A: Search Strategy (continued)
Grey Literature Final Results

Search terms: (“public health” or “population health”) AND (structur* OR organization* OR organisation*)

Source # Hits Scanned # Primary Potentially Full-Text

Relevant Hits Review
Health Systems Evidence 20 0 0
https://www.healthsystemsevidence.org/?lang=en
McMaster Health Forum 53 0 0
https://www.mcmasterforum.org/
Public Health Services and Systems Research (USA) 363 3 0
http://publichealthsystems.org/
NICE Evidence Search 50 14 1
https://www.evidence.nhs.uk/
Turning Research into Practice (TRIP) database 0 0 0
https://www.tripdatabase.com/
Australian National University: Research School of Population Health publications 0 0 0
http://rsph.anu.edu.au/research/centres-departments/australian-
primary-health-care-research-institute/projects

Sax Institute https://www.saxinstitute.org.au/publications/implementation- 50 0 0

research-publications/

Google Custom Search Engine (Ontario Public Health Libraries Association) 40 0 0

https://cse.google.com/cse?cx=007843865286850066037:3ajwn2jlweq

Google Search 50 0 0
DuckDuckGo Search 20 0 0

21
Appendix A: Search Strategy (continued)
Resources/Websites Solicited from Experts

Date Response?
Key Informant Organization/Position Method Recommended Resource(s)
Contacted (Y/N)
Dr. Maureen Scientific Director of the National 07-Jun-19 email & Y Knowledge broker at NCCMT conducted a search for literature on
Dobbins Collaborating Centre for Methods & Tools phone regionalization and population health outcomes – didn’t find any
and Health Evidence, McMaster University relevant review evidence but forward the following:
 Van Aerde (2016) Has regionalization of the Canada health system
contributed to better health?
(Not a review)
 Denis (2015) Is there a future for regionalization in Canada?
Presentation at CAHSPR conference
(Not a review. Seems to be focused on regionalization of entire health care
system and is not public health specific)
 Hays et al. (2012) Public health governance and population
health outcomes.
(Not a review – single study)
Dr. Mike Assistant Director, McMaster Health 03-Jun-19 email & Y Confirmed that McMaster Health Forum does not have the
Wilson Forum, McMaster University phone capacity at this time to create a rapid response brief for our
question.
Provided orientation to searching Health Systems Evidence
repository. No relevant records identified. Shared masters thesis
about public health systems:
 Jarvis (2017) Defining public health systems: A critical interpretive
synthesis of how public health systems are defined and classified.
(not relevant – not about org or structure of PH)

22
Dr. Ross Professor, Institute for Public Health, 12-Jun-19 email Y Provided a data extraction table from his review of reviews. Also
Brownson Washington University, St. Louis, recommended the following articles:
Missouri  Aarons et al. (2014) Aligning leadership across systems and organizations
to develop a strategic climate for EBP implementation.
(not relevant – not about org or structure of PH)

23
 Date Response?
Key Informant Organization/Position Method Recommended Resource(s)
Contacted (Y/N)
 DeCorby-Watson et al. (2018) Effectiveness of capacity building
interventions relevant to public health practice: a systematic review
(not relevant – not about org or structure of PH)
 Duggan et al. (2014) What influences the use of administrative evidence-
based practices in local health departments? (not a review – single study
& duplicate) (not relevant – not about org or structure of PH)
 Gyllstrom et al. (2015) Local health department factors associated with
performance in the successful implementation of community-based
strategies
(not a review – single study)
 Hyde & Shortell (2012) The structure and organization of local and state
public health agencies in the US: A systematic review (duplicate)
 Hilliard & Boulton (2012) Public health workforce research in
review: A 25-year retrospective
(not relevant – not about org or structure of PH)
 Nguyen et al. (2019) Factors associated with continuous improvement by
local boards of health
(not a review – single study)
 Xiao et al. (2018) Development of a survey to assess local health
department organizational processes and infrastructure for supporting
obesity prevention
(not a review – single study)

Dr. Justeen Institute for Community Health and 12-Jun-19 email N Requested data extraction tables from her relevant systematic
Hyde Harvard Medical School review. No response
Dr. Glen Mays Director, National Coordinating Center for 12-Jun-19 email N No response
Public Health Services and Systems
Research and Public Health Practice-Based
Research Networks, University of Kentucky

24
 Date Response?
Key Informant Organization/Position Method Recommended Resource(s)
Contacted (Y/N)
Chris Naylor Senior Fellow, Policy, The Kings Fund 25-Jun-19 email Y Provided additional information about the literature review methods
for the report: The role of cities in improving population health

Searched:
TKF IKS database https://koha.kingsfund.org.uk/cgi-
bin/koha/opac-main.pl

PubMed Google

Scholar

Provided search terms used on each site but no inclusion/exclusion

criteria

25
 Appendix B: Literature Search Flowchart

Bibliographic Database Records Grey Literature Records

n= 644 n= 461

Total Records Duplicates Removed

n=1105 n=3

Records Screened Records Excluded

(1ᵒ Relevance) n=1090
n= 1102

Full-Text Records Excluded with Reasons

Full-Text Review
Not about structure of public health (3)
(2ᵒ Relevance)
Discussion paper/commentary (3)
n = 12
Conceptual article (1)

Total Relevant Records

n=5

Result Type
Review of reviews (2)
Systematic Review (1)
Literature review with expert
consultation (2)

22
Appendix C: Data Extraction Tables
Brownson et al. (2012) Fostering more-effective public health by identifying administrative evidence-based practices: A review of the
literature
Type of Article (Design) Review of reviews
Quality Rating 5/10
 Quality assessment of included articles not completed
 Table of characteristics of included articles and outcome data not included
 Weighting of evidence unclear
 Findings not consistently linked to the supporting evidence and referenced
Objective(s) To identify agency level structures and practices (administrative evidence-based practices) associated with performance measures (e.g. achieving core public
health functions or carrying out evidence-based interventions) for
local public health (PH)
Included Evidence  Reviews (n=30)
o Most reviews were of studies conducted in the USA but also included some evidence from Canada, UK,
Australia and Europe
o Three reviews included studies from the UK only, two reviews included studies from Australia only
 Single studies (n=65)
o USA (n=62)
o Canada (n=2)
o Australia (n=1)
 Quality of included reviews and studies is unknown
Structural/Organizational Administrative evidence-based practices (A-EBPs)
Elements
A-EBPs were not clearly specified, but included:
 Organizational size and structure
 Organizational climate
 Leadership
 Facilities
 Setting (urban, rural, suburban)

23
 Finances
 Resources

24
Brownson et al. (2012) Fostering more-effective public health by identifying administrative evidence-based practices: A review of the
literature
 Workforce
 Partnerships
 Barriers and facilitators to evidence-based practice
Outcomes Any outcome linked to evidence-based decision making Outcomes were
not clearly specified, but included:
 Performance of the local health department or public health system in the National Public Health Performance Standards Program in the United
States
 Implementation of evidence-base practices
 Achieving service objectives for specific program areas (e.g. maternal/child health, immunization etc.)
 Performance of core public health functions and/or CDC’s 10 essential public health services
 Partnership effectiveness
 Workforce capability
 Population health status

No further details were provided about outcomes or how they were defined or measured

25
Findings Macro (system)-level administrative evidence-based practices (A-EBPs)
1. Health department oversight & infrastructure
Jurisdiction
o Population size
o Type (county, city)
Governance & authority
o Presence of local board of health
o Policy-making role (not advisory role) for local board of health (especially in jurisdictions with large population)
o Centralization of authority at state level or shared state & local control
2. Financial processes
Allocation and expenditure of resources
o Local health department (LHD) expenditures per capita
o LHD expenditures per staff FTE
o Diverse funding sources
o Per capita taxes or allocation percentage of local taxes to PH
3. Workforce size and composition
o Staff FTEs per capita

26
Brownson et al. (2012) Fostering more-effective public health by identifying administrative evidence-based practices: A review of the
literature
o Pre-service educational background, licensing and certification
o Mix of disciplines

Review authors classified micro A-EBPs as:

High priority (H) - associated with an outcome of interest in numerous studies or at least one review article; focused on a micro-level administrative or
management change; and deemed modifiable

Moderate priority (M) - associated with an outcome of interest in one study but no reviews; or was deemed to take longer to modify

Micro (local) modifiable A-EBPs

1. Workforce development
Training (H)
o Quality improvement (QI) or EBP in-service training
o Skills-based training
o Multidisciplinary in-service training
o Alignment of training with essential services and usual job responsibilities Access to technical
assistance (H)
o Access to and use of knowledge brokers
o Use of process-improvement activities
o Face-to-face meeting to share lessons, experiences and updates Staff Composition
(M)
o Master’s degree or higher
Staff Competencies (M)
o Ability to communicate research to policymakers
o Economic evaluation skills Staff
Incentives (M)
o Use of incentives & rewards
2. Leadership
Skills & Background (H)
o Leadership skill development
o Leadership experience
27
Brownson et al. (2012) Fostering more-effective public health by identifying administrative evidence-based practices: A review of the
literature
o Leadership quality
o Leadership influence
o Competency to manage change Values &
expectations (H)
o Support for QI, performance standards, EBP, innovation, accreditation
o Intent to hire educated, experienced staff including specialists Participatory
decision-making (H)
o Management team
o Leaders and middle managers seek and incorporate employee input
o Non-hierarchical decision-making
3. Organizational climate, culture & infrastructure
Access and flow of information (H)
o Communication flow
o Tailored messaging for EBP
o Ready access to high quality information
o 360-degree employee performance reviews geared to EBP (with extensive feedback) Support of innovation &
methods (H)
o Leadership and employee training in EBP including new methods
o Employees’ perception that management supports innovation
o Conscious creation of environments conducive to innovation
o Organizational capacity to continue “business as usual” and be in a state of exploration
Learning orientation (H)
o Shared employee perceptions
o Project management teams that encourage communication and collaboration
o Presence of multi-disciplinary, diverse management teams Organization
climate (M)
o Common language and terminology LHD
accreditation (M)
o Identification of gaps
o Participation in accreditation
Information systems (M)
o Tools for EBP
28
Brownson et al. (2012) Fostering more-effective public health by identifying administrative evidence-based practices: A review of the
literature
o Tools for more rapid access to evidence Health
department evidence (M)
o High job satisfaction & morale
o LHD staff certification
o Common language for EBP
o Use of incentives & rewards

4. Relationships and Partnerships

Interorganizational relationships
o Build and/or enhance partnerships with schools, hospitals, social services, community organizations, law enforcement, private businesses
and universities (H)
o Cooperative agreements with state and/or local health departments regarding QI (H)
o Number & diversity of types of collaborating organizations (M)
o Percentage of local public health services & activities provided by non-LHD organizations (M)
o Distribution of authority and effort among collaborating organizations (M)
Vision and mission of partnerships
o Clear vision and aligned mission of partnerships (H)
o Capacity building over time (H)
5. Financial processes
Allocation and expenditure of resources
o Outcome based contracting (H)
o Resources allocated to QI, EBP, innovation, information access, training and implementation (H)
o Diverse funding sources (H)
o Program financial risk (program expenditures/program revenues) (M)
Financial Accountability
o Financial transparency practices (M)

Important to recognize the potential interaction of macro-level elements identified above with the micro-level A- EBPs
Author’s Conclusions Macro (system)-level A-EBPs
 Allocation and expenditure of resources (per capita spending in local health departments) is the strongest predictor of public health
performance

29
Brownson et al. (2012) Fostering more-effective public health by identifying administrative evidence-based practices: A review of the
literature
 Number of full-time equivalents, population size of health department jurisdiction, and presence of a governing board of health were positively
associated with public health performance.
 Centralization of authority within state health department or shared state and local authority had mixed effects on public health performance.
 To influence these macro-level factors, systems change would be required to governance of local health departments; federal, state and local
funding, and/or changes to how schools train public health professionals.
(no further details or supporting data provided)

Micro (local) modifiable A-EBPs

Five key domains
1. Workforce development
2. Leadership
3. Organization climate & culture
4. Relationships & partnerships
5. Financial processes

Across all five domains, organizational-level strategies to increase implementation of A-EBPs could include:
 Performance and quality improvement (QI) initiatives
 Health department accreditation

This would require:

 Workforce training
 Leadership support
 Reliable and valid measures of A-EBPs
Limitations Limitations
 Search of published literature only
 Did not conduct an exhaustive search of complementary disciplines (e.g. business, organizational psychology)
 No quality assessment of included studies or reviews
 Majority of studies were cross-sectional
 Only one reviewer screened for relevance and extracted data
30
Hyde & Shortell (2012) The structure and organization of local and state public health agencies in the US: A systematic review
Type of Article (Design) Systematic review/Review of reviews
Quality Rating 5/10
 Quality assessment of included articles not completed
 Table of characteristics of included articles and outcome data not included
 Weighting of evidence unclear
 Findings not consistently linked to the supporting evidence
Objective(s) To describe the organization and structure of local and state public health agencies in the USA and determine the influence of organizational
characteristics on public health performance and/or health status
Three key questions:
1. What is known about the governance, finance and geographic coverage of public health agencies in the USA?
2. What is the relationship between organization and structure of public health agencies and capacity or performance?
3. What is the relationship between organization and structure of public health agencies and population health outcomes?

Only data answering questions 2 & 3 are relevant and were extracted for this literature review

Included Evidence  Reviews (n=7)

 Single studies (n=54)
o USA (n=54)
o Cross-sectional (n=36)
o Longitudinal (n=8)
o Descriptive/case studies (n=10)
- Quality of included studies is unknown

Studies specific to PH structure & organization and capacity to provide public health services
 Reviews (n=1)
 Single studies (n=20)

Studies specific to PH infrastructure, performance and health outcomes

 Single studies (n=4)
Structural/Organizational  Local public health or state infrastructure
Elements  Public health systems
31
Hyde & Shortell (2012) The structure and organization of local and state public health agencies in the US: A systematic review
 Partnerships
 Financing

No further details provided about organizational or structural characteristics that were relevant, or how they were defined or measured.
Outcomes  Performance was measured as the capacity to provide the 10 essential public health services:
1. Monitor health status to identify and solve community health problems
2. Diagnose and investigate health problems and health hazards in the community
3. Inform, educate and empower people about health issues
4. Mobilize community partnerships and action to identify and solve health problems
5. Develop policies and plans that support individual and community health efforts
6. Enforce laws and regulations that protect health and ensure safety
7. Link people to needed personal health services and ensure the provision of health care when otherwise unavailable
8. Ensure competent public and personal healthcare workforce
9. Evaluate effectiveness, accessibility, and quality of personal and population-based health services
10. Research for new insights and innovative solutions to health problems

 Measured through national performance standards surveys or investigator-developed surveys

 Indicators for each essential service and the number of indicators used to measure performance varied across studies
 Any health outcome

No further details provided about outcomes or how they were defined or measured
Findings Public Health Structure, Organization & Performance
 Jurisdictional size
o Strongest predictor of performance (10 studies)
o Larger jurisdictions performed better than smaller ones. Most studies did not report an optimal jurisdictional size. In studies that did, it ranged
from populations of ≥50,000 to ≥100,000
o Population size was positively associated with performance up to 500,000 at which time performance began to decrease (1 study)
o There was no relationship between population size and performance in one study
 Staffing patterns and characteristics

32
Hyde & Shortell (2012) The structure and organization of local and state public health agencies in the US: A systematic review
o More staff associated with better performance when compared to samples with fewer staff
o A higher proportion of staff per population served associated with better performance on most essential services
 Leadership
o Inconclusive findings on the association between educational background of local health directors and PH performance
 Having a public health director with masters or bachelor’s degree in PH was the strongest predictor of
performance on 6/10 essential services (1 study)
 Having female director (1 study) or a director with a nursing degree (1 study) were the positively associated performance on 5/10 essential
services
o Diversity (not all PH training) and experience of staff was positively associated with performance
 Organizational control
o Mixed findings on the association between organizational control and performance
o Local PH departments with centralized organizational control had significantly higher mean performance compared to organizations with
decentralized or mixed structures (2 studies)
o Mixed or hybrid organizational structures were associated with better performance (2 studies)
o No relationship found in other studies
 Governance
o Jurisdictions governed by local board of health with policy making authority positively associated with performance of some essential
services (3 studies)
 This relationship may not be true for smaller jurisdictions (population ≤ 100,000) (1 study)
 Funding Resources and types
o Mixed findings on the association between public health finances and performance
o Greater taxes per capita associated with performance on 6/10 essential services (1 study)
o Local health department spending a modest predictor of performance in 9/10 essential services (1 study)
o Substantial increases in local government expenditures (1 study) or per capita spending (3 studies) associated with better performance
o Public health spending as a ratio of FTEs in local health departments associated with increased performance (1 study)
o No relationship was found between state and local funding and public health performance at the local level (2 studies)
 Partnerships
o Partnerships with universities and businesses associated with improved performance (1 study)

33
Hyde & Shortell (2012) The structure and organization of local and state public health agencies in the US: A systematic review
o Participation of outside agencies in PH planning and provision of PH services associated with performance (1 study)

Public Health Structure, Organization & Health Outcomes

o For each 10% increase in expenditures per capita, infectious disease morbidity decreased by 1.82% (p=0.037) (1 study)
o For each 10% increase in FTEs per capita, cardiovascular disease (CVD) mortality declined by 0.65% (p=0.014) (1 study)
o Positive associations between local public health performance and county health status (e.g. colon cancer, lunch cancer, CVD, motor vehicle
accidents and homicide) (2 studies)
o Public health network structure was associated with adolescent and senior health in rural communities. The network analysis examined the
relationship between PH system network density and organizational centrality,
public health governance and community size in rural communities (1 study)

Author’s Conclusions  Greater population size served by a public health department associated with increased capacity to provide essential services
o Fragmentation in structure may contribute to inefficiencies in performance of core PH functions.
o Limited evidence available to determine whether regionalization would improve PH performance and capacity.
 Strong evidence that public health expenditures and per capita funding positively associated with performance
 Influence of other structural characteristics such as organization control, leadership, jurisdiction and partnerships on performance and outcomes was
mixed
 Lack of research on small and rural public health departments. In general, they are often found to provide fewer essential services due to lack of
infrastructure and geographic isolation
 Link between organizational structure and performance and health outcomes is unclear. Complex to study due to influence of organizational,
contextual, economic, political and sociocultural factors
 Overall, available evidence is limited, and more research is needed
Limitations  Reliance on published literature
 No quality assessment of included studies or reviews
 Majority of studies were cross-sectional, and more than half relied on the same national profile surveys in the US. Not all local or state health
departments participate in these surveys.
 Outcome were not defined or measured in the same way
 Only one reviewer extracted data and coded findings

34
Dilley et al. (2012) Quality improvement interventions in public health systems: A systematic review
Type of Article (Design) Systematic review
Quality Rating 4/10
 Search was not comprehensive
 Study designs/level of evidence not reported
 Quality assessment of included articles not completed
 Weighting of evidence unclear
 Findings not consistently linked to the supporting evidence
Objective(s)  To identify quality improvement (QI) initiatives implemented in the US public health system and associations with public health performance or health
outcomes
Included Evidence  Single studies (n=15)
o Included 18 separate QI interventions
o USA (n=15)
 Design of included studies is unknown
 Quality of included studies is unknown
Structural/Organizational  Quality improvement (QI) interventions
Elements o QI interventions seek to improve the efficiency and effectiveness of public health programs, services
and organizations
Outcomes  Public Health performance improvements
 Health outcomes
Details not provided about outcomes or how they were defined or measured
Findings Three categories of QI interventions
1. Organization wide QI (Big QI)
2. Specific program/service-oriented QI (small qi)
3. Administrative/management QI (mix of Big & small QI)

 Organization wide QI interventions (7 studies)

o Systems approach (such as having a dedicated team or standard approach to QI) associated with improvements in programs (e.g. reduced
documentation time in tuberculosis clinic), health predictors (e.g. receipt of fluoride treatment) or health indicators (e.g. improvements in 11/14 key
health indicators following statewide QI intervention)
o Organization-wide QI intervention associated with increase in systemwide integration of QI (e.g. increase in
programs meeting the public health standard for establishing quantifiable performance indicators; or

35
Dilley et al. (2012) Quality improvement interventions in public health systems: A systematic review
improvements in program evaluation plans)

 Program or service-related improvement interventions (7 studies)

o QI initiatives targeted a specific program or service such as decreasing wait times for clients, improving information documented in a
communicable disease reporting system, or processes improvements for an influenza responses system.
 Administrative or management function QI activities (4 studies)
o QI interventions targeting administrative or management practices was associated with some improvements
 Reorganizing staffing structure to reduce costs led to reduction in supervisory staff positions by 3.5 FTE (1 study)
 Providing continuing education training for most common skill deficiencies in non-supervisory staff associated with improved knowledge and
skills for specific competencies (1 study)
Findings were promising but not supported by data for:
 Change from activity-based to outcome-based contracting processes improved service delivery (1 study)
 Use of a 360-degree feedback model for performance reviews for staff (1 study)
Author’s Conclusions  Only a small number of studies linked QI interventions to proven predictors of health and these provided weak evidence that QI interventions
improve population health outcomes.

Themes from QI intervention research:

 Detailed assessment of current practice (“process mapping”) is the first step to QI
o Provides a baseline against which improvements could be measured
o Helps to identify points for intervention, opportunities to increase efficiencies and where weaknesses exist
 Engaging leadership is key to success for QI and a barrier when not present or with leadership change.
o Manager engagement could include adding QI or program evaluation to job descriptions or individual outreach to managers about QI projects
 Engaging staff and/or QI experts (e.g. QI team or external advisors) essential to planning and implementing QI activities

Limitations  Clearly valid and reliable measures of improvement were not always provided
o Data used to measure improvement varied across studies and included measures of service or adherence to standards; subjective
measures of satisfaction; or process descriptions or subjective feedback
 All studies (except one) did not measure progress against an external comparison group
o Two studies measured progress in a group receiving a QI intervention compared to a group that did not

36
Dilley et al. (2012) Quality improvement interventions in public health systems: A systematic review
o One study used pre and post tests
o Other studies used internal comparisons to measure progress forward from baseline against a goal
 Relevant evidence may have been missed since grey literature was not searched and literature relevant to QI could be misclassified or described
using numerous terms

37
Carlson et al. (2015) Defining the functions of public health governance
Type of Article (Design)  Literature review with expert consultation (published)
Quality Rating  Unable to critically appraise
 Not a systematic review
Objective(s)  To validate, refine, and update the public health governance functions.
Included Evidence  Articles (n=44)
 Board of health orientation manuals (n=18)
 Study design or type of included articles is unknown Quality of
included evidence unknown
Structural/Organizational  Authors developed draft definitions of governance functions based on the literature review which were sent to individuals with PH backgrounds to
Elements review and provide feedback. A second draft was developed and reviewed by
the same stakeholders (n = 100). Consensus was achieved for the final list of functions and their definitions.
Outcomes  List of governance functions and associated definitions.
Findings Six Functions of PH Governance:
1. Policy Development – Lead and contribute to the development of policies that protect, promote, and improved public health while ensuring
that the agency and its components remain consistent with laws and rules.
2. Resources Stewardship – Assure the availability of adequate resources (legal, financial, human, technological and material) to
perform essential PH functions.
3. Continuous Improvement – Routinely evaluate, monitor and set measurable outcomes for improving
community health status and the PH agency’s or governing body’s own ability to meet its responsibilities.
4. Partner Engagement – Build and strengthen community partnerships through education and engagement to ensure the collaboration of all
relevant stakeholders in promoting and protecting the community’s health.
5. Legal Authority – Exercise legal authority as applicable by law and understand the roles, responsibilities, obligations, and function of the
governing body, health officer and agency staff.
6. Oversight – Assume ultimate responsibility for PH performance in the community by providing necessary leadership and guidance to
support the PH agency in achieving measurable outcomes.

Author’s Conclusions Defined the 6 functions of PH governance so they could be used by PH governing entities alongside the existing, overarching PH materials, such as the 3
core functions and 10 essential PH services, and to provide insight into how a governing entity supports and guides health agency service provision and
participation in the PH system.

38
Limitations Lack of systematic exploration of relationship between PH governing entities and performance of health agency.

Lack of data specific to PH governing entities and their performance required that development of the 6 functions be grounded in work done with
hospital, education and non-profit boards.
Convenience sample for included board of health orientation manuals.
 Included body of knowledge relating to state boards of health for literature review but majority of effort focused on local PH governing
entities.

39
Naylor & Buck (2018) The role of cities in improving population health
Type of Article (Design)  Literature review with expert consultation (grey literature) Three
methodological components:
1. 50 interviews
2. Literature review
3. Expert roundtable discussion
Quality Rating  Unable to critically appraise
 Not a systematic review
Objective(s)  Examine what a place-based approach to population health might look like in a
city context.
 Explore the range of roles that city governments can play in improving population health (directly and indirectly), and
the underlying conditions needed for effective
urban health governance.
Included Evidence  50 interviews (25 based in London, England; 25 international)
 Number of articles and attendees at roundtable not reported
Findings Themes were identified in 3 areas:
1. Why Cities Matter for Health
 City leaders have significant influence over the social determinants of health
 Cities are where most of the population live
 Cities contain significant health needs and inequalities
 Cities are playing a growing role in national and international politics
 Cities are becoming increasingly well connected

2. Conditions for Successful Health Governance in Cities

 Governance – decision-making processes, partnerships, collaboration across and beyond city boundaries
 Leadership – bold, willingness to invest in championing health; understanding that elected leaders have soft powers beyond formal
responsibilities and the ability to use these
 Powers – full use of regulatory and other powers available; devolution of powers for national government to cities
 Expertise – adequate resourcing of PH functions; distribution of PH expertise
 Connectivity – work with and learn from other cities

40
 3. Roles for City Governments in Population Health
 Co-ordinating system-wide action – on population health and adequate investment in program management; explicit methodology for
collaborating effectively
 Promoting innovation – full use of assets available in a city; explore way to stimulate innovation; develop mechanism for sharing learnings
 Using regulatory and legislative levers – evidence-based; know the law and have access to legal advice; use as one component of broader strategy
 Mobilising the population – see communities as an asset and empower citizens; engage people in civic decision-making
Using planning powers to create healthy places – use evidence and data to make informed decisions about the use of public spaces

Author’s Conclusions This report illustrates the important and distinctive role that cities can play in relation to population health improvement. City governments and their
partners are well placed to co-ordinate cross-sectoral activities; create an environment that fosters innovation; mobilise communities to pursue citizen-led
improvement; and to use regulatory levers and planning powers to create health promoting environments. At their best, cities have the clout to bring about
change at scale while managing to retain the local responsiveness and agility that national policy-
making can sometimes lack.
Limitations  None identified by authors
 Methods not identified
 Literature not identified or appraised

41