Evaluation Plan

PUBH 8475/HLTH 8475: Advanced Program Implementation and Evaluation

Charmaine Vierra

Summer Quarter, 2016

Dr. Jirina Foltysova

Walden University

July 30, 2016

1
 1. Evaluation Stakeholders

Program interventions developed without respect to a planned evaluation fail to encrypt

in the plan an accurate tool for measuring outcomes. As with most programming, the

stakeholders need to know if the program achieved its desired results which primarily support

increases in positive health behaviors among the priority population. Table 1 gives the role of the

stakeholders in the evaluation process. In the case of childhood obesity and type-2 diabetes, the

priority population cannot advocate for themselves, and need parents and caregivers as a primary

part of the overall program design and delivery. The outcome evaluation takes the approach that

changes in behavior as a result of the intervention are measurable in terms of new health habits

and skills, or decreased health risks (SOPHE, 2010, Kindle Location 5326).

Table 1. Evaluation Stakeholders

Stakeholder Name Stakeholder Role in the Evaluation How and When to Engage
Category
Parents Primary Parents are integral part
of developing Parents offer feedback to
assessments/surveys as program sensitivities that may
well as articulating impact the priority population.
cultural sensitivities.
Establish strategies for
Local Business support, and work to Visible campaign to support
Community-based Tertiary build on health networks healthy food/choices, sensitive
Organizations that help to ensure advertising as to avoid
options with healthy manipulating vulnerable child
produce, healthy menu consumers, offer incentives to
options using locally make healthy choices.
grown produce.
Faith-based Primary & Volunteers, venue space, Strategize to provide program
communities Secondary resources, dissemination support through venue
of ads. acquisition and other help.
Schools Establish nutritional Utilization of curriculum,
Primary guidelines and healthy hands on applied nutrition,
•
meal options. Reinforce gardening, and increased
health messages. physical activity.
Public health/local Secondary Identify low-risk health Measurable preventative
government Tertiary factors and high strategies that provide health
preventative health and disease preventative tools
•
factors. for educational/promotional
purposes.

2
 2. Evaluation Goals

Health professionals when planning programs need to first consider the purpose of the

program and what they plan to accomplish. Evaluation of the program is as much a part of the

program, as its overall purpose; the first establishes the second. The approaches to evaluation fall

under four different categories that monitor and observe different changes and programmatic

effects. The four types of evaluations are formative, process, impact, and outcome (SOPHE,

2010, Kindle Location 5322). The goals of each are established during preliminary program

planning and described in Table 2.

Formative Evaluation. This approach is integral in understanding whether the program will

reach the priority population. It also describes the benefits and seeks to draw conclusions as to

the purpose of the program, the appropriateness of the content/activities, and the overall potential

to achieve the desired behavioral change.

Process Evaluation. Making observations and monitoring the progress of a program as well as

checking in to determine the necessary adjustments, helps this approach to save funding dollars

by allocating more support to a successful area and pulling funding from failing activities.

Impact Evaluation. Interventions that tend to produce a type of cause and effect reaction that

creates an immediate measurable change is considered “impact” evaluation. These types of

evaluations look to the priority population to see if any immediate behavioral changes occurred

as the result of the intervention.

Outcome Evaluation. This approach measures the results of the program activities and

interventions and their effect on measurable behavioral change or measurable health benefits.

This approach is simplistic in its applications as it considers successful the results that show

positive health promoting behaviors and decreased poor health choices.

3
Table 2. Evaluation Goals
Type of Evaluation Evaluation Goal
To determine whether an online tool is comparable to a print version, as
Formative Evaluation it provides a significant reduction in cost.
Researchers of one obesity prevention program found that switching
schools over to an online curriculum to be more cost effective than a
print version (Welk, Chen, Nam, & Weber, 2015).

To determine which type of intervention or combination of interventions

improve the obesogenic environment addressing the multivariate health
Process Evaluation determinates.
Pettigrew, et al., (2014) describes interventions that propose to focus the
experiential learning through 3 different processes: down stream, mid-
stream, and upstream. Down stream are those interventions that address
behavior through primary or direct approach such as changing the
individuals eating behaviors and activity level. Mid stream includes the
family and addressing those familial patterns or barriers that promote
poor eating choices. Upstream views the rational community as an
environment that provides access to safely make healthy choices
available to everyone without preference to socio-economic factors.

Impact Evaluation To encourage a neighborhood built environment with favorable

influence on nutritional choices and access to physical activity to
decrease the prevalence of childhood overweight or obesity.
One study found that built environments that boasted of many near-by
supermarkets and less fast-food outlets and more opportunities to walk
and engage in physical activity indicated less childhood overweight and
obesity incidence by 59% (Saelens, et al., 2012).

Outcome Evaluation To increase consumption of fruits and vegetables through Junior Chef
classes to instruct in the culinary arts using plant-based foods.

A study reviewing cooking education classes found that among four of the
studies measuring the pre- and post-intervention consumption of daily
fruits and vegetables, one observed significant increases in consumption
(Hersch, Perdue, Ambroz, & Boucher, 2014).

4
 3. Data Collection

Data collection and subsequent analysis provides the information to determine if the

interventions are achieving the desired outcomes. On a daily basis, individuals make choices

based on analyzing information produced from action. Thus most data is observed and not

measured, and can be qualitative or subjective and quantitative or objective. Scientific study uses

both qualitative and quantitative data.

Table 3. Evaluation Data Collection Strategy

Evaluation Evaluation Measures/Indicators Data Source of
Purpose Question Collection Data
Method

Formative
Evaluation Students access the
Does the online online program daily.
curriculum engage Participants are Data will be
the students tracked when collected from
interest more/less Students use the online Logging into the online
or equally as the more that print version. the online program
print version? version. platform.

Process Should the

Evaluation intervention 1. Students attend the 1. Times in Voluntary
address multiple cooking classes. attendance journal/record
socio-economic 2. Parent’s purchases 2. Weekly
factors? increase in fruits and purchases of
vegetables. produce

Impact Are the walking Increase opportunity to Amount of time Voluntary

Evaluation trails and parks physical activity such spent in daily journal/record
promoting as walking exercise
increased physical
activity?
Outcome Do the cooking Students eat more fruits Daily amount Voluntary
Evaluation classes increase and vegetables of fruits and journal/record
consumption of vegetables
fruits and consumed.
vegetables?

5
 4. Evaluation Team

The evaluation team is an integral part of the evaluation process. Team members are

responsible for methods of collecting and analyzing data, reporting data with evidence of the

specific concerns to consider and what issues take priority. The project manager is the liaison

and carries significant responsibility to oversee the day-to-day program operations. This

individual possesses personal strengths of unbiased inquiry honesty, integrity, respect for human

dignity, and honestly cares for the welfare of others (Muriel, 2010, Kindle Locations 523-526).

Stakeholder involvement is important for the benefit of “appropriate administrative,

epidemiological, political, and sociocultural perspectives” (Muriel, 2010, Kindle Locations 523-

526). Medical professionals as part of an evaluation team understand the medically sensitive

issues as well as ethical issues surrounding patient health information, as well as any serious

medical issues or contraindications. The medical professionals may act in the auspices of

program medical director and provide consultation as needed. Community members are that

active arm that gives color and sustainability to the program. The community members are

attuned to the cultural sensitivities and are personally invested to create a successful and

sustainable program.

Table 4. Evaluation Team

Individual Role Responsibilities
Health Educational Project Manager Liaison, data analysis and reports, articulates
Specialist programs progress, fiscal accountability
Stakeholders Stakeholder Helps to design and develop program evaluation,
analysis participates in pertinent decision making, identifies
additional areas of concern, helps determine the tools
utilized in the evaluation process, shares
responsibility of program outcomes, discusses the
evaluation and that participates in partnerships to
support successful outcomes.
Medical Medical Determines the health risks to the priority
Professional Director/Expert populations, advises and counsels to the intervention
standards/measures, gives expert counsel.
Community Program Assistant Data collection, analysis, interpretation, technical
Members writing, reporting, research

6
 5. References

Harris, Muriel J. (2010). Evaluating public and community health programs. Wiley. Kindle

Edition.

Hersch, D., Perdue, L., Ambroz, T., & Boucher, J. L. (2014). The impact of cooking classes on

food-related preferences, attitudes, and behaviors of school-aged children: a systematic

review of the evidence, 2003-2014. Preventing Chronic Disease, 11(2), E193.

http://doi.org/10.5888/pcd11.140267

Pettigrew, S., Borys, J. M., du Plessis, H. R., Walter, L., Huang, T. T.-K., Levi, J., & Vinck, J.

(2014). Process evaluation outcomes from a global child obesity prevention intervention.

BMC Public Health, 14, 757. Retrieved from

http://www.embase.com/search/results?subaction=viewrecord&from=export&id=L6063230

77\nhttp://dx.doi.org/10.1186/1471-2458-14-

757\nhttp://vb3lk7eb4t.search.serialssolutions.com?sid=EMBASE&issn=14712458&id=doi

:10.1186%2F1471-2458-14-757&atitle=Process+evaluati

Saelens, B. E., Sallis, J. F., Frank, L. D., Couch, S. C., Zhou, C., Colburn, T., … Glanz, K.

(2012). Obesogenic neighbourhood environments, child and parent obesity: the

neighbourhood impact on kids study. American Journal of Preventive Medicine, 42(5), e57–

e64. http://doi.org/10.1016/j.amepre.2012.02.008.Obesogenic

Society for Public Health Education (SOPHE). (2010). health promotion programs: from theory

to practice.Wiley. Kindle Edition.

Welk, G. J., Chen, S., Nam, Y. H., & Weber, T. E. (2015). A formative evaluation of the

SWITCH® obesity prevention program: print versus online programming. BMC Obesity,

2, 20. http://doi.org/10.1186/s40608-015-0049-1