MEDS 4053 SU Patient Preferences and Evidence Discussion

Please refer to Slide 7 of the  Week 3 slide deck for this reply.

If you had to make an informed decision, as a health professional, in moving forward on a treatment, intervention, or policy, which 1 or 2 of these considerations would be important for you, and why?

1
Thinking Scientifically:
Evidence-Based Practices
Evidence-Based Public Health (MEDS 4053)
Kelley A. Carameli, DrPH
Week 3
2
What is Evidence?
Defining Evidence
• Evidence: The available facts or information on whether a belief or
concept is true or valid.
▪ Law: expert/witness testimony, forensics
▪ Medicine: epidemiologic (quantitative), experts
▪ Public Health: epidemiologic (quantitative), program or policy
evaluations, qualitative (key informants, document studies, clinical
or public health practice)
• Evidence is used in decision-making and priority setting.
• Evidence is most useful and effective when:
▪ Based on systematic assessment or evaluation
▪ Data are timely or relevant to issue at hand
▪ Real-world application
▪ Triangulated: quantitative, qualitative, and cultural/geographic
3
Evidence: Informed by Science
How do we obtain public health evidence?
• By using science, or scientific methods.
What is science, or the scientific method?
•
A way of systematic thought and investigation to obtain reliable information.
Obtaining knowledge through evidence. Communicating knowledge to
others. Using methods that can be replicated (checked, valid, objective).
•
A process of thinking and answering questions by formulating questions and
using a set of rules for inquiry or answering questions.
Posing questions to understand relationships. Testing the proposed
relationship against reality. Determining if something happened.
• Additional consideration in scientific inquiry:
– Reasoned judgment: using the best available knowledge to inform
decision-making in the absence of complete evidence.
– Opinion: personal view of reality. Custom: social influence on reality.
4
Thinking Scientifically in Health
Thinking Scientifically in Public Health
• A way of systematic thought and investigation to obtain reliable
information.
• A process of thinking and answering questions by formulating questions
and using a set of rules for inquiry or answering questions.
We often start
and stop here
We also need to
progress to here
5
Thinking Scientifically in Health
When to Apply Evidence-Based Approaches
• Use evidence-based (scientific) approaches in public health:
– when conducting literature reviews for grant proposals
– when evaluating the effectiveness and cost-benefit of health programs
– when establishing new health programs
– when policies are being implemented
– when scientific evidence is important to support decision-making.
Are there ever disadvantages to using an evidence-based approach?
Case example: A public health department wants to be innovative in
establishing new programs. It is particularly interested in developing a
program quickly to address transgender bullying in its county even though
evaluative information is not readily available on successful interventions
for this group. Should it proceed without evidence? What are the
advantages and/or disadvantages?
6
When to Balance Science + Action
Balancing Evidence/Scientific Approaches + Public Health Actions
• Be realistic…every decision may not have robust evidence. Instead:
– Use and weight all available information
• Magnitude of the problem, known risk determinants?
• Stakeholder opinion, existing practices/traditions?
– Balance program implementation against its fidelity (original design)
vs. reinvention (new setting, local context)
• Recognize trade-offs – urgent action may limit ability for evidence-based
decisions, but poorly informed action may also be hard to change
• Considerations for decision-makers:
• Size/scope of problem
• Intervention effectiveness
• Intervention cost, value, alternatives
• Equitable
• Preventability
• Benefits and Harms
• Acceptability (culture, values)
• Sustainable, Appropriate
7
When to Balance Science + Action
Considerations in evidence-based decision making:
Size/scope of problem
Intervention
effectiveness
Intervention cost
Equitable
Preventability
Benefits and Harms
Acceptability
Sustainable,
Appropriate
•
•
•
•
•
•
•
•
•
•
•
•
Is it important?
What is the public health burden?
Does it work in real-world settings?
Is there better evidence for an alternative?
Is it affordable?
Does it distribute resources fairly?
What is the efficacy?
Can it work in an “ideal” circumstance?
What are the consequences? Trade-offs?
Is it consistent with community priorities, culture,
values, the political situation?
Are resources and incentives likely needed to
support/maintain the intervention?
Is it likely to work in this setting? Others?
Anderson et al., 2005, American Journal of Preventive Medicine
8
Benefits + Limits of Using Evidence
Benefits
Limits
•
•
Evidence shown for one setting or
time may not translate to another
– Social context/culture shapes
behavior, as does new data
– Pap/prostate screening, DARE
•
Some outcomes easier to measure
– Standard: tobacco use, vaccines
– Mixed: cultural competency,
health literacy
•
Socio-political values shape public
health and what we measure
– Risk-reduction vs. abstinence
(sexual health, substance use)
– Natural experiments vs.
randomized control groups
•
•
Greater likelihood of implementing
more effective policies/ programs
– Better use of resources (staff,
materials, time, etc.)
– More informed (and productive)
public health workforce
Greater likelihood of impacting
change in public health issues
– Change informed by evidence
– Change may be reproducible
(systematic)
Access to higher-quality
information. Learn what works!
9
Looking at the Scientific Evidence
Scientific Evidence and Public Health Action
• The higher-quality information needed for public health action comes from
research studies or program evaluations to learn what ‘works’:
1.
2.
3.
4.
5.
Understand etiologic links between behaviors and health.
Develop testable methods (valid, reliable) for measuring behavior.
Identify the factors that influence the behavior.
Determine whether public health interventions are successful.
Translate/Disseminate research findings into practice.
Most public health actions (programs, policies) are based on the
presumption that the behavior-health relationships
(i.e., etiological links) are causal.
Type 1 Evidence
10
Looking at the Scientific Evidence
Levels of Evidence-Based Data
Type 1: Something should be done.
Most common: Medicine/Epi.
Type 2: Specifically, this should be done.
Public health practitioners have
most interest here.
Type 3: Context for how an intervention is done.
From Brownson RC, et al. Evidence-Based Public Health: A Fundamental Concept for Public Health Practice, 2009.
11
Looking at the Scientific Evidence
Scientific Evidence and Public Health Action
• The higher-quality information needed for public health action comes from
research studies or program evaluations to learn what ‘works’:
1.
2.
3.
4.
5.
Understand etiologic links between behaviors and health.
Develop testable methods (valid, reliable) for measuring behavior.
Identify the factors that influence the behavior.
Determine whether public health interventions are successful.
Translate/Disseminate research findings into practice.
Most public health actions (programs, policies) are based on the
presumption that the behavior-health relationships
(i.e., etiological links) are causal.
Type 1 Evidence
1
2
3
12
Looking at the Scientific Evidence
Levels of Evidence-Based Data
Type 1 “Something should be done”
• Size, strength, or causal relationship between the behavior (or risk
factor/determinant) and health (or disease)
– Magnitude of issue: number, incidence, prevalence
– Severity: morbidity, mortality, disability
– Preventability: deaths averted, effectiveness, economic impact
• Clinical designs (randomized, experimental) focused on evidence of causality:
– Consistency: association is observed in different settings, populations, methods.
– Strength: size of the relative risk estimate.
– Temporality: time relationship between risk factor onset and disease onset.
– Dose-response: dose of the exposure and magnitude of relative risk estimate.
– Biologic plausibility: biological mechanism between risk factor and disease
outcome.
– Experimental evidence: findings from a prevention trial; random assignment.
13
Looking at the Scientific Evidence
Levels of Evidence-Based Data
Type 1 “Something should be done”
• Size, strength, or causal relationship between the behavior (or risk
factor/determinant) and health (or disease)
– Magnitude of issue: number, incidence, prevalence
– Severity: morbidity, mortality, disability
– Preventability: deaths averted, effectiveness, economic impact
From Brownson RC, et al. Evidence-Based Public Health. New York: Oxford University Press, 2003.
14
Looking at the Scientific Evidence
Levels of Evidence-Based Data
Type 2 “This should be done” (specific intervention)
• Relative effectiveness of intervention on the risk factors.
• Population designs (non-experimental) to show evidence of intervention
effectiveness (intervention → increases childhood vaccination)
– Evidence-based (peer-reviewed, systematic, external validity)
– Efficacious (peer review, research-tested, external validity)
– Promising (formative or summative evaluation, theory-consistent, lacks peer
review)
– Emerging (ongoing or in-progress evaluation, theory-consistent)
• Explores which intervention option (or combination) is more effective
and/or cost-effective
– Client reminder, Community edu., Health insurance, Vaccines in schools
15
Looking at the Scientific Evidence
Levels of Evidence-Based Data
Type 3 “How to take specific action”
(context of intervention)
• Adapt/translate evidence into
population-level intervention or policy.
• Program/policy may work in one
context, but not another. Consider
contextual domains (→).
• New programs/policies (i.e.,
innovations) may incur unintended
consequences of action.
– Political, social, or structural
domains
– E.g., school vaccine policy →
reduces measles rates)
16
Sources of Scientific Evidence
Scientific evidence is relative to the time, culture, and context.
– Public health decisions may be based on the ‘best possible’ (reasoned
judgement) and not always the ‘best available’ evidence.
– Important to consider triangulated evidence (e.g., mixed-methods).
– Looking for an intervention’s ‘active ingredients’ – transferability by context.
Quantitative Evidence
•
•
Shows how variables are
related; large sample sizes
Surveys, surveillance:
If X, then Y and Z
Qualitative Evidence
•
•
Explores why relationships
exist; smaller sample sizes
Interviews, case studies:
Why Y? Why Z? What makes
them similar/different?
17
Sources of Scientific Evidence
Analytic Tools for Obtaining Evidence Used in Public Health
Systematic Reviews/Guides
•
•
Synthesis of existing or state-of-the art research or
literature.
Translate evidence to local action
Public Health Surveillance
•
•
Ongoing, systematic data collection and analysis on disease
/ injury.
Data tracking (e.g., tobacco sales)
Economic Evaluation
•
Relative value, cost-benefit of action
Health Impact Assessment
•
•
Probable effect of public health policy/programs in nonhealth sectors
Impact = “5 A-Day” on agri. production
Participatory Approaches
•
Soliciting stakeholder (local) input
18
How to Think Scientifically
Processes for “Thinking Scientifically” in Public Health
1. Define and quantify the issue
–
What is the size of the public health problem?
2. Gather evidence to inform public health action
–
When reviewing evidence consider:
•
What are the results? How precise? Similar across studies?
•
Are the results valid? Is the assessment reproducible?
Was the methodological quality sound?
•
How can the results be applied to public health actions?
Are the benefits worth the costs and potential risks?
3. Translate evidence into practice
–
–
–
Inputs
Are there effective programs for addressing this problem?
What information about the local context is needed?
Is this particular policy or program worth doing?
4. Disseminate evidence-based findings and practices
Process
Outputs
Outcome
Impact
19
How to Think Scientifically
Processes for “Thinking Scientifically” in Public Health
Case Example: State Regulation/Firearm Homicide (Irvin et al., 2014)
20
How to Think Scientifically
Processes in “Thinking Scientifically” in Public Health
1. Define and quantify the issue
Inputs
A. What is the health issue? (problem statement)
–
Develop a concise statement of the issue being considered
▪ What is the issue and why care?
▪ Who is the population(s) affected?
▪ What is the size / scope of the problem?
▪ What prevention opportunities currently exist?
▪ Who are the key stakeholders?
B. Quantify the issue (counts, incidence, prevalence)
–
–
Look to existing research for baseline data – descriptive data, vital
statistics, surveillance systems, surveys/national studies
What patterns exist? By person (gender, race/ethnicity, place
(geography), or time (seasonal variation).
C. Use the literature to shape the issue (logic model)
21
How to Think Scientifically
Processes in “Thinking Scientifically” in Public Health
2. Gather evidence to inform health, program, and/or policy change
–
–
–
Look to existing research literature (peer-reviewed, testable)
Initiate own research or evaluation
Does this practice help alleviate the health issue? How? Why?
Inputs
A. What factors affect the health issue? (hypothesis)
–
–
Descriptive to understand why or how; single concept
Relational to understand connections; multiple concepts
B. How to measure this relationship? (operationalization)
–
–
Concept / Construct (real, phenomena)  Metric / Variable
(validity or ‘accuracy’, reliability or ‘consistency’; IV and DV)
Look to existing research (and theory) for measurement options
C. What is learned from the data? (analysis, interpretation)
–
–
Data relationships or trends/patterns (significance)
Critical review – method, measures, theory, field comparisons
22
How to Think Scientifically
Processes in “Thinking Scientifically” in Public Health
3. Translate evidence into practice (evidence-based decision-making)
A. How can it be applied? (translational research)
•
•
•
What are the “real world” applications learned from the literature?
– Prioritize findings.
Process
– Identify barriers: resources, political, cultural.
Blend what is known from medicine, public health, and
other disciplines
Incorporate input from community-based stakeholders (e.g.,
Outputs
expert panels, policy makers, coalitions.
B. Develop an action plan and implement intervention(s)
•
Consider short- and long-term goals or changes
C. Evaluate program or policy
•
Apply quantitative and qualitative techniques
Outcome
Processes for “Thinking
Scientifically” in Public
Health
Case Example: State
Regulation/ Firearm
Homicide (Irvin et al., 2014)
3. Translate evidence
into practice
• Any patterns?
• Does this affect our
logic model?
23
24
How to Think Scientifically
Processes in “Thinking Scientifically” in Public Health
4. Disseminate evidence-based findings and practices
–
–
–
–
Peer-review journals,
conferences/meetings
Media, local
interactions, word-ofmouth
Policies, programs
Consider these
elements when sharing
findings to enhance
stakeholder decisionmaking →
Outputs
Outcome
Impact
RESEARCH AND PRACTICE
Evaluating the Effect
of State Regulation of
Federally Licensed
Firearm Dealers on
Firearm Homicide
Nathan Irvin, MD, MS, Karin Rhodes, MD, MS,
Rose Cheney, PhD, and Douglas Wiebe, PhD
Effective federal regulation of firearm dealers has proven difficult. Consequently, many states choose to
implement their own regulations. We
examined the impact of state-required
licensing, record keeping of sales, allowable inspections, and mandatory
theft reporting on firearm homicide
from 1995 to 2010. We found that
lower homicide rates were associated
with states that required licensing and
inspections. We concluded that firearm dealer regulations might be an
effective harm reduction strategy for
firearm homicide. (Am J Public
Health. 2014;104:1384–1386. doi:
10.2105/AJPH.2014.301999)
Current federal regulations and enforcement
practices limit the federal government’s ability to
effectively deter illegal firearm sales by federally
licensed firearm dealers.1—4 Several states have
enacted their own firearm laws or require an
additional layer of oversight, but evidence in
support of these laws as injury reduction strategies vary.5– 7 Firearm dealer regulations aimed at
decreasing trafficking have been successful, yet
little data exist regarding the effect of these
regulations on firearm homicides.8 In this study,
we examined state licensing and other lawful
sales promoting dealer regulations, and hypothesized that they decrease firearm homicide.
METHODS
We conducted a state-level panel study to
examine how regulation of federally licensed
firearm dealers related to firearm homicide
during 1995 to 2010.
We used data from the Centers for Disease
Control and Prevention’s Web-based Injury
Statistics Query and Reporting Systems and
Multiple Cause of Death files to identify statelevel firearm homicide totals from 1995 to
2010. Homicide rates were subsequently calculated for each state. We used published
peer-reviewed research that cited the laws
regulating firearm dealers, and characterized
the regulatory status of each state during the
study.9 LexisNexis was used for confirmation.
We performed multivariable Poisson regression analyses controlling for sociodemographic characteristics from the US Census,
burglary and drug arrest rates from the FBI’s
Uniform Crime Report, state firearm regulation
scores from the Traveler’s Guide to Firearm
Laws of the Fifty States, and a validated firearm
ownership proxy measure.10—12
Models were analyzed using state licensing,
theft reporting, allowable inspections, and
mandatory record keeping as categorical independent variables and homicide rates as the
dependent variable. We also constructed
models evaluating interactions between key
variables.
In addition, a model using an overall
strength variable, which equaled the sum of the
4 regulations, was constructed and analyzed.
All analyses controlled for clustering at the
state level.
RESULTS
The characterization of each state’s dealer
regulations are listed in Table 1. Over the years
examined, 195 932 people died by firearm
homicide. The median annual homicide rate
per 100 000 people was 3.66 (interquartile
range = 1.80—5.39).
Lower homicide rates were associated with
states that required licensing and allowed inspections (Table 2). Theft reporting was not
associated with lower homicide rates (incidence rate ratio [IRR] = 1.04; 99% confidence
interval [CI] = 0.95, 1.14), and record maintenance was associated with higher homicide
rates (Table 2). The protective effect was
stronger in states that required both licensing
and inspections (IRR = 0.49; 99% CI = 0.42,
0.58).
Lower homicide rates were associated with
states that had 3 or more laws regulating
1384 | Research and Practice | Peer Reviewed | Irvin et al.
firearm dealers (IRR = 0.76; 99% CI = 0.67,
0.86 [3 laws] and IRR = 0.75; 99% CI = 0.65,
0.86 [4 laws]).
DISCUSSION
Our national study adds to the literature
through a rigorous examination of the effect of
state regulation of firearm dealers on firearm
homicide. Our findings suggest that firearm
dealer licensing and allowable inspections
might save lives. Although limited to association by the observational design and absence of
policy change during the study period, these
findings are promising and warrant further
investigation.
Similar to previous studies, our results varied
based on the type of regulation. State licensing
and authorized inspections were associated with
lower homicide rates, whereas record keeping
was associated with increased homicides. Furthermore, having both licensing and inspections
appeared to be more strongly protective against
homicide than either alone. It makes intuitive
sense for the effect to be stronger for having
both mandatory licensing and allowable inspections because it is important to have a
mechanism by which to evaluate and enforce
compliance with the licensing. The association
between record keeping and increased homicides is less clear. Perhaps this finding exists
because states that have problems with firearm
diversion, and consequently, increased access to
guns that might be used in homicides, have
chosen to enact these laws to attempt to address
these problems. These findings highlight the
complex nature of these associations and suggest that the findings might also be influenced
by other unmeasured covariates, such as enforcement of the law or other unmeasured laws
or variables not included within our models.13
Our findings are compelling, but have limitations. In addition to the study design caveats
mentioned previously, it is important to acknowledge that, as evidenced by the nonlinear
association between increasing laws and decreased firearm injury, all laws are not equivalent, and further research is necessary to
identify the combination of laws that might best
prevent homicide. Furthermore, we were unable to quantify enforcement in our models,
which evidence suggested is an important
factor in determining the effect of laws.8,14,15
American Journal of Public Health | August 2014, Vol 104, No. 8
RESEARCH AND PRACTICE
TABLE 1—Mean Annual Homicide Rates by State and State Laws Regulating Federally
Licensed Firearm Dealers: United States, 1995–2010
Firearm homicide is a persistent threat to
societal well-being. Our study highlights
regulatory approaches states could take to
potentially decrease firearm homicide.
Through tougher regulation of firearm
dealers, it might be possible to prevent
firearm-related deaths. j
Mean
Homicide
Rate
License
Records
Inspections
Thefts
7.27
4.46
Yes
No
Yes
Yes
No
No
No
No
Arizona
6.18
No
No
No
No
About the Authors
Arkansas
5.82
No
No
No
No
California
5.06
Yes
Yes
Yes
Yes
Colorado
2.84
No
Yes
Yes
No
Connecticut
2.4
Yes
Yes
Yes
No
At the time of this study, Nathan Irvin was with the
Department of Emergency Medicine, University of Pennsylvania School of Medicine, Philadelphia. Karin Rhodes
is with the Department of Emergency Medicine, University
of Pennsylvania School of Medicine. Rose Cheney and
Douglas Wiebe are with the Firearm Injury Center at Penn
(FICAP), University of Pennsylvania School of Medicine.
Correspondence should be sent to Nathan Irvin, Johns
Hopkins Department of Emergency Medicine, 5801 Smith
Ave, Baltimore MD, 21209 (e-mail: nirvin1@jhmi.edu).
Reprints can be ordered at http://www.ajph.org by clicking
the “Reprints” link.
This article was accepted March 24, 2014.
State
Alabama
Alaska
Delaware
3.49
Yes
Yes
Yes
No
District of Columbia
Florida
29.01
4.44
Yes
No
Yes
No
No
No
No
No
Georgia
5.62
Yes
Yes
Yes
No
Hawaii
1.09
Yes
No
Yes
No
Idaho
1.98
No
No
No
No
Illinois
5.37
No
Yes
Yes
No
Indiana
4.41
Yes
No
No
No
Iowa
1.25
No
No
No
No
Kansas
Kentucky
3.36
3.78
No
No
No
No
No
No
No
No
Louisiana
10.5
No
No
No
No
Maine
1.31
No
Yes
Yes
No
Maryland
6.79
Yes
Yes
Yes
No
Massachusetts
1.49
Yes
Yes
Yes
Yes
Acknowledgments
Michigan
5.22
No
Yes
Yes
Yes
Minnesota
1.72
No
No
Yes
No
Mississippi
Missouri
7.76
5.3
No
No
Yes
No
Yes
No
No
No
Montana
2.64
No
No
No
No
We would like to acknowledge the Robert Wood
Johnson Foundation Clinical Scholars Program at
the University of Pennsylvania for their support and
assistance with this project.
We would also like to thank Kelly Chen for helping
with the legal research involved in this study.
Nebraska
2.29
No
No
No
No
Nevada
5.7
No
No
No
No
New Hampshire
0.99
Yes
No
No
No
New Jersey
2.6
Yes
Yes
Yes
Yes
New Mexico
5.07
No
No
No
No
References
New York
North Carolina
3.09
5.43
Yes
No
Yes
Yes
Yes
Yes
Yes
No
North Dakota
1.07
No
No
No
No
1. USDOJ, Office of the Inspector General. Review of
ATF’s Federal Firearms Licensee inspection program.
April 2013. Available at: http://www.justice.gov/oig/
reports/atf.htm. Accessed January 10, 2014.
Ohio
3.21
No
No
No
Yes
Oklahoma
4.4
No
No
No
No
Oregon
2.23
No
Yes
Yes
No
Pennsylvania
4.2
Yes
Yes
No
No
Rhode Island
1.89
Yes
Yes
Yes
No
South Carolina
South Dakota
5.58
1.03
Yes
No
Yes
No
Yes
No
No
No
Tennessee
6.07
No
Yes
Yes
No
Texas
4.53
No
No
No
No
Continued
August 2014, Vol 104, No. 8 | American Journal of Public Health
Contributors
N. Irvin helped conceptualize the idea, gathered the
data, analyzed the data, and helped write each version
of the article. K. Rhodes and R. Cheney helped
develop the project idea and assisted in writing and
revising the article. D. Wiebe helped develop the idea,
conduct the analyses, and write and revise all versions
of the article.
Human Participant Protection
This study was deemed exempt by the University of
Pennsylvania institutional review board.
2. Wintemute G. Firearm retailers’ willingness to
participate in an illegal gun purchase. J Urban Health.
2010;87(5):865—878.
3. Wintemute GJ. Frequency of and responses to illegal
activity related to commerce in firearms: findings from
the Firearms Licensee Survey. Inj Prev. 2013;19(6):412–420.
4. Sorenson SB, Vittes KA. Buying a handgun for
someone else: firearm dealer willingness to sell. Inj Prev.
2003;9(2):147—150.
5. Rosengart M, Cummings P, Nathens A, et al. An
evaluation of state firearm regulations and homicide
and suicide death rates. Inj Prev. 2005;11(2):
77—83.
Irvin et al. | Peer Reviewed | Research and Practice | 1385
RESEARCH AND PRACTICE
TABLE 1—Continued
Utah
1.8
No
No
No
No
Vermont
1.31
No
Yes
Yes
No
Virginia
4.32
Yes
Yes
Yes
No
Washington
2.39
Yes
Yes
No
No
West Virginia
3.62
No
No
No
No
Wisconsin
Wyoming
2.46
2.15
No
No
Yes
Yes
No
Yes
No
No
TABLE 2—Adjusted Effect of the State Regulations on Firearm Homicides: United States,
1995–2010
Outcome/Laws
IRR (95% CI)
Homicide rate
34.65
Licensing
0.74* (0.67, 0.81)
Record keeping
1.45* (1.30, 1.61)
Inspections
0.64* (0.59, 0.69)
Theft reporting
Licensing and inspections
1.04 (0.95, 1.14)
0.49* (0.42, 0.58)
Strength
1 law
AIC
34.65
1.10 (0.96, 1.26)
2 laws
0.94 (0.85, 1.05)
3 laws
0.76* (0.67, 0.86)
4 laws
0.75* (0.65, 0.86)
Note. AIC = Akaike’s information criterion; CI = confidence interval; IRR = incident rate ratio. Covariates in the model included
race, percent urban, percent living in poverty, percent male, percent younger than 24 years old, percent college educated,
drug arrest rate, burglary rates,12 scores, and firearm ownership proxy.
*P £ .001.
6. Conner KR, Zhong Y. State firearm laws and rates of
suicide in men and women. Am J Prev Med. 2003;25
(4):320—324.
11. Wiebe DJ. Homicide and suicide risks associated
with firearms in the home: a national case-control study.
Ann Emerg Med. 2003;41(6):771—782.
7. Webster DW, Vernick JS, Zeoli AM, Manganello JA.
Association between youth-focused firearm laws and
youth suicides. JAMA. 2004;292(5):594—601.
12. Kappas S. Traveler’s Guide to the Firearm Laws of
the Fifty States. Covington, KY: Traveler’s Guide Inc;
1997—2010.
8. Webster DW, Vernick JS, Bulzacchelli MT. Effects
of state-level firearm seller accountability policies on
firearm trafficking. J Urban Health. 2009;86(4):525–537.
13. Braga AA, Wintemute GJ, Pierce GL, Cook PJ,
Ridgeway G. Interpreting the empirical evidence on
illegal gun market dynamics. J Urban Health. 2012;89
(5):779—793.
9. Vernick JS, Webster DW, Bulzacchelli MT, Mair JS.
Regulation of firearm dealers in the United States: an
analysis of state law and opportunities for improvement.
J Law Med Ethics. 2006;34(4):765—775.
14. Webster DW, Vernick JS, Bulzacchelli MT, Vittes
KA. Temporal association between federal gun laws and
the diversion of guns to criminals in Milwaukee. J Urban
Health. 2012;89(1):87—97.
10. Miller M, Hemenway D, Azrael D. State-level
homicide victimization rates in the US in relation to
survey measures of household firearm ownership, 2001–2003. Soc Sci Med. 2007;64(3):656—664.
15. Webster DW, Bulzacchelli MT, Zeoli AM, Vernick
JS. Effects of undercover police stings of gun dealers on
the supply of new guns to criminals. Inj Prev. 2006;12
(4):225—230.
1386 | Research and Practice | Peer Reviewed | Tiwari et al.
The Impact of Data
Suppression on Local
Mortality Rates: The Case
of CDC WONDER
Chetan Tiwari, PhD, Kirsten Beyer, PhD, MPH,
MS, and Gerard Rushton, PhD
CDC WONDER (Centers for Disease Control and Prevention
Wide-Ranging Online Data for Epidemiologic Research) is the nation’s
primary data repository for health
statistics. Before WONDER data
are released to the public, data cells
with fewer than 10 case counts are
suppressed. We showed that maps
produced from suppressed data
have predictable geographic biases
that can be removed by applying
population data in the system and
an algorithm that uses regional
rates to estimate missing data. By
using CDC WONDER heart disease
mortality data, we demonstrated
that effects of suppression could
be largely overcome. (Am J Public
Health. 2014;104:1386–1388. doi:
10.2105/AJPH.2014.301900)
CDC WONDER (Centers for Disease Control
and Prevention Wide-Ranging Online Data for
Epidemiologic Research) provides county-level
data on directly age-adjusted mortality rates, and
age- and gender-stratified mortality and population counts.1 To protect against the potential
disclosure of personal health information,
WONDER suppresses any statistic (counts or
rates) calculated using fewer than 10 observations.2 However, such suppression restricts the
utility of WONDER data to compute and map
reliable rates for areas with small populations, for
short time periods, or for rare diseases.3,4 Furthermore, rates that are indirectly adjusted for
age, which are currently not provided by WONDER, can only be calculated for those counties
where count data are not suppressed.5,6 Using an
example of heart disease mortality, we showed
American Journal of Public Health | August 2014, Vol 104, No. 8