NEW_Where to find evidence

BETA Contact us to help improve this site

Where to find evidence

Toolkits

Toolkits distil complex research into clear, actionable insights, making them invaluable for busy practitioners. They are typically created by aggregating high-quality evidence from systematic reviews, evaluations, and meta-analyses. Usually, each intervention is assessed for effectiveness, cost, ease of implementation, and relevance, ensuring users have access to rigorously curated information.

Key toolkits for EBP:

What Works Centre for Crime Reduction (WWCCR) Toolkit: A toolkit that uses the EMMIE framework to evaluate interventions based on their effectiveness, mechanisms, moderators, implementation, and economic cost. It summarises evidence in plain language, with easy-to-read ratings for outcomes like crime reduction or community trust.
Youth Endowment Fund (YEF) Toolkit: A toolkit focused on interventions aimed at preventing youth involvement in crime and violence. It offers detailed evaluations of approaches like mentoring, school-based programs, and family support, complete with cost-effectiveness insights.
The Evidence-Based Policing (EBP) Matrix: A visual tool categorising studies based on proactivity (reactive to proactive), specificity (general or focused), and target (individual or place). It only includes interventions rated Level 4 or 5 on the Maryland Scientific Methods Scale, ensuring high methodological rigor.

Advantages of toolkits:

Accessibility: Toolkits break down technical research into user-friendly formats, using visuals, summaries, and practical recommendations.
Efficiency: They save time by eliminating the need to navigate lengthy academic papers or raw data.
Transparency: Users can see not just what works but also why it works (mechanisms), under what conditions (moderators), and at what cost (economic evaluation).
Tailored guidance: Many toolkits include filters to refine results based on your needs, such as specific crime types, populations, or operational priorities.

Databases

Databases are online repositories that catalogue published studies, evaluations, and research papers. In the context of evidence-based policing, they are essential tools for accessing the latest findings, understanding prior studies, and identifying high-quality evidence to guide decision-making.

Databases offer several benefits:

Comprehensive search capabilities: They include peer-reviewed studies, grey literature, and reports from various disciplines relevant to policing, criminology, and public safety.
Advanced filters: Allow users to refine searches by topic, publication year, methodology, and more.
Access to full-text papers: Many databases provide direct links to full studies, though some require subscriptions.

Key databases for EBP:

Global Policing Database (GPD): One of the most comprehensive collections of intervention studies related to policing. It contains over 7,000 studies covering topics such as crime prevention, procedural justice, and technology use in policing.
Web of Science: A multidisciplinary database that includes high-impact journals in criminology, social sciences, and public policy. It is excellent for identifying foundational studies and tracking citation networks to see how research has evolved. Advanced tools also allow for citation analysis and finding the most influential works in EBP.
Campbell Collaboration Library: Focuses on systematic reviews and meta-analyses across social sciences, including crime and justice. Offers plain-language summaries and detailed breakdowns of evidence quality, making it accessible to non-academics.
PubMed: Though primarily used in medicine, PubMed includes studies on public safety, mental health, and substance abuse that intersect with policing. Useful for exploring topics like trauma-informed policing or interventions related to health and safety.
ProQuest Criminal Justice Database: Specialises in criminology, law enforcement, and security management topics. Features both academic papers and professional reports, bridging the gap between theory and practice.
Google Scholar: A broad search tool for academic content, including peer-reviewed articles, conference papers, and theses. Users can set up alerts for keywords like "evidence-based policing" to stay updated on new research.
SAGE Journals Online: Provides access to a wealth of social science and criminology research, including studies published in leading journals like Policing: A Journal of Policy and Practice. Frequently includes articles with practical applications for police agencies.

Tips for Using Databases

Start Broad, Then Refine: Use general terms like "crime prevention" or "evidence-based policing" initially, then narrow by keywords, regions, or methods.
Check Abstracts First: Abstracts summarize a study's focus, methodology, and findings, helping you determine relevance quickly.
Explore References: Studies often cite other key research, providing a trail to follow for additional evidence.
Leverage Alerts: Many databases let you set alerts for specific topics, ensuring you stay up-to-date with the latest findings.

Databases provide the foundation for evidence-based decisions, offering access to the rigorous research needed to assess and implement effective policing practices.

Systematic reviews

A systematic review is a research method that combines findings from many studies to answer a specific question. By carefully evaluating all available evidence, systematic reviews help identify what works, what doesn’t, and under what circumstances.

How Systematic Reviews Are Done

Define the Question: Researchers start with a focused question, like “Do body-worn cameras reduce complaints against police?”
Search for Studies: They look through databases, journals, and reports to find all the studies on the topic.
Pick the Best Evidence: Only studies that meet strict quality standards are included. For example, studies with robust methods, like randomized controlled trials, are prioritized.
Summarize Findings: Researchers combine the results of these studies to identify patterns and trends.

This process ensures that the review includes a wide range of high-quality evidence, making it more reliable than relying on individual studies.

Why Systematic Reviews Are Highly Trusted

Systematic reviews are near the top of the evidence hierarchy because they:

Summarize Multiple Studies: They look at all the available evidence instead of focusing on just one or two studies.
Reduce Bias: The process is transparent, so there’s less chance of cherry-picking results to support a particular viewpoint.
Provide Clear Answers: By combining evidence, systematic reviews help practitioners make informed decisions about what works in policing.

For example, systematic reviews have shown that hotspot policing is an effective way to reduce crime without pushing it to other areas.

What Is a Meta-Analysis?

A meta-analysis is a part of some systematic reviews that uses statistics to combine results from multiple studies into one clear answer. It calculates an overall “effect size,” showing how strong the evidence is for an intervention.

For instance, a meta-analysis on body-worn cameras might find that their use leads to a 20% reduction in complaints against officers (this reduction is the "effect size").

Why Meta-Analyses Can Be Hard to Understand

Effect Size: This tells you how much of a difference the intervention makes, but interpreting it requires understanding what the numbers mean.
Variability Between Studies (Heterogeneity): This measures how different the studies are from each other. For example, one study might test body-worn cameras in a large city, while another tests them in a small rural town. These differences can affect the overall results.
What is heterogeneity? It’s simply the variation or differences between the studies included in the review. Researchers try to account for these differences, but it’s important to consider when applying the findings.

The Campbell Collaboration

The Campbell Collaboration is a global organization that produces systematic reviews focused on social sciences, including crime and justice. Their reviews are highly regarded for their rigor and practical utility.

Plain-Language Summaries: Campbell reviews are written to be accessible, even for non-academics, making them especially useful for police practitioners.
Examples of Topics: Reviews cover a range of policing issues, such as the effectiveness of neighbourhood watch programs, strategies to reduce gang violence, and interventions to improve procedural justice.

Using resources like the Campbell Library ensures that decisions are grounded in the best available evidence.

How Systematic Reviews Help Policing

Systematic reviews are practical tools for making evidence-based decisions:

Save Time: Instead of reading dozens of studies, a systematic review provides a clear summary of what works.
Reduce Risks: They highlight proven strategies, so resources aren’t wasted on ineffective interventions.
Inform Training and Policy: Reviews can guide officer training programs or shape policies based on what has been shown to work.

For example:

A systematic review might confirm that restorative justice programs are effective in reducing reoffending, helping a department justify their use.
Another review might show that predictive policing tools require careful implementation to avoid bias, guiding agencies on their ethical use.

Systematic reviews and meta-analyses are powerful tools for understanding the effectiveness of policing strategies. While meta-analyses can include complex statistics, focusing on the practical conclusions allows practitioners to confidently apply evidence in the field.

Artificial intelligence tools

There’s a temptation to use AI for everything these days and finding evidence-based policing tactics is no different. In fact, AI tools can be really helpful if used in the right way and with care. Here is a description of some of these tools:

Key AI Tools for Finding Research

ChatGPT:
- What It Does: A conversational AI model that can explain concepts, answer questions, and summarize research in plain language.
- Strengths: Easy to use, great for quick summaries or generating ideas.
- Limitations: It doesn’t access live databases or provide citations unless explicitly connected to curated datasets.

Research Rabbit:
- What It Does: A research discovery tool that allows users to explore related studies by visualizing connections between papers, authors, and topics.
- Strengths: Excellent for building a deeper understanding of a research area and discovering less obvious connections.
- Limitations: Requires some familiarity with academic research to interpret the connections effectively.

Consensus:
- What It Does: Uses AI to scan academic papers and provide direct answers to research questions, often with supporting evidence.
- Strengths: Focuses on providing evidence-backed answers and highlighting the strength of the evidence.
- Limitations: Coverage is still developing, and nuanced topics may yield incomplete results.

Elicit:
- What It Does: Specializes in assisting with literature reviews, identifying relevant papers, and comparing study results.
- Strengths: Designed for systematic reviews, it helps users identify gaps or patterns in the evidence.
- Limitations: Best suited for users familiar with systematic review methods, as it doesn’t replace expert judgment.

Limitations and Potential Problems

Accuracy and Bias: AI tools are only as good as the data they are trained on. If the underlying data is biased or incomplete, the outputs may be misleading. These tools can sometimes generate inaccurate or oversimplified summaries, especially for complex or contested topics.

Lack of Context: AI might miss the nuance or context of a study, such as limitations in methodology or applicability to specific policing scenarios.

Citation Issues: Tools like ChatGPT may not provide citations unless linked to curated sources, making it difficult to verify the origin of the information.

Over-Reliance: Users may overly depend on AI, risking the loss of critical thinking or neglecting deeper dives into primary research.

Using AI Tools Effectively and Responsibly

To maximise the benefits of AI tools while minimising risks:

Validate Information:Always cross-check AI-generated summaries or answers with original sources to ensure accuracy.

Focus on Specific Questions: Use clear and specific queries to get the most relevant and useful results.

Combine with Traditional Research: Use AI tools as a starting point, but follow up with traditional research methods, especially for critical decisions.

Understand Limitations: Be aware of the tool’s scope and avoid using it for highly specialized or nuanced questions without further verification.

Protect Data Integrity: Avoid inputting sensitive or confidential information into AI tools, as they may store or process data in ways that aren’t secure.

AI tools can significantly enhance how policing practitioners engage with research by making evidence more accessible and actionable. When used responsibly, these tools can save time, broaden understanding, and improve decision-making.

In your own organisation

Many valuable sources of evidence exist within your own organisation. From crime reports to internal evaluations, these materials—often called grey literature—can provide insights that academic studies might overlook.

What is Grey Literature?

Grey literature refers to research and reports that are not formally published in academic journals. Examples include:

Internal evaluations of policing interventions.
Crime analysis reports.
Policy reviews or white papers.
Presentations or training materials.

Grey literature is typically created to address immediate, practical questions within organisations, making it highly relevant to local challenges. However, it often lacks the formal peer review process used in academic publishing, so its quality and reliability can vary.

Evaluating Studies and Reports from Your Organisation

When using internal research, it’s essential to critically assess its credibility and applicability. Here are some things to look out for:

Clarity of Purpose: Does the report clearly define its objectives? For example, is it measuring the effectiveness of a new initiative or examining broader trends?

Data Quality: Are the data sources reliable? For instance:

Are crime statistics complete and accurately recorded?
Were survey questions unbiased and clear?
Incomplete or poorly recorded data can lead to misleading conclusions.

Methodological Rigour: Is the study design robust? Common issues to check for:

Lack of a control group to compare outcomes.
Small sample sizes that might not represent broader populations.
Failure to account for alternative explanations (e.g., crime trends unrelated to the intervention).

Transparency: Are the methods and assumptions clearly stated? If it’s unclear how conclusions were reached, the findings may be unreliable.

Bias: Does the report reflect an agenda? For example:

Was it commissioned to justify a policy decision rather than evaluate it impartially?
Are the authors connected to the programme being assessed?

The Role and Merits of Peer Review

One of the key differences between academic research and grey literature is peer review. This process involves independent experts evaluating research before it is published. Peer review increases the value of research evidence because it:

1. Ensures Rigour: Reviewers check that the study’s methods and findings are robust and reliable.

2. Identifies Errors: Peer review catches mistakes, inconsistencies, or oversights that might compromise the study.

3. Reduces Bias: Independent reviewers provide an impartial assessment, minimising the influence of vested interests.

While grey literature often skips this step, that doesn’t mean it isn’t useful—it just requires more critical examination by the user.

Making the Most of Internal Evidence

To use internal studies and reports effectively:

Supplement with External Evidence: Cross-check findings against peer-reviewed studies or systematic reviews to ensure consistency.

Seek Expert Input: Where possible, have internal evaluations reviewed by external experts for an impartial perspective.

Encourage Documentation: Ensure that internal reports clearly document their methods, data sources, and limitations. This increases their reliability and utility.

Balance Local and General Insights: Internal evidence is often highly specific to your organisation or region. While this makes it relevant, combining it with broader studies ensures more robust conclusions.

Grey literature from within your organisation can be a powerful tool when used responsibly. By critically evaluating its quality and combining it with external evidence, you can maximise its value for evidence-based decision-making.