Australian Institute of Criminology

Skip to content

Evaluation methodology

This report presents the findings from research conducted by the AIC, which has involved working closely with ACTP to undertake an evaluation of the operation and effectiveness of policing strategies directed at licensed premises and entertainment precincts in the Australian Capital Territory. This project was undertaken in response to a request from the AIC Board of Management to undertake research into the impact of policing strategies on the harms associated with the excessive consumption of alcohol and was subsequently approved by the ACTP Chief Police Officer.

The purpose of this research was to determine the impact of proactive policing and enforcement strategies in areas with a high concentration of licensed premises on:

  • the perceived risks and costs associated with breaching liquor licensing laws and the actual level of compliance with liquor licensing legislation and regulations;
  • the patterns of consumption among patrons of licensed premises and the impact of those premises on the social and cultural environment in entertainment precincts;
  • the nature and the level of alcohol-related harm in areas with a high concentration of licensed premises; and
  • perceptions and the experience of public safety.

This was intended to help ascertain the most effective methods of policing licensed premises and the relative impact of proactive policing and enforcement strategies. In doing so, the research aimed to determine whether there is an optimal level and nature of policing activity, which maximises the benefits associated with the activity relative to the cost of resources required.

The current project

The AIC undertook a process and outcome evaluation of the ACTP response to alcohol-related crime in entertainment precincts over the 2009–10 summer period. In particular, this current project involves an evaluation of the following strategies:

  • front-line policing, which involved ‘general duties’ officers patrolling entertainment precincts and providing a visible police presence during the peak periods of alcohol service and consumption;
  • the RLLP, which was a four-stage project developed and implemented by ACTP Crime Prevention and was designed to educate, facilitate and enforce responsible liquor licensing within the Civic entertainment precinct;
  • monitoring, regulation and enforcement of licensed premises in partnership with ORS; and
  • intelligence gathering and analysis to identify problematic locations and premises, which was designed to help inform front-line policing, enforcement operations and the RLLP.

In addition, this component of the larger project aimed to provide baseline data to measure the impact of future strategies delivered in the Civic entertainment precinct, particularly through the introduction of legislative changes and reorganisation of police tasked with the responsibility for liquor licensing issues.

This component of the project aimed to address the following key research questions:

  • What is the precise nature and level of alcohol-related crime associated with licensed premises in the Civic and Manuka/Kingston entertainment districts?
  • Were the policing strategies developed by ACTP to address alcohol-related crime in entertainment precincts implemented according to how they were designed and what factors impacted upon the operation of these strategies?
  • What characteristics of the ACTP approach to policing licensed premises contribute to their overall effectiveness as crime reduction strategies?
  • What impact does improved intelligence relating to violence and other alcohol-related offending in and around licensed premises have on the capacity of police to address alcohol-related crime in entertainment precincts?
  • What short-term impact do the proactive policing and enforcement strategies delivered as part of the ACTP response to alcohol-related crime in entertainment precincts have on:
    • the level of compliance with liquor licensing legislation and regulations?
    • the patterns of consumption and problematic drinking behaviour among patrons of licensed premises?
    • the nature and the level of alcohol-related harm in areas with a high concentration of licensed premises?

Research design

To address these research questions, the AIC worked with ACTP to evaluate evidence-informed policing and enforcement strategies aimed at reducing the extent of alcohol-related problems in areas with a high concentration of licensed premises. The AIC identified the Australian Capital Territory as a jurisdiction to be included in this research project as Canberra represents a compact licensed environment, unique in that there are a number of distinct, but relatively small, entertainment precincts amenable to the proposed research project. Recently, these Canberra entertainment precincts have received considerable attention in relation to alcohol-related problems associated with licensed premises; particularly in the local media. For instance, The Canberra Times newspaper conducted a four part series in December 2009 (12–15) reporting on aspects of policing the Civic entertainment precinct. The four part series also coincided with a nationwide police operation (Operation UNITE), which will be discussed in this report.

This project has involved the development of a quasi-experimental research design that achieves level three on the Scientific Methods Scale (Farrington et al. 2006), considered the minimum design for drawing conclusions regarding the effectiveness of crime prevention interventions. This requires that appropriate measures of the dependant variables (including levels of compliance, patron behaviour and rates of alcohol-related harm) are developed to assess the impact of the intervention (policing strategies) before and after the program in both an experimental and comparable control condition.

The Civic entertainment precinct was selected as the intervention area and the Manuka/Kingston entertainment precinct was selected as an appropriate comparison area. These locations were selected in consultation with ACTP. They were identified as having a high concentration of licensed premises, a disproportionate rate of alcohol-related harm and being locations with potential to implement evidence-informed policing strategies targeting non-compliance with liquor licensing laws.

The development of evidence-informed policing strategies

The AIC’s role in this project was to manage the research and evaluation component, identify appropriate measures of performance, design and monitor the implementation of relevant data collection mechanisms, analyse the data collected and to provide feedback at regular intervals as to the efficacy of the interventions being implemented. Primary responsibility for the design and implementation of evidence informed policing strategies rested with ACTP. The AIC sought to contribute in an advisory capacity and to ensure that appropriate consideration was given to the implications for the evaluation in the design of these interventions.

As part of this advisory role, the AIC submitted to the ACTP a list of possible policing strategies and the evidence in support of the different approaches. From this list, ACTP indicated support for the following strategies:

  • Lockouts—involves licensed premises not being able to allow entry to new patrons after a specified time, such as 2 am. This aims to restrict late-night movement of patrons between premises.
  • Improvements to public transport—targeting queues for public transport late at night by increasing the availability of transport for patrons leaving licensed premises, improving the design on public transport facilities and providing greater security to manage these facilities.
  • Liquor accords—collaborative initiatives involving police, licensees and other key stakeholders developing guidelines or codes of practice specifying harm minimisation principles and practices to reduce alcohol-related inappropriate behaviours.
  • Greater enforcement of liquor licensing—targeting licensees who do not adhere to liquor licensing legislation or regulations and prosecuting breaches (in partnership with ORS).
  • High-visibility targeting of drink driving—establishing Random Breath Testing operations in locations around the Civic entertainment precinct, targeting patrons leaving late at night.
  • Increased signage outside of licensed premises—displaying signage such as ‘full house’, which may influence people to move on elsewhere rather than lining up and/or loitering. Capacity signage can also remind staff (including security) of the maximum number of patrons allowed.

While a high-level of support existed for these strategies, it became apparent that there were already a number of strategies planned for the 2009–10 summer period (some of which were consistent with those listed above). The time and resources required to develop and implement additional strategies (and other practical barriers, including limitations with the existing legislation) in time for summer was considered to be prohibitive. Therefore, the AIC and ACTP elected to focus the current evaluation on reviewing the implementation of and, where possible, the effectiveness of strategies that were designed to address problems in Civic during the intervention period.

Data collection methods

A range of research techniques were employed to gather the information required for undertaking the evaluation of policing strategies, approved by the AIC’s Human Research Ethics Committee. These methods included stakeholder interviews, observational research in and around licensed premises, an online survey for the general community, analysis of recorded offence and incident data from ACTP and ‘place of last drinks forms’ developed by the AIC and completed by general duties officers. These are described in more detail below.

Stakeholder interviews

The AIC conducted interviews with a number of ACTP members over the course of the project, including (but not limited to) those working in the crime prevention, drug and alcohol policy, and general duties sections. The AIC also interviewed representatives from ORS, who are responsible for liquor licensing in the Australian Capital Territory. In addition, interviews were conducted with licensees from the Civic and Kingston/Manuka entertainment precincts. The interviews were all semi-structured in nature and covered a range of aspects relating to the project including:

  • current issues regarding the effective management of entertainment precincts;
  • perceptions regarding the role of police in the management of entertainment precincts;
  • views regarding existing policing strategies;
  • key issues to consider with respect to the implementation of policing strategies in the target areas;
  • the relationship between policing and other strategies being delivered in the target area;
  • the impact of policing on patron drinking behaviour, including the pattern and level of consumption, and patron behaviour in and around licensed premises;
  • key indicators of alcohol-related harm, such as alcohol-related violence, property offending and public injury due to assault; and
  • (for police only) current and potential policing strategies, mechanisms for monitoring the implementation and impact of policing.

Participants were asked a standard set of questions relating to the project (see Appendix 1). While direct quotes have been used in this report, they have not been attributed to individuals.

Observations in and around licensed premises

Observational research was used to examine issues relating to the management of licensed premises, problems associated with licensed premises and intoxicated patrons, and to monitor existing police strategies in the entertainment precinct. Direct observation, supplemented by other sources of data, is an effective mechanism for the study of licensed settings and has been used widely in past research in Australia and overseas (eg Graham et al. 1998; Homel et al. 2004; Sim, Morgan & Batchelor 2005). This observational method was suitable for this research project as the behaviour under study occurred openly and in a public place. Non-participant observation, which uses independent researchers rather than individuals involved in the delivery of an intervention to collect data, minimises the chance that research subjects will alter their behaviour due to the presence of the researchers in the research environment. This approach is particularly appropriate for busy, public venues such as licensed premises and entertainment precincts.

This methodology was based on best practice in international research (Sim, Morgan & Batchelor 2005). The observational research focused on the Civic and Manuka/Kingston entertainment precincts and licensed premises within these precincts, as well as the conduct of patrons in and around these premises. This involved observations both outside and (with the consent of licensees) inside licensed premises. A total of 20 hours (pre- and post-intervention) of data collection was conducted, with the findings from the observational research discretely documented by researchers in accordance with the guidelines provided in Appendix 2.

Online survey

The AIC also developed an online survey for both residents and business owners/operators in the Australian Capital Territory. The online survey covered a range of issues relating to the project including:

  • community perceptions of alcohol-related harms in entertainment precincts;
  • views on perceptions of safety and crime in and around licensed premises at various times of the day;
  • experience with responsible service and premise management practices; and
  • personal drinking patterns and attitudes towards alcohol and the licensed environment.

The final questionnaire was developed in consultation with key project stakeholders and based upon a review of other surveys used for similar research projects. The survey was promoted on the AIC’s website and other internet sites, as well as through various print and radio sources. The survey was based on convenience sampling (only people who become aware of the survey and who were willing to participate) and as such the survey was not random and the findings from the survey are not necessarily representative of the views of the wider community.

There were a total of 98 responses to the online survey for ACT residents. The low response rate prohibits any generalisations from the survey to the broader population and the results from the analysis of survey data outlined in various sections of this report should be interpreted with some caution. However, it does provide some useful information regarding perceptions of entertainment precincts among those who responded to the survey. The survey for business owners and operators in the intervention and control areas was discontinued due to a lack of respondents and difficulties promoting the survey to the intended participants.

Analysis of data provided by ACT Policing and the Office of Regulatory Services

Throughout this report, results will be provided from analyses conducted on two datasets provided by ACTP for offences and apprehensions in the ACT region as well as data on breaches by licensees provided by ORS. The de-identified data supplied by ACTP from their Police Real-time On-line Management Information System (PROMIS) database, for offences and apprehensions covered the entire ACT region for the time period 1 July 2005 to 18 April 2010. Most of the analysis conducted was focused on the intervention and control sites (Civic and Kingston/Manuka), as well as specific offences often associated with alcohol and entertainment precincts (eg assault). In addition to the ACTP data, the AIC received data from ORS relating to breaches of liquor licensing legislation. These data were used to provide some examples of the type of breaches that are made by licensees and the frequency of such breaches. Analysis of all the data documented trends and characteristics of recorded crime, apprehensions, alcohol-related incidents and compliance with liquor licensing legislation in entertainment precincts.

Place of last drink forms

Building upon work in other jurisdictions, the AIC worked closely with ACTP Crime Prevention, intelligence and drug and alcohol units to develop and implement a ‘place of last drink’ form to be used by the Beats teams. These forms were based on a model currently utilised by WA Police and were similar to the data collection mechanisms that have been implemented by police in other states, such as New South Wales and Queensland. Completing the forms required officers to record additional information on all incidents attended in which the person of interest or victim had consumed alcohol, including:

  • the type of incident attended;
  • the premise at which the incident took place;
  • details of the person spoken to;
  • whether the person was intoxicated;
  • the location and time of their last drink; and
  • the time of incident.

Completed forms were then submitted to the AIC, for data entry, analysis and to provide regular reports back to ACTP.