Australian Institute of Criminology

Skip to content

The AMCOS performance management framework

The current AMCOS Performance Management Framework is approximately three years old (see Appendix for framework). It was initiated early in 2009 in response to a degree of criticism from within New Zealand Police as to the visibility of AMCOS activities. The rationale for its creation at the local level is succinctly expressed in the words of Sean Price, Chief Constable of Cleveland Police Force in England:

Performance management is an area where there is no single accepted model for the police service, and any such model or framework needs to be fit for purpose (Home Office 2008: np).

The framework rests on a simple concept—first, identify what is important and second, work out how to measure what is important (Braga & Moore 2003a). Therefore, purpose determines the performance indicators chosen. The starting point for the framework was strategic guidance and specifically, the three Strategic Goals and associated outcomes listed in high-level New Zealand Police policy documents. From these, a number of outputs and activities were identified. Table 1 shows this first phase of classification.

Table 1 Initial top level derivation of performance categories
Outputs Activities
Evidence-based policing Intelligence management
Knowledge management
Timely and effective responses Deploy to risk
Manage organised and serious crime investigations
Thorough investigations Forensic support
Strategic collaboration Manage government relationships
Manage agency relationships
Liaise with other agencies

Source: APMF 2009

The authors identified these contributory outputs and activities through an eclectic mix of literature review, first-principles reasoning and consultation. This process of linking outputs to outcomes would not have passed scientific muster, but was the best possible at the time (given resource and time constraints) and as is noted below, has been substantially improved already. As such, the framework began as an output/activity framework linked to outcomes. This was done for the reasons already cited earlier in the paper on the difficulty of developing useful outcome measures for specialist policing activities without being overly prescriptive (see Vollaard 2006).

The value of such a strategic derivation seems obvious, but it appears to be relatively rare within police, although it is common in militaries. At least within New Zealand Police (and presumably overseas, given the literature surveyed), performance measures are largely presented in decontextualised fashion, separated from the strategic goals the organisation pursues except in the broadest possible terms, such as ‘crime reduction’. There is seldom a clear hierarchical chain from outcome through enabler through output, activity and input. This separation makes it difficult to identify why a particular measure has been chosen and therefore divorces frontline practitioners from the strategic outcomes that should be their overall goal. The authors felt that an essential element in ensuring that any performance framework is understood and well-supported, as well as in ensuring that the framework remains outwardly focused rather than self-referential, was to make explicit the contribution to and links between low-level measures, and operational and strategic-level goals and objectives.

Former Commissioner of the New York Police Department, William Bratton, has talked about the value of a ‘decentralized management system with strong strategic guidance at the top’ (Bratton 1999: 16); the same applies to performance measurement. At the national level, the value is in identifying strategic performance areas, outcomes, impacts and Key Result Areas. However, this is not enough; lower levels must then analyse those high-level elements and distil from them the key measures relevant to their own business. The Home Office Police Standards Unit in England and Wales has said exactly this when it emphasised the importance of varying performance frameworks across—and by extension, within—forces to reflect differing circumstances (Home Office Police Standards Unit 2004). Alongside the top-down, strategic goal-driven process, there was a bottom-up element. This was based on an analysis of the literature, which suggested that performance schemes built from the bottom-up—based on the ‘true nature’ of police work—had value (Carter, Klein & Day 1992; van Sluis, Cachet & Ringeling 2008). Flynn (1986: 392) has stated that ‘at their best, performance measures are developed by managers themselves to enable them to do their job better’. Stone and Travis (2011: 19), discussing professionalism in policing, note that ‘careful analysis of local problems and the custom crafting of solutions continue to be necessary’; the development of a performance framework is indeed a solution to a particular set of problems relating to accountability and learning.

One quality of locally developed frameworks is that they are more likely to be perceived by those being measured as being focused on learning and improvement, rather than being strict accountability tools—‘sticks’ to use a colloquial term. Willis, Anderson and Homel (2011: 4) have noted:

measurement systems designed to focus on performance improvements (as opposed to instruments of control) are much more easily accepted than systems designed exclusively for accountability purposes.

This is a particular problem given police traditional culture, which has often been seen as a challenge for New Public Management-style techniques and the accountability that accompanies those techniques (Butterfield, Edwards & Woodall 2004; Fleming & Lafferty 2000). By allowing staff to participate in the development of the framework, not only would the framework be better, but staff would also gain a sense of ownership and hopefully engage with it at a deeper level (Loveday 2006). Developing the scheme with input from the frontline also ensured that the scheme would remain focused on instrumental improvements to performance, rather than merely being a symbolic initiative designed to gain legitimacy for AMCOS (Roy & Séguin 2000). It was also anticipated that such an organic, bottom-up approach would help avoid neo-Taylorism—a separation between management (measurers) and workers (doers)—developing (Loveday 2006), which was felt to be likely if an externally developed framework was imposed without consultation.

Another benefit of developing the framework locally was agility. Any changes could be quickly made at the AMCOS level, involving only 300 staff, rather than having to wait for movement at the national level, where there is a much larger bureaucracy. This would help make the framework particularly flexible and responsive.

The bottom-up element of developing the framework involved visiting all AMCOS units and discussing their work. From this, key activities and potential measures could be identified, and just as importantly, things that need not be measured could also be identified (Carter, Klein & Day 1992). These discussions were vital for two reasons:

  • they helped ensure that any measures chosen were meaningful to the unit; and
  • they also helped the unit understand why the framework was being developed.

As might be expected, there was some minor concern initially about ‘counting everything’, but AMCOS staff have increasingly become more engaged with the framework, seeing it as the best mechanism for expressing the degree of work that they do. Care was taken to ensure that the measures chosen were parsimonious.

The top-down and bottom-up elements were then blended together into an integrated classification scheme of AMCOS performance measures. Where the two elements met, there was a process of creative destruction and rationalisation. The core defining characteristic of each low-level element was identified and it was then placed against what was deemed to be the most fitting higher level category. At times, the nature of the work done on the ground led to a slight re-definition of higher level categories, which being based on inductive reasoning sometimes failed to account for the evidence. If a higher level category seemed to have no available measures, it was left empty for the time being. An Output Dictionary was produced and disseminated, covering the scheme in its entirety. Table 2 shows a simplification of that output schema.

Table 2 Example of low-level measures identified and linked to higher level categories
Top level Intelligence management
Intermediate level Basic products Analytical products Field intelligence processes
Specific measures Notings Strategic assessments Gang events attended
Operation-specific support Tactical assessments Other gang contacts
Problem profiles Community events attended
Subject/offender profiles
Thematic assessments

Source: APMF 2009

Over time, the Performance Framework has evolved (see Table 3). One key change has been a move away from Police Strategic Goals to Police Output Classes as the highest level of the framework (New Zealand Police 2009). This was done due to an increased focus at the national level on those outputs, and has had the benefit of forcing further analysis of the various measures and reconsideration of their best location within the framework. As a result, the Output Dictionary has been revised and updated several times. An example of the types of specific measures that exist under each broader activity category is shown in Table 4.

Table 3 Simplified representation of current derivation of activity categories from high-level guidance
Output classes Specific APMF activity categories
Policy advice and ministerial servicing Intelligence management
Other knowledge management
General crime prevention services Gather community intelligence
Strategic collaboration
Communicate knowledge outside police
Code of conduct activities
Specific crime prevention services and maintenance of order Specialist operational support
Manage VIP and WitPro Ops
Manage civil emergency responses
Police primary response management Specialist operational support
Investigations Forensic support
Active investigation processes
Terminated investigation processes
Covert support
Drug lab services
Case resolution and support to the judicial process Criminal case resolution
Road safety program n/a

Note: Framework has been modified to allow its replication here

Source: APMF 2011

Table 4 Simplified representation of measures linked to higher level activities
Drug lab services
Deployment to suspected clan labs Clan labs dismantled Location assessments High-value chemicals seized

Source: APMF 2011

In developing the framework, thought was also given as to how to gauge the framework’s success. Here, the thinking of Bititci et al. (2006) was crucial—is the performance scheme being used to regularly monitor performance and make decisions, and do staff see the system as valuable? Given the long-term nature of most AMCOS activities, it was intended that holistic formal reports would be produced every six months, with some performance indicators presented on a monthly basis. As noted further below, the framework is now transitioning to a quarterly tempo. It was also decided to survey staff on their opinion of the framework. The results of those surveys are covered later in the penultimate section on challenges and successes.

Overall, by developing the framework internally within AMCOS, it was felt that several key benefits would result. First, it would ensure that the framework was accurate, meaningful and engaged with the unique nature of specialist policing. Second, it would ensure that staff would gain a sense of ownership over the scheme and thus it would be supported. Next, it would ensure that the scheme remained focused on real performance, rather than merely symbolism. Finally, it would allow the framework to be flexible and responsive to changing circumstances. As a result of all the above, it was felt that the framework would overcome many of the challenges noted earlier in relation to performance measurement in general, be less vulnerable to perverse behaviour and therefore be a better tool for management.

The performance framework in detail

The value of the AMCOS Performance Framework comes primarily from its overall structure, rather than its individual components. These individual measures are, for the most part, not unique. However, when linked with other measures and activities, one gains a better appreciation of the quality and quantity of specialist policing services delivered by AMCOS and their potential impact on policing outcomes.

In the following sections, the performance measures are described according to four specific areas that fall within the broader concept of technical and niche units introduced earlier—forensics, operations support, investigations and intelligence. Before discussing those specific measures, however, it is useful to first consider some performance measures common to all AMCOS units, measures that are among some of the most innovative in the entire framework.

As was noted earlier, professionalism in policing rests upon a commitment to continuous learning and innovation, a willingness to reflect, and thus the creation and sharing of knowledge (Bradley 2005; Stone & Travis 2011). It is an area that police agencies have seldom been good at (Pendleton 2005). One common set of performance measures for all AMCOS units relates to the sharing of knowledge, both within and outside police, and the maintenance of good working relationships with governmental, non-governmental, and international partners and agencies. Knowledge sharing cannot and perhaps should not, be easily quantified; the number of meetings attended may be a simple indicator, but it lacks meaning. Rather, performance in these areas is reported on in a descriptive, qualitative manner, which provides a rich source of data for evaluation and comparison.

Another set of common measures relates to training and development activities, including levels of compliance with required training categories. It would be useful in future to expand this to something akin to a Military Essential Tasks List, which is a detailed method of identifying the level of preparedness and capability of a particular military unit (Global 2011; Tritten 1997).

Key to the learning and innovation aspect of the framework is the Centre for AMCOS Lessons Learned (CALL; Alach 2010a). CALL is an evaluation system that closely examines AMCOS activities, either individual operations or a closely linked series of occurrences. As a performance measurement tool, CALL is too resource intensive to be applied to all activities—a single investigative operation CALL report can take a month and AMCOS terminates more than 50 investigative operations a year—but it provides a useful adjunct to the broader, more easily reported elements of the framework. In particular, it contributes to our understanding of the relationship between outputs and outcomes (Boba 2003; Bradley 2005; Braga & Moore 2003a; Haberman & King 2011; Hughes, McLaughlin & Muncie 2001; Lum 2009; Vollaard 2006; Weisburd & Neyround 2011).

Because of this, CALL is vital to the future simplification and enhancement of the performance framework, not only for AMCOS and specialist policing, but also for New Zealand Police and law enforcement in general. This is because, as understanding of the linkages between outputs and outcomes improves, there is less worry about actual outcome measures. This may seem counterintuitive, so deserves further explanation.

Let us consider an outcome goal, Y. Measuring Y may be difficult, expensive and result in massive fluctuations over short time periods, thus making frequent measurement difficult. However, if we can be confident in a link between output X and outcome Y, then we can focus our measurements on output X, which is likely to be far easier to do, less expensive and more amenable to frequent reporting. It will still be necessary to occasionally confirm that the output–outcome relationship remains, but this can be done for auditing and reassurance reasons, rather than strictly for performance measurement. As such, the more CALL does to link specialist policing outputs with policing outcomes, the greater the future benefits for performance measurement are likely to be, both in effectiveness and efficiency.

Forensics performance measures

Forensic units fall squarely within the concept of technical units raised earlier, where the problem is that while contributory to major police outputs and outcomes, they are not responsible for such outcomes. As such, the challenge is to identify how much they have contributed and how important that contribution has been to overall results. The AMCOS forensics group consists of:

  • forensic imaging (photography and videography);
  • fingerprints; and
  • criminal profiling.

AMCOS once included a DNA collection squad, but that has since been disestablished; however, performance measures for that squad were developed and are mentioned below.

Forensic imaging

For forensic imaging, performance measures focus on the number of jobs (outputs) attended where photographs are taken and/or video is collected and analysed, for both standard jobs (attended when resources are deployed per a standard roster) and emergency jobs, where an immediate response is required, such as a major motorway accident. It was felt that focusing on jobs was the best way of identifying the overall impact of the unit, rather than focusing on (for example) the number of pictures taken. The next phase for forensic imaging will be to shift towards some outcome (from the perspective of the unit) measures by examining courtroom and evidential results, such as the number of cases supported by photographic evidence and potentially even the number of cases where photographic evidence was vital to success. Gathering the data for these next-level measures will be very difficult, as cases are the responsibility of investigators separate to the forensic imaging unit.

Fingerprint unit

The fingerprint unit already had the basics of a performance measurement scheme in place, focusing on the life cycle of a print:

  • the number of cases received (an indicator of workload);
  • the number processed/finalised (an indicator of available resources);
  • the number retained (an indicator of sample quality); and
  • the number of identifications made, both via a computerised system and also via any priority suspect list provided by investigators.

The next step was to include the number of partial court case files prepared by fingerprint officers, who unlike forensic imagers are often required to testify themselves. Other additional measures included the number of crime scenes attended, as well as the actual number of physical exhibits examined. In the future, as with forensic imaging, new measures will be developed to identify the success rate of cases with fingerprint evidence against those without, as well as identify changes in efficiency (the number of cases processed correlated against staff numbers), as well as highlight areas of external concern (if the number of cases retained drops, it may indicate poor work at crime scenes by first attenders).

Criminal profiling unit

The criminal profiling unit was treated similarly to the forensic imaging unit, in that its primary performance measures were requests received (demand indicator) and products produced (primary outputs). It was decided not to split ‘products produced’ into separate categories, largely due to the fact that there is no formal typology of profiling products and as such, it is easier to merely list overall production and then describe the details separately. A second set of performance measures relate to the ViCLAS database system, the projected centrepiece of the unit into the future. These show inputs into the systems as well as outputs (potential linkage reports, both successful and unsuccessful). The next step will involve greater focus on courtroom results, which as might be noted, will be a common factor in future performance measurement across all forensics groups. It is likely that an in-depth CALL report will be required to develop the analytical basis for such indicators.

DNA collection squad

Performance measures for the now-disestablished DNA unit were simple—the number of samples taken (both voluntary and involuntary), and the number of offenders and crimes linked to those samples. Had the DNA unit remained part of AMCOS, future work would have focused on the evidential value of identifications.

Specialist operational support performance measures

Specialist operational services also fall within the technical unit category. From a thematic, if not organisational perspective, AMCOS has four full-time specialist operations groups:

  • the VIP protection squad;
  • the maritime unit;
  • the dog section; and
  • the air support unit.

It also has a number of part-time groups whose performance measures have not received the same degree of attention.

VIP protection squad

From a VIP protection perspective, the most obvious performance measure is a negative one, namely nil harm to VIPs under protection. However, this is an unsatisfactory measure, because the absence of harm cannot be reliably construed as resulting from the presence of protection personnel unless the outcomes can be compared against the outcomes for a control group of VIPs who are not protected. Nor can the occurrence of harm automatically be assumed to be the result of poor VIP protection (ie non-responsibility). For example, someone shooting a VIP from a range of 1,500m with a military-grade sniper rifle is simply outside the ability of close-quarters VIP protection personnel to prevent.

Given this problem, outputs are the focus of VIP protection performance measures, namely the number of operations undertaken (this incorporates the fact that an operation is defined by particular quality standards, such as the publication of an operations order and adherence to standard operating procedures) and the number of person-hours of protection provided. There is no easy way to improve the measures in terms of outcomes. As such, any future enhancements of measures for VIP protection may borrow from Mackenzie and Hamilton-Smith (2011) and instead seek to ‘benchmark’ the unit against overseas best practice, which presumably has been derived from the evaluation of protection failures.

Maritime unit

The maritime unit presented great difficulties in identifying measures due to the sheer multiplicity of roles performed by the unit. It acts as a general duty response capability (in New Zealand Police terminology, a ‘marine I-car’), a search-and-rescue (SAR) manager, an investigator into stolen outboards and similar equipment, and also works with other government agencies on lengthy, offshore patrols. As such, its performance measures are also varied. Inspiration from naval and air force performance schemes around the world led to the inclusion of sea hours as a measure. The number of jobs attended is another key measure, as is the number of people apprehended at or near the sea; this latter element therefore allows for the measuring of results from investigations into stolen maritime equipment. Further measures examine seizures of weapons and drugs, the first of which is particularly important given that foreign sailors usually have limited understanding of New Zealand weapon laws. The number of SAR operations managed, and people located and recovered is another performance measure. Lastly, the soft performance measures relating to liaison with other agencies are vital to the Maritime Unit and thus receive substantial qualitative description.

Dog section

Similarly to the fingerprint section, the dog section already had a performance system in place, focusing on deployments to jobs (divided into those in which a dog was used and those in which it was not), the number of apprehensions and arrests, the number of incidents cleared and the total number of offences cleared. While these measures are similar, they each have a subtly different nuance and therefore changes in their inter-relationships over time can be particularly informative. Another set of performance measures examine the number of times specialist detector dogs (drugs, firearms, explosives and bodies) have been deployed and is logically followed by measures of the quantity and type of drugs and weapons seized.

Air support unit

The air support unit (ASU) has a clear role, which made development of performance measures relatively simple. As with the dog section, the primary output is the number of deployments to jobs, followed by the number of flying hours provided. The next group of measures can be regarded as outcomes from the ASU perspective and include the number of offenders detected by both day and night, in both raw terms and also as a percentage of total deployments. This measure shows changes over time in the efficiency and effectiveness of the ASU and can stimulate more in-depth consideration of the causes behind any changes. The final performance measure for the unit concerns the number of times that the ASU has acted as a personnel carrier and deployed a unit such as the Special Tactics Group.

Next steps in developing specialist operational support performance measures

The next stage for both air support and dog performance measures will be a greater focus on outcomes and in particular, the added value of the two units. With a large enough sample size, the difference in detection rates between live burglary (or other live criminal events, such as armed robbery) occurrences with and without air support can be identified; a similar calculation could be done for the dog section. While it is anecdotal, information received when the ASU briefly deployed to Christchurch in the aftermath of the recent earthquakes indicated that its influence may actually have been greater than is commonly perceived in Auckland; familiarity may well have bred a certain degree of contempt. Something akin to military-style Operational Analysis may be required to model the optimum mix of general duties branch staff, dog units and air support over a full range of policing scenarios. This might also include modelling the impact of advanced technologies, in that enhanced sensors and data links might provide a quantum leap in the degree of value provided by any sort of aerial platform.

For the other part-time operational units—SAR, specialist search, negotiation, emergency management and armed response—performance measures are focused on primary outputs, such as the number of operations undertaken and number of person-hours provided, as well as a small element of outcome focus in terms of drugs and weapons seized and the number of people located.

Intelligence performance measures

The final component of the technical unit category consists of intelligence units. The AMCOS intelligence performance framework is posited on Crous’ (2010) two-fold typology of the internal activities of intelligence units (activities/outputs) and the effect of those activities (outcomes). The former, which primarily comprises intelligence products, is simple to measure. The New Zealand Police National Intelligence Centre has already set out definitions and quality standards for a range of analytical products and as such, it is easy to simply count the number of each type of report produced during a particular reporting period. Also measured quantitatively is the number of intelligence notings (a raw intelligence product) produced. Other output measures include the number of investigative operations that provided organic intelligence support, where an intelligence analyst brings their specific skillset into the investigation team.

The next step in measuring the internal aspect of intelligence performance will involve more closely tracking the eight fundamental elements often seen as central to good intelligence practice (Crous 2010):

  • executive leadership;
  • intelligence leadership;
  • commitment;
  • collaboration, coordination and partnerships;
  • tasking and coordination;
  • collection management;
  • analytical capabilities; and
  • training and education.

This is not performance measurement in the way described in the rest of the paper and so will not be described further here.

The second aspect of intelligence performance measurement focuses on outcomes and specifically, the difference that formal intelligence support has made in terms of outcomes for the police as a whole (Crous 2010). This is a difficult task and one that requires a substantial quantity of data that is not currently available; as such, this part of the performance scheme is as yet dormant. It is not enough to merely show that an intelligence-led operation has good results; this does not validly separate out the value of intelligence from that of other policing contributors. Ideally, intelligence-led and non-intelligence-led operations undertaken by units that are otherwise similar would be compared. Such comparative evaluations would help identify the relative utility of formal intelligence support and processes compared with informal, traditional approaches. There would also be a place for specific identification of those occurrences where intelligence made the crucial difference between success and failure.

Such a specific and focused approach to intelligence performance measurement is possible within AMCOS, as it would be easy in a relatively small group to either provide or not provide formal intelligence support to squads that are otherwise similar in terms of experience and expertise, and then evaluate the results. It would be difficult on a larger scale; in such cases it might be more useful to conduct a large scale evaluation of crime rates before and after the introduction of intelligence-led processes, corrected for other factors.

Once the impact that intelligence processes have had on operational outcomes has been identified, the next step is to identify the costs involved and thus identify the benefit-to-cost ratio of formal intelligence support. Anything less than this would give only a partial answer to the question of how formal intelligence contributes to policing outcomes.

Investigations performance measures

For the most part, developing a performance framework for forensics, specialist operational units and intelligence units—using the earlier typology, technical units—was relatively easy. Almost all had a good base of existing activity and output measures that could be better aligned with strategic guidance and combined into a coherent, holistic scheme and that led easily into potential outcome measures. However, much greater difficulties were encountered with the specialist investigations components focused on drugs, organised crime and national security targets (niche units using the earlier typology).

As noted earlier, measuring performance in this field is difficult due to the lack of agreed definitions and reliable data (Mackenzie & Hamilton-Smith 2011). Because of the relatively small number of such investigations compared with overall crime rates and investigations, statistical approaches lack some validity when measuring outcome effects; variations in operation numbers are much more likely to represent the level of police attention rather than the actual level of organised crime, for example. In drug law enforcement, existing mixtures of activity, output and outcome measures from varying perspectives (police and government) have led to a distinct lack of clarity (Willis, Anderson & Homel 2011) and limited progress towards the identification of any sort of best practice in terms of performance measurement.

It was decided that, as a start, the performance scheme for this area would focus on outputs and more specifically on operations. The term operation is commonly used to describe proactive investigations targeting a particular group of drug/organised criminals and thus had the value of common understanding. The next step was to divide operations into two types, borrowing from the British National Intelligence Model:

  • Level 1 local operations; and
  • Level 2/3 complex and organised investigations (HMIC 1997; National Centre for Policing Excellence 2005).

Level 1 operations are the vast majority of criminal investigations, involving small-scale criminality in a geographically focused area, whereas Level 2 involves cross-border offending and Level 3 involves serious, national and organised offending. Initially, a trifold division into Levels 1, 2, and 3 was attempted, but whereas distinguishing between Level 1 and 2 was relatively easy, distinguishing Level 2 from Level 3 was immensely difficult. As such, the core measurement for investigation teams is the number of Level 1 and Level 2/3 operations active as at the performance reporting date and the number of operations terminated (arrests made) during the preceding reporting period. The ratio between active and terminated operations in turn provides some information about the ‘chunkiness’ of investigation workloads. Linked to these primary outputs are measures relating to the number of arrests made, the quantity and type of drugs seized, and the quantity and type of weapons seized.

An outcome measure derived from the quantity and type of drugs seized is the amount of social harm to New Zealand avoided. This relies on an independently developed Drug Harm Index, which uses econometric analysis to arrive at a quantification of drug harm (BERL 2008). Another outcome measure used intermittently is the post-operation disruption assessment (Mackenzie & Hamilton-Smith 2011). These are assessments carried out once an operation has terminated that evaluate the impact that a particular operation had on the criminal environment, based on intelligence gathered during the termination and post-termination phase. Essentially the police equivalent of the military ‘Battle Damage Assessment’ (Diehl & Sloan 2005), these assessments are extremely valuable, but time consuming.

A next group of outputs in this area relate to judicial processes. They cover the number of trials completed during a reporting period, as well as the number of trials being actively prepared for. The number of witnesses managed during the period is also measured.

Future performance measures for investigative units will focus on two key elements—outcome harm reduction and investigative efficiency. In the first element, the primary focus will be better understanding the impact that police activities have on the availability of drugs (Willis, Anderson & Homel 2011). As such, substantial data on the price, purity and availability of various illicit drugs in the Auckland region will need to be gathered (Mackenzie & Hamilton-Smith 2011). A framework that has been developed in Australia (see Willis, Anderson & Homel 2011) appears to be an excellent base to build from, although in requiring a substantial amount of data from non-police agencies, it will require close coordination with information sharing efforts at the national level. If resources allow, more analysis into other effects of organised crime will also be undertaken, such as corruption and distortion of markets.

Investigative efficiency and quality is an area of performance that seems to have been largely ignored in the literature. This seems rather odd, as poor investigative techniques can have multiple negative effects, including wrongful convictions, wrongful arrests, avoidable acquittals and false prosecutions (Canadian Police College Council of Investigative Excellence 2004). These can, in turn, lead to Commissions of Inquiry and even the payment of substantial compensation to those who have been wronged (Canadian Police College Council of Investigative Excellence 2004). It therefore seems essential to factor in investigative efficiency, perhaps in the form of compliance with quality standards, into the future investigative performance framework (Braga & Moore 2003a). This may involve formally identifying some principles or key elements of investigative excellence (Canadian Police College Council of Investigative Excellence 2004) and then randomly auditing a number of cases at any one time to identify the degree to which actual practice matches best practice. There would thus be a requirement for continual research and analysis to ensure that any best practice criteria remain accurate and relevant.

Investigation support components also require performance measurement. Informant management is measured in a number of ways. At the first level of complexity are some basic activity measures, such as the number of informants recruited and registered. The primary outputs measured are intelligence releases. In the near future, an effort will be made to trace the impact that informant-derived intelligence has on investigations, a form of outcome measurement.

Other covert capabilities, such as technical and surveillance support, are also measured on the basis of outputs—Level 1 and Level 2/3 operations supported, and the total number of person-hours of support provided. As with informant management, the next stage of performance measurement for these units will be to identify the impact that they have had on investigative success through comparisons with operations in which such support was not provided.

A last set of measures covered under the investigative section could just as easily have been discussed under the section on intelligence. This set relates to intelligence gathering activities, also known as field intelligence, such as attending gang or community events. At this stage, these involve simple activity measures only and it is difficult to identify how these measures could be developed further.

Future development of the framework

There are several ways the AMCOS Performance Framework might be enhanced as a whole, above and beyond the individual enhancements noted above. Before describing these potential enhancements, it is important to briefly reiterate why a relatively small unit (300 out of 12,000 total staff in New Zealand Police) needs to develop and improve its own framework and how doing so counters many of the difficulties related to performance measurement.

As noted earlier, one of the values in developing performance frameworks internally is creating a sense of ownership (Loveday 2006). Another is meaningfulness. This is particularly the case for AMCOS, which is unique within New Zealand Police; while centrally developed measures focusing on crime rates and resolutions may be relevant across the 12 general purpose Police Districts, they are less so for AMCOS. Local development can also ensure that managers involved in the production of the framework are more aware of the potential for those being measured to engage in perverse behaviour (Loveday 2005) and can intervene to prevent this occurring.

The next evolution of the AMCOS Performance Framework will be focused on enhancing cohesion, clarity and outcomes. Good performance measurement is facilitated by a coherent strategy (Collier 2006; Home Office Police Standards Unit 2004; Mackenzie & Hamilton-Smith 2011; Smith 1995a) and AMCOS is already working towards aligning and integrating performance measurement into its business planning, project management and risk management frameworks, using a program logic approach. The end goal is therefore a centrally directed, but locally managed, performance framework based on core strategic goals, which is integrated into planning and project management processes. Such a scheme is hoped to:

provide the integrated management framework necessary to achieve the output and outcome performance required to fulfil organisational goals and objectives (Barrett 2000: 64).

Once fully implemented, the framework will enable true professionalism by facilitating learning from the past and thus improving the future. There will be enhanced clarity and accountability, as managers will only be evaluated against the performance indicators they have agreed to in the ex-ante planning process (Flynn 1986). It is also hoped to better link performance indicators with managerial incentives and controls, further clarifying expectations (Smith 1990). Internally, a truly integrated and meaningful performance measurement system may even lead to a more participative and consultative management style (Bititci et al. 2006), as managers continually seek information and insight into the reasons for changes in performance.

In identifying meaningful objectives for the planning process, the military-derived concept of Effects Based Operations (EBO) could be particularly useful (Alach 2010b). EBO is about identifying the effects required to achieve a particular goal, then tracing back the causal chains required to achieve those effects. As such, it rests on inductive reasoning backed up by research. By applying an EBO approach to any AMCOS strategy, the causal chains required to reach desired outcomes (strategic goals modified to the AMCOS geographic sphere) could be identified. These chains could then be turned into tiers of performance measures, some at the outcome/strategy level, the majority at the output/activity level and even some at the input level.

For example, one desired outcome for AMCOS might be reduced use of methamphetamine across Auckland. The effects required to achieve this might be:

  • fewer first time users;
  • a higher price per pure gram;
  • reduced availability;
  • a higher risk to consumers;
  • improved social happiness; and
  • better drug treatment.

From this, AMCOS-specific measures might be identified, such as increasing the price per pure gram by a certain percentage, seizing a certain amount of the drug and arresting a certain number of suppliers.

It is intended that the framework will be continually reviewed to ensure that measures do not become ossified. One mechanism of balancing the need for consistency (and thus comparability) over time with the need to be flexible has already been noted earlier—the use of quality standards. For example, the measure might remain ‘number of operations terminated’, but the standards used to define an operation might be modified to adapt to evolving circumstances.

Core to the future review of the framework will be rigorous analysis and comparative research undertaken by CALL. CALL will be able to build from post-operation assessments and identify elements of best practice. This will assist in the development of standards against which to evaluate future operations. CALL will also evaluate the outcomes achieved by particular activities and as such, help clarify the relationship between activities, outputs and outcomes, potentially simplifying and enhancing the performance measurement framework.