Australian Institute of Criminology

Skip to content

The DUMA program

DUMA was established in 1999 by the AIC as a national monitoring program of illicit drug use among people who have been arrested and detained by police (detainees) at police stations and watchhouses around Australia (Sweeney & Payne 2012; Makkai 1999). Data are now collected nationally at nine sites on a quarterly basis from alleged offenders who have been detained in police custody making DUMA the largest and most comprehensive program of data collection on alleged offenders in Australia. Approximately 4,000 detainees are interviewed annually at nine sites across six states (Sweeney & Payne 2012).

DUMA data is comprised of an interviewer-administered survey and a urine specimen, which is analysed for seven different classes of drugs. The survey is divided into two parts—core questions that are asked every quarter and addendum questions that vary each quarter to allow data collection on different topics. Participation in the research is voluntary and confidential. The unique aspects of the DUMA program are its ability to quickly highlight changes in drug use or drug market patterns and corroborate self-report information with urine analysis (for more information see Sweeney & Payne 2012 and http://www.aic.gov.au/about_aic/research_programs/nmp/duma/about.html ).

Data collection covers the areas of drug and alcohol use, drug purchasing patterns, medication use, and drug and alcohol treatment. It also develops offending profiles, as well as recording demographic information such as age, gender, marital status, employment, education level, income and housing.

The detainees who participate in the DUMA program represent a very broad range of alleged offenders including people who have been detained for the first time, through to those who have been through the criminal justice system many times previously, as well as those detained for minor offences through to those detained for the most serious offences. Many of the DUMA detainees spend only a matter of hours in police custody, while some are refused bail and remanded in custody until their matter appears before the court.

A further unique feature of the DUMA program is that it provides a large enough dataset to allow gender-based analysis. Women form a minority of offenders in the Australian criminal justice system, with estimates suggesting they constitute approximately seven percent of the prison population (AIC 2008) and approximately 17 percent of police detainees (Sweeney & Payne 2012). Most criminology studies are either based exclusively on male offenders or do not include enough women to allow gender-based analysis (Mazerolle 2008). As poor mental health and in particular substance abuse, have been shown to be more strongly related to offending among female compared with male offenders (Byrne & Howells 2002; Johnson 2004; Loxley & Adams 2009; Teplin et al. 2007), DUMA data provide a unique and valuable vehicle through which to improve our understanding of issues that may be associated with women’s offending behaviour.

It should be noted that as the DUMA program is based on a cross-sectional design, there is no facility for offender tracking. Information about past events is elicited from detainees by self report. Therefore, associations between variables are largely temporally bound to the current custody episode. Relationships between factors (such as causal or mediating) may be suggested by the data but inferences that can be drawn are limited by the research design.

Quite early during the DUMA program, the prevalence of two psychological factors among detainees became evident—drug and alcohol dependence (as opposed to use/abuse) and mental disorder. From 2003, a drug and alcohol dependency measure was incorporated into the core DUMA survey (Schulte, Mouzos & Makkai 2005) and in more recent years is included on a rotational basis. However, the inclusion of mental health measures remained limited.

Early mental health measures in DUMA

From the inception of the DUMA program, there were two questions in the core survey that provided some information about mental health.

  • detainees were asked whether they had ever been in a psychiatric hospital for at least one overnight stay; and
  • detainees were asked whether they had taken any prescription or over the counter medications during the past fortnight. If they had, the names of such medications were recorded.

The information elicited by these questions can provide an indicator of whether the detainee may have experienced a mental disorder. However, having stayed overnight in a psychiatric hospital does not necessarily indicate a mental disorder, as a person may be admitted for an assessment and then not diagnosed with any disorder. Conversely, it may indicate a very serious mental disorder episode and involuntary confinement. Similarly, the reported use of psychoactive medications may be indicative of a diagnosed mental disorder; however, these medications may also have been prescribed for pain relief or other conditions that are not necessarily classified as mental disorders. Further, taking psychoactive medications may result in amelioration of the disorder, therefore, it cannot be assumed that the person was unwell at the time of detention.

Overall, these existing DUMA questions provide quite limited information about mental health. For detailed analysis of the mental health information produced by these measures see Forsythe and Adams (2009).

How else to measure mental health?

As a part of this review process that aimed to improve mental health measurement for the DUMA program, a literature review was conducted to identify instruments and questions that had been utilised in comparable studies. Researchers working on relevant criminological studies were also contacted regarding their experiences and suggestions. Instruments and questions that appeared at face value to be suitable were evaluated from a conceptual and pragmatic perspective; the findings of these reviews are outlined and discussed below.

Timeframe

When measuring mental health/disorder, a timeframe needs to be defined, depending on the purpose of the research. For example, is a disorder being experienced now, during the past 12 months or over a person’s lifetime? If an objective of the research is to estimate the mental health services likely to be required by the population, then lifetime prevalence may be of most relevance in order to enable estimation of service needs. If, however, the research aim is to identify the mental health service needs of newly incarcerated prisoners, then current incidence and unmet need may be more relevant.

Standardised measures

While in clinical practice a variety of methods for diagnosis may be appropriate, a standardised method is required for research to ensure comparability of diagnostic classifications. Typically, in research, a structured interview such as the Composite International Diagnostic Interview (CIDI) is utilised (Andrews & Peters 1998). However, many of the standardised clinical interview protocols require significant time to complete and require interviewers to have some clinical training. These features mean that the CIDI or other standardised measures are not feasible to incorporate into criminological research projects such as DUMA, where time is limited (detainees are typically available for DUMA participation for approximately half an hour) and data collectors typically do not have clinical training.

Researchers working on large-scale epidemiological research projects have developed a variety of short instruments to estimate the prevalence of mental disorders in populations. Some of these instruments have been validated. Validation refers to a process where the short instrument and a full diagnostic instrument (such as the CIDI) are both administered to the same group of people and the results are compared. The three key factors of a validation study are whether the short instrument correctly identifies the people who met diagnostic criteria for a mental disorder (referred to as sensitivity), whether the short instrument correctly identifies people who did not meet diagnostic criteria for a mental disorder (specificity) and the characteristics of the population of people the validation study was based on (eg see Ford et al. 2009; Furukawa et al. 2003). Several short instruments will be discussed, focusing on their conceptual underpinnings and feasibility for use in the DUMA program.

Instruments such as the Mental Health Inventory (MHI-I), Kessler 6 (K6) and the Kessler 10 (K10) comprise between five to 10 questions, measure non-specific psychological distress and have been developed as indicators of mental disorder (Andrews & Slade 2001; Kessler et al. 2002; Rumpf et al. 2001). Elevated levels of psychological distress have been found to be associated with the presence of diagnosable mental disorders in general population studies when they have been compared with the results of a full diagnostic interview (Andrews & Slade 2001; Furukawa et al. 2003; Kessler et al. 2002; Rumpf et al. 2001). While these psychological distress measures have been validated as indicators of mental disorder in the general population and have been widely used in criminology research, they do not appear to have been validated for offender populations. This raises the question of whether they accurately identify diagnosable mental disorders among offenders.

The K10 was trialled on two occasions with subsets of detainees the DUMA program (Mouzos et al. 2007; Schulte, Mouzos & Makkai 2005) and its use in this context was found to be problematic. First, one of the questions (regarding feelings of worthlessness) elicited an emotionally upsetting response in some detainees, leading to the discontinuation of the use of the K10 on ethical grounds. Second, detainees typically participate in the DUMA research within a few hours of being placed into police custody. The question arises as to whether, in this context, a measure of psychological distress may be measuring the immediate psychological distress in response to the arrest and custody situation rather than longer term and continuing psychological distress indicative of a mental disorder. Therefore, despite widespread use of the K10 in criminology research it was found to be problematic in the DUMA context, highlighting the importance of contextual factors in the use and validity of measurement instruments.

Another mental health indicator used in a variety of studies is a measure of functional impairment. Instruments such as the SF-36 and SF-12 canvas the extent to which a person’s ability to engage in everyday activities has been impaired due to physical and mental health problems (Sanderson & Andrews 2002). The SF-12 has been found to correlate with diagnosable mental disorders in studies of the general population (Gill et al. 2007; Sanderson & Andrews 2002); however, it is unclear whether these measures are equally indicative of mental disorder in offender populations. For example, it could be argued that functional impairment may be reflective of lifestyle factors among people who are heavily involved in illicit drug use and criminal activity rather than necessarily reflecting mental disorder. Without validation studies based on appropriate populations, it is difficult to tease out these distinctions and have confidence in the validity of the concepts being measured.

Screening instruments provide another standardised method of obtaining a measure of mental disorder; they are evidence-based sets of questions designed to identify apparently well people who probably have a diagnosable disorder and therefore warrant a comprehensive assessment (Alexander et al. 2008). Some screening instruments have been developed to screen for specific disorders (eg alcohol or drug dependence) while others are designed to screen more broadly for the likely presence of any mental disorder. Screening instruments vary substantially in length and the skills required for administration and interpretation (Dawe et al. 2002). One important benefit of screening instruments is that they identify current morbidity (or unmet need) as distinct from needs that may have already been identified and addressed (Andrews & Slade 2001).

Several screening instruments have been developed specifically for people entering prison, to facilitate identification of those requiring a comprehensive mental health assessment (Gonzales, Schofield & Hagy 1997). Some of these instruments have been validated against full diagnostic interviews and have been found to accurately classify people. These include the Jail Screening Assessment Tool (JSAT; Grisso 2006), Modified Mini Screen (MMS; Alexander et al. 2008), Brief Jail Mental Health Screen (BJMHS; Steadman et al. 2005) and the Corrections Mental Health Screen (CMHS; Ford et al. 2009). While not designed specifically for research purposes, the fact that these instruments have been validated on offender populations (and thus have demonstrated accuracy in indicating mental disorders among offenders) is an appealing feature for their use in criminological research. However, as all of these measures have been validated on US populations, the degree to which these results are transferable to an Australian context is largely unknown.

Only one Australian study could be found that utilised the JSAT and the BJMHS; it was conducted with detainees in Melbourne and found that both performed well in identifying Australian detainees with serious mental disorders (Baksheev, Ogloff & Thomas 2011). However, this study did not analyse the results by gender, which leaves the question of their effectiveness for use with female detainees unresolved. Differential results were reported for the BJMHS in US studies, which found that this instrument did not screen as effectively for women as it did for men (Steadman et al. 2007, 2005). Additionally, questions have been raised about the effectiveness of the BJMHS among Australian Indigenous people.

A recent Western Australian pilot prison health study (Kraemer, Gately & Kessell 2009) incorporated the BJMHS and found that Indigenous women interpreted two questions differently than intended. The first two questions of the BJMHS are designed to screen for symptoms of psychosis; they ask about mind control, other people knowing your thoughts and putting thoughts into your mind (Steadman et al. 2005). The Indigenous women in the Western Australian pilot prison health study overwhelmingly answered yes to these questions and their comments to interviewers suggested that they had interpreted these questions in a literal way. That is, because they were in prison, these women considered that every aspect of their lives was controlled, including their mind (Jason Payne personal communication 2009). This demonstrates how different interpretations of a concept on which questions are based may lead to invalid results—in this instance, false positive results.

Another issue that suggests caution in assuming that tools validated in the United States will be equally valid in the Australian context arises from jurisdictional differences. When the CMHS was administered by Ford et al. (2009), the inmates had been in jail for between 24 and 72 hours; presumably intoxication and distress levels may have reduced compared with earlier during the custody episode. In the DUMA research, many participants are interviewed within the first few hours following detention. While severe intoxication is an exclusion criteria from the DUMA program, some intoxication may remain and the effect of this on the validity of the screening tool is unknown. While a recent trial of the CMHS on the DUMA program did not test validity, in other respects, it does appear suitable for both the Australian context and the DUMA program (Forsythe & Gaffney 2012); the process of this trial will be discussed in more detail below.

Purpose-designed questions

Mental health information can also be elicited with purpose-designed questions embedded within the broader survey instrument. While this approach can enable flexibility in investigating mental health concerns or histories, there are clear issues of validity to resolve. Question wording can unintentionally influence what information is reported and cultural factors and ethnic background have been found to influence the interpretation of questions (Warnecke et al. 1997).

For example, mental health has different facets and the terminology used in a question may mean different things to different people. The DUMA program recently piloted a free recall and cued recall version of a question asking detainees to identify mental health problems with which they had been diagnosed. In response to the cued recall version, Attention Deficit Hyperactivity Disorder (ADHD) was frequently mentioned, while it was less often mentioned in response to the free recall version. Several reasons may account for such a discrepancy, but it is likely that some respondents did not consider ADHD a mental disorder until cued with a list in which it was included. This highlights the importance of careful and considered question design, as well as a comprehensive piloting and revision process. Needless to say, the question could only identify diagnosed ADHD, as opposed to ADHD that a detainee may have experienced but that was undiagnosed.

In summary, each way of measuring mental health and mental disorder has benefits and limitations. The choice of instrument should be informed by what information is deemed of most use and will by necessity be constrained by conceptual, pragmatic and psychometric limitations that are important to acknowledge.

Mental health measures trialled in the DUMA program

Taking into consideration time limitations, the police custody environment within which DUMA operates and our research experience, a combination of a screening instrument and purpose-designed questions were chosen for piloting.

Following a comprehensive literature review of mental disorder screening instruments, all those discussed so far in this report were considered. The CMHS instrument developed by Ford et al. (2009) was selected for trial for the following reasons:

  • It had demonstrated good psychometric properties with offender populations (Ford et al. 2009).
  • It was designed to be administered by non-clinical staff (DUMA interviewers are drawn from a variety of backgrounds and most are not clinically trained; Ford et al. 2009).
  • Being a screening instrument, it was designed to identify people with undetected mental disorders and therefore could generate data on current and unmet need.
  • The question wording seemed, at face value, suitable for DUMA. Specifically, there were no questions about feelings of worthlessness (which had previously led to emotional upset among some detainees in response to the K10). There were no questions about mind control (the mind control questions on the BJMHS had been found problematic when asked of Indigenous women in a prison context).
  • The CMHS included a question about ever having been in a hospital for non-medical reasons, such as a psychiatric hospital. This would allow some comparability with data generated by a similarly worded question that had been incorporated into earlier versions of the DUMA survey.
  • The CMHS comprised a gender-specific format—the CMHS-M for men and the CMHS-F for women. Gender-specific questions are important, as research indicates that symptoms of mental disorder can manifest differently in men and women (Fleming 2004; Smith et al. 2008; Tolin & Foa 2008) and gender-neutral screening instruments have been found to perform differentially for men and women (Steadman et al. 2007, 2005).

In addition to the CMHS, a question was designed and inserted into the core DUMA survey asking whether the detainee had ever been diagnosed with a mental health problem by a doctor, psychiatrist, psychologist or nurse. Mental health professionals were specified in the question to try and avoid self-diagnosis and/or diagnosis by family and friends.

The term mental health problem was selected instead of mental disorder, as the latter is not frequently used in colloquial language and it was thought that it may be misunderstood by detainees. This question was designed to collect information about mental health needs already identified by mental health professionals, as well as data about diagnoses received. If the detainee responded that they had been diagnosed by a mental health professional, they were asked to specify what they were diagnosed with; all diagnoses they mentioned were recorded. The question was trialled in two versions—the free recall version just described and a cued recall version that was administered using a response card that listed categories of mental health disorders based on the DSM-IV-TR. Detainees were first asked the free recall version of the question and then showed a cue card listing categories of mental disorders and asked to indicate any they had been diagnosed with. Finally, people who indicated that they had been diagnosed with a mental health problem were asked how old they were the first time they were diagnosed (see Appendix 1 for a copy of the pilot questions). The age at first diagnosis was included in order to allow temporal comparisons with age of first drug and alcohol use and first arrest.

Related links

Measuring mental health in criminology research: Lessons from the Drug Use Monitoring in Australia program: