Women's experiences of male violence : findings from the Australian component of the International Violence Against Women Survey (IVAWS)

Jenny Mouzos and Toni Makkai
ISBN 0 642 53842 5 ; ISSN 1326-6004
Canberra: Australian Institute of Criminology, 2004
(Research and public policy series, no. 56)

Appendix 1: Methodology of IVAWS

This appendix provides a description of the methodology employed to measure women's experiences of violence in Australia. It includes an outline of how women were selected, how interviewers were selected, the response rate, a description of the sample and a discussion of some of the methodological issues to arise during or following the fieldwork.

Through a tendering process to select the data collection agency, the Australian Institute of Criminology (AIC) commissioned Roy Morgan Research¹⁵ to conduct the Australian component of the IVAWS. All the interviews conducted by Roy Morgan Research were completed using a Computer Aided Telephone Interview (CATI) questionnaire developed by the International IVAWS Working Group and slightly modified by the AIC for the Australian environment.

Selection of the sample

There were two main sample selection methods employed in the IVAWS. The first involved selecting households with eligible women and the second involved actually selecting the eligible respondent. These two methods are outlined below.

Household selection method

The sampling methodology for selection of households used for IVAWS was 'White Pages plus one'. This is an adaptation of random digit dialling that involves selecting residential telephone numbers at random from the White Pages directory and incrementing the last digit by one (to get unlisted or not yet listed numbers). Both sets of numbers (that is, the white pages number and the incremented number) were then used for sampling. Using the 'White Pages plus one' method no doubt reduced the sampling bias in regards to unlisted private residential telephone numbers. This is important given that the results of the ICVS showed that there is a correlation between crime victimisation and unlisted numbers.

The 'White Pages plus one' method has all the advantages of random digit dialling, but at a lesser cost, as it produces fewer unusable numbers than random digit dialling. This method was also used in the ICVS (Carcach & Makkai 2003). Households were defined as any private household, including people living in shared or group households and those living or boarding with friends. A private household does not include businesses, schools, hospitals, nursing homes, prisons, military barracks, or other non-private households.

In addition, a pre-survey letter on AIC letterhead was mailed out to households whose phone numbers were selected. Results from the first pilot study indicated that the response rate was higher when letters were sent to selected households beforehand than when no letters were sent. Because of unlisted numbers, not all households were mailed an introductory letter from the 'plus one' sample. However, Roy Morgan Research was able to minimise the number of households that did not receive a letter by electronically matching the 'plus one' sample with the 'white pages' sample and sending out letters where numbers matched and names and addresses were available. All letters were addressed to Ms (surname from white pages) to minimise the probability of a male member opening the envelope.

Respondent selection method

Once a private household was contacted by telephone, the respondent was randomly selected using the nearest birthday method. The respondent to be selected was the female aged between 18 to 69 years normally living in the private household whose birthday was nearest to the date of the telephone call. If this person was absent for the duration of the survey, or was either incapable of responding, deaf, or suffering from an illness or disability, then the female with the second-nearest birthday was selected. If this female also happened to be absent from the survey or was incapable of responding, then the youngest female in the private household was selected.

The birthday method is a rigorous method of respondent selection. However, if used on its own, without sufficient call backs, it results in under-sampling of younger people and over-sampling of older people, simply because younger people are less likely to be available. To account for this, Roy Morgan Research ensured that whilst using the birthday method, they also conducted a sufficiently high number of call backs. As many as six call backs were made to speak to an eligible respondent to ensure that the expected impact on survey contact rates and response rates was minimal.

Six call backs were also made when there was no initial contact with the household (that is, there was no reply on the number). Call backs were made at different times of the day and different days of the week to maximise the chances of contact. Special arrangements were also put in place to deal with the potential bias that might occur with some women experiencing language difficulties. When there was a build-up of more than five non-responses due to problems with a language other than English, Roy Morgan Research completed these interviews with a bilingual interviewer.

Sample design

The IVAWS was designed as a stratified random sample. To ensure that the sample generated was representative of the population, strict quotas were imposed for the areas, and 'loose' quotas were imposed in terms of age and sex. Any deviations were then corrected by applying weights to the sample data.

Selection and training of the interviewers

Roy Morgan Research in conjunction with the AIC undertook a careful screening and selection process of the interviewers. Interviewers were selected with the following attributes:

• Female;
• Comfortable discussing issues related to violence against women;
• A non-judgemental and empathetic attitude;
• Highly developed interviewer skills;
• A good telephone interviewing manner;
• Sufficient prior interviewing experience, especially in conducting lengthy surveys;
• Sensitivity and maturity;
• Professionalism; and
• Prior experience of handling similar sensitive research studies, or other experiences that made them suitable for this project.

The AIC was also provided with the Curriculum Vitae of all intended interviewers in order to assess their eligibility. All interviewers and supervisors assigned to monitor the interviewing team undertook one full day of extensive training before fieldwork began. The training session included information on the background and purpose of the study, procedures for respondent selection, appointments and call backs, specific techniques for developing rapport, instilling trust in women and assuring them of confidentiality, a review of the survey design, and practice interviews using the CATI terminal to become familiar with the flow and content of the survey.

In addition, Forsythe Consultants were employed to undertake a review of the training process and first pilot of the survey. In terms of training, the Consultant recommended the following:

• The structure of the survey be explained to interviewers using a flow chart;
• The use of role-plays to go through the questionnaire (including anticipating potential difficult situations);
• Directions on how responses should be coded; and
• Interviewing techniques, including examples on how to build rapport on the phone, how to respond to upset women, and how to ensure the interviewers' well-being during IVAWS (debriefs, etc).

In addition, the AIC participated in an audit of a closely monitored interviewer session.

Structure of IVAWS

The structure of the Australian component of the IVAWS consisted of two pilot studies followed by two stages of the main fieldwork (Figure 46). This was conducted during December 2002 and March 2003 using the CATI questionnaire provided by the international coordinators.

Pilot testing of the CATI questionnaire

There are a number of objectives in pilot testing the IVAWS. These include:

• testing the appropriateness of the proposed CATI questionnaire, and ensuring that questions are clear, precise and without any ambiguities;
• estimating the actual length of interview for different types of women;
• assessing any cultural sensitivities that may exist;
• assessing the likely magnitude of non-response; and
• identifying any potential problems or concerns associated with the conduct of the survey.

The first pilot of IVAWS, comprising 125 interviews, was conducted in August 2002. The average interview length for the first pilot was 37 minutes. The first pilot also examined whether the introductory letter made any difference to the response rate, as not all households received the introductory letter. The response rate for the households that received the introductory letter was 57 per cent, compared to 27 per cent for households that did not it.

The first pilot was extremely useful in refining the questionnaire. Changes included deleting questions that were irrelevant to the Australian context, and wording changes to make the questionnaire more fluid and understandable. A meeting with participating countries was held in October to discuss the results of pilot testing in all countries and to make final adjustments to the questionnaire.

The second pilot test of the revised IVAWS was conducted between 6 and 8 November 2002, with 110 interviews completed. Due to the revisions following the first pilot, the average interview length decreased to 25 minutes. No introductory letters were sent out for the second pilot study, which elicited a response rate of only 23 per cent.

The questionnaire was further refined following the second pilot. Changes included additional questions on childhood victimisation to elicit the respondent's view of the seriousness of the incident(s), and deletion of the question in relation to 'drug facilitated sexual assault'. Roy Morgan Research, the AIC and the International Working Group worked closely to refine the questionnaire following both pilots and following each stage of fieldwork.

Main fieldwork

The first stage of fieldwork was conducted between December 2002 and March 2003. A total of 3630 interviews were completed, with an average interview length of 25 minutes. Following the first stage of fieldwork changes to the questionnaire included reinstating the question on 'drug facilitated sexual assault', as it was considered important to obtain some measure of prevalence. The question as to 'where did it happen' was also included in the questionnaire, as some women had indicated that the incident had occurred in a country other than Australia (for example, whilst overseas on holidays).

The second stage of fieldwork was conducted between end April and June 2003. A total of 3047 interviews were completed and the average length of the interview was 25 minutes. The second stage of fieldwork also included the interviews with non-English speaking background (NESB) women conducted by the bilingual interviewers.

Response rate

A total of 34,882 telephone numbers were dialled. Of these 13,015 were classified as 'ineligible' (Table 20). The main reason for ineligibility was that the telephone number was disconnected or not working, which accounted for 68 per cent of the ineligible telephone numbers.

The 21,867 'eligible' telephone numbers were further reduced by households that were called back more than six times (4538), or where there were no replies or engaged (82). The end result of 17,247 eligible contacts resulted in a contact rate of 79 per cent (eligible contacts to eligible numbers).

Table 20 : Ineligible contacts

Reasons for ineligibility	Number	Percentage
Disconnected/ not working	8225	68
Business number/not a private household	2157	16
Fax / modem / mobile number	1595	12
Does not meet introductory criteria	841	6
Language / hearing difficulty / very elderly	167	5
Respondent requested details to be deleted	30	0
Total	13,015	100
Source: Australian Institute of Criminology, IVAWS 2002/03 [computer file]

After taking account of refusals, terminations, appointments not met, and those who failed to give the number of women in the household, a total of 6677 completed interviews were obtained - a response rate of 39 per cent (relative to the total number of eligible women) (Table 21).

Table 21 : Eligible contacts

	Number	Percentage
Total eligible contacts	17,247	100
Minus
Failed to give number of women in the household	67	0
Appointments not met	34	0
Eligible women	17,146	99
Minus
Refusals	10,029	59
Terminations	440	3
Total final sample	6,677
Response rate relative to eligible contacts	39%
Response rate relative to eligible women	39%
Source: Australian Institute of Criminology, IVAWS 2002/03 [computer file]

This effective response rate of 39 per cent is lower than that indicated by the results of the first pilot. It is also lower than the response rate of 57 per cent achieved for the Australian component of the 2000 International Crime Victim Survey (ICVS). Compared to the ICVS, where terminations contributed 44 per cent to the total non-response, terminations only contributed three per cent of the non-response total. Refusals accounted for the majority of the non-response total. While the geographic distribution was relatively the same between those who refused to participate and the women who were interviewed (Table 22), a higher proportion of refusals came from women who lived in metropolitan Sydney (26%) or metropolitan Melbourne (21%).

There are a number of reasons that could account for the low response rate. Table 23 indicates that women aged less than 35 years were under-represented in the IVAWS sample, suggesting that this age cohort were less available to be interviewed. A possible explanation is the greater use of mobile phones than landline telephones among young adults, which means that telephone based surveys will exclude this cohort to some extent. This issue needs consideration, especially as more and more people rely on mobile phones.

Given that refusals accounted for the majority of non-responses, the length of the questionnaire could not have been a significant factor. The sensitive subject area may however have been an issue. Previous research (although contradictory) offers some insights into the reasons why women may not be willing to participate in telephone surveys:

• women who have been victimised may refuse to participate due to safety concerns (Straus & Gelles 1986), particularly concerns that their intimate partner will find out that they have disclosed the violence to a third party (Schwartz 2000; Waltermaurer, Ortega & Mcnutt 2003);
• women who have not been abused may not be interested in participating (Mcnutt & Lee 1999);
• severely abused women may be much less likely to participate than women who have been subjected to lower levels of violence (Mcnutt & Lee 1999); and
• cultural factors may mitigate against participation as women in some ethnic minority communities may be suspicious of interview questions and interviewers (Intimate Partner Abuse and Relationship Violence Working Group 2001).¹⁶

Unfortunately, it was not possible to gauge the exact reasons for terminating the interview early or refusing to participate, as insufficient information was provided on these women.

Description of the sample

The final sample of 6677 female respondents was adequately representative of the geographic and age distribution of the Australian adult female population, although residents in metropolitan Sydney, rural areas of Queensland (Table 22), and females in the younger age groups (less than 35 years) were somewhat under-represented (Table 23). The sample was weighted to adjust for these biases.

A total of 91 women who completed the survey indicated that they were Aboriginal and Torres Strait Islanders (1.4%), and a further 1113 women described themselves to be from a non-English speaking background (17%).

Table 22 : Sample and population counts by region, percentages

Region	Unweighted sample	Weighted sample	Difference
Sydney	21	22	-1
Rest of NSW & ACT	13	13	0
Melbourne	19	19	0
Rest of Victoria	7	6	+1
Brisbane	9	9	0
Rest of Queensland	10	10	0
Adelaide	6	6	0
Rest of South Australia & Northern Territory	3	3	0
Perth	8	7	+1
Rest of Western Australia	3	3	0
Tasmania	2	2	0
Australia	100	100
Source: Australian Institute of Criminology, IVAWS 2002/03 [computer file]

Table 23 : Sample and population counts by age, percentages

Age group	Unweighted sample	Weighted sample	Difference
18-24	7	15	-8
25-29	7	11	-4
30-34	11	12	-1
35-39	12	11	+1
40-44	14	11	+3
45-49	12	11	+1
50-54	12	10	+2
55-59	10	9	+2
60-64	8	6	+2
65-69	6	5	+1
All women	100	100
Source: Australian Institute of Criminology, IVAWS 2002/03 [computer file]

Two-thirds of the women interviewed were currently married (67%), with a further nine per cent living with a man without being formally married to him. In terms of the highest educational level attained by the women, close to a third had obtained some type of degree from a university or college of advanced education (CAE) (29%). A further 15 per cent of the IVAWS women had indicated that their highest level of education was HSC/VCE or Year 12. Only one per cent had no post-primary school education. The majority of women who completed the IVAWS were working for pay either outside the home (58%) or both at home and outside the home (3%).

Data processing

Roy Morgan Research was responsible for the editing of the raw data for integrity, coding errors and internal consistency. All questions on reasons for reporting or not reporting a crime, and reasons for dissatisfaction with the police were left open ended to allow women's exact answers to be recorded. The answers to these questions were coded according to coding frames prepared jointly by staff at Roy Morgan Research and the AIC. In addition, Roy Morgan Research conducted appropriate checks to ensure consistency and quality of the data before it was provided to AIC. The AIC received a clean data file in ASCII format containing a total of 39,480 unit records.

Weighting

As the sample of women interviewed tended to under-represent women living in metropolitan Sydney and rural Queensland, as well as women aged less than 35 years, survey weights ('post stratification weights') were used to correct these imbalances. In general, survey weights are an estimate of the inverse of the probability associated with selecting a particular household or person in the sample. They serve two major purposes:

1. They enable the survey estimates to be corrected for non-response and coverage problems; and
2. They enable the survey estimates to conform to a known distribution of the population of interest, often defined in terms of sex, age group and geographic region (for further information see Carcach & Makkai 2003).

Roy Morgan Research calculated the weights used in the IVAWS based on weighting specifications provided by the AIC (Roy Morgan Research 2003). Weights were attached to the responses to the survey as follows:

A. Notation and definitions

h =	Stratum indicative
i =	Person indicative,
k =	Household indicative,
a =	sex by age by part of state cell for sampling quotes,
p =	sex by age by part of state post-stratum,
T =	Total number of telephone numbers listed in the frame (White Pages And White Pages plus One),
t =	Total number of interviews,
M =	Number of persons in the household who are 16 years and over,
Qa =	Sampling quota for the a-th cell, first stage (2,005 interviews),
Np =	Benchmark - estimated number of residents in private households in the p-th sex by age by part of state post-stratum, first stage (2,005 interviews)

Person weights

The survey selected telephone numbers at random within specific regional strata. Based on the assumption that there is one telephone number per dwelling, then the probability of selecting a specific telephone number equals the probability of selecting the dwelling. Assuming that there is one household per dwelling, it will also approximate the probability of selecting a private household. One person who was in-scope of the survey was randomly selected from the household.

Selection weighting - persons

Within a given regional stratum, designated by h, a person-selection weight was obtained in the following way:

Non-response adjusted weighting - persons

Sampling quotas were determined for cells defined according to sex-age groups. Let a designate one of these cells and denote the sampling quota in the a-th cell as Q_a. The non-response-adjusted weight was calculated from:

Post-stratification weighting - persons Let Np denote the known population aged 16 years and over for the p-th sex by age by part of the state post-stratum obtained from the Australian Bureau of Statistics. The final person weight, after adjustment for post-stratification, was calculated from:

Household weights Household weights must be derived from person weights as there are no benchmark figures for numbers of households in Australia. One simple way to obtain the weight for the k-th household is to adjust the final person weight by the ratio of the k-th household's size to the sum of household sizes over the sample. This gives the following expression for the post-stratified household weight:

No non-response and selection weights were required for households.

---

• ¹⁵ Roy Morgan Research had also been commissioned to undertake the Australian component of the 2000 International Crime Victimisation Survey (ICVS).
• ¹⁶ Noteworthy is that of the 476 foreign language callbacks made by a bilingual interviewer, only 14 per cent resulted in an interview in a foreign language.