Home → Publications → Reports → Technical and background paper series → 16 → Sample design and selection (in: The Australian component of the 2004 International Crime Victimisation Survey)
The Australian component of the 2004 International Crime Victimisation Survey
Graham Challice and Holly Johnson
ISBN 0 642 53889 1 ; ISSN 1445-7261
Australian Institute of Criminology, 2005
(Technical and background paper series, no. 16)
Download
- PDF print version: sample design and selection (PDF 223kB)
- PDF print version: full report (PDF 560kB)
Sample design and selection
Interviewing for the ICVS was contracted to the Social Research Centre in Melbourne. It was completed between August and November 2004.
The survey was designed as a stratified random sample. There were 15 geographic strata: seven states and territories stratified by capital/rest of state plus the Australian Capital Territory, which was treated as a single area. The first phase consisted of household selection and the second phase consisted of selecting one respondent from all eligible respondents in the household.
Methods of respondent selection and interviewing procedures differed for the two sample groups: the main community sample and the migrant sample.
3.1 Household selection -main community sample
The in-scope population for the main portion of the 2004 ICVS was persons 16 years of age and older who are residents of private households in Australia. Data were collected by computer assisted telephone interviewing. Residents of institutions, refuges and hospitals were excluded, as were homeless people and households with mobile phones only. The sampling technique for the main community survey was random digit dialling (RDD) which improves the probability of selecting households with unlisted or newly listed numbers. Previous research shows that these groups are disproportionately likely to be single or divorced, are relatively transient and have higher victimisation rates. It was important to ensure these groups were adequately represented in the achieved sample.
A total of 29,676 records were randomly selected from the electronic White Pages and were used as the 'seed' numbers for random number generation. This involved retaining the six-digit exchange prefix of the listed number (for example, 02 6260) and randomly generating the last four digits to create a new randomly generated ten-digit telephone number. The listing of randomly generated numbers was then compared against the White Pages directory to identify which numbers could be matched to the White Pages listings (the matched sample) and which numbers could not be matched (the unmatched sample).
The matched sample was then divided into full matches, where both a full postal address and telephone number were listed, and partial matches, where only a telephone number was listed. A total of 10,409 numbers were matched to a full postal address. An approach letter, addressed to 'the householder', was mailed to each of these fully matched selections one week before the initiation of calls. The approach letter was designed to introduce the survey, encourage response, outline respondent selection procedures and help establish the legitimacy of the survey.
3.2 Respondent selection -main community sample
A disproportionate chance of selection methodology was used to select respondents within selected households, with no substitution permitted. The chance of selection in households with persons under 30 years of age was increased by a factor of 2. The chance of selection for males aged 30 and over was 1.5, while females aged 30 and over had no increased chance of selection. While this approach tends to marginally inflate refusals (as young persons and males tend to have higher refusal rates than older and middle-aged females), the main advantage is that it overcomes biases in the achieved sample age and gender distribution that would otherwise occur. It also eliminates the need to discard households as 'quota full' if specific age and gender quotas are applied.
A range of strategies were adopted to maximise participation, including:
- an approach letter;
- an extended call regime of up to 15 calls to establish contact with the household (20 per cent of interviews were achieved at the ninth call attempt or later);
- conversions of 'soft' refusals (15 per cent of interviews were from refusal conversions);
- unlimited calls to complete an interview where contact had been established;
- appointments taken to conduct interviews;
- offering to mail, fax or e-mail an approach letter to sample members who refused to participate without having first sighted a letter of introduction;
- a focus on interviewer training and respondent liaison techniques;
- the operation of a 1800 number by both the AIC and the Social Research Centre, set up to respond to queries about the survey; and
- interviewing in seven languages in addition to English (Vietnamese, Arabic, Turkish, Serbo-Croatian, Mandarin, Greek and Italian).
An extended call cycle and the disproportionate chance of selection methodology helped improve the representation of hard-to-reach populations, such as young people, single person households, employed persons, apartment dwellers and those living in large cities. As shown in Table 1, the achieved sample also matched well on Australian Bureau of Statistics (ABS) benchmarks for Indigenous background and speaking a language other than English at home.
3.3 Household selection -migrant sample
The RDD method of sample generation used for the main community component of the ICVS was not cost effective for the immigrant sample due to the relatively small number of persons of Vietnamese or Middle East origin living in Australia, even in areas with high concentrations of these populations. A surname-based approach to the migrant sample generation was therefore used. Initially, this involved sample purchases from Cultural Perspectives, a company specialising in multicultural and Indigenous research. A total of 3100 sample records were provided, including 1200 records of persons with Vietnamese surnames, 1500 with Arabic/Middle Eastern surnames, and 400 with Turkish surnames.
Whilst the Vietnamese quota was achieved from the sample provided by Cultural Perspectives, further Middle East and Turkish numbers had to be generated to achieve quota, given lower overall participation rates amongst these two groups and the lower proportion of in-scope contacts for Middle Eastern members, in particular. This involved using the same surnames as the original sample provided by Cultural Perspectives, selecting all records with these surnames from the electronic White Pages, de-duplicating against the sample provided by Cultural Perspectives, and randomly selecting a further 6000 records with Arabic/Middle Eastern surnames and 1030 records with Turkish surnames.
The surname-based approach to sample selection has a number of limitations:
- households with silent telephone numbers will be excluded;
- females in the target groups who married into other ethnic groups (such as a Vietnamese women marrying a non-Vietnamese man) will not be included in the sampling frame; and
- there was a low level of precision in identifying Middle Eastern surnames.
However, the surname-based approach does have certain advantages:
- a national sample could be selected as opposed to the clustering that targeted geographic sampling would have entailed; and
- being based on the electronic White Pages, address details were available for the mailing of an approach letter to all selected migrant households. Approach letters were sent in bilingual format, containing English and one of Vietnamese, Arabic or Turkish.
3.4 Respondent selection -migrant sample
The same age criterion (persons aged 16 or over) was used for the migrant component as for the main community component of the survey. However, respondents self-selected into the survey through the following question:
We are particularly interested in speaking with people who were born overseas. Were you or your parents born in any of the following regions?
- Vietnam
- Middle East
- None of these
If required, the definition of Middle East was given as including: Bahrain, Gaza Strip and West Bank, Israel, Iran, Iraq, Jordan, Kuwait, Lebanon, Oman, Qatar, Saudi Arabia, Syria, Turkey, United Arab Emirates and Yemen (consistent with the ABS classification). However, if someone from North Africa self-identified as Middle Eastern, they were included. Similarly, anyone who self-identified with the relevant culture (e.g. ethnic Vietnamese who was born in China) progressed through screening. From this point, the respondent selection methodology was similar to the main community component, with the same disproportionate chance of selection based on age and gender.
Once the total number of persons in the household had been established, the number of in-scope persons was confirmed using the following question:
How many of those are aged 16 years or over and were born in (Vietnam/the Middle East/Turkey), or whose parents were born in (Vietnam/the Middle East/Turkey)?
No formal refusal conversions were attempted for the migrant sample. Other refusal avoidance tactics were utilised, such as offering to call back with an interviewer of the same sex as the respondent. Other call procedures were similar to the main community component of the survey, including an extended call regime to households.
As shown in Table 2, the demographic distributions of the migrant samples were less comparable to ABS benchmarks than the main community sample.
3.5 Call summary and response rate -main community sample
For the purposes of determining the response rate for the ICVS, the following were excluded:
- numbers that were not connected or not residential numbers, and therefore unusable (44.3 per cent of numbers initiated);
- where no contact could be established within the call cycle (14.2%);
- those that resulted in a contact confirming that the selected respondent was out of scope (3.6%), including:
- where there was contact with the household but no interview could be achieved within the call cycle;
- where no one over 16 years of age was resident;
- where the interview couldn't be conducted due to language difficulties; and
- where the selected respondent was too old, frail or incapacitated.
The final overall response rate for the main community sample was 55.4 per cent, where response rate is defined as completed interviews (6000) as a proportion of sample members who could be contacted within the call cycle and were not identified as out of scope (11,244). The average interview length was 17 minutes. Table 3 shows in detail the result of almost 30,000 calls made.
Households that could be matched to a complete mailing address and who were sent approach letters achieved a response rate of 58 per cent compared with a 41 per cent response rate for the unmatched portion of the sample.
3.6 Call summary and response rate -migrant sample
Table 4 presents the final call result for all numbers initiated for the migrant portion of the sample. The call summary profile for the migrant sample is somewhat different from that of the main community sample due to the altered sampling frames. There was a lower proportion of unusable sample records for the migrant sample (19.8 per cent compared with 44.3 per cent for the main sample). There was also a lower proportion of unresolved numbers at the end of the call cycle (6.7 per cent compared with 14.2 per cent), and a higher proportion classified as out of scope contacts (35.1 per cent compared with 3.6 per cent), due mainly to the 'no one of target background in household' outcome.
The final overall response rate for the migrant sample was 45.7 per cent, where response rate is defined as completed interviews (1001) as a proportion of sample members who could be contacted within the call cycle and were not identified as out of scope (2188). The average interview length was 19 minutes.
There were significant differences in response rate for the two migrant groups: 75.4 per cent for the 400 Vietnamese respondents, and 36.3 per cent for the 601 respondents from the Middle East. Middle Eastern migrants were more likely to be suspicious of being contacted by a survey research company, having had very little experience with survey research in their countries of origin. Among this group there seemed to be a lack of familiarity with participating in government survey research, the associated privacy and confidentiality issues and knowledge of how data are used. Refusals for Middle Eastern respondents were compounded by the fact that the survey was in the field during Ramadan, where this group tended to busier, tired or otherwise less positively pre-disposed to participate in the survey than might have otherwise been the case. Among migrants groups in general there may be a general mistrust of authority influenced by experiences in their home country. Coupled with this is the possibility that migrants may be less willing to discuss crimes of a personal nature, as evidenced by the lower victimisation rates for these types of crimes (Johnson 2005). Some migrants may avoid answering questions that reflect badly on their communities.
- Next section: Analysis of the sample
- Previous section: Questionnaire
- Contents