New South Wales Population Health Survey

The NSW Ministry of Health has conducted the Adult Population Health Survey (since 1997) and the Child Population Health Survey (since 2001) through the New South Wales Population Health Survey, an ongoing survey of the health of people in NSW using computer-assisted telephone interviewing (CATI). The main aims of the surveys are to provide detailed information on the health of adults and children in NSW and to support planning, implementation and evaluation of health services and programs in NSW.

Survey instrument

The survey instruments include question modules on health behaviours, health status, and other associated factors. The methods and all questions are approved for use by the NSW Population and Health Services Research Ethics Committee. While some questions are collected annually, other questions are collected less frequently. The instrument is translated into 5 languages: Arabic, Chinese, Greek, Italian and Vietnamese.

Survey sample

The target population for the survey is all state residents living in private households. The target sample was approximately 1,000 persons in each of the health administrative areas (total sample 8,000-16,000 depending on the number of administrative areas).

From 1997 to 2010 the random digit dialling (RDD) landline sampling frame was developed as follows. Records from the Australia on Disk electronic white pages (phone book) were geo-coded using MapInfo mapping software. The geo-coded telephone numbers were assigned to statistical local areas and area health services. The proportion of numbers for each telephone prefix was calculated by area health service. All prefixes were expanded with suffixes ranging from 0000 to 9999. The resulting list was then matched back to the electronic phone book. All numbers that matched numbers in the electronic phone book were flagged and the number was assigned to the relevant geo-coded area health service. Unlisted numbers were assigned to the area health service containing the greatest proportion of numbers with that prefix. Numbers were then filtered to eliminate continuous non-listed blocks of greater than 10 numbers. The remaining numbers were then checked against the business numbers in the electronic phone book to eliminate business numbers.

From 2011 onwards the RDD landline sampling frame was developed as follows: Australian Communications and Media Authority exchange district and charge zone prefixes were generated for each of the strata (that is Local Health Districts introduced in January 2011) using “best fit” postcode (ACMA 2011). All prefixes were expanded with suffixes ranging from 0000 to 9999. The sample was then randomly ordered within each stratum. The estimated numbers required for each stratum was then forwarded to Sampleworx, who used proprietary software to test each numbers current status (valid, invalid or unknown and business, non-business or unknown). The resulting valid non-business and valid unknown numbers were used for the survey.

From 2012 onwards mobile only phone users were included into the surveys using an overlapping dual-frame design, which incorporates three groups of respondents: landline only users, mobile only users and landline and mobile users.

The RDD mobile sampling frame was developed by Sampleworx and included using all known Australian mobile prefixes. Sampleworx used proprietary software to test each number to identify valid and invalid numbers. A random sample of valid mobile numbers was then provided for use for the survey.

The introduction of this design was prompted by the increasing numbers of mobile-only phone users in the general population. Because this design increases the representativeness of the survey sample the production of unbiased estimates over time is also improved. This improvement has been confirmed by an analysis of unweighted estimates, which indicated that a greater proportion of younger people, of males, and of people born overseas participated in the mobile sample compared with the landline sample. Further, comparison of the demographic characteristics of the survey sample for the first quarter of 2012 with the NSW population showed that the NSW Population Health Survey was more representative of the NSW population than the previous sample (Barr et al. 2012).

Due to this change in design, the 2012 NSW PHS estimates reflect both changes that have occurred in the population over time and changes due to the improved design of the survey.

When considering significant differences over time excluding the 2010 and 2011 data points ensures that all estimates are from sampling frames that had adequate coverage of the population, that is 85% or more.

When the Australia on Disk electronic white pages became available, reliable introductory letters were sent to the selected households (1997 to 2008). Households were contacted using random digit dialling. Depending on the frame either one person from the household was randomly selected or the mobile phone holder was selected for inclusion in the survey.

Due to increases in mobile phone usage, and reduction in landline usage, in 2021 the survey moved to a 100% mobile sampling frame. Participants are sent an initial SMS message to advise them of the survey and ask if they are NSW residents. Participants can opt out of the survey by return SMS if they are not NSW residents.

Interviews

Interviews are carried out continuously between February and December each year. An 1800 freecall contact number and website details are provided to potential respondents, so they can verify the authenticity of the survey and ask any questions regarding the survey. Trained interviewers at the Health Survey Program CATI facility carried out interviews until the end of 2014. For 2015, the NSW Population Health Survey was outsourced to McNair Ingenuity Research Pty Ltd, which is a social and market research company. All protocols related to the collection of respondent data have been implemented by McNair.

Up to 7 calls are made to establish initial contact with a household, and up to 5 calls are made to contact a selected respondent. Respondents reached by a landline phone number undergo a within-household selection process, where each member of the household has an equal chance of selection for interview. Respondents reached via mobile phone do not undergo this household selection process. Where a child under the age of 16 has been chosen within the household, the parent or main carer for that child completes the interview on their behalf. When an adult respondent that lives in a household with a child or children is selected for interview, at the end of their interview, they are offered to opportunity to complete a secondary interview about one of their children. In 2015, approximately 41% of all primary adult respondents living in households with at least one child under the age of 16 took up this option. If a parent completing an interview about their children is unsure of their child’s height and/or weight, the respondent is offered the opportunity to be contacted later for this information.

Data analysis

For analysis, the survey sample was weighted to adjust for differences in the probabilities of selection among respondents. Post-stratification weights were used to reduce the effect of differing non-response rates among males and females and different age groups on the survey estimates. These weights were adjusted for differences between the age and sex structure of the survey sample. Population data based on Australian Bureau of Statistics estimates and population projections based on data from the NSW Department of Planning and Infrastructure have been used to calibrate weights to the population within each health administrative area.

Call and interview data were manipulated and analysed using SAPHaRI and SAS version 9.4 (SAS). The Taylor series expansion method was used to estimate sampling errors of estimators based on the stratified random sample. The 95 per cent confidence interval provides a range of values that should contain the actual value 95 per cent of the time.

Trend analysis

A proportional hazard regression model with time equal to one unit using PROC SURVEYREG in SAS software was fitted. PROC SURVEYREG produces relative risks while taking into account the complex survey design; the strata and weights. The strata are a combination of Local Health District and year. The weights are based on the probability of selection in the survey and the age and sex structure of the population each year. For trend analysis, the weights are recalibrated to the 2001 Australian Standard Population by five year age groups to age standardise the analysis. Age-sex standardisation was implemented across the complete survey file (both adult and child records). Separate models were fitted for each sex, and each model had year as the independent variable and the binary indicator as the dependant variable.

Estimated annual rates of change for health indicators (and associated 95% confidence intervals) were calculated from these models as relative differences. If the confidence intervals for the relative difference did not overlap a value of 1, the change was considered statistically significant.

In the reporting of trend analysis results on the topic page data summary tables, up and down arrows are used to show statistically significant increasing and decreasing annual rates of change. If the rate of change is not statistically significant, the trend is considered stable as illustrated by horizontal arrows. For statistically significant trends, the percentage point difference between modelled prevalence for the most recent year of data and that for 5 (short term trend) or 10 years (long term trend) prior. The short term percentage point difference is based on the model incorporating the 5 most recent years of data. The long term trend analysis is based on the model incorporating the 10 most recent years of data. For context, the raw prevalence (and associated 95% confidence interval) estimated from the survey for the most recent year is also reported in the data summary tables.

References

Australian Bureau of Statistics. Standard Population for Use in Age-Standardisation Table (Cat. no. 3101.0), 2013

Australian Communications and Media Authority (ACMA). Communications report 2010-11 series: Report 2 – Converging communications channels: Preferences and behaviours of Australian communications users. Commonwealth of Australia, 2011. Available at http://www.acma.gov.au/

Barr ML, Ritten JJ, Steel DG, Thackway SV. ‘Inclusion of mobile phone numbers into an ongoing population health survey in New South Wales, Australia: design, methods, call outcomes, costs and sample representativeness’. BioMed Central: Medical Research Methodology 2012, 12:177 (22 November 2012). Available at www.biomedcentral.com/1471-2288/12/177.

Centre for Epidemiology and Evidence. NSW Adult Population Health Survey Methods. CEE, NSW Ministry of Health. Available at http://www.health.nsw.gov.au/surveys/adult/Pages/default.aspx

Centre for Epidemiology and Evidence. NSW Child Population Health Survey Methods. CEE, NSW Ministry of Health. Available at http://www.health.nsw.gov.au/surveys/child/Pages/default.aspx