|
Evaluation
of the Northern Ireland Census Coverage Survey
The
Aim of the Survey
The main aim of the CCS was to collect the information required to
estimate the coverage of the 2001 Census. The coverage itself will
be estimated by comparing the information collected in the CCS with
the Census returns from the Survey areas.
The results will be used to calculate the adjustments
required in the “One Number Census” (ONC) outputs.
A further aim of the CCS was that it should also yield some
information about the quality of the 2001 Census data.
Scope
This report is limited in scope to comment on the practical aspects
of the survey.
Full analyses of response rates and other measures can only
be done once the CCS and Census processing is complete.
As a result this report concentrates mainly on the
qualitative aspects.
The
Nature of the Survey
The Survey took the form of an independent, intensive re-enumeration
of a sample of postcodes involving a short doorstep interview with
each household that could be contacted in the sampled postcodes. The
Survey Questionnaire used for the interviews was designed to be as
concise and manageable in the field as possible. The CCS
questionnaire, like the Census form itself, was designed for capture
by scanning and recognition.
The
Interviewers firstly checked if the people they contacted were
resident at the address in question on Census Day.
They then carefully checked on who was in the household to
ensure that no households or residents were missed.
They asked probing questions about those groups likely to be
under-reported, such as babies, students and young people, and the
elderly. The Interviewers also enquired about visitors as an aid in
identifying the household correctly but the information about
visitors was not recorded.
Sample
Design
For the purposes of the CCS, Northern Ireland was divided up into 3
Estimation Areas (EAs). Belfast was considered as a design group on
its own. The rest of the LGDs were grouped into a standard East and
West classification. Using 1991 Census figures, the population of
the Belfast design group was 279,215, the population of the East
design group was 700,364, and the population of the West design
group was 598,111.
Enumeration
District Type (EDT) Index for Northern Ireland
The stratification of 1991 EDs in Northern Ireland was based on
observed response rates to Northern Ireland’s (voluntary) 1997
Census Test, which was designed as a fractional replicate of a 2x2x3
experiment where EDs were the sampling units, classified by
predominant religious background (3 levels), urban/rural and
deprived/non-deprived.
The
number of Northern Ireland EDs sampled was specified using the same
sampling fraction as for England and Wales (approximately 3.6 per
cent). For Northern Ireland this implied a first-stage sample of 134
EDs.
In
England and Wales, simulation work
suggested that a random selection of five postcodes from each
sampled ED (or less in situations where the ED does not contain
five
postcodes) was
a good compromise between clustering for cost efficiency and
spreading the sample of postcodes for statistical efficiency.
Northern Ireland’s experience in the 1999 rehearsal was that the
distribution of postcodes (in terms of population size) was very
skewed with many postcodes having very small numbers of households.
Accordingly, in Northern Ireland, postcodes were successively drawn
at random until the number of households in the sample set reached
at least 70, when sampling stopped. In practice, most of the sampled
postcodes contained approximately 75 households.
The
Data Collected by the Survey
The CCS interviewers collected the following information:
 |
the
address including the postcode; |
|
basic
information about the house; |
|
information
on each person;
|
name; |
|
relationship
within household; |
|
date
of birth; |
|
sex; |
|
marital
status; |
|
whether
a student and if so whether at term-time address; |
|
religion; |
|
whether
the person had a different address one year ago; and |
|
economic
status. |
|
The
Survey used pre-coded classifications similar to those used in the
Census. No write-in responses were required for the “Other”
category in the religion question.
At
the end of the fieldwork a self-completion questionnaire was left at
each address where no contact has been made, in a final attempt to
secure a response. A stamped and addressed envelope was provided for
sending the questionnaire back to NISRA.
The
CCS Management Structure
The survey was directed by the CCS Project Manager who was located
in Demography and Methodology Branch, NISRA. In addition,
there were 3 NISRA HQ based CCS Team Managers – one for
each of the Estimation Areas. The Team Managers were trained by the
GRO Scotland CCS Project Team with regard to fieldwork management
procedures, interviewers training and management, fieldwork progress
reporting and logistics.
There
were also 100 Central Survey Unit interviewers used in the CCS.
Timing
The survey was carried out as soon as practicable after the Census
in order to minimise the effect of migration and the problems faced
by respondents in recalling their details on Census Day.
Interviewer field duties (geography checks and property
listing round) began on 17 May; actual interviewing began on 21 May.
Interviewing duties ceased on 27 June.
Progress
Reporting
Interviewers were supplied with paper pro-formas to record their
progress, which they completed weekly on both an actual and
cumulative basis. These were then posted into the Survey HQ weekly
for collation by the Team Managers and reporting to senior
management.
Emergencies
and Changed Procedures
The Foot and Mouth outbreak in the UK was an unforeseen problem that
coincided with the timing of the CCS.
Emergency procedures had to be compiled and disseminated to
field staff quickly to ensure the adverse impact of the disease was
kept to a minimum.
At
all times Headquarters staff were able to devise alternative
interviewer plans within a very short timeframe and to disseminate
them into the field. The field staff were, likewise, highly
receptive to the necessary alterations to survey procedures.
They provided useful feedback on the state of the situation
at all times.
Field
Methodology
Most of the field methodology used in the CCS was developed as a
direct result of the findings of the 1999 CCS Rehearsal and other
pilot exercises. All the methodology was geared towards achieving a high
response rate to a voluntary survey and therefore finding as many
people as possible who had been missed by the Census and there is
real evidence that the interviewers found whole properties that were
missing from other address lists during their on-the-ground property
listing phase.
Having
a public interface in the form of the interviewer meant that any
questions members of the public had about why the survey was being
run could be answered straight away.
Persuasion could be used to keep compliance rates up, and the
form-filling burden for the public was non-existent.
The
varied calling strategy limited the number of wasted visits. The
postback "last chance" option, involving a self-completion
questionnaire for all households not contacted by the last day of
the survey, resulted in a further 2 per cent response. There is
anecdotal evidence that the probe questions on the survey form
worked well, with people who would have otherwise been missed.
Maps
Interviewers were provided with street maps for their postcode
areas, but this was only to be used as a guide.
On-the-ground checks by the interviewers were used to
identify households within the selected postcodes rather than
identification from the maps or other means.
This meant that if any of the maps were out of date in any
way, the property listing round managed to account for them.
Geography
and Workload Planning
This was a complex area that involved manually grouping selected
postcodes together into interviewer workloads based on a number of
criteria (e.g. expected household count, distance apart).
Training
Training was delivered by GRO Scotland in a two part residential
course to Team Managers and passed on by the Team Managers to the
Interviewers. All training material was developed by GRO Scotland and the
Northern Ireland CCS Headquarters team.
Logistics
A survey operation the size of the CCS required some careful
logistical planning on both the delivery and the pick-up of supplies
in the field. It required close liaison with ONS and GROS on forms design
and delivery. All completed
survey forms were delivered to the processing centre in time for
processing to commence. TNT,
the delivery contractor, was efficient and flexible. Any problems
were dealt with swiftly and efficiently.
Public
Forms
The survey questionnaires were developed and tested during the
rehearsal and other pilot exercises carried out by ONS on behalf of
the three UK Census Offices. Although
the forms were designed to be completed by the interviewers there
was still a requirement for clarity in order that
information recording mistakes did not occur.
The choice of questions, particularly the inclusion of
probing questions, were designed to find people within households
who would otherwise have been missed.
Therefore as well as finding missing households (using the
property listing field methodology) the nature of the questioning
meant that it was also possible to find missing people within
households. In addition to this the employment of clear and simple
showcards kept the interview time short - multiple choice responses
could be displayed to the householder rather than asked. The late
decision to merge individual country forms into a single common UK
form for contractual reasons did not present any difficulties for
the experienced interviewers.The colour of the form however did
present some problems in that a number of people got it confused
with their Census form and wondered why they were being asked to
complete a second Census form.
Public
Helpline
A helpline number was provided that allowed members of the public to
phone in if they had any questions regarding the purpose or
legitimacy of the CCS. Of the 40 calls received, all were from
people who had been uncontactable and had therefore received their
CCS form in the post. About
three quarters of these were from people who had got the CCS form
confused with their Census form, the remainder were complaints,
predominantly from people asking why they had to answer the CCS
questions when they had already given their answers to these
questions on their Census forms.
Overall
Survey Performance
The number of people found who would otherwise have been missed by
the Census is the key measure of the Census Coverage Survey’s
success, and although this will not be known until processing is
complete evidence gathered so far indicates that the survey will
have met its objectives. Of particular note is the response rate
(number of successful interviews achieved as percentage of
properties found) of 92 per cent.
Key
Points
|
The
field staff structure, although not optimal, was effective. |
|
The
interview questionnaire worked well with no reports of
systematic interviewer or interviewee error. |
|
Maps
were clear and, for the most part, up to date. |
|
Training
programmes for Team Managers and Field Staff were well
received. |
|
Progress
reporting arrangements were slow – an electronic means of
reporting progress, as employed in England and Wales, would
have worked better. |
|
New
procedures were developed to enable the survey to continue in
areas affected by the foot and mouth disease outbreak. Centred
around telephone interviewing where face to face interviews
could not take place, these worked well particularly given
they were unplanned. |
|
The
Public Helpline worked well. |
|
There
was a general feeling that administrative tasks of field staff
were over-bureaucratic, especially for experienced
interviewers. |
|
Extensive,
good and early planning is essential. |
|
Good
communications and current management information are
essential in a survey of this size and timing. |
|
The
team working ethos of the CCS was a real strength.
|
Conclusion
The
Census Coverage Survey achieved its primary objective in terms of
overall response rates, whilst keeping the variation in the response
between areas as low as possible.
|
