About web surveys
The popularity of web surveys
Under-coverage
Self-selection
Weighting adjustment
Measurement errors

The popularity of web surveys

Web surveys have quickly become a very popular means of collecting survey data. This popularity is not surprising. A web survey is a simple means of getting access to a large group of potential respondents. Questionnaires can be distributed at very low costs. No interviewers are needed, and there are no mailing and printing costs. Surveys can be launched very quickly. Little time is lost between the moment the questionnaire is ready and the start of the fieldwork. Web surveys also offer new, attractive possibilities, such as the use of multimedia (sound, pictures, animation and movies).

At first sight, web surveys seem to have much in common with other types of surveys. It is just another mode of data collection. Questions are not asked face-to-face or by telephone, but over the Internet. Indeed, with a well-designed web survey it is possible to collect data that allow for drawing accurate and reliable conclusions about the target population of the survey. Unfortunately, not every web survey is well-designed. If the scientific principles of survey methodology are not applied, web surveys results may easily be substantially biases. The most important issues are under-coverage, self-selection and measurement errors.


Under-coverage

Under-coverage means that the sample selection mechanism of the survey is not able to select specific parts of the population. If data is collected by means of the Internet, only respondents with Internet access can complete the questionnaire form. The target population of a survey is, however, usually wider than just those with Internet. This means that people without Internet are excluded from the survey.

Research shows that people with Internet access differ, on average, from those without Internet access. As a consequence, web survey results only apply to the sub-population of people having Internet. These results cannot be used to say something about the target population as a whole. Or, to say it differently, web survey based estimates of population characteristics are biased.


Self-selection

Self-selection means that it is completely left to individuals to take the initiative to participate in the survey. In case of web surveys, the survey questionnaire is simply put on the web. Respondents are those individuals who happen to have Internet, visit the website and decide to participate in the survey. The survey researcher is not in control over the selection process.

The fundamental scientific principles of survey taking require that samples are selected by means of probability sampling. Each person in the population must have a known and positive probability of being selected in the survey. Only then can unbiased estimates of population characteristics be computed. In case of self-selection, selection probabilities are completely unknown. Therefore, it is impossible to compute unbiased estimates.

Weighting adjustment

Under-coverage and self-selection leads to samples that are not representative for the population the came from. One could attempt to repair this lack of representativity by applying some kind of weighting adjustment techniques. Such a technique assigns weights to respondents. Under-represented groups are assigned a weight larger than 1, and over-represented groups get a weight smaller than 1.

Application of weighting adjustment requires auxiliary variables. These variables must have been measured in the sample and also their population distribution must be available. Weighting adjustment works only if there is a strong relationship between the survey variables and the auxiliary variables, and also between the auxiliary variables and the participation behaviour.

Measurement errors

If high quality data is required, it is recommended to conduct a face-to-face survey or a telephone survey. Interviewers do the data collection for these types of surveys. They can persuade persons to participate. They can also assist the respondents in understanding the questions and giving the right answer.

There are no interviewers for web surveys. The respondents are on their own. On the one hand this may reduce the quality of their answers. On the other hand, the answers to sensitive questions may be closer to the truth. To avoid incorrect answers, it is important that a lot of attention is paid to the design of the web survey questionnaire.