(2004; 98 pages)
5.5 Bias in sampling
Bias in sampling is a systematic error in sampling procedures that leads to a distortion in the results of the study. Bias can also be introduced as a consequence of improper sampling procedures that result in the sample not being representative of the study population. For example, a study to determine the drug information needs of a rural population and plan a community drug use intervention failed to give a picture of the health needs of the total population because a nomadic tribe, which accounted for one-third of the total population, was left out of the study.
There are several possible sources of bias in sampling. The best known source of bias is non-response. In a survey trying to establish how men treat sexually transmitted infections (STIs), it was found that many men refused to answer certain questions, such as whether they had attended an STI clinic in the past month. It is possible that these men feared the consequences of disclosing such sensitive information to an outsider. The researchers may therefore not get a realistic picture of the treatment of STI in the community. Non-response is encountered mainly in studies where people are being interviewed or asked to fill in a questionnaire. They may refuse to be interviewed or forget to fill in the questionnaire. The problem lies in the fact that non-respondents in a sample may exhibit characteristics that differ systematically from the characteristics of respondents. There are several ways to deal with this problem and reduce the possibility of bias:
• data-collection tools (including written introductions for the interviewers to use with potential respondents) have to be pretested. If necessary, adjustments should be made to ensure better cooperation.
• if non-response is due to absence of the subjects, follow-up of non-respondents may be considered.
• if non-response is due to refusal to cooperate, a few extra questions to non-respondents may be considered to discover to what extent they differ from respondents.
• another strategy is to include additional people in the sample, so that non-respondents who were absent during data collection can be replaced. However, this can only be justified if their absence was very unlikely to be related to the topic being studied.
The bigger the non-response rate, the greater the need to take remedial action. It is important in any study to mention the non-response rate and to discuss honestly whether and how it might have influenced the results. Other sources of bias in sampling may be less obvious, but are at least as serious:
• Studying volunteers only. This produces selectivity (or bias) in assigning subjects to various groups. The fact that volunteers are motivated to participate in the study may mean that they are also different from the study population in the factors being studied. It is better to avoid using non-random procedures that introduce the element of choice.
• Sampling of registered patients only. Patients reporting to a clinic are likely to differ systematically from people using self-medication.
• Seasonal bias. It may be that the problem under study exhibits different characteristics in different seasons of the year. For this reason, data on the prevalence and distribution of malnutrition in a community, for example, should be collected during all seasons rather than just at one time. When investigating health services’ performance, to give another example, one has to take into account that towards the end of the financial year shortages may occur in certain budget items which may affect the quality of services delivered.
• Tarmac bias. Study areas are often selected because they are easily accessible. However, these areas are likely to be systematically different from more inaccessible areas.
If the recommendations from a study will be implemented in the entire study population, you should aim to draw a sample from this population in a representative way. If part way through the research new evidence suggests that the sample was not representative, this should be mentioned in any publication concerning the study, and care must be taken not to draw conclusions or make recommendations that are not justified.