- Medicine Information and Evidence for Policy > Information and Publications
- Medicine Access and Rational Use > Rational Use
(2005; 165 pages)
12.5.5 How many replies are enough?
Sample size and selection of respondents need careful consideration. The simplest option is to send the survey to everyone on the mailing list. A response rate of no more than 20 - 25% is a realistic expectation. Where the mailing list is large, resources may not permit analysis of thousands of responses. Alternatively, you can select and send the survey to a representative sample of those on the mailing list.
In constructing a representative sample, you may want to consider demographic factors such as age, sex, individual or group practice, medical specialty, rural or urban location. This will help to make sure that you are reaching a range of different types of readers. The variables used depend on what you know about your readers. For example, if you know the location and specialty of those on the mailing list, select the sample by first grouping men and women separately, then subgroup by primary care doctor and specialist, then further subgroup by rural and urban location. If your mailing list is large, you can divide it into many subgroups. If it is small you may want to stick to larger groups of different types of readers. You can then randomly select names from each group. Most computerised statistical programmes include a random number generator. Consult a statistician on the number of individuals to be selected from each group.
Strategies for increasing the response rate include:
• A follow-up telephone call/letter to non-responders. Contacting non-responders is good if you know who they are. (This needs to be taken into consideration if the survey is confidential i.e. you have to be able to identify who has responded without knowing their name); or
• Replacing non-responders with another group of readers who meet the same randomisation criteria;
• Always assume you will get a poor response and send out more questionnaires than you think you need.
Box 12.2 Readership survey checklist
Purpose of the survey
• What are the bulletin’s aims? E.g. convey current drug information, advocate changes in regulation, etc.
• What readers are you targeting? E.g. all, or a subgroup such as primary care doctors, hospital doctors, pharmacists, the public.
• What sort of information do you need from this survey? e.g. extent bulletin is read, usefulness of the articles, suitability of electronic publishing.
• Whose opinion do you want? E.g. all on mailing list, or a particular subgroup.
• What will the survey achieve? E.g. ideas for improving the format, evidence to support continued funding.
• Have similar surveys been done? Review the literature for published reports.
• Ask editors for survey reports and sample questionnaires via the ISDB network
• What errors were made and what problems were encountered in other similar studies? Talk to other editors and ask for a copy of their survey.
• Who could give advice? Ask an experienced editor or researcher.
• Will you let participants know who the research is being conducted for? This has potential advantages and disadvantages.
• What questions need to be asked?
• How many readers should be sent the survey?
• How will they be selected?
• What demographic variables (profession, location, etc.) should be considered?
• Should respondents be anonymous? This is usually recommended to encourage a full and honest response.
• What is the likely response rate? Remember you may be working with busy medical professionals so this may not be as high as for other types of survey.
• Will non-responders be followed-up; if so, how will they be contacted?
• Is an inducement needed/appropriate to ensure a good response rate?
• Is it necessary to include a stamped self-addressed envelope?
• If the service is available, consider reply-paid envelopes - you only pay the postage for those returned.
• What is the cut-off date for processing responses?
• Is it sufficient for a few readers to review the questionnaire before full distribution or is a pilot study needed?
• How will the data be collected, processed and analysed?
Administration, workplan and budget
• Who has overall responsibility for the survey?
• Who will monitor activities/progress?
• Who will process and analyse the results?
• How long will it take? Is the estimated length of time realistic?
• What resources are needed (staff/money/data processing software etc.)?
• Do you have the appropriate analysis tool?
• Do you have the necessary expertise or access to expertise?
• Who could provide funding and equipment?
• Was the response rate adequate?
• Can valid conclusions be drawn from the results?
Publication and use of results
• Who needs to see the final report? Who is interested in your results? You may want to publish a summary of the results in the bulletin. This is a way of showing respect for respondents and valuing their contribution, as well as allowing readers to compare their attitudes to the overall trend.
• What actions are needed in response to the survey results? Who will do this?
• Who else could benefit from this research? You may want to submit an article on your survey and what you learned from it to the ISDB Newsletter.
• How could you have improved the survey?
• When should you next undertake an evaluation?
If you have divided people into subgroups using demographic variables, such as whether they are male or female, a primary care doctor or specialist, etc., include a question on these variables in the survey. This will help to show you whether a representative sample of people from each of these groups responded to your survey. Also, bear in mind that, in order to draw valid conclusions from the survey findings, you may need to be able to identify respondents (e.g. which responses are from general practitioners) so build this into the survey design.