Starting or Strengthening a Drug Bulletin - A Practical Manual
(2005; 165 pages) View the PDF document
Table of Contents
View the documentPreface
View the documentHow the manual was produced
View the documentAbout ISDB
View the documentExecutive summary
Open this folder and view contents1. Introduction
Open this folder and view contents2. Rational use of medicines
Open this folder and view contents3. What are drug bulletins?
Open this folder and view contents4. Defining aims, target and type of bulletin
Open this folder and view contents5. Planning resources
Open this folder and view contents6. Planning bulletin production: schedules and timing
Open this folder and view contents7. The editorial process
Open this folder and view contents8. Reviewing a new drug: is it a therapeutic advance?
Open this folder and view contentsAnnexe to Chapter 8: Evaluating harm
Open this folder and view contents9. Design and production
Open this folder and view contents10. Dissemination
Open this folder and view contents11. Organizational and legal issues
Close this folder12. Evaluating quality and usefulness
View the document12.1 Introduction
View the document12.2 Evaluation brings many benefits
View the document12.3 Three approaches: audit, feedback and impact assessment
View the document12.4 Start with your own evaluation of the bulletin
Close this folder12.5 Assessing readers' opinions of the bulletin
View the document12.5.1 Methods of assessing readers’ opinions
View the document12.5.2 Potential problems
View the document12.5.3 Selecting the data collection method
View the document12.5.4 Planning the survey
View the document12.5.5 How many replies are enough?
View the document12.5.6 Drafting the questions
View the document12.6 Evaluating the impact of the bulletin
View the document12.7 Feedback is achievable and invaluable
View the document12.8 Simple observations can tell a lot
View the document12.9 References
Open this folder and view contents13. Partnership and collaboration
Open this folder and view contents14. Keeping records and creating a memory
Open this folder and view contentsAppendix: Electronic sources of information

12.5.5 How many replies are enough?

Sample size and selection of respondents need careful consideration. The simplest option is to send the survey to everyone on the mailing list. A response rate of no more than 20 - 25% is a realistic expectation. Where the mailing list is large, resources may not permit analysis of thousands of responses. Alternatively, you can select and send the survey to a representative sample of those on the mailing list.

In constructing a representative sample, you may want to consider demographic factors such as age, sex, individual or group practice, medical specialty, rural or urban location. This will help to make sure that you are reaching a range of different types of readers. The variables used depend on what you know about your readers. For example, if you know the location and specialty of those on the mailing list, select the sample by first grouping men and women separately, then subgroup by primary care doctor and specialist, then further subgroup by rural and urban location. If your mailing list is large, you can divide it into many subgroups. If it is small you may want to stick to larger groups of different types of readers. You can then randomly select names from each group. Most computerised statistical programmes include a random number generator. Consult a statistician on the number of individuals to be selected from each group.

Strategies for increasing the response rate include:

• A follow-up telephone call/letter to non-responders. Contacting non-responders is good if you know who they are. (This needs to be taken into consideration if the survey is confidential i.e. you have to be able to identify who has responded without knowing their name); or

• Replacing non-responders with another group of readers who meet the same randomisation criteria;

• Always assume you will get a poor response and send out more questionnaires than you think you need.

Box 12.2 Readership survey checklist

Purpose of the survey

• What are the bulletin’s aims? E.g. convey current drug information, advocate changes in regulation, etc.

• What readers are you targeting? E.g. all, or a subgroup such as primary care doctors, hospital doctors, pharmacists, the public.

• What sort of information do you need from this survey? e.g. extent bulletin is read, usefulness of the articles, suitability of electronic publishing.

• Whose opinion do you want? E.g. all on mailing list, or a particular subgroup.

• What will the survey achieve? E.g. ideas for improving the format, evidence to support continued funding.

Literature review

• Have similar surveys been done? Review the literature for published reports.

• Ask editors for survey reports and sample questionnaires via the ISDB network

• What errors were made and what problems were encountered in other similar studies? Talk to other editors and ask for a copy of their survey.

• Who could give advice? Ask an experienced editor or researcher.


• Will you let participants know who the research is being conducted for? This has potential advantages and disadvantages.

• What questions need to be asked?

• How many readers should be sent the survey?

• How will they be selected?

• What demographic variables (profession, location, etc.) should be considered?

• Should respondents be anonymous? This is usually recommended to encourage a full and honest response.

• What is the likely response rate? Remember you may be working with busy medical professionals so this may not be as high as for other types of survey.

• Will non-responders be followed-up; if so, how will they be contacted?

• Is an inducement needed/appropriate to ensure a good response rate?

• Is it necessary to include a stamped self-addressed envelope?

• If the service is available, consider reply-paid envelopes - you only pay the postage for those returned.

• What is the cut-off date for processing responses?

• Is it sufficient for a few readers to review the questionnaire before full distribution or is a pilot study needed?

• How will the data be collected, processed and analysed?

Administration, workplan and budget

• Who has overall responsibility for the survey?

• Who will monitor activities/progress?

• Who will process and analyse the results?

• How long will it take? Is the estimated length of time realistic?

• What resources are needed (staff/money/data processing software etc.)?

• Do you have the appropriate analysis tool?

• Do you have the necessary expertise or access to expertise?

• Who could provide funding and equipment?

Data analysis

• Was the response rate adequate?

• Can valid conclusions be drawn from the results?

Publication and use of results

• Who needs to see the final report? Who is interested in your results? You may want to publish a summary of the results in the bulletin. This is a way of showing respect for respondents and valuing their contribution, as well as allowing readers to compare their attitudes to the overall trend.

• What actions are needed in response to the survey results? Who will do this?

• Who else could benefit from this research? You may want to submit an article on your survey and what you learned from it to the ISDB Newsletter.

• How could you have improved the survey?

• When should you next undertake an evaluation?

If you have divided people into subgroups using demographic variables, such as whether they are male or female, a primary care doctor or specialist, etc., include a question on these variables in the survey. This will help to show you whether a representative sample of people from each of these groups responded to your survey. Also, bear in mind that, in order to draw valid conclusions from the survey findings, you may need to be able to identify respondents (e.g. which responses are from general practitioners) so build this into the survey design.

to previous section
to next section
The WHO Essential Medicines and Health Products Information Portal was designed and is maintained by Human Info NGO. Last updated: December 6, 2017