Once planning is complete, develop the questions. Consider what questions to ask, how they will be asked, and the order (see Box 12.3).
Box 12.3 General rules for drafting survey questions2
|
1. Avoid asking two questions at once e.g. 'Do you consider articles are accurate and unbiased?' These two points need to be divided into two questions.
2. Word questions so that all respondents understand the same meaning.
3. Be as precise as possible in the framing of questions e.g. 'Have you recently changed the advice you gave a patient because of an article?' would be clearer if worded as 'In the last six months, have you changed the advice you gave to a patient because of an article?'
4. Tell respondents when a question can have more than one response, e.g. 'You can tick more than one box'.
5. Design the survey to make the task of reading questions, following instructions and recording answers as easy as possible.
6. Word questions so that clear, unambiguous answers are more likely to be given.
7. Simple tick/check boxes work well for answer options, along with providing space for respondents to write additional comments.
8. Consider using questions with similar scales to one another so that respondents can become familiar with the task they are undertaking.
9. If using agree/disagree statements, try to ensure a mix of positive and negative statements.
10. Remember to allow for all possibilities in your responses: this may include having a ‘don’t know’ or ‘none of these’ for some questions.
|
The questions asked will depend on the information being sought. Here are some ideas from which to develop questions:
• The extent to which the bulletin is read.
• What happens to each issue?
• Factors preventing more articles being read.
• General readability of the bulletin.
• Preferred authors (national/international, specialists/primary care doctors etc.).
• Level of detail required in articles.
• Usefulness of various articles or sections.
• Content of articles (e.g. should clinical advice be given? If it is already, is the level of advice adequate?).
• Perceived influence of the articles (on prescribing practice, advice given to patients etc.).
• Acceptability of format, suggestions for improving design.
• Frequency of bulletin editions.
• Comparison with other journals.
• Suggestions for future articles.
Many questions merely require a yes or no response, or a comment. Those that measure degrees of frequency, satisfaction or other such parameters require presentation of a scale from which the respondent can select.3 Appropriate options have to be provided. Where there may be interpretation problems, consider using scales respondents are familiar with. It is always helpful to design your survey form with enough space for people to enter comments about the bulletin, which could be unrelated to the questions you asked them.
Measuring frequency
One approach is to ask the respondent to estimate the frequency of a certain action e.g. 'In the last six months, how many times did you refer back to previously published articles?' However, respondents may not remember how often they did something so providing a response scale with a rough quantification may be useful e.g. 'How often do you refer back to a previously published article?'
Every few weeks
Every few months
Less often than every six months
Never
Avoid using a response scale of ‘often’, ‘sometimes’, ‘occasionally’, ‘rarely’ etc. as they are difficult to interpret.
Asking evaluative questions
Questions asking respondents to say how they rate specific aspects of a bulletin are common in readership surveys, e.g. what do you think of the standard of the book reviews? A numerical scale from 0 to 5, where 5 is excellent and 0 is poor, could be used or a scale such as:
Excellent
Very good
Good
Fair
Poor
To encourage accurate and full completion of the survey, keep scales simple (i.e. restrict the number of different answer options) and ensure there is a clear distinction between scale levels. If the differences are too subtle, this will affect the quality and accuracy of the survey results.
Measuring feelings
The following scale is often used to rate how respondents feel about something:
Very positive
Generally positive
Mixed: about equally positive and negative
Generally negative
Very negative
A numerical scale can also be used to measure feeling, for example, 'Do you feel the bulletin improves your ability to recognise an adverse reaction in a patient?' (on a scale of -2 to +2, with 0 being the midpoint). Note: while every point on the continuum does not need to be defined, there should be a midpoint where positive feelings turn to negative.
Rating agreement
Often readership surveys ask how much the respondent agrees with certain statements e.g. 'Some readers have suggested including a summary at the beginning of each adverse reaction article. How do you rate this suggestion?' An appropriate scale may be:
Strongly agree
Agree
Disagree
Strongly disagree
While it may be appealing to offer a middle category (neither agree or disagree) it may be more useful to make the respondents commit themselves.
Ranking the bulletin
If, for example, you want respondents to prioritise your bulletin against three others publications for usefulness, list the publications and ask the respondent to rank them from 1 to 4, where 1 is the publication they find most useful, and 4 is the publication they find least useful.
Data processing and analysis
While data from simple readership surveys can be processed manually, a software package may be needed to process more complex survey responses. There are many data processing software packages available - ask a local researcher for advice about which to use.
Unlike clinical studies or full impact evaluations, statistical analysis of readership surveys is not usually necessary. If statistical analysis is required, ask a statistician for advice when planning the research.
At the end of this chapter there is an example of a readership survey questionnaire together with the results, from the French bulletin Bulletin d’Information du Médicament et de Pharmacovigilance.
Loading...
