- Medicine Information and Evidence for Policy > Medicines Policy
- Medicine Information and Evidence for Policy > Monitoring and Evaluation
(1999; 250 pages) [French]
Step 7: Develop data collection forms
The data collection forms which will be used to collect and record the data must be designed, pretested and finalized before starting the data collection process. Data collection forms are necessary for collecting data both from interviews and record reviews and from surveys. The first are mainly checklists; the monitoring unit can adapt for this purpose the summary forms in Annex 1. The second are more complex to design as the same form should be used to collect data for several indicators; a set of model data collection forms has been developed to assist countries. These forms can be used as they are and therefore directly copied from this manual or adapted to national situations (see Annex 1). When adapting or designing new forms, it is important to consider the following:
• the precise objective of the indicators for which data are collected: lack of understanding of the precise question asked in the indicator can lead to the collection of unnecessary data and to delays and additional work later;
• the exact data which need to be collected: as a general principle, the minimum amount of data sufficient for monitoring national drug policy should be collected in the simplest possible way;
• the design of the form: the text should be easy to understand and enough space should be provided on each form for data collectors to be able to note questions or comments;
• facilities for analysis: computer or manual; if computers are used, data processing personnel should be asked for advice on the best way to design the data collection forms and to code the answers.
Step 8: Select and train staff
A key step in preparing for field work is for the monitoring unit to identify and train data collectors. In this manual, the majority of indicators are collected at central level from interviews and records, and the remaining ones at central and peripheral levels through surveys. It is therefore advisable, as long as a monitoring system is not in place, to set up two groups of data collectors: one to collect information from records and interviews, and the second to carry out the surveys. The first group should be familiar with pharmaceutical terms to be able to extract information reliably from records and to note it accurately during interviews. Two people (for example, one pharmacist and one economist) are normally sufficient, as the monitoring unit should also participate in the data collection at central level. The composition of the second group will depend on the number of sites where surveys are to be carried out and their distance from the capital city; each team may include one supervisor (if possible one member of the monitoring unit as well as the first group of data collectors) and two other people: one familiar with surveys but not necessarily a health/pharmaceutical specialist and one familiar with drug issues and drug names.
To ensure consistency in results, the monitoring unit should carefully train all data collectors. Data collectors should be thoroughly familiar with the objectives of the work and with the methodology. Therefore, it is recommended that the following be discussed with them: objectives of the work; rationale; data collection tools to be used; sampling procedures; plan for data collection, and analysis. For each indicator, they should know why the information is required and how it will be collected. They should also be taught how to fill in the data collection forms, which part they need to fill in, who will carry out the calculations, etc. Although each data collection form lists the data needed to obtain the indicators, some instruction should be given on how to handle any missing information so that this will be done consistently by different data collectors in the field. They should be requested to note anything which is unusual, as well as comments from staff in the places surveyed and reasons for modifying forms and methods during the field work.
Collectors should receive clear instructions on what to do when a facility is not operational or when it is impossible to obtain 30 prescriptions/sales in one day. They should also be taught basic interview and record review techniques so that there is no misunderstanding over how to record answers, observations and data. Finally, explanations should be given concerning how the data collector should introduce himself or herself (a survey is not an inspection), how to explain the objective of the work, how to ask for consent, etc.
Step 9: Conduct the pretest
A pretest should always be conducted. This serves two purposes: (i) to test the relevance of the data collection forms, and (ii) to train the data collectors. The pretest also enables the monitoring unit to revise the methods and logistics of data collection before starting the actual field work. As a result, a good deal of time, effort and money can be saved in the long run.
Pretesting should determine whether the data collection forms allow collection of the information needed, whether the forms are reliable, how much time is required to administer the various checklists and questions, and whether the forms need to be revised (e.g. wording unclear, coding system inadequate, space for answers insufficient, etc.).
Involving the data collectors in the pretest will provide them with a practical training and will strengthen their interest in and commitment to the work. It will also provide them with some basic experience on how to organize their work before starting the real data collection.
Finally, training and pretesting allow the central team to check the accuracy of the budget and of the estimated time required for collecting data at the various sites.
Step 10: Schedule data collection
Although the work is usually performed by the ministry of health, it is important that all relevant authorities are adequately notified of the purpose and methods of the exercise. This is particularly true for surveys which mobilize regional administrative staff as well as local authorities. The monitoring unit should inform these authorities of the objectives of the data collectors' visit and the dates proposed, in order to promote their active cooperation. A schedule of visits should therefore be prepared with the dates of every visit, whether at central or peripheral level. Logistical arrangements (transport, accommodation, etc.) should also be made in advance, not forgetting such factors as appropriate seasons, public holidays and vacation periods, and accessibility and availability of the key people to meet.
A critical factor for the success of any survey is the cooperation of the owner/person in charge and personnel of the facility surveyed. In the research discussed (see footnote 1), a few countries reported problems in obtaining cooperation at private drug outlets/pharmacies; in most countries, however, this was not the case. Countries where private sector cooperation was readily obtained used one or several of the following strategies:
• data collectors had an official letter of introduction addressed to the facility; this letter also stated briefly the purpose of the survey;
• an official letter announcing the survey and asking for the collaboration of the facility was sent in advance (but data collectors should also have a copy of the letter with them);
• data collectors were well aware that they were not inspectors, and therefore that they had no right to impose themselves, but had to request (and depend on) cooperation;
• it was clearly explained that results would be presented in a randomized way and therefore be "anonymous", so that it would be impossible to relate specific data to a particular facility.
The same strategies are useful in the public sector.
Step 11: Collect data
Data should be entered on the spot on the relevant data collection form. When data are incorrectly entered the data collector should be told what to do, e.g. to cross them out neatly, sign and enter the correct answer alongside.
Additional information should be given if the data collectors are also requested to code the data and to fill in summary forms. This can be done only in countries where the level of knowledge and training of data collectors will be sufficient for them to code indicators reliably, e.g. people who are not familiar with drug names may be confused by different ways of referring to the same product. The possibility of assigning incorrect codes in such situations is greater than if better-trained people assign all codes at a later stage.
Step 12: Ensure quality
It is extremely important that the data collected are of good quality, i.e. accurate, reliable and valid. To ensure this, supervision and evaluation of the data collection should be done by the monitoring unit in addition to proper selection and training of data collectors and pretesting of data collection forms and procedures.
This ongoing supervision of the data collectors - in the field for the surveys and at central level for the other methods - should aim to ensure that the agreed procedures and methods are being followed and the data collected make sense, and are complete and of good quality. It should also assist in solving problems (e.g. incomplete answers, unclear statements, omissions, etc.) and in handling and storing data collection forms before they are processed and analysed.
In addition, it is recommended:
• to cross-check data whenever possible;
• to use data from (official) reports, if possible;
• to note down not only the data themselves, but also the source and the period to which they apply;
• to ascertain whether the figure given is an actual figure, a forecast or an estimate;
• to use data from the same source when comparing one year with another;
• to use data from a source that (most likely) will be updating its information regularly, so that next time you can use data from that source again;
• in some cases, to consider using data from "international sources", such as the World Bank and/or the IMF, instead of local data.
Biased data are also an important problem; bias can be intentional or subconscious. Deliberate biases can be introduced when staff involved in the survey are very motivated to show that their programme or region is performing very well, or when personnel in the facility investigated have the feeling they are being inspected. On the other hand, some people may have an interest in introducing a negative bias.
Whilst not a great deal can be done about bias that one is unaware of, some "tips" can help: have a supervisor in the field often, and ask data collectors to note down anything unusual they see, as well as remarks from personnel in the facilities investigated. It should be clear to anyone involved in collecting, analysing or presenting the data that the purpose of the exercise is to measure what is really going on, in order to improve the situation, and not to control anybody.