Indicators for Monitoring National Drug Policies
(1999; 250 pages) [French] View the PDF document
Table of Contents
View the documentACKNOWLEDGEMENTS
Open this folder and view contentsCHAPTER I: Introduction
Open this folder and view contentsCHAPTER II: Development of the manual
Open this folder and view contentsCHAPTER III: Model lists of indicators
Close this folderCHAPTER IV: Methodology for indicator calculation
View the documentOrganizing the data collection
View the documentCollecting data
View the documentAnalysis and reporting
View the documentConducting surveys
View the documentCalculating the value of a basket of drugs
Open this folder and view contentsCHAPTER V: Detailed presentation of indicators
Open this folder and view contentsANNEX 1: Data collection forms
View the documentANNEX 2: Glossary
View the documentANNEX 3: Table of random numbers
View the documentBACK COVER
 

Conducting surveys

Although most of the data for calculating the indicators can be obtained from the existing monitoring systems and from reviews of documents/records and interviews at central level, data for some process and outcome indicators will need to be collected in health facilities and drug outlets through special surveys. These surveys can be organized to collect data for several indicators at the same time. This section provides the list of indicators which require surveys, guidelines on survey design and implementation including sampling issues, and gives examples of sampling procedures.

List of indicators which require surveys

For the proposed list of indicators, the data for nine process indicators and nine outcome indicators will need to be collected through special surveys in the majority of countries.

Process indicators

PR9:

Number of drugs from the national essential drugs list (EDL) prescribed, out of total number of drugs prescribed.

PR10:

Number of drugs from the national essential drugs list (EDL) sold, out of total number of drugs sold.

PR27:

Average time between order and delivery from central store to remote facilities in the last year, out of average time between order and delivery in the past three years.

PR29:

Average stockout duration for a basket of drugs in a sample of remote facilities in the last year, out of average stockout duration for the same basket in the past three years.

PR30:

Value of a basket of drugs, out of CIF/ex-factory value of the same basket.

PR31:

Average expenditure per prescription, out of average expenditure per prescription in the past three years.

PR32:

Value of a basket of drugs, out of value of the same basket in the year of reference.

PR33:

Number of prescribers having direct access to a (national) drug formulary, out of total number of prescribers surveyed.

PR35:

Number of prescribers who have attended at least one training session in the last year, out of total number of prescribers surveyed.

Outcome indicators

OT1:

Number of drugs from a basket of drugs available in a sample of remote health facilities, out of total number of drugs in the same basket.

OT2:

Number of drugs at the lowest price from a basket of drugs, out of total number of drugs in the same basket.

OT3:

Average retail price of standard treatment of pneumonia, out of the average retail price of a basket of food.

OT4:

Value of a basket of drugs, out of the value of the same basket with the cheapest drugs.

OT5:

Number of drugs/batches that failed quality control testing, out of the total number of drugs/batches surveyed.

OT6:

Number of drugs beyond the expiry date, out of the total number of drugs surveyed.

OT7:

Average number of drugs per prescription.

OT8:

Number of prescriptions with at least one injection, out of the total number of prescriptions surveyed.

OT9:

Number of children under five with diarrhoea receiving antidiarrhoeal drugs, out of the total number of children under five with diarrhoea surveyed.

Survey design and implementation

The various steps which have been described above (see pages 29 to 43) apply also to surveys and need to be carefully followed. However, additional specific steps are necessary and are reviewed below.

Selecting the type of facilities

The data needed for calculating the 18 indicators should be collected in two types of facilities:

• drug outlets (public and private);
• health facilities (public, private and remote).

Table 1 gives the sites for which data should be collected for the 18 indicators requiring surveys.

Table 1: Survey sites for the 18 indicators

Indicators

Private drug outlets

Public drug outlets

Private health facilities

Public health facilities

Remote health facilities

PR9




PR10





PR27





PR29





PR30





PR31




PR32




PR33




PR35




OT1





OT2





OT3




OT4





OT5




OT6




OT7




OT8




OT9





Selecting the type of survey

Most of the data will be collected through cross-sectional surveys carried out in drug outlets and health facilities. Data will be collected from current patients as they present to the drug outlet or from prescribers/dispensers on the day of the data collector's visit. For process indicators PR27 and PR29, data will be collected from existing records and inventory controls.

The literature shows13 that individual health providers tend to exhibit more or less consistent practices over time. Therefore, a sample drawn at one point in time will provide basically the same results as a sample that covers a longer period. However, since data will generally be collected over a short period, they may suffer from bias due to seasonality, variations in staffing, inconsistencies in the supply cycle, or the fact that providers are aware that their behaviour is being observed. Data collectors should be trained to try to guard against these possible sources of bias.

13 For more information, see the document How to investigate drug use in health facilities (WHO/DAP/93.1), in which some of the methodological principles used in this manual are described.

Drawing a sample

To estimate indicators accurately, it is important to follow specific procedures for drawing samples of health facilities and drug outlets. These procedures will vary depending on the context of each country and the availability of data.

The selection of samples should be made in such a way that each sampling unit has an equal chance of being included in the sample. In this way, selection bias can be avoided and the study population will be representative of the reference population. However, there are often major logistical constraints in carrying out surveys, such as transport, time, budget and lack of data collectors with specific skills. The best design for a particular survey therefore depends not only on statistical theory, but also on the practical aspects of collecting the data.

Different strategies for sampling drug outlets and health facilities can be used:

Simple random sampling

This is the simplest form of probability sampling. To select a simple random sample, the first step is to make a numbered list of all the units from which the samples are to be drawn (sampling frame), e.g. all the drug outlets or health facilities. The next step is to decide on the size of the sample, i.e. the number of units that need to be randomly chosen from the sampling frame (see page 48). Then the required number of sampling units is selected using a "lottery" method or a table of random numbers (see Annex 3). The lottery method assigns numbers to all the units; these numbers are then mixed and the required number drawn at random (without replacement). With a table of random numbers, each sampling unit is assigned a number and the required numbers are selected from the table. All units with these numbers constitute the samples. Simple random sampling ensures that the indicators are unbiased, but may not be the most efficient procedure.

Systematic sampling

Sometimes the most convenient way of obtaining a sample is by choosing the sampling units directly from the sampling frame (e.g. taking every n drug outlet from a list of all drug outlets). Ideally a random number is used to decide where to start. The sampling interval will be calculated by dividing the total number of drug outlets existing in the country by the desired sample. For example, if a systematic sample is to be selected from 360 drug outlets existing in the country and a sample size of 20 has been chosen, the sampling interval will be 360/20 = 18. In other words, one in every 18 drug outlets will be included in the sample until 20 drug outlets have been obtained. This strategy is used in the model sampling procedures described below (see pages 49 to 58).

Stratified sampling

In this type of sampling, units (e.g. health facilities/drug outlets) are put into groups according to a characteristic (such as urban/rural area), and the sample is apportioned among these groups according to a set plan, chosen to ensure their representation in the sample. Stratification, along or combined with systematic sampling, can achieve very efficient designs. This strategy is used in the model sampling procedures described below (see pages 49 to 58).

Cluster sampling

It may be difficult or impossible to take a random sample of sampling units (e.g. health facilities/drug outlets) in the study population, either because a list of all the drug outlets or health facilities does not exist, or because of other logistical difficulties (e.g. visiting drug outlets which are scattered over a large area may be too time-consuming). However, when a list of groupings of sampling units - clusters - is available (e.g. districts or provinces) or can be easily compiled, then a random sample of clusters can be selected. Within the clusters that are finally selected, the sampling units (e.g. drug outlets, health facilities) are listed and sampled.

Multi-stage sampling

A multi-stage sampling procedure is carried out in phases and usually involves more than one sampling method. A first stage sampling could, for instance, be cluster sampling of districts and the next stage, sampling of drug outlets within the selected districts. The strategy proposed as an example in this manual is a multi-stage sampling using systematic sampling for selection of drug outlets and health facilities.

Determining the sample size

One difficult aspect of designing a sample is deciding how many health facilities, drug outlets, prescribers and prescriptions to include.

The sample size is usually a compromise between what is desirable and what is feasible. The sample size should be the smallest one that will give an estimate of proportion within the desired degree of precision. The size of the sample is also determined by the availability of time, human resources, transport and money.

Countries with large human and financial resources may wish to make their own decision on a representative sample size. In such cases this should be done with the support of statisticians. The sample size depends on the degree of precision needed and the anticipated proportion of the characteristics under study.

For countries with limited human and financial resources and major logistical constraints on carrying out surveys, a recommended sample size of health facilities, drug outlets, remote health facilities, prescribers and prescriptions has been determined:

• For collecting data for indicators PR9, PR10, PR31, OT7 and OT8, 20 drug outlets or health facilities should be selected randomly and 30 prescriptions or drugs sold per health facility or drug outlet should be collected, amounting to a total sample size of 600 prescriptions or 600 drugs sold (see details on pages 49 to 58). For indicator OT9, it is suggested that the first 5 prescriptions seen for children under 5 years old with diarrhoea should be collected in 20 health facilities, amounting to a total sample size of 100 prescriptions.

Because the treatment practices of individual providers are consistent and similar among providers within the same facility, in-facility sources of variation will tend to be reduced, and after a certain point, adding prescriptions to a sample within a facility provides very little new information. The principal source of variation will tend to be differences in practice between health facilities14. Increasing the number of facilities in a sample will be the best way to obtain more accurate and reliable estimates and will be better than increasing the number of prescriptions sampled within facilities. Because of the substantial variations in practice among facilities for many indicators, it would be hazardous to generalize about a large population of facilities from a sample that includes fewer than 20 facilities.

14 See footnote 13.

• For collecting data for indicators PR33 and PR35, it is suggested that 40 public sector health facilities should be selected randomly (see details on pages 49 to 58). Within these facilities a minimum of 100 prescribers should be included in the sample to achieve any statistically significant comparison from one year to the other.

• For collecting data for indicators PR27, PR29, PR30, PR32, OT1, OT2, OT3, OT4, OT5 and OT6, it is suggested to use the previously selected 20 private drug outlets, to randomly choose 20 of the 40 previously selected public sector health facilities and to randomly select 20 remote health facilities (see details on pages 49 to 58).

Proposed sampling procedures

The following procedures are intended to assist national managers in selecting representative samples of health facilities, drug outlets and prescribers, for collecting data needed for the calculation of the 18 indicators. They are designed as examples for countries with limited resources. However, as stated previously, countries can use other methods and sample size as long as the samples are representative of the country's characteristics.

These procedures are based on the assumption that owing to logistical constraints, surveys should preferably be conducted in a limited number of regions, to the extent that an acceptable sample can be drawn. The selection of the regions where the surveys will be conducted depends on the sites of the surveys: private drug outlets, public drug outlets, health facilities and remote health facilities.

Therefore the procedures described below include two stages: the selection of the regions where the surveys will be conducted and the selection of a representative sample of drug outlets, health facilities, prescribers and prescriptions.

Select the regions where the surveys will be conducted

Selection of regions for the surveys in private drug outlets

In most developing countries, private drug outlets are mainly in the capital city area and in the major urban areas. This should be taken into account when selecting regions in which to conduct the survey.

First, divide the country into geographical units based on administrative regions/districts, with one unit being the capital city area/region. Each unit should include at least one significant urban area (according to the size of the population of the country). Therefore a rural region/district should be merged with one adjacent region/district which includes a significant urban area. A basic principle should be to merge a rural region/district with an adjacent one where there is a reference hospital or an active regional capital with trading activities. Five to 20 geographical units is an acceptable range.

Second, owing to the particular distribution of private drug outlets in most developing countries, it is suggested to select a sample of geographical units as follows:

(a) one geographical unit will be the capital city area/region, where a large number of private drug outlets (G0) is usually concentrated;

(b) three geographical units will be randomly selected, with a probability for each unit to be selected proportionally to the number of drug outlets per geographical unit.

To achieve this, list all private drug outlets in the country, excluding those located in the capital city area/region, and number these drug outlets from 1 to n2. Use a table of random numbers (see Annex 3) to draw a number between 1 and n2 and select the geographical unit which corresponds to that number. Repeat the procedure until three different geographical units are selected (G1, G2and G3) (see Figure 1).


Figure 1: Example of selection of geographical units for determining a sample of private drug outlets

Selection of regions for the surveys in public sector drug outlets and health facilities

The selection of regions where the surveys will be conducted in public sector drug outlets and health facilities can follow two different procedures:

(a) In order to facilitate the logistics and to reduce time and cost, the regions sampled for the private drug outlets (the capital city region G0plus the three geographical units G1, G2 and G3) can be selected. Such a choice is acceptable for countries with limited human and financial resources. But it is based on the assumption that the distribution of public health drug outlets and health facilities is the same as that of private sector drug outlets. This is rarely the case. However, in order to simplify the procedures, it is wise to use this method for the first years in most developing countries.

(b) Because the public sector drug outlets and health facilities are usually not concentrated in the major cities, the grouping of regions in geographical units based on major cities is not necessary. Therefore the administrative division in regions/districts can be used for establishing the list from which a sample of four regions will be chosen. The decision to automatically include the capital city region/district is the responsibility of the central team. It will depend on the country's context. However, in some countries, such choice would reduce the logistical problems and is therefore relevant.

In most of the countries, depending on the size of the regions/districts, four to five geographical units are normally sufficient. For selecting the geographical units, list all the drug outlets/health facilities of the country and number them from 1 to n. Use a table of random numbers (see Annex 3) to draw a number between 1 and n and select the region which corresponds to that number. Repeat the procedure until four different geographical units are selected.

Selection of regions for the surveys in remote health facilities

Remote health facilities can be defined as health facilities located more than 100 km from the capital city or from any city with more than 100,000 inhabitants. In order to facilitate the logistics and to reduce time and cost, the regions sampled for the public sector drug outlets/health facilities can be selected for the remote health facilities, as these are mainly public sector health facilities and some may have already been selected by randomization when selecting health facilities.

In conclusion, it is suggested to use the same regions for all the surveys, at least during the first years of the monitoring process. This will reduce logistical problems and budget.

Select a representative sample of facilities where information will be collected

Private drug outlets

In order to collect the data needed for calculating process indicators (PR9, PR10, PR30, PR31 and PR32) and outcome indicators (OT2, OT3, OT4, OT5, OT6, OT7 and OT8), a sample of at least 20 private drug outlets is required (see page 49). Some countries may prefer to select a larger sample to get more accurate figures. However, such a sample size can be considered acceptable for providing the data needed to calculate indicators with a reasonable level of accuracy in most developing countries (especially those with a low income and few resources). For selecting such a sample of private drug outlets, which should take into account specificities of the drug sector, different procedures could be used. The following is an example which is reasonably easy to implement.

After selecting the geographical units (see page 50), list all private drug outlets located in the capital city area/region (list L1) and number these drug outlets from 1 to n1.

The proportion (p = n1/n) of private drug outlets in the capital city area/region (n1) out of the total number of private drug outlets in the country (n) should be used for defining the size of the sample of drug outlets which will be selected from the capital city area/region (sa). For a total sample (S) of 20 drug outlets, the number of drug outlets which will be selected from the capital city area/region will be sa = p x 20. Then the number of drug outlets which will be selected from the other geographical units will be sb= 20 - sa.

Use a table of random numbers to draw a number between 1 and n1 in the list L1. The number obtained will correspond to the first drug outlet selected. Then move down the list using a sampling interval of i1 = n1/sa to find out the second private drug outlet. When you reach the bottom of the list, go back to the top and repeat the operation. Use the same method until you select the required number of private drug outlets in the capital city area/region (sa). This can be considered an acceptable sample of private drug outlets in the capital city area/region where data for calculating indicators can be collected.


Figure 2: Proposed sampling procedure for selecting a sample of 20 private drug outlets

(p = n1/n)

List all private drug outlets of the three selected geographical units (list L2) and number these drug outlets from 1 to n2. Use a table of random numbers to draw a number between 1 and n2 in the list L2. The number obtained will correspond to the first drug outlet selected. Then move down the list using a sampling interval of i2 = n2/sb to determine the second private drug outlet. When you reach the bottom of the list, go back to the top and repeat the operation. Use the same method until you select the required number of private drug outlets in the three selected geographical units (sb). Such a sample can be considered an acceptable sample of private drug outlets in the selected regions/districts where data for calculating indicators can be collected.

Then add the sample of drug outlets located in the capital city area/region (sa) to the sample of private drug outlets in the selected regions/districts (sb) for obtaining a sample (S) of 20 private drug outlets. Using this methodology, such a sample can be considered an acceptable sample of private drug outlets where data for calculating indicators can be collected (Figure 2). If when collecting data a drug outlet does not exist any more, replace it by the next one on the list.

Public sector drug outlets and health facilities

In order to collect the data needed for calculating process indicators (PR33, PR35) a sample of at least 40 health facilities is required (see page 49); for process indicators (PR9, PR31, PR32) and outcome indicators (OT3, OT5, OT6, OT7, OT8, OT9), only a sample of 20 public drug outlets/health facilities is needed (see page 49). This second sample can be selected from the first using simple randomization, as in many countries most public sector drug outlets are located within health facilities. Some countries may prefer to select a larger sample to get more accurate figures. However, such a sample size can be considered acceptable for providing data to calculate the indicators with a reasonable level of accuracy in most developing countries (especially those with a low income and few resources). For selecting such a sample of public drug outlets and health facilities, which should take into account specificities of the drug and health sectors, different procedures could be used. The following is an example which is reasonably easy to implement (Figure 3).

After selecting the regions (see pages 50 to 52), a sample of public sector drug outlets and/or health facilities can be selected using the same procedures as for private drug outlets. Once a sample of 40 health facilities has been obtained, randomly select 20 public drug outlets and/or health facilities for the indicators where this size of sample is sufficient.


Figure 3: Proposed sampling procedure for selecting a sample of 40 health facilities and 20 public drug outlets

(p = n1/n)

The question of the respective proportion of urban and rural drug outlets and/or health facilities has to be addressed only if it is considered that practices are different in the two situations. In this case a stratified sampling strategy should be used to ensure a minimum number of drug outlets and/or health facilities from urban and rural areas. In the four selected geographical units, list all the public sector drug outlets or the health facilities located in urban areas (L1) and those located in rural areas (L2). Using the same strategy as above in each area, select a sample of drug outlets or facilities for each area.

Remote health facilities

In order to collect the data needed for calculating process indicators PR27 and PR29 and outcome indicator OT1, a sample of 20 remote health facilities is required (see page 49). Some countries may prefer to select a larger sample to get more accurate figures. However, such a sample size can be considered as acceptable for providing data needed to calculate the indicators with a reasonable level of accuracy in most developing countries (especially those with a low income and a population of less than 20 million). For selecting such a sample of remote health facilities, which should take into account specificities of the health sector, different procedures could be used. The following is an example which is reasonably easy to implement.


Figure 4: Proposed sampling procedures for selecting a sample of 20 remote health facilities

After selecting the regions (see pages 50 to 52) list all the remote health facilities in these regions. Consider the remote health facilities already selected in the previous selection of health facilities (sa) as part of the sample of remote health facilities (S). Number the remaining ones from 1 to n1. Use a table of random numbers to draw a number between 1 and n1. The number obtained will correspond to the first remote health facility selected. Then move down the line using a sampling interval of i = n1/sb (sb is the number of remote health facilities which have to be selected according to the size of sample of remote health facilities needed S = sa + sb). When you reach the bottom of the list, go back to the top and repeat the operation. Use the same method until you select the required number of remote health facilities in the selected geographical units. They can be considered an acceptable sample of remote health facilities in the selected regions/districts where data for calculating indicators can be collected (Figure 4). If when collecting data a health facility is not operational, replace it by the next one on the list.

Select a sample of data sources

Prescriptions or drugs sold

In order to collect the data needed for calculating process indicators (PR9, PR10 and PR31) and outcome indicators (OT7 and OT8), a sample of 600 prescriptions or 600 drugs sold is required (see page 49). The following is a procedure which is easy to implement. In each of the 20 drug outlets/health facilities already selected, take the first 30 prescriptions or 30 drugs sold. Although the selection of the 30 first prescriptions or drugs sold is convenient, it may introduce some bias. If the number of prescriptions or customers buying drugs anticipated per day is high, you may choose to take one prescription or one customer in every three. On the contrary, if it is difficult to obtain 30 prescriptions or drugs sold in one day, you may return the following day to complete the data. However, a reasonable time limit should be set for staying at one unit (see box).

Example 7: Data collection in Guinea

A data collector in one remote facility reported: "I have spent two full days in this health centre now, and so far I have seen 5 prescriptions only."

Obviously, it would not be very realistic to require the data collector to stay until having collected 30 prescriptions in this health centre, which would take more than 2 weeks.

Possible actions to reduce the risks of being short of data in any one facility include:

• On the local market day, the number of patients can be substantially higher, so consider surveying remote facilities on that day.

• In cases where there is a good, reliable system of recording all prescriptions, data recorded the previous day could be added to data collected during the data collector’s visit, to make 30.

Prescribers

In order to collect data needed for calculating process indicators PR33 and PR35, a sample of at least 100 prescribers is required (see page 49). The following is a procedure which is easy to implement. Select all prescribers up to five in each of the 40 selected health facilities. If there are more than five prescribers in some health facilities, list all of them and select five randomly. If there are fewer than five prescribers, include all of them in the sample.

Drugs for quality control testing

In order to collect data needed for calculating outcome indicator OT5, a sample of at least 20 drugs is needed. The following is a procedure which is easy to implement. The monitoring unit should indicate on the data collection form which of the drugs from the basket should be collected at each outlet. When visiting the drug outlet, the data collector should pick the selected drug randomly.

to previous section
to next section
 
 
The WHO Essential Medicines and Health Products Information Portal was designed and is maintained by Human Info NGO. Last updated: November 5, 2014