Indicators for Monitoring National Drug Policies
(1999; 250 pages) [French] View the PDF document
Table of Contents
View the documentACKNOWLEDGEMENTS
Open this folder and view contentsCHAPTER I: Introduction
Open this folder and view contentsCHAPTER II: Development of the manual
Open this folder and view contentsCHAPTER III: Model lists of indicators
Open this folder and view contentsCHAPTER IV: Methodology for indicator calculation
Close this folderCHAPTER V: Detailed presentation of indicators
View the documentStructural indicators
View the documentProcess indicators
View the documentOutcome Indicators
Open this folder and view contentsANNEX 1: Data collection forms
View the documentANNEX 2: Glossary
View the documentANNEX 3: Table of random numbers
View the documentBACK COVER
 

Process indicators

These 38 indicators provide quantitative information on the mechanisms and activities by which a national drug policy is implemented. They monitor the main aspects of each key strategy/component of drug policy:

• legislation and regulation;
• essential drug selection and drug registration;
• drug allocation in the health budget/public sector financing policy;
• public sector procurement procedures;
• public sector distribution and logistics;
• pricing policy;
• information and continuing education on drug use.

Within each component/strategy, the process indicators allow assessment of the effectiveness and efficiency of the mechanisms and activities which have been put in place. This effectiveness is measured according to standards and targets set at national level, and which may be modified over time. For instance, a country may decide that in order to improve its drug procurement procedures in the public sector, 90% in value of the drugs will be bought by tender by the year 1999 (PR19); another country may decide to set its target at 70%.

The process indicators are measured by a percentage, using information available at the central level and/or obtained through surveys. For some indicators in certain specific situations, the figures used for calculating the percentages can also provide useful information. These indicators can be used for assisting national decision-makers and senior management staff in monitoring progress in the implementation of the national drug policy, by providing quantitative measures of achievement of targets set at the national level. Each year, the percentage measured should come closer to the national standard. Indeed, they allow comparison between the situation at the time the indicator is calculated the situation a few years previously or an ideal situation (e.g. PR23), and therefore give information on the progress achieved.

The results of the process indicators should be analysed together with the results of the structural indicators and this in the framework of each of the seven key strategies/components. It is considered that if for each component/strategy all the structures and systems are in place (structural indicators) and if they function properly (process indicators), it should be possible to achieve the main objectives of drug policy.

Field experience shows that data which have to be collected at central level for calculating process indicators can be obtained within two weeks - if the data collection is well organized. However, for some indicators marked by an asterisk in Chapter III, special surveys may be necessary. These surveys can be organized to collect data for several process and outcome indicators at the same time (see Chapter IV) and can take up to six weeks. Data needed for the denominator may be difficult to collect the first year (e.g. PR29). It is, however, useful to collect data for the numerator as they can be used in the following years as denominators, and they can also provide interesting information on the year they are collected. Some indicators can be applied separately for the public and private sectors, while others can be subdivided by level of care, etc. These decisions should be taken in relation to targets and standards decided at national level (Chapter III). Model sampling procedures, data collection forms (DCF) for field work and a model summary (Summary Form 3) are provided in Chapter IV and Annex 1. In this chapter, each indicator is described as follows: definition; use; description; sources and methods of data collection and indicator calculation; and limitations.

Legislation and regulation

Indicator PR1:

Number of drug outlets inspected, out of total number of drug outlets in the country.

Use: To assess the capacity of the inspectors to control the distribution of drugs at the retail level. Inspection is an effective tool to ensure that drugs reaching the patient are safe and effective.

Description: A drug outlet is defined as a place (public or private) where drugs are legally dispensed or sold. The indicator should normally be applied in both the public and the private sectors; results can be presented separately, if needed. However, in certain countries, government inspectors are not supposed to visit drug outlets in the public sector (hospitals, pharmacies, etc.). The indicator should be calculated for one year. Similar indicators can be developed for drug manufacturing units and wholesalers. The higher the percentage of drug outlets inspected, the better the result, as drug outlets should be inspected at least once every two years.

Sources and methods of data collection and indicator calculation: The figure for the numerator derives from the number of full inspection reports for drug outlets. The figure for the denominator derives from the official list of drug outlets in the country. Both are normally available from the ministry of health and/or the national drug authority. The indicator is obtained by simple calculation:

Example: 33% of drug outlets have been inspected.

Limitations: The aggregate figures in this indicator do not provide information on the geographical coverage of the inspection. If drug outlets are inspected only in the capital, this indicator will not reveal the geographical pattern of inspections. In addition, the figures do not provide information on the quality of the reports. These can be assessed through interviews or analysis of inspection reports, and the results can be included in the final report.

Legislation and regulation

Indicator PR2:

Number of drug outlets in violation, out of total number of drug outlets inspected.

Use: To assess the effectiveness of the inspection system. The indicator also measures compliance of the distribution system with regulations. A drug distribution system which does not comply with the regulations can create major risks for the patient.

Description: A violation is defined as any specification or action not conforming to the regulations. In most countries, only major violations should be monitored. Each country should specify an appropriate definition of "major violation" and the monitoring unit should establish a list of major violations for the data collectors. As for indicator PR1, the indicator should be applied in both the public and the private sectors. The indicator should be calculated for one year. Similar indicators can be developed for drug manufacturing units and wholesalers. The rate should be close to 0%, as all drug outlets should comply with regulations.

Sources and methods of data collection and indicator calculation: Numerator and denominator derive from the inspection reports. The denominator is the same figure as the numerator of indicator PR1. Both are normally available from the ministry of health and/or the national drug authority. The indicator is obtained by simple calculation:

Example: 26% of the drug outlets inspected were in violation.

Limitations: The reliability of the indicator depends on the quality and objectivity of the reports, e.g. the indicator can be close to 0% if violations are not listed correctly in the reports. In addition, when no drug outlets are inspected, this indicator cannot be calculated. However, this result, which shows the absence of inspection, provides important information on the functioning of the regulatory system and should be taken into account when assessing or monitoring the pharmaceutical system.

Legislation and regulation

Indicator PR3:

Number of sanctions and administrative measures implemented, out of total number of violations identified.

Use: To assess the capacity of the national authorities to enforce regulations and support the inspection system. Major violations should lead to sanctions or administrative measures to be issued by the legal authority with enforcement power. Often at country level, sanctions are ordered but are not implemented, as the inspection body has no legal power to do so. If the identification of major violations is not followed by sanctions, the functioning of the inspection system is jeopardized.

Description: A sanction or an administrative measure is defined as any measure which should have been taken according to the regulations for each of the major violations identified by inspectors and reflected in the inspection reports. This indicator should be used in conjunction with indicators PR1 and PR2, as it is the logical follow-up to these. As for indicators PR1 and PR2, this indicator should be applied in both the public and private sectors. The indicator should be calculated for one year. When more than one major sanction has been observed in an establishment, only one will be considered for the calculation of the indicator. Similar indicators can be developed for drug manufacturing units and wholesalers. The rate should be close to 100%, as major violations should lead to sanctions or at least some administrative measures.

Sources and methods of data collection and indicator calculation: The numerator derives from a review of the appropriate reports to check if any sanction has been ordered and implemented for each violation identified in indicator PR2. These reports should be available at the drug inspection unit or from the enforcement authority. The denominator is the same figure as the numerator of indicator PR2. The indicator is obtained by simple calculation:

Example: A sanction has been implemented in 50% of the drug outlets where a major violation was identified.

Legislation and regulation

Indicator PR4:

Number of samples routinely collected, out of total number of planned collected samples.

Use: To assess the capacity of the inspectors to collect samples on a routine basis. The collection of samples for analysis is an important means of controlling the quality of drugs and ensures that drugs reaching the patient are safe and efficacious.

Description: In most inspection systems, there is a plan to collect samples on a regular basis. This plan does not always cover all the country or all drugs every year. It can proceed by region or by therapeutic classes. The samples to be analysed should be collected in the various manufacturing and distribution sites: manufacturing units, customs, wholesalers and pharmacies. Only samples routinely collected should be counted (e.g. the samples collected because of suspicion of fraud should not be counted). The indicator should be calculated for one year. The rate should be close to 100% if plans are respected. The indicator is completed by indicator PR5.

Sources and methods of data collection and indicator calculation: The numerator derives from the inspection reports. The denominator derives from the plan of work for the inspection body. Both are normally available in the inspection unit from the ministry of health and/or the national drug authority. The indicator is obtained by simple calculation:

Example: 50% of the samples which were planned to be collected have been collected.

Limitations: In many countries, samples are not collected routinely, and therefore it will not be feasible to calculate this indicator; depending on the national context, other indicators can be identified. In the absence of a national quality control laboratory, inspectors will probably rely more on other quality assurance methods than on testing of drugs. Again, the indicator should be replaced or adapted to the national situation. The aggregate figure does not provide information on the ways in which samples are collected, or therefore on the technical knowledge of the drug inspectors.

Legislation and regulation

Indicator PR5:

Number of samples tested, out of total number of samples collected.

Use: To assess the efficacy of the drug quality control laboratory or in the absence of a national quality control laboratory, the efficacy of any system implemented to control the quality of pharmaceutical products. The control of the quality of pharmaceutical products is a major aspect of a national drug policy; this control is part of the overall quality assurance system.

Description: In an effective quality control system, all the samples collected by the inspectors should be analysed. This indicator should be used in conjunction with indicator PR4. It seeks to ascertain the percentage of samples collected by drug inspectors (PR4) that have been analysed. It should be calculated for one year. The rate should be close to 100%, as all samples collected should be tested. If samples are not routinely collected, but only collected in special circumstances, the indicator can be adapted and used without PR4.

Sources and methods of data collection and indicator calculation: The numerator derives from data/reports available from the quality control laboratory and the ministry of health and/or the national drug authority. The denominator is the same figure as the numerator of indicator PR4. The indicator is obtained by simple calculation:

Example: 80% of the samples which have been routinely collected have been tested.

Limitations: Most of the limitations of indicator PR4 are valid for this indicator. If very few samples are collected, this indicator will not be valid as a percentage. At least 100 samples should be collected. This indicator does not provide information on the speed with which the results of the tests are obtained. If necessary in the national context, such an indicator can be added.

Legislation and regulation

Indicator PR6:

Number of advertisements in violation of regulations on the ethical promotion of drugs, out of total number of advertisements monitored.

Use: To assess the compliance of drug manufacturers or others with the regulations. Drug promotion is an important determinant of drug use; experience shows that without control, drug promotion is sometimes unethical and can lead to misuse of drugs, with negative health and economic consequences.

Description: When advertisements are not monitored, it will be impossible to calculate the indicator. This will show the drug administration's lack of control of drug promotion. Advertising is defined as any activity used by manufacturers and distributors, the effect of which is to induce the prescription, supply, purchase and/or use of drugs. It can be geared to physicians, health-related professionals and the general public. When there is a monitoring system for drug promotion at country level, there should be a list of violations. This list should be provided by the monitoring unit to the data collectors. The indicator should be calculated for one year. The rate should be close to 0%, as all advertisements should be in line with regulations.

Sources and methods of data collection and indicator calculation: The numerator and the denominator derive from the reports available from the ministry of health and/or the national drug authority and/or any institution in charge of monitoring drug promotion. The indicator is obtained by simple calculation:

Example: 45% of the advertisements which have been monitored do not comply with the regulations on the ethical promotion of drugs.

Limitations: The indicator is reliable only if the monitoring system is reliable.

Legislation and regulation

Indicator PR7:

Number of sanctions implemented for advertisements in violation of regulations, out of total number of violations identified.

Use: To assess the capacity of the national authorities to enforce regulations. Major violations should lead to sanctions or administrative measures, and withdrawal of or changes in the advertisements for and promotion of drugs. If the identification of violations is not followed by sanctions, it means that the functioning of the national drug authority is not adequate and that prescribers and the public are exposed to unethical drug promotion.

Description: Advertising is defined as any activity used by manufacturers and distributors, the effect of which is to induce the prescription, supply, purchase and/or use of drugs. It can be geared to physicians, health-related professionals and the general public. When there is a monitoring system for drug promotion at country level, there should be a list of violations. This list should be provided by the monitoring unit to the data collectors. A sanction is defined as any measure which should have been taken according to the regulations for each of the violations identified. The indicator should be calculated for one year. The rate should be close to 100%, as all violations should result in sanctions. This indicator should be used in conjunction with indicator PR6.

Sources and methods of data collection and indicator calculation: The numerator derives from a review of the appropriate reports to check if any sanction has been implemented for each violation identified in indicator PR6. The denominator is the same figure as the numerator of indicator PR6. Both reports should be available from the ministry of health and/or from the national drug authority and/or from any institution in charge of monitoring drug promotion. The indicator is obtained by simple calculation:

Example: In 60% of the drug advertisements which did not comply with the regulations on the ethical promotion of drugs, a sanction or some other measure has been implemented.

Essential drug selection and drug registration

Indicator PR8:

Value of drugs from the national essential drugs list (EDL) procured in the public sector, out of total value of drugs procured in the same sector.

Use: To assess the level of implementation of the national essential drugs list. If the national essential drugs list is seen by senior managers and policy-makers as a major tool to improve the drug situation, it should be used by the public sector. In addition, the indicator measures the effectiveness of the procurement system for the public sector, as in an effective system, only drugs on the national essential drugs list should be procured.

Description: The national list of essential drugs is the list which has been defined, adopted and published at country level. It normally covers all health facilities, including the main hospitals. This indicator should be calculated for one year. The rate should be close to 100%, as all drugs procured should be from the EDL. If the value of drugs from the EDL is not known or too complicated to calculate, the number of drugs from the EDL can be used instead of the value and the indicator adapted accordingly.

Sources and methods of data collection and indicator calculation: The numerator is obtained by adding the value of all the drugs which have been procured during the year and which are on the national essential drugs list; the denominator is the total value of drugs procured during the year for the public sector. These figures are normally available from the drug procurement unit. If there are several procurement units, e.g. if hospitals procured themselves, the figures from these various units should be added. The indicator is obtained by simple calculation:

Example: 93% in value of the drugs procured in the public sector are from the EDL.

Essential drug selection and drug registration

Indicator PR9:

Number of drugs from the national essential drugs list (EDL) prescribed, out of total number of drugs prescribed.

Use: To assess the compliance of prescribers with the national EDL. In an effective pharmaceutical system, prescribers are trained to use drugs included in the different lists of essential drugs designed for each level of health facility.

Description: The indicator can be used in the public as well as the private sector but should be used separately in the two sectors. In the private sector, prescribers are rarely obliged to prescribe only those drugs which are on the national EDL. However, the indicator is useful to assess the impact of a national EDL on prescribing practices, since an essential drugs list is developed to cover most of the therapeutic needs of the population. If an effective rational drug use policy is implemented in the public sector, the indicator should be close to 100%, as all drugs prescribed should be from the EDL. In the private sector, the closer the rate is to 100%, the better the result. Countries without a national list of essential drugs can use the WHO model list of essential drugs.

Sources and methods of data collection and indicator calculation: Numerator and denominator derive from a survey in a sample of drug outlets. A model for sampling drug outlets is provided in Chapter IV and data collection forms in Annex 1 (see Data Collection Forms 1 and 2). For this indicator, the national essential drugs list and a cross-reference index (generic and brand names) are needed when calculating the number of drugs from the EDL. The indicator is obtained by simple calculation (see example in Chapter IV):

Example: 40% of the drugs prescribed in private drug outlets are from the EDL and 75% of the drugs prescribed in public drug outlets are from the EDL.

Essential drug selection and drug registration

Indicator PR10:

Number of drugs from the national essential drugs list (EDL) sold, out of total number of drugs sold.

Use: To assess the compliance of the public at large with the national EDL. In most developing countries, drugs are often consumed without prescription. It is therefore important to have information on the percentage of essential drugs consumed by the public.

Description: The indicator covers all brand name or generic drugs sold with or without prescription; every dosage form should be included. For combination drugs, only the ones from the EDL should be counted in the numerator. If they contain one active ingredient which is on the EDL and other active ingredients which are not on the EDL, they should not be counted. The indicator applies only to private drug outlets as, in most countries, public drug outlets provide only drugs prescribed in the nearby public health facility and do not sell drugs without prescription to the public. The closer the rate is to 100%, the better the result, as ideally most of the drugs sold should be from the EDL. Countries without a national list of essential drugs can use the WHO model list of essential drugs.

Sources and methods of data collection and indicator calculation: Numerator and denominator derive from a survey in a sample of private drug outlets. A model for sampling drug outlets is provided in Chapter IV and a data collection form in Annex 1 (see Data Collection Form 1). For this indicator, the national essential drugs list and a cross-reference index (generic and brand name) are needed when calculating the number of drugs from the EDL. The indicator is obtained by simple calculation:

Example: 74% of the drugs sold in private drug outlets are from the EDL.

Limitations: If the national essential drugs list has not been revised for a long time, or if the list is only for certain levels of care, the result may be lower.

Essential drug selection and drug registration

Indicator PR11:

Number of locally manufactured drugs sold in the country from the national essential drugs list (EDL), out of number of drugs from the national essential drugs list.

Use: To assess the compliance of the local drug industry with the national EDL. This indicator provides an indication of the impact of the EDL and of the national drug policy on the drug sector, because local manufacturers should produce drugs from the national essential drugs list as a priority.

Description: Only drugs which are sold/registered in the country should be taken into account. In some cases, it can happen that drug manufacturers produce drugs only for export; these drugs should not be included in the list of locally manufactured drugs for the purpose of this indicator. The indicator can be calculated on the basis of the number of drugs as chemical entities (INN) in the EDL or of the number of dosage forms, depending on availability of data. It is important to note how the calculation was done in order to proceed identically in the future for the purpose of comparison. However, it is more accurate to take the number of dosage forms. The higher the rate, the better the results, as priority in locally manufactured drugs should be given to those from the EDL. Countries without a national list of essential drugs can use the WHO model list of essential drugs.

Sources and methods of data collection and indicator calculation: The numerator derives from a review of the drugs produced locally compared with the EDL. The numerator is normally available in the registration unit or from the local manufacturers. The national essential drugs list is needed with the generic name of the locally produced drugs. The indicator is obtained by simple calculation:

Example: 30% of the drugs on the national essential drugs list are produced locally and sold in the country.

Essential drug selection and drug registration

Indicator PR12:

Number of combination drugs newly registered, out of total number of newly registered drugs.

Use: To assess the effectiveness of the registration process. Only a few combination drugs can be considered essential, as reflected in the WHO Model List of Essential Drugs. The indicator provides information on the rationality of the criteria used for registration.

Description: A combination drug is a drug with more than one active ingredient. A newly registered drug is a drug registered for the first time. Every drug has to be registered before being authorized to be sold on the market. This indicator should be measured only for the oral and injectable forms and for a period of one year. The rate should be relatively low, in most cases less than 10%.

Sources and methods of data collection and indicator calculation: The numerator can be obtained by reviewing the registration files of the preceding year and noting the number of combination drugs registered. The denominator derives from the same files. The files are normally in the registration unit. The indicator is obtained by simple calculation:

Example: 5% of drugs registered during the last year were combination drugs.

Essential drug selection and drug registration

Indicator PR13:

Number of registered drugs which are banned in other countries, out of total number of registered drugs.

Use: To assess the effectiveness of the registration process. In an effective drug registration system, drug registration should be based on efficacy, safety and quality. Drugs which have been banned on these grounds in other countries should not be registered.

Description: A list of drugs which have been banned in a number of countries is published by the United Nations and WHO: "Consolidated list of products whose consumption and/or sale have been banned, withdrawn, severely restricted or not approved by governments". The rate should be close to 0%, as drugs registered should be safe and efficacious.

Sources and methods of data collection and indicator calculation: The numerator derives from the list published by the UN and from the national list of registered drugs. The denominator derives from the national list of registered drugs. Both are normally available in the registration unit. The indicator is obtained by simple calculation:

Example: 5% of the drugs registered are included in the UN list and are banned in other countries.

Drug allocation in the health budget/public sector financing policy

Indicator PR14:

Value of public drug budget spent per capita in the last year, out of average value of the same budget during the past three years.

Use: To assess government commitment to supporting the health services. The drug budget, which has often decreased dramatically during the last decade, should at least remain stable if equitable access to drugs is to be achieved.

Description: A minimum per capita budget should be provided from general government revenues. There is no absolute figure, but one US dollar per year/per capita has often been quoted as a minimum level to reach. The target amount should be defined at country level, taking into account socioeconomic data. Public drug budget is defined as the total amount of money spent on pharmaceutical products by the government through the ministry of health, other ministries and hospitals. Public finance is understood as general government revenues and compulsory health insurance (sometimes known as social insurance) that is either publicly managed or heavily regulated by governments (see footnote 7, page 18). The indicator should therefore include all the funds spent for drugs. In a normal situation, the rate should never be less than 100%, as the public drug budget should not decrease. When compulsory insurance exists, the indicator should be adapted to include this public financing.

Sources and methods of data collection and indicator calculation: The numerator is obtained by dividing the total drug budget spent in the various institutions during the previous year by the population. The denominator is obtained by adding budgets spent in the past three years in real terms (i.e. including the annual rate of inflation for each year) and by dividing the result by the sum of the average population for each year during the same period. It is important to divide the yearly drug budget by the average population in the same year, particularly in countries where population increases rapidly. This average population figure may be difficult to obtain; however, some estimates are normally available in the planning office of the MOH. Numerator and denominator are normally available from the ministry of health and/or the ministry of finance. The indicator is obtained by simple calculation:

Example: 80% of the public drug budget spent per capita in the past three years has been spent this year; this means that the public drug budget has decreased.

Drug allocation in the health budget/public sector financing policy

Indicator PR15:

Value of public drug budget spent by major hospitals, out of value of public budget spent.

Use: To assess the allocation of resources to primary health care and the commitment of the government to greater equity. Hospitals often consume more than 40% of the total government drug budget. With primary health care as the main objective of the health policy, there should be a more equitable allocation of resources between the various levels of the health system.

Description: Public drug budget is defined as the total amount of money spent on pharmaceutical products by the government through the ministry of health, other ministries and hospitals. Public finance is understood as general government revenues and compulsory health insurance (sometimes known as social insurance) that is either publicly managed or heavily regulated by governments (see footnote 7, page 18). In certain countries, hospitals have a budget provided by the state, but this budget is separate from the ministry of health budget. Figures should therefore be carefully compiled in order not to miss any data. Major hospitals are defined as national and/or regional hospitals, but the list should be made at national level. The drug budget is the one which is actually spent, not the one which is allocated or planned. The indicator should be calculated for one year. The rate should be decided at national level. When compulsory insurance exists, the indicator should be adapted to include this public financing.

Sources and methods of data collection and indicator calculation: The numerator is obtained by adding the drug budgets spent by the main hospitals. The denominator is the total drug budget spent during the same year in the public sector. These figures are normally available from the ministry of health, from the main hospitals, from the central procurement unit and/or from the ministry of finance. The indicator is obtained by simple calculation:

Example: 35% of the total drug budget is spent by the three major hospitals.

Drug allocation in the health budget/public sector financing policy

Indicator PR16:

Value of international aid received for drugs, out of value of public drug budget.

Use: To assess the dependence of the country on international aid for its drug supply. International aid often represents an important share of the public drug budget in many developing countries, especially the least developed ones.

Description: International aid should be defined as any major support for the provision of drugs whether in cash or in kind. Public drug budget is defined as the total amount of money spent on pharmaceutical products by the government through the ministry of health, other ministries and hospitals. Public finance is understood as general government revenues and compulsory health insurance (sometimes known as social insurance) that is either publicly managed or heavily regulated by governments (see footnote 7, page 18). If aid is already included in the denominator, it should be deducted for the purpose of the indicator. International aid through decentralized projects should also be included in the numerator. The indicator should be calculated for one year. There is no ideal rate for this indicator. In certain cases, international aid is a very useful component of the drug budget, but in the long term this percentage should decrease. The rate will depend on each country's situation. When compulsory insurance exists, the indicator should be adapted to include this public financing.

Sources and methods of data collection and indicator calculation: The numerator is the same as BG18. It can be obtained by adding all the funds given by international aid for drugs and in certain cases the amount given in kind should be calculated. It happens that drug donations are valued at a very high price (for instance, pharmaceutical companies often give drugs under brand name; the price of these drugs may be much higher than existing alternatives on the market). In this case, it is desirable to calculate the value of the donation at the average price of the same product on the international market. These data are normally available from the planning department of the ministry of health, and sometimes from the donor agencies or from the central procurement unit. The denominator is available from the ministry of health. The indicator is obtained by simple calculation:

Example: International aid represents 20% of the public drug budget.

Drug allocation in the health budget/public sector financing policy

Indicator PR17:

Value of revenue generated for drugs through additional financing systems, out of value of public drug budget.

Use: To assess the capacity of households to cover some of the drug needs in the public sector through various financing systems and therefore to complement the government's drug budget. This indicator is also useful for providing a global picture of the amount spent on drugs in the public sector and on the respective share of households and the state. In many developing countries and especially the poorest ones, the government budget is insufficient to cover the drug needs of the population. A number of financing systems based on out-of-pocket payments by the population (cost recovery mechanisms, etc.) have been implemented in the public sector to complement government budget. In certain countries such programmes cover a significant amount of the total drug expenditure in the public sector.

Description: Revenues can be defined as the total amount of money collected from the households in the various health programmes based on these financing schemes in the public sector. Only the amount of money used for the purchase of drugs should be included. The public drug budget is understood as general government revenues and compulsory health insurance (sometimes known as social insurance) that is either publicly managed or heavily regulated by governments (see footnote 7, page 18). If revenue from additional financing systems is already included in the denominator, it should be deducted for the purpose of the indicator. The indicator should be calculated for one year. Experience has shown that communities can cover only part of the health expenditures. Probably 20% of the health expenditures in the public sector is the maximum which can be recovered through community financing without the risk of increasing inequity and excluding the poorest from access to health care.

Sources and methods of data collection and indicator calculation: The numerator is obtained by adding the various revenues generated for drugs in the public sector; these data are normally available at central level, and from the various programmes/projects implementing these financing schemes. The denominator is available from the ministry of health and is the same as in indicators PR15 and PR16. The indicator is obtained by simple calculation:

Example: The revenues generated for drugs through additional financing schemes represent 50% of the drug budget.

Limitations: This indicator is useful only in countries where such financing schemes are operating and where it is feasible to determine the total revenues collected through these schemes.

Drug allocation in the health budget/public sector financing policy

Indicator PR18:

Public drug budget spent, out of public drug budget allocated.

Use: To assess the efficiency and effectiveness of the ministry of health, the central procurement unit and/or the health facilities in using the funds allocated for purchasing drugs.

Description: The drug budget allocated is normally the budget which is available to the ministry of health for buying drugs for the public sector. In many countries the allocated drug budget is often underspent because of poor planning and management in the ministry of health or lack of foreign exchange. This contributes to a shortage of drugs in health facilities, as the budget allocated for drugs is normally already less than is needed to cover basic drug requirements. The indicator should be calculated for one year. It should be as close as possible to 100%, as the drug budget allocated should all be spent.

Sources and methods of data collection and indicator calculation: The numerator is obtained from the ministry of health and/or from the procurement unit and/or the health facilities allowed to purchase directly. Data for the denominator are normally available from the ministry of health and from other ministries in charge of budget matters. The indicator is obtained by simple calculation:

Example: Only 50% of the budget allocated for drugs by the government has been spent.

Limitations: In certain cases, the reasons for not spending the funds do not lie with the ministry of health but with the procurement agency. The indicator will then provide information on the efficiency and effectiveness of the procurement agency, and not on the ministry of health, and should be part of the next section: "Public sector procurement procedures" (see indicators PR19 to PR26). The reasons for not spending funds can also include late availability of the funds, delays in obtaining hard currency, etc. In these cases, some explanations should be provided in the final reports to clarify the result.

Public sector procurement procedures

Indicator PR19:

Value of drugs purchased through competitive tender, out of value of drugs purchased.

Use: To assess the efficiency of the procurement procedures used in the public sector. The indicator also measures the management capability and the willingness of the ministry of health, other concerned ministries and the procurement unit to implement a procurement system able to procure drugs at low cost. Drugs bought through competitive tender are usually less expensive than drugs bought directly from a single supplier at his/her quoted price.

Description: Competitive tender is defined as a procedure for procuring drugs which puts a number of suppliers into competition. Purchasing is done on the basis of quotations submitted by suppliers in response to a public notice. When drugs are procured through IDA or UNICEF or with a supplier who ensures prices at the same level as the international one, the indicator should be slightly modified and the amount procured this way should be included in the numerator. The indicator should be calculated for one year. The higher the rate, the better the results.

Sources and methods of data collection and indicator calculation: The numerator derives from the total value of the drugs bought for the public sector through procedures such as limited consultation or open tender. The denominator derives from the total value of the drugs purchased by the public sector/central procurement unit and major health facilities, if they purchase directly, during the same period. Data are normally available at the central procurement unit as well as the ministry of health and other relevant ministries. The indicator is obtained by simple calculation:

Example: 75% of the drugs purchased are purchased through open tenders.

Public sector procurement procedures

Indicator PR20:

Value of drugs purchased from local manufacturers through competitive tender, out of value of drugs purchased through competitive tender.

Use: To assess the competitiveness of local producers compared with the international market. Local purchasing has several advantages provided that the quality is ensured (ease of procurement in case of shortages, no need for foreign currency, increased self-reliance).

Description: Competitive tender is defined as a procedure for procuring drugs which puts a number of suppliers into competition. The purchase is done on the basis of quotations submitted by suppliers in response to a public notice. The indicator should be calculated for one year. The rate depends on each country's situation. If the rate is high, it suggests that local producers are a good source of drugs at low cost.

Sources and methods of data collection and indicator calculation: The numerator derives from the total value of the drugs bought by the public sector from local producers during the previous year, through procedures such as limited consultation or open tender. The denominator derives from the total value of all the drugs purchased by the public sector during the same period through procedures such as limited consultation or open tender. It is the same figure as the one for the numerator of indicator PR19. Both are normally available at the central procurement unit and at local producers if health facilities are allowed to purchase directly. The indicator is obtained by simple calculation:

Example: 40% of the drugs purchased through competitive tender were purchased from local producers.

Limitations: In some countries, the purchase of certain drugs is limited to local production. In such cases, this indicator will be of no value.

Public sector procurement procedures

Indicator PR21:

CIF/ex-factory value of a basket of drugs, out of CIF/ex-factory value of the same basket in the year of reference.

Use: To assess the effectiveness of the procurement procedures. In a supply system which is improving, the price of drugs purchased should decrease as procurement procedures improve. If the system is already effective, the indicator may be used to assess price trends.

Description: A basket of drugs is provided in this manual as an example (see Chapter IV, page 59). It can be adapted to the country's context. It is preferable that it remains the same for all the indicators. However, if it is not the same for all the indicators, it should remain the same for the different years. The indicator is calculated on a basket of drugs which takes into account the level of consumption of each of the drugs of the basket. A method for calculating the value of such a basket is provided in Chapter IV (see page 60). The value should be calculated from the CIF/ex-factory prices. CIF price is the price of the imported drugs at the port of entry. Ex-factory price is the price of locally produced drugs when leaving the factory. The indicator should be calculated once a year, preferably at the same time of the year. If prices have not changed over the years, the indicator will be 100%; if prices have decreased, the indicator will be below 100%; if prices have increased, it will be over 100%.

Sources and methods of data collection and indicator calculation: The numerator and the denominator are obtained by calculating the value of the drug basket for the year under study and for the year of reference. These calculations are explained in detail in Chapter IV, page 60, and should be carefully followed. The price for each drug of the basket for calculating the value of the basket is the CIF/ex-factory price. CIF/ex-factory prices are usually available at the central procurement unit or the ministry of health. The indicator is obtained by simple calculation:

Example: The value of the drug basket at CIF/ex-factory price is 110% of the value of the same drug basket in the year of reference; this means that it has increased by 10%.

Limitations: The increase or decrease of the indicator can also be due to an increase or decrease in international and national prices of finished products and/or active ingredients or to an increase or decrease in import taxes and duties. In this case, the indicator should take this increase or decrease into account, otherwise it will provide a false picture of the procurement procedures. However, evidence suggests that the price of drugs on the international market is relatively stable and that changes in import taxes and duties are relatively rare.

Public sector procurement procedures

Indicator PR22:

CIF/ex-factory value of a basket of drugs, out of "reference" value on the international market of the same basket.

Use: To assess the effectiveness of the procurement procedures in the public sector. The indicator measures the difference between international "reference" prices and CIF or ex-factory prices obtained by the procurement unit. In a properly functioning system, the value of a basket of drugs purchased should be comparable to the value of a basket of drugs bought at reference prices on the international market.

Description: A basket of drugs is provided in this manual as an example (see Chapter IV, page 59). It can be adapted to the country's context but should remain the same for all the indicators. The indicator is calculated on a basket of drugs which takes into account the level of consumption of each of the drugs of the basket. A method for calculating the value of such a basket is provided in Chapter IV (see page 60). The value should be calculated from the CIF/ex-factory prices and the international reference prices. CIF price is the price of imported drugs at the port of entry. Ex-factory price is the price of locally produced drugs when leaving the factory. International "reference" prices can be calculated from price lists, such as those of UNICEF or IDA. Such prices do not reflect the reality of the prices in the international market but can be used as tentative benchmark prices. The indicator should be calculated once a year, preferably at the same time of the year. The rate will usually be around 100%.

Sources and methods of data collection and indicator calculation: The numerator is the same as in indicator PR21 and can be calculated the same way. The denominator is obtained by calculating the value of the same basket using international prices. The calculation is explained in detail in Chapter IV, page 60. International prices can be obtained from two or three lists from international procurement agencies (IDA, UNICEF, Echo, etc.); 10 to 15% should then be added to prices for insurance and freight. For more precise figures, the agencies can be asked for the cost of insurance and freight for the country. It is important to use the same percentage throughout the years. CIF/ex-factory prices are usually available at the central procurement unit or the ministry of health. The indicator is obtained by simple calculation:

Example: The value of a basket of drugs bought by the CMS is 120% the international "reference" value of the same basket. This means that the cost of a selection of drugs bought by the CMS is 20% higher than the cost of the same selection on the international market.

Public sector procurement procedures

Indicator PR23:

Average lead time for a sample of orders in the last year, out of average lead time during the past three years.

Use: To assess the effectiveness of the procurement procedures in the public sector. The indicator measures if the procurement unit manages all the steps in the tendering process properly. In an effective procurement system, lead time should be as low as possible.

Description: Lead time is defined as the interval in months between the beginning of the procedures for tender by the procurement unit and the arrival of the drugs in the warehouses. If procurement is done by international tender, the lead time will be a minimum of six months. For national tenders, it can be much shorter. The indicator should therefore be calculated separately for national and international tenders. Only important tenders should be included in the calculation of the average lead time. Emergency orders should be excluded. If there are improvements in the tendering process, the rate will be less than 100%.

Sources and methods of data collection and indicator calculation: The numerator is obtained by adding all the months between the beginning of the procedures for tender and the delivery of the orders and dividing the total by the number of orders. The denominator derives from previous years' figures. If the indicator is calculated for the first time, data should be sought in the procurement unit; if they are not available, an estimate should be made. The data needed for numerator and denominator are normally available from the procurement unit. The indicator is obtained by simple calculation:

Example: The average lead time in months last year was 95% of the average lead time in the three previous years.

Public sector procurement procedures

Indicator PR24:

Average time period of payment for a sample of orders, out of average time period of payment stated in contract.

Use: To assess the capacity of the procurement unit and/or the major health facilities to respect the terms agreed in the contract. Not respecting such terms will have a negative impact on the reliability of the drug procurement unit among suppliers. It can decrease the number of reliable suppliers willing to tender in the future.

Description: Every contract includes specific dates for payment. Periods for payment differ according to countries and procedures. For this indicator, all important orders should be included with similar time periods for payment stated in the contracts. The indicator can be calculated for the major health facilities if they purchase drugs directly. The indicator should be calculated for one year. If the time period of payment stated in contracts is respected, the rate will be around 100%.

Sources and methods of data collection and indicator calculation: The numerator can be calculated by adding the time period of payment for the main orders during the last year and dividing the total by the number of main orders. The denominator can be obtained the same way with the time period of payment stated in each contract. Data for the numerator and denominator are normally available from the procurement unit and/or the major health facilities. The indicator is obtained by simple calculation:

Example: The average time of payment for the main orders was 125% of the time stated in the contracts, which means that orders have on average been paid for after a delay of three months compared with what was stated in the contracts.

Limitations: The indicator will not be reliable if the banking system does not function properly or if foreign exchange is not supplied in time to the procurement unit by the Central Bank.

Public sector procurement procedures

Indicator PR25:

Number of drugs/batches tested, out of total number of drugs/batches procured.

Use: To assess the capacity of the procurement unit to control the quality of drugs purchased. When a procurement unit operates effectively, the selection of suppliers is done carefully, decreasing the risk of low-quality drugs. However, testing of drugs/batches is still necessary.

Description: When ordering drugs, the procurement unit requests a number of specifications, e.g. pharmacopoeias' norms. Some basic tests should be performed on the arrival of the drugs in order to ensure that each batch complies with the specifications. These tests can be done in the unit itself, in the national laboratory or in any other quality control facility inside or outside the country. The indicator can be calculated for each consignment of drugs or on a one-year basis. The rate should be as high as possible. The norm can be defined at country level taking into account each specific context.

Sources and methods of data collection and indicator calculation: The numerator is obtained by adding the number of drugs/batches procured during the year which have been tested. The denominator is the total number of drugs procured by the procurement unit. The data are normally available from the procurement unit. The indicator is then obtained by simple calculation:

Example: 65% of the batches have been tested.

Limitations: The indicator will be valid only if batches are tested. This information is not always available and often least developed countries are not in a position to test every batch. In certain cases, other methods which can be very effective are applied to ensure the quality of drugs. Therefore this indicator should be analysed in conjunction with others and should be accompanied by some qualitative statements. In some countries, especially the least developed ones, another indicator can be created - for example, the number of drugs procured for which a WHO certificate is attached, out of the total number of drugs procured during the year.

Public sector procurement procedures

Indicator PR26:

Number of drugs/batches that failed quality control testing, out of number of drugs/batches tested.

Use: To assess the quality of the procurement process and the drugs purchased in the public sector. This indicator should be used in conjunction with indicator PR25.

Description: This indicator refers only to the drugs procured in the public sector. The outcome indicator OT5 provides information on the quality of the drugs in the two sectors. A definition of a substandard drug is needed at country level to effectively calculate the number of drugs which failed quality control. The indicator is based on randomly collected samples; if quality control is done only on drugs under suspicion, it should be clearly indicated in the final reports, as the percentage obtained will certainly be higher. The indicator should be calculated for one year. If procurement is done in a proper manner - good selection of suppliers, extensive specifications, etc. - the indicator should be close to 0%. The indicator should be read in conjunction with the previous one - PR25.

Sources and methods of data collection and indicator calculation: The numerator is obtained by adding the number of batches which failed quality control. The denominator is the same figure as for the numerator of indicator PR25. Data are normally available from the procurement unit or from the quality control laboratory. The indicator is obtained by simple calculation:

Example: 5% of the drugs tested failed quality control testing.

Limitations: The indicator is meaningful only if the quality control laboratory functions properly. In addition, if the denominator is too small compared with the total number of drugs procured, the indicator will not give a good picture of the real situation.

Public sector distribution and logistics

Indicator PR27:

Average time between order and delivery from central store to remote facilities in the last year, out of average time between order and delivery in the past three years.

Use: To assess the performance of the distribution system in the public sector. In an effective distribution system, the time between order and delivery is supposed to be approximately the same for each order and as short as possible, depending on the distance and the number of intermediate levels.

Description: Remote health facilities have been used for this indicator as it is assumed that if the average time for distribution to these facilities is short and is improving over time, the distribution system is quite effective. The indicator can also be used for monitoring the effectiveness of each level of the distribution system. In this case, the definition of the numerator and the denominator would be changed according to the level monitored. A remote health facility is defined as a facility which is situated at a distance of more than 100 km from a city of more than 100,000 inhabitants. However, the definition can be adapted to each country context. The time to be considered is the number of days/weeks/months between the moment the order is sent by the remote health facility and the moment the same facility receives the drugs. The indicator should be calculated for one year. If there are improvements in the distribution system, the rate will be less than 100%.

Sources and methods of data collection and indicator calculation: The numerator is calculated by adding the number of days/weeks/months between order and delivery for the main orders during the year and dividing the result by the number of orders. The denominator is obtained from aggregated figures collected during the surveys done in previous years. If the indicator is calculated for the first time, an estimated guess should be made for the previous years. Data for the numerator and the denominator should be collected through a survey in remote health facilities. A model for sampling health facilities is provided in Chapter IV and a data collection form in Annex 1 (see Data Collection Form 4). The indicator is obtained by simple calculation:

Example: The average time between order and delivery in remote health facilities was 95% of the average time in the past three years.

Limitations: In certain cases, it should be noted that different levels can be responsible for delaying the distribution of drugs. Long distribution time can be due to causes other than failures in the distribution system, such as delays in procurement, financing problems, etc.

Public sector distribution and logistics

Indicator PR28:

Average stockout duration for a basket of drugs in the central and/or regional stores in the last year, out of average stockout duration for the same basket in the past three years.

Use: To assess the efficacy of the stock management of drugs at central and/or regional warehouses. If drugs are managed properly, they should be in stock most of the time.

Description: Stockout duration is calculated in days/weeks/months for each of the drugs in the basket. A basket of drugs is provided as an example in this manual (see Chapter IV, page 59). The indicator should be calculated for one year. If stock management has improved the rate will be lower than 100%.

Sources and methods of data collection and indicator calculation: The numerator and the denominator are calculated by adding the stockouts in days/weeks/months for each drug in a basket and dividing the result by the number of drugs in the basket. The denominator is obtained from aggregated figures collected the previous years. The data for the numerator and the denominator are normally available from the central store. The indicator is obtained by simple calculation:

Example: Average stockout duration is 110% of average stockout duration in the past three years.

Limitations: Stockouts can be due to different causes: procurement or financing problems, delays in distribution, etc. Therefore, the indicator is not very specific and does not always measure only the performance of stock management.

Public sector distribution and logistics

Indicator PR29:

Average stockout duration for a basket of drugs in a sample of remote facilities in the last year, out of average stockout duration for the same basket in the past three years.

Use: To assess the performance of the distribution system and of the management of drugs at local level. If the logistics system is operational and if drugs are managed properly, they should be in stock most of the time.

Description: Remote health facilities have been used for this indicator as it is assumed that, if there is no stockout at this level, the distribution and management system is operational all over the country. A remote health facility is defined as a facility which is situated at a distance of more than 100 km from a city of 100,000 inhabitants. However, the definition can be adapted to each country. The stockout duration is the number of days/weeks/months when drugs of the basket are out of stock. A basket of drugs is provided as an example in the manual (see Chapter IV, page 59). The indicator should be calculated for one year. If the rate is lower than 100%, it suggests that the distribution system and the management of drugs at local level are improving.

Sources and methods of data collection and indicator calculation: The numerator is obtained by adding days of stockout for each drug in the basket during the year in all the facilities of the sample and dividing the total by the number in drugs in the basket multiplied by the number of health facilities in the sample. The denominator is obtained from aggregated figures collected during the surveys done the previous years. If the indicator is calculated for the first time, an estimated guess should be made for the past three years. Data for the numerator should be collected through a survey in remote health facilities. A model for sampling remote health facilities is provided in Chapter IV and a data collection form is provided in Annex 1 (see Data Collection Form 4). The indicator is obtained by simple calculation:

Example: The average stockout duration was 95% of the average stockout duration in the past three years, which means that the situation has improved slightly.

Limitations: Different factors can be responsible for stockouts, e.g. bad management at the central medical stores, or, at peripheral level, weakness in logistics and delays in delivery, etc.

Pricing policy

Indicator PR30:

Value of a basket of drugs, out of CIF/ex-factory value of the same basket.

Use: To assess the compliance of wholesalers and pharmacists with official margins, when they are fixed through regulation, or with an average level of margins, when they are not fixed. The indicator could also be used for monitoring the degree of success of any intervention on margins carried out at wholesaler and retailer level, in order to improve drug affordability through a decrease in drug prices.

Description: A basket of drugs is provided as an example in Chapter IV. It can be adapted to a country's context but should remain the same for all the indicators. The indicator is calculated in a basket of drugs which takes into account the level of consumption of each of the drugs in the basket. A method for calculating the value of such a basket is provided in Chapter IV. Value should be calculated from the retail and the CIF/ex-factory prices. The CIF price is the price of the drugs at the port of entry; ex-factory price is the price of locally produced drugs when leaving the factory. The indicator is intended for use in the private sector, but it can also be used in the public sector. In certain countries, drugs are free to the patient in the public sector, but the health facilities pay the procurement unit for them; in such cases, the indicator can be adapted. The indicator should be calculated once a year, preferably at the same time of the year. The rate is normally less than 140%, and can be less for certain drugs or in the public sector.

Sources and methods of data collection and indicator calculation: The numerator is obtained by calculating the value of the drug basket at retail prices. The denominator is obtained by calculating the value of the drug basket at CIF/ex-factory prices. These calculations are explained in detail in Chapter IV. The two values should be calculated for a given year, even if the drugs from the basket have been bought in previous years. Data for the denominator are available from the ministry of health, customs authority, wholesalers, etc. Data for the numerator should be collected through a survey in a sample of drug outlets. A model for sampling drug outlets is provided in Chapter IV and a data collection form in Annex 1 (see Data Collection Form 1). The indicator is obtained by simple calculation:

Example: The value of the basket of drugs at the pharmacy level is 140% of the CIF/ex-factory value. Therefore, the total margin for the wholesaler and the pharmacist is 40%.

Limitations: If local duties and inland transportation costs increase, e.g. if the price of fuel increases, this will in some circumstances lead to a deterioration of the indicator, which does not automatically mean an increase in the margins of wholesalers and pharmacists. If used in the public sector, when the cost of drugs covers other costs (e.g. the Bamako Initiative), this should be taken into account when analysing the results.

Pricing policy

Indicator PR31:

Average expenditure per prescription, out of average expenditure per prescription in the past three years.

Use: To assess trends in prices and the evolution of relative drug affordability. Even when essential drugs under generic name are relatively affordable, prescriptions can be very expensive. They often contain many products, some less essential than others and at very high prices. The indicator will also measure the impact of interventions, such as training of prescribers, as the average price of a prescription should decrease if other factors remain the same.

Description: This indicator can be measured in the public sector and the private sector. In this case, results can be presented separately. In certain countries, drugs are free to the patient in the public sector, but the health facilities pay the procurement unit for them; in such cases the indicator should be adapted. Ideally it should not increase more than the cost of living. However, effective intervention on procurement procedures, prescribing practices and margins could even lead to a decrease, i.e. the indicator would be <100%.

Sources and methods of data collection and indicator calculation: The numerator is obtained by adding the cost of each prescription and dividing the result by the number of prescriptions. The denominator is obtained from figures collected the previous years. If the indicator is calculated for the first time, an estimate should be made for the past three years. The drug price of the previous years has to be calculated in real terms. Data for the numerator should be collected from a survey in a sample of public or private drug outlets. A model for sampling drug outlets is provided in Chapter IV and data collection forms are provided in Annex 1 (see Data Collection Forms 1 and 2). The indicator is obtained by simple calculation:

Example: The average expenditure per prescription was 120% of the average expenditure per prescription during the last three years; this means that people have to pay more for their prescriptions than in the past.

Limitations: In the public sector, the indicator is only relevant when patients pay for drugs. The indicator can be influenced by two parameters: an increase or a decrease in the cost of the drugs, or a change in the prescribing practices of the health personnel. More specific information on these factors will be needed for targeted action.

Pricing policy

Indicator PR32:

Value of a basket of drugs, out of value of the same basket in the year of reference.

Use: To assess trends in prices and the evolution of relative drug affordability on the basis of a limited number of drugs.

Description: A basket of drugs is provided as an example in Chapter IV (see page 59). It can be adapted to a country's context but should remain the same for all the indicators. The indicator is calculated on a basket of drugs which takes into account the level of consumption of each of the drugs in the basket. A method for calculating the value of such a basket is provided in Chapter IV (see page 60). The value should be calculated from the retail prices. Retail price is the price of the drug to the consumer. The indicator can be used in the public and in the private sectors. In certain countries, drugs are free to the patient in the public sector, but the health facilities pay the procurement unit for them; in such cases the indicator should be adapted. The indicator should be calculated once a year, preferably at the same time of the year. The indicator should not increase more than the cost of living. However, effective intervention on procurement procedures and pricing policies could lead to a decrease, i.e. the indicator would be <100%. This indicator can be considered as an acceptable drug price index throughout the years (see Chapter IV, page 61).

Sources and methods of data collection and indicator calculation: The numerator is the same as in indicator PR30. Numerator and denominator are obtained by calculating the value of the drug basket at retail prices in the year under study and in the year of reference. These calculations are explained in detail in Chapter IV, page 60, and should be carefully followed. Data for the numerator should be collected through a survey in a sample of drug outlets. A model for sampling drug outlets is provided in Chapter IV and data collection forms are provided in Annex 1 (see Data Collection Forms 1 and 2). The indicator is obtained by simple calculation:

Example: The value of the drug basket is 130% of the value of the same basket in the year of reference; therefore it has increased by 30%.

Limitations: A number of factors can influence the results, e.g. changes in CIF prices, exchange rates, post-procurement costs (taxes, transport, etc.), margins and procurement methods. However, the indicator provides a signal and can be a measure of affordability when compared with the cost of living. To be useful for action it should be supplemented by more specific information.

Information and continuing education on drug use

Indicator PR33:

Number of prescribers having direct access to a (national) drug formulary, out of total number of prescribers surveyed.

Use: To assess the availability of objective information on drugs at prescriber level. It is assumed that a prescriber who has a drug formulary on his or her desk is more likely to use it and therefore will prescribe more rationally. The indicator can only be measured if a formulary exists (see indicator ST42).

Description: Every prescriber should have a national or regional formulary or any manual containing as a minimum the following impartial information for each drug: active ingredient, dosage, indications, contraindications, side-effects, precautions. The exact definition of what can be considered as a manual of objective information should be established at country level. Prescribers include every health worker who prescribes: doctors, nurses, midwives, etc. This indicator can also be used for the private sector. The rate should be as close as possible to 100%, as every prescriber should have direct access to such information. It will often be lower in the private sector, as in many countries these kinds of manuals are primarily produced for health workers in the public sector. The indicator can also be used for dispensers, since they will be more able to provide accurate information to the customers if they have a formulary at their disposal. It should then be calculated separately.

Sources and methods of data collection and indicator calculation: The numerator and denominator derive from a survey in a sample of health facilities. If the indicator is used for the private sector, a survey in a sample of private prescribers will be necessary; the survey should be done according to procedures described in Chapter IV. A model for sampling health facilities is provided in Chapter IV and a data collection form in Annex 1 (see Data Collection Form 3). The indicator is obtained by simple calculation:

Example: 65% of prescribers have direct access to a drug formulary in the public sector.

Information and continuing education on drug use

Indicator PR34:

Number of training sessions on drug use for prescribers in the last year, out of average number of training sessions organized in the past three years.

Use: To assess commitment of government (or any other institution in charge of providing objective information on drugs) to rational use of drugs. Countries' experiences suggest that training and continuing education are interventions which improve prescribing practices.

Description: Training session is defined as any meeting, workshop, seminar, etc. in which problems related to the rational use of drugs are discussed. Only those organized by the public sector (i.e. ministry of health) and/or non-profit institutions and/or professional organizations (i.e. medical associations) are considered as training sessions. In the specific context of developing countries, any meeting organized/supported by the pharmaceutical industry cannot be considered as a training session. Prescribers include every health worker who prescribes: doctors, nurses, midwives, etc. This indicator can be used for the public sector as well as the private sector. The indicator can also be used for dispensers. It should then be calculated separately. If the rate is more than 100%, it suggests an increased commitment to improve prescribing practices. In countries where a plan for training prescribers exists, the indicator can be modified and the number of sessions which took place can be compared with the number of planned sessions.

Sources and methods of data collection and indicator calculation: The numerator derives from data available from the various bodies in charge of the training sessions. The denominator derives from data collected for the past three years as an average. If there is a plan for training and continuing education, the denominator can derive from the targets decided for the year. Numerator and denominator should then be calculated for the same year. The indicator is obtained by simple calculation:

Example: The number of training sessions on drug use for prescribers was 120% of the average number per year during the past three years; therefore it increased by 20% last year.

Limitations: The indicator does not assess the quality of the training.

Information and continuing education on drug use

Indicator PR35:

Number of prescribers who have attended at least one training session in the last year, out of total number of prescribers surveyed.

Use: To assess the number of prescribers who have been exposed to training sessions and in certain cases, when training is not mandatory, the commitment of the prescribers and others to improving their own practice.

Description: Training session is defined as any meeting, workshop, seminar, etc. in which problems related to the rational use of drugs are discussed. Only those organized by the public sector (i.e. ministry of health) and/or non-profit institutions and/or professional organizations (i.e. medical associations) are considered as training sessions. In the specific context of developing countries, any meeting organized/supported by the pharmaceutical industry cannot be considered as a training session. Prescribers include every health worker who prescribes: doctors, nurses, midwives, etc. The indicator applies to the public sector, but it can easily be adapted to the prescribers of the private sector. The indicator can also be used for dispensers. It should then be calculated separately. A rate close to 100% suggests a high commitment by the medical community to improving prescribing practices. In certain countries where there is a scheme for training personnel, the indicator can be modified as follows: the number who attended can be compared with the number planned. In this case, there is no need for a survey.

Sources and methods of data collection and indicator calculation: The numerator and the denominator derive from a survey in a sample of health facilities. If the indicator is used for the private sector, a survey of a sample of private prescribers will be necessary; the survey should be done according to procedures described in Chapter IV. A model for sampling health facilities and prescribers is provided in Chapter IV and a data collection form in Annex 1 (see Data Collection Form 3). The indicator is obtained by simple calculation:

Example: 40% of prescribers have attended at least one session in the last year.

Limitations: The indicator can only be used if a significant number of training sessions are organized each year throughout the country.

Information and continuing education on drug use

Indicator PR36:

Number of issues of independent drug bulletins published in the last year, out of average number of issues of independent drug bulletins published per year in the past three years.

Use: To assess the quantity of objective drug information available to prescribers and dispensers. The absence of objective information is one of the main reasons for irrational prescribing practices. Some country experiences show that an increase in the availability of objective information improves prescribing practices. Therefore an increase in the number of issues of drug bulletins should lead to more rational prescribing.

Description: An independent drug bulletin is any bulletin which is recognized by the national scientific community as not being influenced by the pharmaceutical industry, and as providing unbiased drug information. If the availability of independent information has increased, the rate will be more than 100%. If several independent drug bulletins exist, it may be better to present the results separately, particularly if the bulletins do not reach the same target audience.

Sources and methods of data collection and indicator calculation: The data needed for the numerator are normally available from the ministry of health, from universities and/or from other bodies in charge of drug information. The denominator derives from data collected during the past three years. The indicator is obtained by simple calculation:

Example: The number of issues of independent drug bulletins published last year was 90% of the average number per year during the past three years; therefore it has decreased by 10%.

Limitations: This indicator does not provide information on the number of issues published and sent to prescribers; it should be used in conjunction with indicator PR37. Indeed, in some countries a drug bulletin is published regularly but is disseminated only to a few prescribers (financing, distribution problems, etc.).

Information and continuing education on drug use

Indicator PR37:

Average number of copies of independent drug bulletins sent to prescribers, out of total number of prescribers.

Use: To assess the availability of objective information to prescribers. One of the main constraints on rational prescribing is the lack of objective information on drugs. Receiving this information regularly should improve prescribing practices.

Description: An independent drug bulletin is any bulletin which is recognized by the national scientific community as not being influenced by the pharmaceutical industry and as providing unbiased drug information. If the number of prescribers who receive existing drug bulletins is very low, the impact on prescribing practices cannot be expected to be high. Ideally every prescriber should receive a copy of each issue of existing drug bulletins. The indicator should be calculated for one year. It can also be used for dispensers. The higher the percentage, the better the results. It is 100% if all prescribers receive a copy of each issue published. If several independent drug bulletins exist, it may be better to present the results separately, particularly if the bulletins do not reach the same target audience.

Sources and methods of data collection and indicator calculation: The numerator is obtained by adding the number of copies of each issue sent during the year and dividing the total by the number of issues. These data are normally available from the ministry of health, from universities and/or from other bodies in charge of drug information. The figure for the denominator is normally available from the ministry of health. The indicator is obtained by simple calculation:

Example: On average, the bulletins have been sent to 30% of the prescribers.

Information and continuing education on drug use

Indicator PR38:

Amount spent on public education campaigns on drug use, out of total amount spent on public health education campaigns.

Use: To assess the commitment of the government to promote rational drug use by the public. Irrational drug use is widespread and public education is essential for people to understand how to use medicines wisely.

Description: A public education campaign on drugs is defined as any programme or campaign conducted at local or national level by the ministry of health, by other ministries or by other bodies, aimed at increased awareness of drug issues and improvement in the use of drugs by the public. General public health education campaigns are campaigns oriented to general health issues; public education campaigns on drugs can be part of broader health education campaigns. The indicator should be calculated for one year. The target amount for public education campaigns on drugs should be defined at country level taking into account country priorities and level of implementation of the national drug policy.

Sources and methods of data collection and indicator calculation: The numerator is obtained by adding the amount of resources spent by the various programmes/projects/ministries on public education campaigns on drug use during the year. The denominator is obtained by the same calculation. These data are not always easy to obtain and should be sought in various places: the ministry of health, other ministries, projects, etc. The indicator is obtained by simple calculation:

Example: The resources spent on public education campaigns on drug use represent 20% of the resources spent on health education campaigns.

Limitations: In many developing countries, there are no education campaigns on drugs. In such cases, indicator ST49 will be negative and it will not be feasible to measure this indicator.

to previous section
to next section
 
 
The WHO Essential Medicines and Health Products Information Portal was designed and is maintained by Human Info NGO. Last updated: March 20, 2014