Again, the reported differences between developing and developed country projects were relatively few. A greater proportion of reported developed country projects have been completed; proportionately more developing than developed country projects were based on research; and the principal themes varied slightly between the two categories of countries. Developing countries were more likely to use a mix of channels for the activities, while developed countries had a greater tendency for single-channel approaches and for innovation.
Although planning groups were nearly universal, planning itself was often quite loose. Most projects selected fairly general themes; these were most often selected on some basis of perceived need. Project by project, it may appear that those whose development and theme selection were based on research were better planned and conducted. Nevertheless no correlation was found between having carried out preliminary research and the self-assessment of having met project objectives.
Target groups were usually extremely broad, more often than not simply "general public". Similarly, expected outcomes or behaviour changes were also broad (changes in general knowledge) and numerous. The number of multiple responses suggests that these issues may not have been clearly defined before the project was underway. This is an important finding with respect to the possible outcomes of public education.
A common goal of public education and communication is to change some behaviour or set of behaviours of a particular group of people. Behaviour change, however, takes place along a continuum that begins with awareness of an issue or problem and culminates in adopting a new behaviour that will address or solve that problem. The stages between these poles are described in various ways, and include such steps as acquiring necessary knowledge, feeling responsible, acquiring skills and means, feeling capable or empowered, and being predisposed to try and then to adopt a new behaviour. The experience of communication and marketing demonstrates that interventions to raise awareness and change knowledge - early stages in the continuum - may be effective even if aimed at a large, unspecified target audience. However, having a very specific target group is a predictor of a successful intervention to change behaviours. It may be surmised that most of the responding projects in this study have too broad a target audience to be particularly effective. Alternatively, it could also be that the projects are at a relatively early stage in the continuum of behaviour change, or were intended primarily to empower communities rather than to change specific behaviour; which would be consistent with the expected outcomes described below.
Similar to the choice of a broad, general target audience this emphasis on very broad project objectives may indicate lack of definition or where most projects find themselves along the continuum of behaviour change. As discussed earlier, in the description of stages of behaviour change, positive changes in general knowledge and attitude are most likely to be prerequisites of changes in behaviour. However, as is known from the vast experience with public education on smoking cessation, seatbelt and helmet wearing, and drinking while driving, correct knowledge and attitude are only the beginning, and are not sufficient in themselves to produce behaviour change.
Activities were numerous and varied, as were the number and types of materials produced. The most common activities were the distribution of printed materials, conducting training courses, and promotion via mass-media. From the questionnaires it was not possible to judge the quality of the activities carried out, but about half of the projects reported having pretested their materials with the target audience, which may be a good indicator of quality control, if the process was methodical and rigorous.
The proliferation of materials in developing country projects may be cause for concern. Although a multi-channel approach to public education is commendable and may lead to an increased exposure to messages, there is often a tension between having a few, well-developed and used complementary materials, and a large number of relatively unconnected ones. Experience shows that many public education projects consider the production of materials as an end-point rather than a middle stage in the development and implementation of project activities. Such a focus on materials’ production for its own sake may be inadvertently promoted by funding and supporting agencies, and senior managers, for whom it may represent a concrete and tangible "proof" of activity. The final evidence, of course, is in the use made of the materials rather than the materials themselves, but an assessment of this would require visits to projects and observation of activities.
Although just over half of projects reported pre-testing materials, a substantial proportion omitted this critical step in materials development. The importance of pretesting was highlighted by the fact that almost all materials undergoing this process were subsequently revised. Lack of pretesting is a common weakness in health education programmes, probably partially at least linked to the lack of training that health professionals receive in the need and skills to communicate information in lay language.
The responses do not permit an assessment of the quality of the pretesting methodology. Nevertheless, accumulated field experience demonstrates that this methodology is often lax and rarely rigorous. This may be due to the lack of qualified staff available, especially in developing countries, or to the lack of clear standardized methodologies, or to a combination of both. The development of relatively simple methodologies may be helpful in improving this process.
It is difficult and perhaps inappropriate to make a judgement on the quality of an educational material divorced from the setting and manner in which it is used. Nevertheless, the quality of the materials sent by respondents varied widely when judged by recognized communication principles: common weaknesses included too many messages and the overuse of scare tactics; others appeared to require a good deal of training in their use.
Project evaluation usually focused on the progress of activities rather than on their impact, and was rarely carried out in a systematic manner. The weakness of evaluation, as reported by respondents, confirmed the findings of the literature review. Many projects estimated through informal assessments that they had reached or partially reached their objectives. The only correlation found between characteristics of planning and an assessment of success was between perceived need and having met objectives: those projects based on perceived need were much more likely than other projects to report having met the objectives of the project. There was no association between project success and the reported reasons for project development, nor with the various themes, general or specific.
The above correlation is interesting, but it may in reality demonstrate the situation that arises when a project neglects the crucial step of defining objective criteria, standards, or indicators for evaluation. If these criteria are not defined (and ideally measured) before project activities are implemented, any ensuing evaluation of impact risks being biased. This risk is especially high if the evaluations are conducted as informal assessments.
The lack of submitted project and evaluation reports reinforce the observation that lack of documentation makes it difficult, if not impossible, for projects to learn from each other’s experience, whether nationally, regionally or internationally. Although some projects operate in politically sensitive areas in which non-documentation of activities and approaches may be a deliberate decision; for most projects this would not be so.