Monitoring & Evaluation
Monitoring and Evaluation go hand in hand. Not one is more important than the other. Monitoring ensures that the right thing is done, while Evaluation ensures that the right outcomes are achieved. Take for example a child who is having fever. He/she is taken to the doctor. The doctor does some oral examination and recommends some tests/diagnosis. Apart from the treatment that is rendered, the doctor tells the parents to check his temperature every 3 hours or so and prepare a chart. Finally, after few days when it appears like the fever is gone, the doctor carries out some tests again.
In the program context, the first tests are like the baseline measurements. The frequent measurement of temperature is like monitoring and the final test is like an evaluation. The cycle may or may not be repeated (with or without modifications) depending upon the need, both in the case of the child’s treatment and in development programs/projects.
M&E Systems and Methods
A strong M&E system is required for a program to be predictable and successful. M&E systems have changed much in the last two decades. Earlier, monitoring was confined to mostly quantitative recording of activities. This kind of system was helpful but in a very limited manner. While activity records would tell what has been done and how much, they would not reflect what has been delivered or what changes have taken place or how much has changed.
The earlier system registered results, more in qualitative terms through narratives and success stories with a few figures stuffed in here and there. But a good M&E system demands much more. It should not only record activities and inputs, but also reflect what has been delivered, how much of and what kind of changes have occurred and also a quantitative measure of achievement of the goal or the lack of it.
Few experienced and technically sound organizations are spending lot of resources on M&E these days, in terms of time, money, capacity building and manpower. At the same time there is a vast number of other organization who are struggling to even monitor their programs/project, let alone evaluation.
To have a strong M&E system in place, a program/project would need:
1. Technical Expertise including tools and methods
2. Financial Resources allocated to M&E
3. Manpower to support and implement it
4. Time dedicated to M&E work
And more
There are various methods employed in M&E systems. Some of the means of monitoring could be like field reports, document review, record verification, etc. And for evaluative purpose, sample based surveys could be used.
M&E Plans and Execution
M&E plan is ideally a document that details the plan for Monitoring & Evaluation. It should show the indicators that will be in use for the purpose, proper definition of the indicators (i.e. meaning of the indicators), methods by which they will be measured, means of verification, frequency of measurement and the person or body responsible for its measurement.
As important it is to have the M&E Plan, so important is it to execute it. Unless the program/project follows the M&E plan, there is neither any use of having it nor will the program/project be under control. Few things that the M&E plan helps in are:
1. Monitoring of inputs, activities and outputs
2. Evaluating outcomes and impact
3. Finding out what is working and what is not
4. Reminding where to look for and what to do when we need data
5. Indicating the simplicity or the complexity of indicators
6. Verifying data
M&E Tools
Though there are many M&E tools in use today and many of them and industry proven, it is not necessary to lay hand on all of them. A few wisely chosen tools can help us develop a strong and meaningful M&E system. A few very effective M&E tools are:
1. Field Reports
2. Registers and other records
3. Sample based surveys (with different sampling methods depending upon the purpose)
4. Focused Group Discussions (FGDs)
5. Purposefully designed community exercises
Data Collection
Data collection can be carried out in many ways, depending upon the significance of the role that the data will play in the program/project. Data can be collected from secondary sources like government reports, reports of other organizations working in the same sector, articles and research works if the data is to be used for preliminary purposes like pre-assessment studies or just to have a rough idea about a place. Such data is useful when you have the task of selecting one particular geographic division (like a district) out of many. Secondary data can help in comparing several such divisions with each other. But such secondary data may not be very useful for specific programming purposes when you have to decide the course of work and the quantum of it. It will not help much in determining the specific needs of the communities there.
Primary data is required when you need very specific information on particular sectors, sub-sectors, needs, progress or an area of your specific program interest which may not have ever been studied by others. Primary data also ensures that the program/project or its staff take direct responsibility of the information reported. Ther is no room for denying the authenticity or the accuracy of the data.
Primary data can be collected thorough various means. Some idea is given by the M&E Tools mentioned above. Here we will elaborate a little bit on the third category of M&E Tools mentioned above. Sample based surveys are effective means of data collection. The following are some important points on sample based surveys:
1. Only proven sampling methods should be used
2. Samples should be large enough to represent the whole and small enough to be managed by the program/project team in terms of time, resources, manpower, etc
3. Methods employed should be valid for the level of indicator for which measurement is being done and the population covered; some sampling methods are good for Output level measurement but invalid for Outcomes, some methods are good for small population like 20,000-50,000 but not applicable for large populations in the tune of millions
4. Survey methods need to be clearly distiguished from sampling methods
5. Survey can involve combination of several survey methods but it should not blend two or more sampling methods
6. Each person involved in a survey should have at least some understanding of the methodology being used and the survey coordinator (the one actually leading the survey) should have very clear and deep understanding
In the program context, the first tests are like the baseline measurements. The frequent measurement of temperature is like monitoring and the final test is like an evaluation. The cycle may or may not be repeated (with or without modifications) depending upon the need, both in the case of the child’s treatment and in development programs/projects.
M&E Systems and Methods
A strong M&E system is required for a program to be predictable and successful. M&E systems have changed much in the last two decades. Earlier, monitoring was confined to mostly quantitative recording of activities. This kind of system was helpful but in a very limited manner. While activity records would tell what has been done and how much, they would not reflect what has been delivered or what changes have taken place or how much has changed.
The earlier system registered results, more in qualitative terms through narratives and success stories with a few figures stuffed in here and there. But a good M&E system demands much more. It should not only record activities and inputs, but also reflect what has been delivered, how much of and what kind of changes have occurred and also a quantitative measure of achievement of the goal or the lack of it.
Few experienced and technically sound organizations are spending lot of resources on M&E these days, in terms of time, money, capacity building and manpower. At the same time there is a vast number of other organization who are struggling to even monitor their programs/project, let alone evaluation.
To have a strong M&E system in place, a program/project would need:
1. Technical Expertise including tools and methods
2. Financial Resources allocated to M&E
3. Manpower to support and implement it
4. Time dedicated to M&E work
And more
There are various methods employed in M&E systems. Some of the means of monitoring could be like field reports, document review, record verification, etc. And for evaluative purpose, sample based surveys could be used.
M&E Plans and Execution
M&E plan is ideally a document that details the plan for Monitoring & Evaluation. It should show the indicators that will be in use for the purpose, proper definition of the indicators (i.e. meaning of the indicators), methods by which they will be measured, means of verification, frequency of measurement and the person or body responsible for its measurement.
As important it is to have the M&E Plan, so important is it to execute it. Unless the program/project follows the M&E plan, there is neither any use of having it nor will the program/project be under control. Few things that the M&E plan helps in are:
1. Monitoring of inputs, activities and outputs
2. Evaluating outcomes and impact
3. Finding out what is working and what is not
4. Reminding where to look for and what to do when we need data
5. Indicating the simplicity or the complexity of indicators
6. Verifying data
M&E Tools
Though there are many M&E tools in use today and many of them and industry proven, it is not necessary to lay hand on all of them. A few wisely chosen tools can help us develop a strong and meaningful M&E system. A few very effective M&E tools are:
1. Field Reports
2. Registers and other records
3. Sample based surveys (with different sampling methods depending upon the purpose)
4. Focused Group Discussions (FGDs)
5. Purposefully designed community exercises
Data Collection
Data collection can be carried out in many ways, depending upon the significance of the role that the data will play in the program/project. Data can be collected from secondary sources like government reports, reports of other organizations working in the same sector, articles and research works if the data is to be used for preliminary purposes like pre-assessment studies or just to have a rough idea about a place. Such data is useful when you have the task of selecting one particular geographic division (like a district) out of many. Secondary data can help in comparing several such divisions with each other. But such secondary data may not be very useful for specific programming purposes when you have to decide the course of work and the quantum of it. It will not help much in determining the specific needs of the communities there.
Primary data is required when you need very specific information on particular sectors, sub-sectors, needs, progress or an area of your specific program interest which may not have ever been studied by others. Primary data also ensures that the program/project or its staff take direct responsibility of the information reported. Ther is no room for denying the authenticity or the accuracy of the data.
Primary data can be collected thorough various means. Some idea is given by the M&E Tools mentioned above. Here we will elaborate a little bit on the third category of M&E Tools mentioned above. Sample based surveys are effective means of data collection. The following are some important points on sample based surveys:
1. Only proven sampling methods should be used
2. Samples should be large enough to represent the whole and small enough to be managed by the program/project team in terms of time, resources, manpower, etc
3. Methods employed should be valid for the level of indicator for which measurement is being done and the population covered; some sampling methods are good for Output level measurement but invalid for Outcomes, some methods are good for small population like 20,000-50,000 but not applicable for large populations in the tune of millions
4. Survey methods need to be clearly distiguished from sampling methods
5. Survey can involve combination of several survey methods but it should not blend two or more sampling methods
6. Each person involved in a survey should have at least some understanding of the methodology being used and the survey coordinator (the one actually leading the survey) should have very clear and deep understanding