The choices involved in using the ‘Observational method’ include the number of times to observe, the number of observers and the duration of observation.
In constructing a survey, evaluators should frame questions using the languages of the prospective respondents; ensure that each survey item is a single, simple question; include several questions (three to five) to obtain information on a single variable; make the survey as brief and straight-to-the-point as possible, giving brief, clear instructions; ensure consistency in formatting and wording the survey; use complementary data entry methods to complement the survey; and test the survey with a pilot study sample to ensure that it is appropriate for use.
Selecting a sample is applied in cases where it is impracticable to evaluate an entire population, due to factors such as financial and time constraints. A sample represents the entire population.
Describing what each number means in a Likert-type scale is essential to help the participants know what their responses mean. This avoids vagueness.
Purposive sampling is important to ensure that the sample selected for the study suits the purpose of the study. To this end, an evaluator would not survey patients if they want to study the perceptions of drivers towards traffic rules. On the contrary, convenience sampling does not emphasize the purpose of the study, but selects samples based on the convenient and ease of reaching the participants within the population.
Advantages of conducting focus group interviews include: the interviewer can establish a personal connection and rapport with the participants, increasing chances of getting more quality data; access to more detailed information; ability to discover unexpected information; ability of the researcher to record personal observations relevant to the study; and the possibility of interviewing illiterate participants and including everyone.
The disadvantages include the facts that some groups may decline to participate, for example if they are busy; it involves scheduling which may be difficult; it is costlier than other methods since it utilizes a trained interviewer among other factors; possibility of getting varied response to similar questions as interviewees may interpret questions differently; and the qualitative data obtained is more challenging to analyze.
Where the aim of an evaluation is learning, effectively communicating and reporting about an evaluation to stakeholders ensures attainment of the objective. It also fosters collaboration between evaluators and stakeholders to ensure accomplishment of the goals of the evaluation.
Archival data may be obtained from management files, databases and personnel files. The advantages of such sources include the facts that the data is relevant and objective; it is easy to collect; and is often quantitative hence easy to analyze. The disadvantages include that the data may not be representative and accurate; does not allow follow-up or use of other methods; and may be difficult to access or organize the information in the preferred way.
The evaluator must select the interviewers carefully since they are central to the success of the evaluation. The interviewers must possess strong observational and communication skills, able to listen effectively. They should be neutral and unbiased. The interviewer should be able to handle unanticipated difficulties during the interviews. The evaluator should opt for someone who is experienced in related activities such as journalism and counseling. The interviewers should be able to follow specific instructions and be available for the duration of evaluation. Lastly, they should possess training in related fields such as behavioral science, research or any other relevant area.
Unstructured, semi-structured and structured interviews are the three interview approaches.
Unstructured: used with no known questions and the conversation directs the interview
Semi-structured: entails structured questions but interviewer allows for extra questions in the process.
Structured: uses orderly, pre-determined questions with minimal additional queries.
Quantitative data can be converted to numerical formats such as averages, percentages and frequencies, as it involves limited responses. Conversely, qualitative data are often open-ended and cannot be represented numerically. Instead, evaluators analyze the data and make conclusions.
Assuming a sales training program increase sales by $ 200, 000 annually, and the program costs $ 10, 000; Benefit/Cost Ratio = Program Benefits/Program Costs= $ 200, 000/$ 10, 000 = 2.
Therefore, Benefit/Cost Ration is 2:1.
The observational method is valuable as it is accurate in examining processes, relationships, as opposed to self-reporting methods. Observations are objective, for example is better suited to report the extent to which a new employee has learnt a process, than self-reporting.