Some Thoughts on Evaluation

Evaluation is a vital component to nearly any product, service, or item. Evaluation is present in many ways from customer service surveys to end of semester course evaluations. Product designers, businesses, and instructors use feedback from these short evaluations to tailor their products, materials, or delivery to provide a better experience to all involved.

At a more granular level, evaluation is an important component of program and policy development. Successful programs and policies often include an evaluation component as measures of success are often found at later time points. For example, a medical school might wish to evaluate a new research program they have implemented within their institution. They might select a process evaluation in which they monitor the implementation of the program and gather feedback on if the program has been implemented as originally intended. They could also choose to have an outcome evaluation completed to better understand if their program is meeting its intended goals.

My experience in evaluation has included process, outcome, impact, and performance monitoring frameworks. These evaluations have used a variety of evaluation methods, such as focus groups, stakeholder interviews, large data analysis, social network analysis, and survey analysis. These methods are employed to better understand the data that is present and what it is trying to tell people about the program or policy. For example, an education program needs an impact evaluation to understand the magnitude of impact a program is having on the intended population. The data available may be baseline test scores and post-program implementation test scores. In addition, interview and focus group data and artifact data regarding policy guidance might be collected. These data sources and the evaluation methods used (e.g., a paired t-test for differences between pre- and post-test measures) can be used to understand that the program had a significant impact on student achievement based on a significant improvement in student test scores.

A simple, core knowledge of evaluation is important for everyone. This can be from the consumer understanding that evaluation is essential to product development and improvement to larger stakeholders looking to monitor their developed program or policy.