Mythbusters: Evaluation Edition

Whenever I tell people that I am an evaluator, I typically get one of three reactions:

  1. A blank stare (because they have no idea what I’m talking about)
  2. Excitement (because it’s someone who also loves evaluation and we have found each other)
  3. Confusion (because they have a misconception about what evaluation is)

A lot of people have a misunderstanding of what evaluation is – either because they don’t know what it truly is or they had a bad experience with someone calling themselves an evaluator or any other number of reasons.

So let’s clear up some of those misconceptions now.

5-evaluation-myths

Myth #1: Evaluation and research are the same thing.

While evaluation and research are not mutually exclusive, evaluation and research can also be drastically different.  A lot of it comes down to intention.  Evaluators and researchers often use similar methods, but the intention is different.  For researchers, the intention is primarily knowledge generation; for evaluators, the intention is primarily to improve programs and organizations.  Evaluation is also much more likely to engage stakeholders in the process, whereas research is usually focused on the hypothesis of one or a small group of researchers.  And so on. The CDC has a good guide for the distinguishing factors of evaluation and research.

Myth #2: Evaluation isn’t rigorous.

I used to work in academia. There was often the misconception there that evaluation was not rigorous and therefore wasn’t worth focusing on. While the rigor of the evaluation often depends on the budget available, evaluators use the most rigorous option possible for any given evaluation (depending on the size of the budget, the size of the organization, the data available, etc.).  Just because evaluation results are not always published in a peer-reviewed journal, does not mean they are not rigorous.

Myth #3: Evaluators only tell you what you are doing wrong

I can’t tell you how many times I have started working with a client and they are guarded because they think I am there to point out all the things that they are doing wrong. It takes a long period of rapport building for clients to get over this assumption and buy in to the evaluation process.  Evaluation focuses on strengths and areas for improvement.  Programs, policies, and organizations cannot improve and grow without identifying areas for growth, but it is also important to look at what is working well and the strengths so that the overall impact can be deepened.

Myth #4: Evaluators are auditors

Similarly, the other reaction that I will typically get from clients is that evaluators are just there to make sure that they are doing everything they are supposed to be doing and everything was tracked and collected properly. To just check off the boxes. While there are some similarities between audits and evaluations, the two are distinct. Audits check to see if activities were conducted, protocols were followed, information was tracked correctly, etc. Evaluations assess more questions, such as efficiency, effectiveness, impact, sustainability, and relevance, in order to inform decisions and program planning.

Myth #5: Evaluation should only happen at the end of a project

Many funders require an evaluation. As a result, organizations will typically find an evaluator at the end of grant funding to make sure that they are fulfilling their requirements.  Most of my experiences with this approach have been difficult. It is hard to go in at the end of the project and try to collect data about what happened. Evaluation should be happening at all phases of a project or program.  Evaluators can help in the planning phase, the implementation phase, and at the end.  Evaluations help organizations learn from what they are doing and correct course (if necessary) or incorporate lessons learned.

bonus

Bonus Myth: Social workers can’t do evaluation

When people think of evaluation, social work is not typically a field that gets included in the list of professionals that do evaluation.  However, social workers are uniquely qualified for evaluation, and especially to evaluate social service organizations. Social workers have interpersonal skills that help with stakeholder engagement and building rapport. Social workers are trained to use a systems approach to examine how different systems interact and impact individual organizations, programs, and people. Social workers have a code of ethics that they must follow.  And social workers know how other social workers in non-profits work.  So, in fact, social work is an important field in evaluation.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s