Using Systems Thinking in Your Evaluations

This is the third part in a series of blog posts about the intersection between evaluation and social work.

A lot of people have been talking about systems thinking in evaluation for a while now. But there still seems to be a disconnect between understanding systems thinking principles and incorporating systems thinking into your evaluation practice.

As a social worker, I am a trained systems thinker. In social work, we are trained to think about the systems in which we work and our clients live. How do these systems impact individuals? How do these systems impact our work? How can we work within these systems to improve services for our clients? These are the questions that social workers are trained to answer.

What is Systems Thinking?

A system is defined as “an interconnected set of elements that is coherently organized in a way that achieves something,” (Meadows, 2008). So, a system is essentially a bunch of elements that work together to achieve a result. These elements can be individual programs, organizations, policies, legislation, or cultural norms or standards.habits-of-a-systems-thinker

The Waters Foundation developed a handy cheat sheet for determining if you are naturally
a systems thinker. But don’t worry, these habits are easily cultivated and trained, even if you don’t do them naturally.

Why is Systems Thinking Important in Program Evaluation?

Understanding the system that you are working in helps you better understand what facilitates or hinders the success of an organization, program, community change effort, or policy. Not only do you have to think about the individual program, but you have to think about the system in which it operates for the evaluation and the program to be effective. How much of the system is included in the evaluation is dependent on the scope of work, the budget, and the client or funder.

Some of the key questions that I ask to find a systems thinker in evaluation are:

  • Do you try to understand how external factors influence individual programs?
  • Do you think about how individual programs are connected?
  • Do you look at issues or problems from multiple perspectives?
  • Do you understand that clients participate in more than one program within a system?

The social ecological model (which comes out of the work of developmental psychologist Urie Brofenbrenner) provides one simple framework for incorporating systems thinking into your evaluation. Brofenbrenner’s work is based on child developmental psychology, but has been broadened to apply to organizational development, programming, and evaluation.

social-ecological-model

Many theorists have developed their own ecological model with different levels. You can use the social ecological model in a couple of ways from an evaluation perspective:

  1. One way is to think about the individual clients of a program. You have the individual client in the middle, their personal relationships with their family, friends, school, work, etc.; then you have the community that they live in; within the society that they live in. All of these systems impact the individual client in the program that you are evaluating.
  2. Another way you can use this model is to think of the model at the program level. You have the individual program in the middle; the program or organization’s relationships with other programs/organizations, their immediate network; within the community in which they operate; within the society.

Each of these levels, whether you are looking at it from the client perspective or the program perspective, has their own sets of norms, beliefs, traditions, and structures that all influence each other.

The Centers for Disease Control and Prevention has developed a helpful social ecological model that demonstrates how this model can be adapted within different contexts.

Key Systems Thinking Principles

These are three of the main concepts in systems thinking that you have to keep in mind whenever you are doing a systems-level evaluation, whether you are evaluating a system or evaluating a program within a system.

Interrelationships: Interrelationships refer to how things are connected. These things can be individuals, programs, organizations, policies, and so on. How do these things fit together? What is the nature of the interrelationships within a situation? What is the structure of these interrelationships? What are the processes between them? What are the patterns that emerge from those processes, with what consequences for whom? It is important to understand how programs are connected, who the stakeholders are, and how the individuals involved in the programs are connect.

Perspectives: Perspectives refers to the different ways in which a situation can be understood. Individuals often understand the same thing in different ways, depending on their own values and beliefs. How will these different understandings affect how people judge the success of an endeavor? How will they affect behavior, and thus the behavior of the system, especially when things go wrong from their perspective? With what result and significance? So programs and systems also approach situations in different ways based on their own norms, values, and beliefs. As evaluators, it is important for us to incorporate different perspectives in order to fully understand a situation.

Boundaries: Boundaries refers to what is considered part of the system and what is not. What is deemed relevant and irrelevant? What is important and what is not? Who benefits and who is disadvantaged? As evaluators, it is important for us to understand not only the boundaries of individual programs, but also the boundaries of systems. Working from a systems-level perspective, you have to understand the boundaries of individual programs and how those boundaries interact with each other and are permeated or not. For the system itself, it is important to understand what is and what is not a part of the system, or else you will end up incorporating everything into your evaluation and that is not reasonable or impactful.

Lessons Learned in Systems Thinking

  1. Define Boundaries: Defining the boundary of the evaluation is important, and oftentimes, outside of the boundary of the individual program to include some elements of the system.
  2. Zoom In, Zoom Out: Look at the big picture of the system and how that relates to the details of the program, and vice versa.
  3. Use Appropriate Language: Within the same system, there can be different language used to describe the same things. Make sure that you are using the appropriate language with the right audience.
  4. Contextual Information is Important: Systems thinking is all about understanding the contextual information of a program or organization. You can to understand the context in which the program operates.
  5. Identify Leverage Points: These are the points where programs can have the most impact on a system. They can be important for the evaluation, but also for the organization to identify focus areas.

You can read the previous posts in the series on the relationship between evaluation and social work here:

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s