42: The answer to every bad evaluation question

In the novel Hitchhiker’s Guide to the Galaxy, a group of hyper-intelligent pan-dimensional beings build a supercomputer to ask the “ultimate question…the answer to life, the universe, and everything.” After waiting millions of years, the supercomputer tells them the answer to life, the universe, and everything is…42! Some might disagree, but the lesson here is simple—if you want a useful finding, you have to ask the right evaluation question! And you don’t have to be hyper-intelligent, pan-dimensional beings to learn how to do it.

Evaluation questions are developed to guide your evaluation. They allow you to focus your study, clarify your program/service outcomes, and help check or authenticate your work. Your mission is to answer them by collecting and analyzing data. Your findings should give you important insights about your program or service. Depending on the size of your evaluation, you may have anywhere from one to five main questions. To get started, follow these four steps:

Step 1: Clarify goals and objectives of program

You can’t develop an evaluation question if you aren’t clear about the intended outcomes of your program. Otherwise, you might research a question that ends up being entirely irrelevant. Also take the time to review your logic model—you want to ensure that each question ties to one of its components. For instance, if an activity on your logic model is STEM instruction, you might ask “to what extent did staff have adequate training and support to implement proper STEM instruction to children ages 6-12.” Luckily, we’ve recently covered both of these topics more in-depth. Learn more about outcomes here or logic models here. 

Step 2: Identify key stakeholders and audiences

It’s helpful to make a list of your evaluation’s stakeholders and audiences, including taking note of their “stakes.” From this stakeholder list, identify who your evaluation serves. Is it to provide data to your library board? Do you intend to use the information to improve a program for library users? Think about whether your evaluation questions will give you answers that serve these groups of people. Additionally, consider whether your key stakeholders or audiences should have an opportunity to provide feedback on your evaluation questions.

Step 3: Write a list of evaluation questions

Now it’s time to put pen to paper and write some questions. Write as many as you can think of and then we’ll eliminate some in the next step. Here are some examples that frame the question around the objective of your evaluation:

Objective: To review the summer reading program.
Question: In what ways are participating children demonstrating interest in reading at home? 

Objective: To provide information on non-library users in the community.
Question: For what reasons do residents within our library service area not use the library?

Objective: To examine library services directed at library users being affected by housing insecurity.
Question: To what extent are library programs and services directed at housing insecure patrons meeting their direct needs?
Sub-question: You can also have a sub-question like, “What need-gaps still exist that library services could provide?”   

Step 4: Evaluate your evaluation questions

Ok, I know this might be starting to feel like the movie Inception, but bear with me. Now we need to evaluate each evaluation question based on these criteria:

Relevant: Does the question clearly apply to an aspect of the program (i.e. design, activities, outcomes)? Does it contribute valuable information to stakeholders?

Answerable: Is it possible to answer this question via empirical research methods? Can you obtain the necessary information ethically and respectfully? 

Reasonable: Can the question be addressed given the resources and constraints (time, budget, staff, etc.) of your evaluation? Is it worth the effort?

Specific: Does the question distinctly target a program component? Are there any ambiguous phrases or undefined target groups? 

Evaluative: Will data related to the evaluation question provide either formative information about the program or service for decision-making and improvement purposes, or summative information to determine the effectiveness? Is your question phrased objectively so that you are not making assumptions about your program or service prior to evaluating it? 

Complete: Will the evaluation question give ample information for stakeholders to move forward?

If questions on your list don’t meet all of these criteria, consider revising or eliminating them. It’s possible you still have too many to be able to accomplish them all within your constraints. If so, go through each one and score them based on the criteria (1 = not very relevant, 5 = very relevant, etc.). Prioritize the questions that score the highest.

 Still have questions about evaluation questions? Feel free to reach out to us at LRS@LRS.org. We’re always happy to shop talk and help you reach your library evaluation goals!   

 

 

 

The Logic Model: Take it one step at a time

When your organization designs a program, service or experience, it’s helpful to think intentionally. What do you hope happens? How would you know if it did? We wrote about determining the outcomes for your efforts last time. Identifying outcomes is an important first step in planning and evaluating a program, service or experience. What do you need to do after you’ve identified outcomes? It’s helpful to have a model to guide you through your questions, what you hope will happen, how to best collect data, and how it all connects.

There are different types of guides for this process in the evaluation world. The logic model is the one most frequently used in nonprofits and libraries, so we’ll be focusing on it. The key to this process, no matter the model, is to think carefully about the outcomes you have specified, how those outcomes will be achieved, and how success will be measured.

The logic model outlines each component of a program, service, or experience. We’ll discuss each component of the logic model using storytime programming as an example, which is shown below. Keep in mind that terminology and some of the components vary in different versions of the logic model, so what we’re sharing here is not the definitive, one and only way to create a logic model. It’s one example.

Inputs

Inputs are the resources that go into making programs, services, and experiences possible. Almost anything we do in libraries requires staff time, funding or supplies. Staff training or background research could also be inputs.

Activities

Activities include the events, services, or experiences that you hope will achieve the outcome. One of the most important steps of this process is making sure the activities could realistically lead to the outcome. For example, in our logic model our outcome is “Caregivers and children learn early literacy skills.” What activities would make it possible for this outcome to happen? The storytime would need to include instruction on early literacy skills for children and parents to be able to learn them. Logical, right?

Outputs

Outputs are the concrete results of the activities. They are usually things we can count, like the number of attendees at a storytime or circulation statistics. 

Outcomes

Outcomes are how the participants are affected by their participation. Does something change for them? Do they know, believe or can they do something differently from before they participated? Many logic models distinguish between short-term, medium-term and long-term outcomes (also called impacts). In our example, a short-term outcome is the one shown in the diagram: caregivers and children learn early literacy skills. A medium-term outcome would be that caregivers and children enjoy reading together more. A long-term outcome or impact would be that children’s literacy skills improve. The outcomes build on each other over time.

Assumptions & External Factors

The programs, experiences, and services libraries provide exist within the complicated context that is our world. Assumptions and external factors are a place to capture some of that context. Assumptions are just that—the underlying ideas and values that come with us wherever we go. How do we think things work? We often share assumptions as a profession and questioning them can be uncomfortable. It is still important to explicitly discuss our assumptions because the project could go very differently than we planned due to a faulty assumption. External factors are those elements of the world that may play a big role in how the program, experience, or service works in real life. You can think of this as the environment where the project lives. In our case, right now the pandemic has an impact on all our projects. 

Conclusion

I hope this post gives you a useful bird’s eye view of the planning and evaluation process. Using a guide like the logic model can help you identify each component of the process and how it leads to the next step. Looking at everything sequentially helps you ensure that each piece works together to achieve your outcomes. 

Further reading

I used several sources to inform this post. First, I’d like to credit them for their thoughtful and easy to follow explanations of the logic model. I’d also like to refer you to them if you want more information:

 

What’s your goal here?

Every day we assess the world around us. We ask ourselves whether that decision we made was a good idea, what makes that person trustworthy, why we should or should not change something. We form a question in our head, collect data, analyze the information, and come to a conclusion. In short, we are all experienced evaluators!  

However, that doesn’t mean setting up an outcome-based evaluation is a cake walk. It’s important to apply structure to the subconscious process occurring in our head. So where should you start? At the end. That might sound counterintuitive, but the first step in an outcome-based evaluation is figuring out how you define success for your program or service—what do you hope to achieve? 

Think of a program or service you want to evaluate. It could be something already being offered in your library, or something new. What do you want your users to know/do/understand/believe after participating in the program or experiencing the service? Remember that outcomes are goals framed around your users. It’s the impact you hope your service or program has on the people participating—the big “why” of your work.

I’m going to ask you to take a few minutes to think through some potential outcomes, but before I do, we need to talk briefly about outputs. Outputs are the tangible and intangible products that result from program/service activities. If we were talking about summer reading, an example of an output is the number of children who complete the program. So, we may aim to increase the number of completions this year by 20 percent. That’s a great goal right? Yes, but be careful not to confuse it for an outcome. Increasing completions, even though it’s addressing users, does not capture the impact we hope summer reading has on children who participate in the program. 

So what would be a good outcome for a summer reading program? Here are some potential ideas:

  • Children choose to engage in a reading activity every day.
  • Children believe that reading is an important part of their daily routine
  • Children return to school without exhibiting effects of “summer slide”

In each example, the “who” is the user (children) and the “what” is the impact we hope the experience has on them. 

Now it’s your turn! Take a few minutes to write down some potential outcomes for the program or service you’re thinking about. As you’re doing it, remember to ask yourself:

  1. Is it achievable? It’s great to have aspirational goals, but we want to choose something that can be achieved by the program, service, or experience you are offering. We all want to alleviate poverty, but a much more achievable goal might be to create economic opportunity or increase wage-earning potential for a certain target group.  
  2. Is it framed around the user? Think about who you want to have an effect on. Be as specific as possible. 
  3. Does it capture impact? Make sure to be clear in your outcome about what you want your user to know, do, believe, or understand by the end of your program or service.

Congrats! You’re on your way to being an expert evaluator. Having clear and defined outcomes is the first step to designing your evaluation plan. In our next post, you’ll use these outcomes to develop a logic model. Until then, if you have any questions, feel free to reach out to us at LRS@LRS.org

Finding your way: the difference between research and evaluation

Sign posts on the top of an alpine peak

Have you ever stayed up late, staring up at the night sky, wondering “What is the difference between evaluation and research?” No?! Well, even if you haven’t lost sleep pondering this, we think it’s an important topic. Why? In this blog series, we’ll be focused on how to do an evaluation: how to determine the value and impact of programs, services, and experiences. At the same time, we’ll be talking a lot about methods from social science research because those are our tools for collecting and analyzing data. 

Knowing how evaluation and research relate to each other gives you a better understanding of where you are now, where you’re going, and how to get there as you work on a project. It’s like having a map in your head with a little star that says “you are here!”

Let’s start with clarifying what we mean by research. We might say that we’re going to research some recipes for dinner, or some interesting STEM activities for kids. In that context, research means “go find more information about.” When we talk about research in this post, we mean original research: when a study is designed to answer a question by methodically collecting and analyzing data.

Often original research happens at a university, within a specific discipline like physics, psychology, or history. In general, original research

  • aims to answer a question
  • is based in a theory (a set of related ideas about how something works)
  • tests a hypothesis (an idea about what will happen this time)
  • comes to a conclusion that can be applied in a lot of situations (generalized)
  • increases our overall knowledge on a topic

Evaluation and research do have commonalities. They’re both processes of inquiry, or ways of finding out more information in order to answer a question. So what makes them different? The answer to that can depend a bit on who you ask (a recent survey of 522 researchers and evaluators found that they had several ways of thinking about how research and evaluation relate). 

For our purposes you just need to know which it is you are doing—evaluation or research? A broadly accepted way of thinking about how evaluation and research are different comes from Michael Scriven, an evaluation expert and professor. He defines evaluation this way in his Evaluation Thesaurus: “Evaluation determines the merit, worth, or value of things.” He goes on to explain that “Social science research, by contrast, does not aim for or achieve evaluative conclusions…Social science research does not establish standards or values and then integrate them with factual results to reach evaluative conclusions. In fact, the dominant social science doctrine for many decades prided itself on being value free.” This definition and more information are available at the Evaluation Exchange.

Put another way: evaluation and social science research use the same strategies to collect and analyze data, but the goals of each are different. A useful visualization of this concept, created by John LaVelle, is below.

An hourglass showing evaluation and research

Essentially evaluation aims to do exactly what it says—determine value. Did it work? Should we keep doing it or do something else instead? What was the value of what we did? Social science research, on the other hand, aims to maintain a more impartial stance—describe what is happening, as it is, and generally not judge or evaluate it as valuable or not.

As we move forward and learn more about the evaluation process, keep this idea in the back of your mind—that little “you are here!” star. We usually start an evaluation because we want to know if something is working and providing value in the way we hoped. Remembering that’s why you started and where you’re going can help you orient yourself throughout the project. We look forward to seeing you back here next time!

 

Between a Graph and a Hard Place Chapter Two: Do it yourself

Research can be a scary word that comes with a lot of fear about our own skills. We think of experts conducting field work, gathering data, and writing long, technical reports. Like reading a foreign language, it’s easy to feel ill-equipped for deciphering what it all means.

Chapter one of Between a Graph and a Hard Place gave you the lexicon for understanding existing data and research. We covered a myriad of topics from checking sources to reading data visualizations that you can find here in case you missed any. Now, we’re excited to introduce chapter two of our blog series in which you—the reader—will go out and conquer your own research and evaluation projects. You don’t need to be a researcher, just curious about how to gather insights about the work you’re doing in a library.

Every other week we’ll cover new topics that build your research “vocabulary,” starting with forming a research question or goal. On our journey we’ll talk about issues like researching vulnerable populations and other ethical considerations. Using real world examples from Library Land, chapter two will also cover the basics of collecting and analyzing data, including how to do it without ever leaving your desk—or home. We’ll talk about surveys, focus groups, and observations. You’ll learn how to code data and run simple analyses. It’s a lot to tackle, but it’s easier than you think!

No matter what position you have in a library and no matter what kind of library it is, having the skills to collect and interpret data and evaluate the work you’re doing is critical. Like I said earlier, you don’t have to be a researcher to conduct research. You work in libraries, which means you probably like discovering information and communicating it to others—the foundation is already there! So join us on the next chapter of our data journey. We’ll get out from between a graph and a hard place and onto a path toward research fluency.

Your Ruby Slippers: five key data takeaways

Hi there, readers! We have so enjoyed having you on this data journey with us. The posts we’ve shared since March are an introduction to data literacy, and we’re wrapping up that theme today. Fear not! This series—Between a Graph & a Hard Place—isn’t going anywhere. We’re just starting a new theme, like the next chapter in a book. (We’re data people, but who can resist a book metaphor?)  

We hope that you’ve learned something—preferably lots of things—and will join us on the next leg of our journey. Based on surveying you, our readers, the new direction we’re taking is to share how you can actually do research and evaluation in the library. After today’s post, we’re going to post every other week. We love writing these, and they take time to write well. If you’re worried you’ll forget when we’re posting again, it’s easy to sign up here to get notified when we have a new post.

We’d like to give a good send off to this chapter and show you how all the posts tie together. As we review each post today, I want you to keep five big themes in mind. These key ideas apply to every area of data literacy and each post from the series connects to them. 

Five themes in data literacy:

  • The quality of research varies. Details matter, so take the time to think about them. 
  • Your common sense will take you far. Does what you’re reading make sense?
  • Our human brain has feelings, biases, and preferences. Stay aware of yours.
  • Researchers are also human. They have feelings, biases, and preferences too.
  • When considering what data mean, err on the cautious side. What do we know from these data? What is more of a guess?

These themes are your data literacy ruby slippers. You have them with you all the time, and if you start to feel lost or confused, they can show you the way home. You just have to remember you have them! With these big themes in mind, we’re ready to review data literacy.

How to compare apples to oranges

  • When data are compared, think carefully about what two things are being compared and if they are truly similar to each other.
  • One way to make things more comparable is to use per capita, or per person, data.
  • Comparisons can be messy. Keep your thinking cap on.

Habits of mind for working with data

  • Give yourself permission to struggle and get help.
  • Acknowledge your feelings about the topic.
  • Whether you like the data or not, that information gives you an opportunity to learn.

Do the data have an alibi?

  • The quality of the data matters.
  • Where were the data published and when were they collected?
  • Who the authors are is also important. What is their area of expertise? Why did they publish this?

What’s typical and why does it matter?

  • Means and medians are measures of what’s typical.
  • Knowing what’s typical can be very helpful for comparisons.
  • The mean (average of a data set) is impacted by extreme values, so sometimes the median (middle value in a data set) is more representative. 

Correlation doesn’t equal causation

  • Correlation is one way that two variables relate to each other.
  • A strong correlation is when we can predict with a high level of accuracy the values of one variable based on the values of the other. They co-occur. 
  • Causation is different because it’s a cause and effect relationship: we know that A leads to B. 

The right data for the job – part 1

  • Do the data collected make sense based on the research question?
  • What data were collected and how they were collected are both important.

The right data for the job – part 2

  • Definitions impact what data are collected and how they are interpreted. 
  • The data collected for research are usually a sample of a larger population.
  • To be representative, the sample needs to reflect the population in key ways.

Visualizing Data: a misleading y-axis

  • The y-axis (vertical axis) does not always begin at zero on a chart.
  • The y-axis may be shown on a larger or smaller scale (zoomed in or out).
  • Depending on how the y-axis is displayed, the data will look different—which can highlight or obscure differences between groups or changes over time.

Visualizing Data: the logarithmic scale

  • Logarithmic (or log) scales are another way to display the y-axis. 
  • On a log scale, the distances between intervals increase by a percentage: multiplying by x each time.
  • Log scales are useful because they show rates of change—the percent something increases or decreases.

Visualizing Data: color

  • Color can help you understand visual information, but it can also confuse or mislead you.
  • We have feelings about colors and their meanings, which are not always conscious.
  • Red holds a special place in our brain. It says “pay attention.” 

Visualizing Data: choosing the right chart

  • The best chart for showing change over time is a line or bar chart. 
  • The best chart for showing multiple variables is a bar chart.
  • The best chart for comparing something to the total is a pie chart.

Here we are, at the end of this chapter! We are delighted to have come this far. Knowing that these blog posts have been useful for you all makes us so happy. Please join us on July 29th to continue the journey. We look forward to seeing you then!

Visualizing Data: choosing the right chart

If you walk into a hardware store, you might see an entire aisle of screws—short ones, long ones, phillips head, flat head, ones with weird little anchors on the ends. They might all be screws, but they each serve a specific purpose—for wood or cement, for different screwdrivers, for thick or thin materials. It’s the same with data visualizations. They might all be charts, but pie charts, bar charts, and line charts all serve a different purpose. When data visualizers use the wrong one (often unintentionally), you’re left with a chart that doesn’t really make sense. 

Below are charts using the same data—the number of reference questions, by topic, asked each month from January through April. Let’s take a look at what information we can gather based on how those data are displayed in the visualization.

Line ChartsLine charts are commonly used to track changes over a period of time. They have a y-axis (up and down) and an x-axis (left to right) to plot two different variables. While a bar chart can also be used for this purpose, a line chart is particularly helpful when smaller changes exist or when you’re comparing changes over the same period of time for more than one group, like in the chart above. 

Here we can see that something might have happened in February to cause healthcare, business, and employment to all increase. Homework questions dropped off a bit though. Did schools give kids time off before online learning started? We know to investigate those questions because the line chart helps us identify trends. 

Pie/Donut ChartsPie/donut charts should only be used to compare parts to a whole. Each category is associated with a slice of the pie which corresponds to that category’s proportion (or percentage) of the total.  We can see that the majority of questions asked during this time period were about employment because it’s the largest slice. The least amount of questions were about genealogy. However, there’s a lot we can’t see. For instance, we have no idea how many reference questions in each category were asked in each month. We can’t see if there was a spike in healthcare questions in February when flu season hit its peak.

If you added up the values of each slice, they would equal 100 percent because each slice of the chart is determined by dividing the whole (total number of reference questions) by the part (question topic). As a reader, a huge red flag should go off if they don’t (unless the chart states it’s due to rounding). Sometimes pie charts will only have a legend that tells you what each slice represents, rather than data labels. In these cases, it’s even harder to discern how slices compare to one another because our brains are terrible at making spatial comparisons between circular areas. In general, pie charts should not contain more than five slices. When they do, it becomes difficult to read and some slices might be so small that you can’t interpret them anyways, rendering the data visualization pretty much useless. 

Bar ChartsBar charts are used to compare things between different groups or to track changes over time. They can also be used to present data that sum to more/less than 100 percent because, unlike pie charts, they aren’t limited to presenting parts to a whole. Like a line chart, they have an x-axis and y-axis, but bar charts aren’t confined to using a unit of time across the x-axis. For instance, a bar chart could use a demographic variable like age group. They can also be stacked, like in the example below. Conclusion

When looking at charts, think about whether the one the creator chose makes sense for the data story they’re trying to tell. Are they talking about changes over time, comparisons between multiple groups, or how much something makes up of the total? If the story doesn’t match the visual, be careful to draw any conclusions based on the chart. In addition, 3D renderings of any of these charts are likely to cause distortion and be visually inaccurate, even if it’s the right type of chart for the job. Here’s a nifty cheat sheet that always helps me recall when each chart should be used, and some important notes to remember: 

  • If it’s talking about something changing over time, it should be a line or bar chart 
  • If it’s talking about multiple variables, it should be a bar chart
  • If it’s talking about comparing something to the total, it should be a pie chart.

LRS’s Between a Graph and a Hard Place blog series provides strategies for looking at data with a critical eye. Every week we’ll cover a different topic. You can use these strategies with any kind of data, so while the series may be inspired by the many COVID-19 statistics being reported, the examples we’ll share will focus on other topics. To receive posts via email, please complete this form.

Let us know what you think!

When the COVID-19 pandemic began a couple of months ago, we at LRS began thinking about how we could help. What skills could we share that might be useful to library staff and our communities?  So many different sources were releasing charts and graphs to help us all understand what was happening, and we were all trying to process a lot of data every day. LRS created the Between a Graph and a Hard Place blog series to provide strategies for looking at all kinds of data with a critical eye—strategies that could be used in a library or in our everyday lives. 

We are wrapping up the first part of that series and we would love to get your feedback about what worked, what didn’t, and what you think we should do next. Don’t worry—we’re going to keep writing these posts for you! However, in lieu of publishing a post this week, we have created a survey to collect your thoughts to help guide our future posts. If you have ten minutes, we would greatly appreciate it if you’re able to fill it out. 

Thank you so much and see you next week! 

Visualizing Data: Color

I love color. As long as I can remember, I have kept my crayons organized in rainbow order. It makes me happy to see them that way! It’s a little tedious with the magical 64 pack of crayons, but totally worth it. I am an extreme example, but humans in general are visual creatures. Color impacts how we perceive and understand visual information—including graphs, charts, and infographics. 

A good data visualization combines a thoughtful display of the data with strong art and design principles, including color. Our brains are wired to pay attention to color, even if some of us perceive it differently (read more here). While color can help you understand visual information, it can also confuse or mislead you. Understanding the principles that data visualization designers use can give you insight into the role that color plays when you process visual information. 

When we make charts at LRS, we try to use several different shades of one color or one main color and a highlight color. Why just one or two colors? Believe me, if it worked, I would make all of our charts look like rainbows. The problem is that for each color you use, a viewer has to process how they personally feel about it and what that color symbolizes in our culture. Then they have to sort out what that color means in the chart. 

Our emotional reactions to color are not always conscious. If I went to the dentist and found myself sitting in a neon yellow waiting room, I would become incredibly anxious, but I may not know why. Designers spend a lot of time studying color and use it strategically, which is both good and bad for you, the viewer. The power of color can help you understand and it can emotionally manipulate you. 

What’s your favorite color? Do you know why? What about your least favorite color? Why? You carry around those preferences in your brain all the time. We’re going to look at some examples now, and I want you to keep track of how you feel about the colors.

Look at that beautiful rainbow! These pale shades of basic colors makes me think about spring and a happy version of childhood where nothing ever goes wrong—like a fanciful children’s book about talking animals. As a designer, I would use these colors to evoke viewers’ sense of nostalgia about childhood before I talk about children’s programming at the library. 

As a viewer, I’m distracted by the colors even though I like them. I really like that shade of green, so I just want to think about that column. Is the green column the most important data in this chart? I have no idea. My eye also keeps getting drawn to the red color—is that where I’m supposed to focus? While these colors are all different, they still have a similar level of saturation or brightness. What happens when that is not consistent?

This chart is really hard for me to interpret as a viewer. I think I’m supposed to focus on 2010—I can barely pull my eyes away from it. The data from 1980 is bright too. I don’t know why the data for 1980 and 2010 are shown in brighter, more saturated colors. I’m losing track of 2020, even though it has the largest value. 

Two color choices are creating a lot of confusion here. One is the use of red. Red holds a special place in our brain (read more here). It’s one of our brain’s priority colors—meaning that we are particularly skilled at perceiving it and its different shades. The cultural symbolism of red is also important. Think about the places red shows up in our world: stop signs, stop lights, warning symbols, and sports cars. Red says to us: pay attention. And sometimes also “bad” or “danger.” We can’t help but stare at the saturated red color and assume it’s important. 

The second confusing choice is the saturation of the colors. The green column is as saturated as the red column, which makes me assume it is the second most important data here. My intuition thinks more color = more attendance, but in this chart the two most saturated columns are not the ones with the highest values. Overall, color is not helping here.

Ah, ok. I’m still not sure what the takeaway message from this chart is, but at least I don’t want to run away from it. It’s easier for me to think about the data now that I’m not distracted by the colors. I can focus and develop some questions. The one thing that is missing is a visual cue about where I should focus or what is most important.

Ah, there’s my cue! This chart provides both a cohesive experience and a good indication of where the viewer is supposed to focus. I don’t need to spend a lot of energy deciphering it. I still don’t know what happened in 2010, but I feel curious and ready to find out more. The use of color augments my understanding of the data.

Out there in the wilds of the internet, there are some data visualizations where color is a barrier to understanding the data or used to elicit an emotional response. As a viewer, you don’t get to change the colors to be less distracting or add in a helpful cue about where to focus. If only we could! Instead, notice if the colors are distracting you or producing a strong emotional reaction and do your best to work around it. Often that means focusing on the data in spite of the colors. I have also printed data visualizations in grayscale to strip the color out myself. 

I could go on about color, but I want you to get back out there, using these skills! If you want to learn more about color, I recommend this episode of the podcast Radiolab.

 

Visualizing Data: the logarithmic scale

Welcome to part 2 on data visualizations. If you’re just joining us, we talked last week about how the y-axis can be altered to mislead a reader about the data. You can find that post here. Now, let’s jump right back into another big data visualization misunderstanding. 

The goal of data visualizations is to allow readers to easily understand complex data, but sometimes it’s the data visualization that we don’t understand. Certain techniques are utilized because they are the best fit for the data—not the best fit for the reader—and that can cause quite a bit of confusion if we don’t know what we’re looking at! Such is the case with logarithmic scales, which most people are unfamiliar with, but encounter all the time. Let’s break it down together.

That scale is growing out of control! 

Logarithmic (or log) scales are simply another way to display your y-axis. Unlike linear scales, where the distance between each interval increases by the same amount (adding x each time), the distance between each interval in a log scale increases by the same percentage (multiplying by x each time). Log scales are useful because they show rates of change—the percent something increases or decreases.

Imagine your library grows its print collection yearly by 100 percent. That means every year you double the number of books on your shelves. The first year you have only 192, the next year 384, then 768…1,536…Fifteen years later you’d have more than 3 million books! Good luck using a linear scale to show that kind of growth in your annual report. A better option would be to use a log scale where you can show your collection has grown annually by 100 percent. Take a look at the same data using a linear scale versus a log scale. Can you tell which one is which?

That’s right, the one on top uses a log scale (x10) and the one on the bottom uses a linear scale (+1 million). As you can see, the linear scale makes these data look like you didn’t get any books until eight years after you opened! However, if you weren’t familiar with log scales you might also think you increased your collection by the same number of books every year, instead of at the same rate

Let’s say instead of expanding your book collection by 100% annually, that growth rate begins to slow down after eight years. You still increased your collection by 27,000 books in the last year, but the log scale might make you assume you got less books than you did the first couple of years. This flattening effect is often misleading, but it simply shows a decrease in the rate, not in absolute numbers. 

Log scales have their advantages and are often used to display data that cover a wide range of values or numbers that are growing exponentially. For epidemiologists who study disease spread, log scales allow them to chart the first outbreak (often a couple of people) up to community or global spread. The volcanic explosivity index and the Richter scale, which measures earthquakes, are other common uses of a log scale.

Conclusion

Like we mentioned last week, data visualizations are all about conveying the data’s story. When you see a log scale, remember that the story is about the rate of change, not the absolute numbers. Understanding how and why certain data visualization tactics are used will help you read any data story. Next week we’ll cover some new tactics so be sure to join us! 


LRS’s Between a Graph and a Hard Place blog series provides strategies for looking at data with a critical eye. Every week we’ll cover a different topic. You can use these strategies with any kind of data, so while the series may be inspired by the many COVID-19 statistics being reported, the examples we’ll share will focus on other topics. To receive posts via email, please complete this form.