PRISM Project Toolkit

• Welcome
• Project Overview
• Data Collection
• Data Analysis
• Research Findings
• More Information

Welcome!

This project was funded by the Institute of Museum and Library Services (IMLS) with one goal of the project being to develop and share the research design for other libraries to adapt and use. This toolkit includes some background information, along with information about research methods and logistics. Please help yourself to this information, including our survey and focus group questions, as it is available under CCY BY-NC-SA 4.0 (Attribution-Noncommercial-Sharealike 4.0 International).

What is PRISM?

General project timelines

History

Logistics

Ethics

People and Organizations Involved

Focus Groups and Surveys

Focus Group Tips

Selection

Participants for our focus groups were randomly selected from a roster of the current facility population divided by custody levels. The Institutional Library Development (ILD) team assisted us in obtaining the rosters and completing the random selection. The number of people selected from each facility was based on the number of people in each custody level at the facility. We invited people from every custody level in each facility and normally held a separate focus group for each custody level because they often could not be grouped together. Once participants were randomly selected, we mail merged their name, cell location, and DOC number with our invitation letter and sent these letters to the facility librarians to distribute. Some librarians chose to send the letters through the facility mail, and some librarians chose to deliver them to the cells themselves.

After a couple of focus groups, we updated our invitation letter because we heard from a focus group participant that it could easily be mistaken for a letter from the Colorado Department of Corrections (CDOC). This could have been a deterrent for some of the people randomly selected because there may be reluctance to voluntarily participate in something run by CDOC possibly due in part to fear of authority and the power structures incarcerated people endure daily. We changed the text of our invitation to ensure it was clear that we are not affiliated with CDOC.

Once people received the invitation, they could select whether they would like to participate in the study and return the form to the facility librarian. We did hear from a couple of participants while conducting focus groups that this part of the process was occasionally difficult to navigate for some who received a letter. We heard secondhand from participants of people who wanted to participate but were unable to return the form to the librarian, sometimes due to lack of cooperation from correction officers.

Once the facility librarians received the interest in participating from people randomly selected, they communicated with the liaison from Institutional Library Development (ILD) assigned to that facility to start planning the focus group schedule. How the schedules were set up varied from facility to facility, but we always completed the focus groups from each custody level at a facility in one to two days, sometimes conducting as many as six focus groups in a day.

Conducting a focus group

Most of the time focus groups were conducted in the facility library. The library served as a comfortable meeting place because it often felt like one of the most normalized places in the facility and interruptions were limited. There were a handful of times when the focus groups for certain custody groups could not be conducted in the library due to security concerns or lack of access to the library. In these cases alternative meeting spaces were used for the group, or private visitor booths were used when the interviews needed to be conducted individually for the highest custody levels.

To protect participants’ privacy, when focus groups were held at the library, the facility librarian stayed in their office during the focus group. This was also important to ensure participants’ responses were not influenced by the librarian’s presence. The only people present during the focus groups were the participants, members of the research team (usually one taking notes and one asking questions) and a member of the ILD team. After the focus group was over and the recording stopped, the ILD staff member helped answer any questions about facility libraries that arose throughout the conversation. This also provided an opportunity for ILD to spread the word about new or existing services at that facility. We made sure that the ILD staff member present had not previously worked at that facility library and did not know any of the participants present.

An ILD staff member was also present during focus groups to follow safety protocols. The ILD team had previous experience working in prison libraries, DOC badges, and the necessary training completed to act as our chaperones within facilities. Members of the research team did not walk around the facility without a member of ILD or other facility staff person escorting them.

To begin the focus groups, a member of the research team read the entire consent to participate form out loud to the group. There was then time for participants to ask any questions they might have, and if they agreed to participate they signed and turned in the consent form. We were careful throughout this process to make clear that participating would not have any impact, positive or negative, on participants’ terms of incarceration and that participants could refrain from answering questions or leave the focus group at any time. We also made sure that participants had enough information and a clear understanding of the study for them to be capable of giving true informed consent.

The research team provided pens to participants for the consent form but were required to collect them all back from participants. Food and water bottles were set out before the start of the focus groups along with paper plates, napkins and plastic utensils, so participants could help themselves throughout the focus group. We also needed to collect the leftover food and anything we distributed with the food at the end of the focus groups so participants could not bring any of it back to their cells.

Once participants had signed the consent form, we started the recording and began asking our questions. Our outline was set up with seven main questions and multiple follow up questions under each. Not every follow up question was asked in each focus group depending on whether the participants readily volunteered information. Focus groups were semi-structured, with the main questions and question order staying the same from focus group to focus group, but unscripted follow up questions were also often asked for clarification or to pursue an interesting topic presented by participants.

Surveys

Survey Design

The basic design of the survey was created during the planning stages of the project. Before administering the surveys, however, we made updates to improve the questions, organization/formatting, and language used. The whole team discussed these changes during multiple meetings, and we also had a sensitivity reader review the survey. The sensitivity reader helped ensure we were approaching topics through a trauma-informed lens for people who are or were incarcerated.

We administered surveys to people outside of facilities who were previously incarcerated before administering surveys inside the facilities. Surveys for people who were previously incarcerated were distributed by our partner Carol Peeples, founder of the reentry organization Remerg. Carol brought printed surveys to parole offices around the state and returned the completed surveys to the Library Research Service (LRS). LRS then manually entered the 271 completed surveys into an online version of the survey created in the survey platform Alchemer. This allowed us to easily export the data to spreadsheets or build reports within Alchemer to filter and analyze the data. We did not collect any PII, so the identities of the survey respondents were protected.

Although we exceeded our goal of 264 completed surveys from formerly incarcerated people, Carol communicated that there was sometimes reluctance from people to fill out the survey. We contributed this in part to the length of the survey. Large font size and spacing for the open-ended questions (both of which are crucial to distributing an accessible survey) also lengthened the survey. The resulting thick, stapled packet may have been viewed as intimidating and time consuming by potential survey takers. There was room for formatting improvements, so before distributing surveys inside facilities, we spent time redesigning the second survey for currently incarcerated people and focused on condensing the design while keeping it accessible. We removed questions such as, “Have you ever been incarcerated by the Colorado Department of Corrections?”, “Year of most recent release:”, “Year of most recent entry:” and “Please describe your current living situation” because these questions are not relevant for surveys distributed within prisons. We also improved the wording of several questions. These changes were all guided by our experience with the first survey distributed outside of facilities. We identified questions within the survey that were confusing to respondents and places where the wording needed to change for currently incarcerated people. These differences in surveys distributed outside and inside of facilities make it challenging to directly compare answers to multiple questions between the two surveys; however, we determined that these changes were necessary to ensure that we were distributing a clear and accessible survey and collecting quality data from within prisons.

Logistics of Administering Surveys in Prison

When possible, both printed and online surveys were administered within the facilities. Printed surveys were sent to a randomly selected sample of people at each facility. Our goal was to receive enough surveys for a representative sample, which was calculated at 267 surveys from currently incarcerated people. This target sample size was used to calculate how many people to randomly select and send surveys to within facilities. Those randomly selected received both an English and Spanish version of the survey, a cover letter that also provided a form for requesting the surveys in additional languages, and a pre-addressed and stamped envelope for returning the surveys to LRS. We manually entered the printed surveys we received back into another online version of the survey in Alchemer for analysis.

In facilities that allowed it, we also had the survey available on the library computer as an option for people to fill out during their time in the library. This online version was open to any currently incarcerated person who wanted to fill it out while in the library, regardless of whether they were randomly selected. Online responses were recorded in the same survey where we manually entered the printed surveys received from currently incarcerated individuals, and the text was identical between formats. We also created an online survey in Spanish for people to complete.

Focus Groups

Codebook

Coding is an approach to qualitative analysis in which words or short phrases are applied to pieces of the data. Before we could start coding the transcripts we needed to begin building our codebook, or in other words, the list of codes that we would apply and their definitions. This process truly evolved through grounded theory. Conducting focus groups left us with a preliminary idea of the themes brought up by participants most often, such as barriers to library use and the library services available. This experience alongside a careful review of our first focus group transcripts, laid the groundwork for the codebook we started with. Inevitably, as we read through more transcripts, we identified additional recurring topics relevant to our analysis and added them to our codebook. We structured our codebook with parent codes and child codes, parent codes being an overarching topic and child codes being components that fall under a parent code, such as “information access” as a child code for “library services.”

Even as we finished coding all the transcripts, we recognized in hindsight that there were elements of our codebook that could be improved. For example, we would suggest the addition of the code “books” with child codes “fiction” and “nonfiction”, and we would suggest incorporating the code “self-help books” as a specific example of the code “self-led learning”. We would also suggest the addition of the code “suggestions for improvement” so long as this works with the goals of your project. We would also like to emphasize the importance of clearly defining codes in the codebook. It is sometimes necessary for a code’s definition to evolve throughout analysis, but making sure the entire team has a similar understanding of how each code should be applied from the start can save a lot of time later on.

Qualitative Analysis

For our analysis of this qualitative study, we used the software Dedoose. Dedoose is a relatively low cost analysis tool and is robust enough for our scope of work. It allows users to code transcripts and other data as well as add descriptors. Descriptors are characteristics assigned to each transcript, and can be helpful for identifying differences between groups. For example, our descriptors were gender, security level of facility, facility name, custody level of each focus group, facility size, collection size, and the number of full-time and part-time staff.  These categories added an extra layer of analysis beyond code count and co-occurrence. Codes were added to excerpts from transcripts and we tried to make sure excerpts were at least a few sentences long so as not to lose too much context when reviewing code application.  Dedoose also enables teams to check inter-rater reliability through their “training” function. This allowed our team to make sure we were on the same page in terms of how and when we were applying codes. We did encounter a few drawbacks of using Dedoose; the software can be slow and clunky at times, requiring lots of updates and patience. It is not the most intuitive software, but a robust tutorial library is available as are trainings on platforms such as Udemy and LinkedIn Learning.

Teamwork and collaboration were vital to our success in Dedoose. We had 56 transcripts (called “media” in Dedoose) to code, which would be a very heavy lift for one person. We divided the transcripts between three team members. We originally divvied the transcripts by number of transcripts themselves, but learned that some people were getting very short transcripts while others had more lengthy ones. We reassessed and divided transcripts roughly by number, but paid more attention to length, which made the share of work much more equal.

In order to check each other’s work and make sure we agreed on code application, we used Dedoose’s training (also called testing) function. We began by having one person test the work of another person, but realized that discussing each excerpt would take far too long and was not a great use of time since we had 2409 excerpts to go through. Instead, we deciphered what the six most disagreed upon codes (as in they were applied differently when testing) were, and tested just on those. The best way we found to do this was to have one person run a test that included excerpts in which any of those six codes had been applied. This ended up being 873 excerpts which was still about a third of all our excerpts! We found that testing 100 excerpts took about an hour. It’s tedious work. Doing all 873 excerpts at once was not possible, so the tester did about 100 excerpts in a sitting. Ideally when testing you would be looking for a Kappa score which indicates agreement between raters. Since we had to divide the 873 excerpts into chunks, we were not able to use Kappa (another downfall of Dedoose). In order to find agreement, we went back to our original method of going through each of the 873 excerpts as a team and discussing any disagreement between the original coder and the tester. We would apply new codes to excerpts if we all agreed, but we did not remove any original codes unless the group felt very strongly that a code did not fit. This again was a long process: at our fastest we could discuss about 75 excerpts per hour. Despite this being a tedious task, the end result was a high rate of agreement on coding as well as rich discussions on etymology and library theory.

Notes/slides from ALA

[Coming soon]

Listserv for community discussion

[Coming soon]

Public libraries serving incarcerated patrons at the Franklin County Jail in western Massachusetts have adapted a version of the survey instrument for people who are currently incarcerated, which they then used to collect data to report on the outcomes of a separate grant funded by the LSTA (Library Services and Technology Act) through the MBLC (Massachusetts Board of Library Commissioners).

Citations

1. Maruschak, Laura, and Emily Buehler. 2021. “Bureau of Justice Statistics · Statistical Tables.” https://bjs.ojp.gov/content/pub/pdf/csfacf19st.pdf.
2. Carson, E. Ann, and Rich Kluckow. 2023. “Prisoners in 2022 – Statistical Tables.”
3. Bureau of Justice Statistics. November 2023. https://bjs.ojp.gov/library/publications/prisoners-2022-statistical-tables. “Monthly Population and Capacity Report,” Colorado Department of Corrections, last modified April 30, 2024. https://drive.google.com/file/d/1nv7vMGbRwqO56x4mZyynIN21wTBgbKvR/view.