Guest Post: Thoughts on Teaching Critical Information Literacy and Thinking Outside the Box*

by Latia Ward, Research Librarian, University of Virginia School of Law

Using generative and extractive artificial intelligence (AI) tools in research is an effective way to help students develop critical information literacy skills. Extractive AI is an algorithm (a step-by-step process) that retrieves information from a source (LexisNexis, 2023). Brief Analysis in Lexis and Quick Check in Westlaw, which pull legal concepts and citations from uploaded briefs, are examples of extractive AI tools. Generative AI is an algorithm that creates new information, such as text and images (LexisNexis, 2023). ChatGPT and Perplexity AI, which provide information in response to prompts, are examples of generative AI tools. Both extractive and generative AI are trained on sets of information (LexisNexis, 2023).

Some librarians and other knowledge management professionals are concerned about whether generative AI tools will make their jobs obsolete. In a blog post for the San Jose State University School of Information, Hannah Nguyen asserted that “There seems to be a general consensus that certain human qualities, like empathy and critical thinking, cannot be replicated by AI, and they are essential for the work that we do.” However, Bob Ambrogi has noted that a survey of legal professionals by Wolters Kluwer found that two-thirds of respondents believed generative AI could replace those in knowledge management, including librarians. ABA Standard 302(b) requires law schools to establish learning outcomes that include competency in legal research, therefore, librarians and teachers of legal research should stay up-to-date with research trends and include new strategies and tools in instruction to prepare students for their future work.

As librarians, we can use AI in different forms (extractive and generative) to teach patrons and students how to evaluate information critically.

Critical Information Literacy

Eamon Tewell states that people are information literate when they can use information to complete their work and fulfill their civic duties. However, critical information literacy goes beyond using information for daily needs, and requires interaction and engagement with the powerful systems that create and distribute information (Tewell, 2015). Critical information literacy requires asking, “Why?” and figuring out what information is missing (Ward, 2022).

Critical information literacy has its origins in critical pedagogy. In his book Pedagogy of the Oppressed, Paulo Freire explains critical pedagogy by contrasting the banking model of education with problem-posing. In the banking model, the teacher tells students information, which they repeat back to the teacher. In contrast, a problem-posing approach to education engages the teacher and students in a dialogue. Critical pedagogy is based on dialogue. Therefore, from a critical perspective, a dialogue about the information found is essential as students complete and reflect on legal research exercises.

I teach Advanced Legal Research at the University of Virginia School of Law. This semester, we discussed how databases can be biased, how algorithms impact research results, and how information may be missing. In-class exercises reinforced these points. Discussions and exercises like these are examples of legal information literacy and critical legal research.

Critical legal research involves deconstructing and reconstructing (or recreating) legal concepts, using alternative legal resources (free and government resources), pulling sources from various fields, and stopping to think (Stump, 2015). Thinking or reflecting on search results was integral to a lesson I created called Thinking Outside the Box in Legal Research (Ward, 2023).

Before class, students read Yasmin Sokkar Harker’s Invisible Hands and the Triple (Quadruple?) Helix Dilemma: Helping Students Free Their Minds. The Quadruple Helix consists of 1) the Library of Congress Subject Headings, 2) the Index to Legal Periodicals, 3) the West Digest System, and 4) algorithms and artificial intelligence that people use in their legal research. Sokkar Harker states that all four items affect what researchers find and write about and that the “invisible hands” that create and maintain databases influence legal research results.

To begin the lesson, I asked students to provide written answers to discussion prompts. Then, we discussed concepts from the Harker article. Some students indicated that they had not previously considered invisible hands and how these hands influence databases and search results. Students wanted to know how to conduct searches that algorithms wouldn’t hamper. I suggested three practices for conducting searches designed to get more information:

1. Search several databases (because content in one database is not always identical to content in another).

2. Think about how the legal issue you are researching may be like another legal issue. See if the way to handle that other legal issue can apply to the issue you are researching.

3. Review the West Key Numbers or topics and ask if you can describe the concepts using different words. If so, what words?

I asked students to think about an assertion by Richard Delgado and Jean Stefancic: “A computer is good at showing you what is. It cannot show you what might be” (Delgado and Stefancic, 2007, p. 328). In wrapping up our discussion, I mentioned how Delgado and Stefancic suggested turning off the computer and thinking about your research. Thinking or reflecting without the aid of a machine is yet another way of thinking outside the box.

For the corresponding in-class exercise, students worked in small groups on research problems I developed beforehand. As part of their work, they compared what they found in subscription databases with what they found in the generative AI tool of their choice. They used free tools and were not required to purchase a subscription. They also had to think about whether they were getting complete information and whether they trusted the information. At the end of each exercise, the groups reported what they had found using presentation slides and sharing their screens on large monitors in the classroom. While some groups found the generative AI tool to be helpful, others indicated it would give false citations (hallucinations) or provide details of situations with no citation or way to verify the information. Sometimes, generative AI provided no answer at all (an error message).

In keeping with the practice of evaluating search results, I shared questions Safiya Noble discusses in her book Algorithms of Oppression. This exercise focused on one of Noble’s questions: “Who is the intended audience for this information?”

LexisNexis and Westlaw released their generative AI products shortly after we did this exercise. I am interested in seeing how future students complete the exercise with generative AI tools created by legal information vendors.

One student said the AI tools were intended to assist people in representing themselves in legal matters. The student’s observation about generative AI tools and self-represented litigants highlighted another class theme: critical information literacy and access to justice. Self-represented litigants may face barriers to accessing justice. According to the Legal Services Corporation, 92% of “low-income Americans do not get any or enough legal help for their civil legal problems” that have a major impact on their lives. Access to justice relates to legal research because people who cannot afford lawyers may not be able to access subscription databases; they are likely to use free legal information resources instead. Librarians serving the public concerning legal information must be ready to suggest open-access resources.

Likewise, some non-profits and small organizations involved in legal advocacy may have limited or no access to subscription databases (Valentine, 2011). To prepare for their future work, students need to know how to access and evaluate information in different formats and places. In whatever context they find themselves, knowing how to use both subscription and open-access databases and generative AI tools is imperative. For research exercises, I use both subscription-based and open-access databases and now, generative AI tools. Open-access resources should not be overlooked, even when subscription-based databases are available.

Conclusion

Thinking about and discussing search results is key when teaching legal research. Librarians and others who teach research can create a dialogue and ask questions to facilitate the critical evaluation of resources. They can also consider barriers to accessing information for both legal professionals and the public. Some questions may not have readily apparent answers. Hopefully, asking questions creates additional perspectives from which to view both the research process and research results.

*I adapted this blog post from a presentation I gave to the Clinical Legal Education Association on October 11, 2023. The presentation was part of the Building on Teaching Critical

Legal Research with Law Librarians webinar and is available at: https://youtu.be/vTt3IlcLne8?si=ipZPxP1JPVLcl1Ac.

Here is a copy of the exercise I used:

Latia Ward, Think Outside the Box (In-Class Exercise), ACRL Framework for Information Literacy Sandbox, (2023), https://sandbox.acrl.org/library-collection/think-outside-box-class-exercise [https://perma.cc/PT4X-F8JU].

Sources (in order of appearance):

Algorithm, Merriam-Webster (2023), https://www.merriam-webster.com/dictionary/algorithm [https://perma.cc/79U9-748T].

LexisNexis, AI Terms for Legal Professionals: Understanding What Powers Legal Tech, (Mar. 20, 2023), https://www.lexisnexis.com/community/insights/legal/b/thought-leadership/posts/ai-terms-for-legal-professionals-understanding-what-powers-legal-tech [https://perma.cc/7UKU-59L7].

Hannah Nguyen, Should Librarians Be Worried About Artificial Intelligence?, SJSU School of Information Career Blog (May 29, 2023), https://ischool.sjsu.edu/career-blog/should-librarians-be-worried-about-artificial-intelligence [https://perma.cc/YW52-XU8F].

Bob Ambrogi, Survey Says Gen AI Puts Law Librarians and KM Professionals At Risk of Obsolescence, LAWSITES (July 6, 2023), https://www.lawnext.com/2023/07/survey-says-gen-ai-puts-law-librarians-and-km-professionals-at-risk-of-obsolescence.html [https://perma.cc/GM9G-NNGV].

American Bar Association Section of Legal Education and Admissions to the Bar, American Bar Association 2023-2024 Standards and Rules of Procedure for Approval of Law Schools, Chapter 3: Program of Legal Education, (2023), https://www.americanbar.org/content/dam/aba/administrative/legal_education_and_admissions_to_the_bar/standards/2023-2024/23-24-standards-ch3.pdf.

Eamon Tewell, A Decade of Critical Information Literacy: A Review of the Literature, 9 COMMUN. INF. LIT. 24-26 (2015), https://pdxscholar.library.pdx.edu/comminfolit/vol9/iss1/2 [https://perma.cc/2F9A-FXVQ].

Latia Ward, A Librarian’s Experience Teaching Critical Information Literacy, 41 LEG. REF. SERV. Q. 52, 64 (2022) https://doi.org/10.1080/0270319X.2022.2089808.

PAULO FREIRE, PEDAGOGY OF THE OPPRESSED 71-72 (30th Anniversary ed. 2000).

Nicholas F. Stump, Following New Lights: Critical Legal Research Strategies as a Spark for Law Reform in Appalachia, 23 AM. UNIV. J. GEND. SOC. POLICY LAW 573, 618-23 (2015) https://digitalcommons.wcl.american.edu/cgi/viewcontent.cgi?article=1670&context=jgspl [https://perma.cc/5DGH-MLEG].

Yasmin Sokkar Harker, Invisible Hands and the Triple (Quadruple?) Helix Dilemma: Helping Students Free Their Minds, 101 BOSTON UNIV. LAW REV. ONLINE 17-20 (2021) https://www.bu.edu/bulawreview/2021/04/06/invisible-hands-and-the-triple-quadruple-helix-dilemma-helping-students-free-their-minds/ [https://perma.cc/3LWJ-WXVW].

Richard Delgado & Jean Stefancic, Why Do We Ask the Same Questions – The Triple Helix Dilemma Revisited, 99 LAW LIBR. J. 307, 328 (2007).

SAFIYA UMOJA NOBLE, ALGORITHMS OF OPPRESSION: HOW SEARCH ENGINES REINFORCE RACISM 5 (2018).

LexisNexis, LexisNexis Launches Lexis+ AI, A Generative AI Solution with Linked Hallucination-Free Legal Citations, (Oct. 25, 2023) https://www.lexisnexis.com/community/pressroom/b/news/posts/lexisnexis-launches-lexis-ai-a-generative-ai-solution-with-hallucination-free-linked-legal-citations [https://perma.cc/Z4W5-TMNY].

Thomson Reuters, Thomson Reuters Launches Generative AI-Powered Solutions to Transform How Legal Professionals Work, (Nov. 15, 2023), https://www.thomsonreuters.com/en/press-releases/2023/november/thomson-reuters-launches-generative-ai-powered-solutions-to-transform-how-legal-professionals-work.html [https://perma.cc/KSH7-PGPB].

Legal Services Corporation, The Justice Gap: Executive Summary, https://justicegap.lsc.gov/resource/executive-summary/ [https://perma.cc/92US-SZNV].

Sarah Valentine, Leveraging Legal Research, in Vulnerable Populations and Transformative Law Teaching: A Critical Reader 145, 155 (Society of American Law Teachers and Golden Gate University School of Law eds., 2011).