Same Old Song & Dance: Rolling Out a New Product

by Erik Adams

Recently it was reported that celebrity chef Jamie Oliver had incurred the wrath of the people of Spain for a new recipe: a paella that included chorizo. Paella, it seems, is considered the national dish of Spain, and although there are many ways to make it, none of them include pork sausage seasoned with paprika. He was publicly chastised for thinking outside the box, trying something new.

I was reminded of this controversy as I sat through the kick-off meeting for a project we are embarking on at my firm. We are rolling out a new research product, and the vendor has proposed a fairly standard plan to introduce it to the firm and ensure that it gets used. The basic steps are:

  1. Kick off meeting with major parties (vendor representatives and librarians, in this case)
  2. Survey the firm to identify hot spots
  3. Recruit champions among the attorneys
  4. Introduce the product to the firm with demonstrations of basic features
  5. Provide more advanced training targeted at practice groups
  6. Review usage and identify areas for improvement

We’ve been down this road before with varying degrees of success and feel that there are some areas that need improvement. Therefore, we have proposed some slightly different steps to make the rollout quicker and more effective.

First, we have found it very hard to get attorneys to respond to surveys. In my personal experience, the number of responses to a survey is inversely proportional to the length of time it takes to complete; I got the highest number of responses from a survey that had only two yes/no questions. In a law firm that is as large as mine, with a diverse number of practice areas, any survey that really digs into attorney needs is going to be too long. Fortunately, we are not rolling out this product completely blind; we already have a competing product from another vendor. Thanks to feedback we have received, data we already have in our reference question tracking software, and web analytics, the attorneys have already told us what is hot and what is not.

Recruiting champions has always been difficult. I am reminded of a Dilbert cartoon where a manager asks employees to volunteer for a new project, but with the secret goal of identifying which workers don’t have enough to do and can possibly be laid off. We do need engagement from attorneys, but rather than brainstorm about who doesn’t have enough to do, we thought we could get our champions to self-select by making a change to the introductions.

Rather than making the initial product demonstrations a discussion of basic features, we are going to have the introductions targeted to specific practice areas. In practice, 80% of the presentations will be the same, but the examples will be tailored to practice groups. For example, securities people won’t have to sit through litigation examples. This will hopefully serve two goals: attorneys will be more likely to participate if they know it will directly apply to their practice area; and, the attorneys who attend are more likely to be interested in helping make the project a success. The attorneys who attend the initial demonstrations can be the group from which the champions are selected.

These are pretty small changes, but when we proposed them to the vendor, they were receptive. They’ve been down this road before (probably more times than us) and know what works and needs improvement. Our suggestions aren’t wild, and ultimately we have the same goal: getting attorneys to use the tools to be more effective in their work.

Posted in Customer Service, Issues in Law Librarianship, Legal Research Instruction, Platform change | Leave a comment

Charting a New Course: A Foray into Instructional Design

by Ashley Ahlbrand

In my last post, I mentioned that I have started taking courses to earn my certificate in instructional design. I received a lot of feedback on this and thought I would post periodically on the topic. I am fortunate to have an online program available at the university where I work, so I am enrolled in Indiana University School of Education’s Instructional Systems Technology Certificate program. These programs take different shapes at different universities. Mine, for example, puts an emphasis on technology use in instructional design for various course environments. To see the range of programs available, I recommend this article from eLearning Industry.

I am currently in my first course, Instructional Design & Development, which takes us through the phases of instructional design using the ADDIE model:

  • Analysis
    • Concept and Procedure Learning – what is to be taught?
    • Learner Analysis – student characteristics, prerequisites needed, special requirements
    • Contextual Analysis – needs analysis, instructional context, transfer context
    • Topic Analysis – detailed breakdown of concept/procedure
    • Instructional Objectives, Strategy, and Sequencing
  • Design – designing instructional materials (for both instructor and student)
  • Development – testing out the instructional materials and revising as needed
  • Implementation – developing a procedure for training both the instructors and learners on any procedures or tools they’ll need for instruction
  • Evaluation – how instruction and instructional materials will be evaluated using both summative and formative assessment

Our first project for this class asked us to focus on the analysis portion of ADDIE, by choosing a concept relevant to our own careers and designing instruction on that concept. While considering legal research instruction, I always think of teaching how to research a particular topic or source. However, we learned in the course that these are not concepts – they’re procedures. As a rule of thumb, a procedure is anything that can start with “how to.” A concept is a classification used to group similar things, so it begins more with “what is.”

Step 1: Select a Concept

In January, I will teach a 1-credit, one-week course for our international LLM students called “Introduction to American Legal Research,” so I decided to focus on that audience. My concept was the foundational primary versus secondary sources. It occurred to me that this is a concept that we always talk about in legal research instruction, but, in my own teaching, I don’t traditionally delve deeply into. Yet this is a critical concept when you’re planning to teach a population of students who come from countries with legal structures far different from the U.S.

Step 2: Learner Analysis

This step looks at factors that could impact the instructional experience, such as the age of the learners and their attitudes and expectations for the instruction. For my population, I also analyzed cultural backgrounds and language skills. This analysis also includes any prerequisites (i.e. base understanding of American law) and special needs.

Step 3: Contextual Analysis

This begins with a simple, yet critical question: Is there a need for this instruction? I then had to describe the instructional environment (i.e. that it would be an in-person class, in a classroom with typical technology offerings) and the transfer context – where the students would use these skills later on. For this I discussed the fact that there are very few required courses for our graduate students, many write LLM theses, most will at least have seminar papers to research, and the students either remain in the U.S. after school to practice law or return to their home countries as scholars, practitioners, or both. Most will, therefore, use their legal research skills as an integral part of their careers and could use this foundational instruction.

Step 4: Topic Analysis

The next step was the topic analysis that broke down my concept into small sub-concepts with definitions and examples. I gave a lengthy description of each, and provided the following diagram:

concept chart

I also included an appendix that delves further into the concepts and examples, accessible here, if you’re interested.

Step 5: Description of Instructional Objectives, Methods, and Assessment

Here I stated that students should be able to identify each type of source, first as primary or secondary, then between the sub-concepts. To assess this skill, we would begin with an identification exercise for each sub-concept. For example, if we were studying statutes, I might give them an exercise with ten samples, nine of which are statutes and one of which is a regulation; students should be able to identify that the regulation does not belong. We would then progress to a labeling exercise, wherein students would get a set of ten samples that include everything from statutes to articles in a constitution to excerpts from a case opinion to regulations, and they should be able to identify each. If we can progress to this level, the students have mastered the underlying concept.

For this first project, we did not progress to the design, development, implementation, or evaluation phases, but we did have to describe what we thought would be included in the instructional materials. I discussed introducing the concept through both a graphical handout, describing all concepts and sub-concepts, and a brief lecture overview as reinforcement.  To engage the students, I discussed ideas such as beginning the instruction with the “Three Ring Government” video from Schoolhouse Rock and showing examples of citations in a case and a law review article to demonstrate how primary and secondary sources are used in different kinds of research. Finally, I discussed culminating the exercises with a research prompt where the students are asked to identify what sources they might use to answer the research question and why, showing not only that they know what the sources are, but also why each is helpful.

I received positive feedback from both classmates and from my professor on this project, so I intend to utilize this instruction in some form for my upcoming course in January.  Perhaps I’ll write a follow-up post at that time to let you know how it went!

Posted in Continuing Education, Legal Research Instruction, Uncategorized | Tagged , , | Leave a comment

Designing a Legal Research Course: Phase 2-?

by Tig Wartluft

In a previous post, I discussed three phases of designing a legal research course. During Phase 2, the question became: how do we organize the obvious content that we must include without allowing that content to override and muddy the course we’ve painstakingly conceptualized? This post continues that conversation.

Kurt Koffka, one of the founders of Gestalt psychology wrote that an individual who has a lot of knowledge knows many facts, but the individual who knows the singular system containing those facts knows and perceives more. The goal of Gestalt psychology was to determine what happens when “thinking really works productively” to solve a problem. (Wertheimer, 1966). This makes the Gestalt approach easily applicable to legal research, as legal research is “in essence a process of problem solving.” (McCrate Report, 1992).

The concept of ‘the whole is greater than the sum of the parts’ originated in Gestalt psychology. Gestalt theory has found that “structured wholes . . . are the primary units of mental life.” (Wagemans, et al., 2012). Our brains recognize patterns; too many facts and our brains start to look for the simplifying singular structure containing those facts. Our brains, like all physical systems, show a tendency to maximize stability with a minimum of energy. In other words, our brains are lazy. We can harness that laziness to our advantage through thoughtful organization of the information we present.

Gestalt investigations into visual perception led to identification of several principles of organization: proximity, similarity, continuation, and common region, among others. Organizing the skills we present to our students using these principles reinforces learning and improves recall, but presenting legal information skills contrary to these innate patterns will camouflage the very skills that we are trying to highlight. An organizational pattern that our brains use as shorthand is proximity.


Figure 1

For example, Figure 1 shows a series of eight dots in a line. By varying the spaces between the dots (line two shows proximity) or the color of the dots (line 3 shows similarity) our brains interpret the information differently. Each line contains eight dots, but at no time does our brain encode ‘eight dots.’ It recognizes a line of dots, several sets of pairs, or alternating sets of black and grey dots. If we’re trying to teach the concept of eight black dots, our students’ brains are obfuscating that point by looking for and encoding the most easily digestible pattern.

If we want to teach research literacy skills, we must organize the information so that the skill is the overriding pattern. I take a skill, like using a citator to find cases interpreting a specific statute, and ask the students to use that same skill in multiple databases. This highlights the similarities even if the individual user interface steps change when moving between databases.

On the other hand, if we present information in a way that camouflages the information we’re presenting, the organization of our instruction could be working against our objectives and lowering retention and recall. To this end, one of the organizational shortcuts that Gestalt research has shown our brains take is grouping items contained within a common region.

In legal research terms, by focusing on a singular database for several skills, students may be encoding the information based upon the common region (the singular database), mistakenly learning that ‘[insert database name here] has everything!’ instead of learning the showcased skills in a transferable way. By focusing on the database, we have unintentionally camouflaged our instructional goal due to the learner’s propensity to ascribe meaning through the common region, the database.

By applying the Gestalt organizational principles to bolster the learning objectives in our course, the syllabus was organized around research skills being applied across research tools instead of focusing on the use of the individual tools. Using the brain’s default encoding practices to highlight the transferable skills required to be a literate legal researcher allowed our learning objectives to inform the content of the course hopefully allowing for greater retention and recall by our students.


Am. Bar Ass’n Section of Legal Education and Admissions to the Bar, Legal Education and Professional Development – An Educational Continuum: Report of the Task Force on Law Schools and the Profession: Narrowing the Gap, July 1992. (Known as the MacCrate Report).

K.Koffka, Principles of Gestalt Psychology (1935).

Irvin Rock & Stephen Palmer, The Legacy of Gestalt Psychology, Sci. Am., December 1990.

Johan Wagemans et al., A Century of Gestalt Psychology in Visual Perception: I. Perceptual Grouping and Figure-Ground Organization, 138 Psychol. Bull. 1172 (Nov. 2012).

Max Wertheimer, Productive Thinking (Michael Wertheimer ed., enlarged ed., 1968).

Posted in Information Literacy, Legal Research Instruction, Teaching (general) | Tagged , , , | 1 Comment

Uncertainty and Legal Research

by Paul Gatz

Whenever I prepare to teach legal research to law students, I try my best to imagine what learning legal research is like from their perspective. Admittedly, this was much easier to do when I first got the chance to teach and was myself only a few years out of law school. As my own experiences drift further into the past, however, my memories naturally fade. Moreover, as new online products alter the array of tools that are used for research, I sometimes wonder whether the current law student’s experience of learning legal research is fundamentally different than it was a mere 5-10 years ago.

For this reason, when I try putting myself into the shoes of a student learning how to research, I try not to rely on just my memories of the databases or the cognitive processes involved in my first legal research experiences. Rather, I try to think about how those first legal research experiences felt. Not incidentally, those memories of the frustrations, anxieties, and rare joys of legal research remain distinct, and of course it is the frustrations and anxieties – and not the joys – that really stand out.

I imagine most experienced legal researchers would recognize these sorts of frustrations. From the beginning to the end of a research project, the researcher is confronted by the unknown. How and where do I start? How do I figure out the best secondary source to use? Is this Boolean search doing what I think it should? Which of the cases in this massive list of results is actually relevant to my problem? How do I know that I haven’t missed a key case?


One of Laredo Taft’s Sons of Deucalion and Pyrrha, deep in the throes of research despair.

With greater knowledge of legal doctrine and more experience in legal research, the
anxieties raised by these questions diminish considerably, but for the novice researcher, these questions can lead to a great deal of frustration and confusion. It is worth noting that all these questions, like a multitude of shadows cast by a single object, have their origins in the same cognitive state: uncertainty.

Over 20 years ago, Carol Kuhlthau published “A Principle of Uncertainty for Information Seeking” in the Journal of Documentation. Based on her empirical studies of information seeking behavior based on the user’s perspective, Kuhlthau developed a model of the information search process consisting of a series of stages and posited an uncertainty principle as the underlying conceptual framework for this process. Each stage can be characterized as playing a unique role in the user’s progression from uncertainty at the initiation stage to understanding at the presentation stage.

Uncertainty has both cognitive and affective aspects. The cognitive aspect is “a lack of understanding, a gap in meaning, or a limited construct.” [347] The affective aspect includes feelings of “uncertainty, confusion, and frustration.” [Id.] My memories of how I felt during my first legal research experiences are memories of this affective aspect of uncertainty, and those feelings were likely closely related to my struggles with the cognitive aspect, my own lack of understanding of both the substantive law and the tools and methods of legal research.

The principle of uncertainty has an intuitive appeal as a foundational concept for information seeking and research. After all, if you already understand something, you have no need to research. It may even be that those symptoms of frustration and confusion are a good thing, since they help the researcher to recognize precisely where those gaps in understanding are. Put another way, if you’re not getting frustrated when you’re researching, you’re doing it wrong.

Every legal research project begins with uncertainty about how the law applies to a particular set of facts. The researcher’s task is to find information and authority that will lead to a practical understanding of the law and its application. Uncertainty may decrease as the researcher skims secondary sources and peruses statutory language, only to increase again when the researcher confronts a large variety of cases or happens upon a source that changes the analysis completely.

But does the cognitive aspect of uncertainty necessarily entail the affective aspect? The researcher can easily access a wealth of information in multiple formats online. What is there to be frustrated or confused about when you can ask a search engine for the elements of burglary and get a reliable answer? Or when you can upload a brief and be automatically directed to important cases you missed?

But algorithms and artificial intelligence, like databases and books, are merely tools. They make legal research easier, but cannot remove the frustration and confusion caused by uncertainty. So long as laws need to be applied to novel situations, the researcher will need to make that progression from uncertainty to understanding.

Based on his or her own experiences, the legal research instructor can point out a few places where the novice researcher is likely to encounter frustration, confusion, or anxiety, but the experienced guide cannot hope to alert the tenderfoot to every danger. The best we can do is prepare them to be ready when, in the depths of their research, surrounded by conflicting cases, dense statutes, and bizarre Boolean operators, uncertainty casts its shadow.

Posted in Legal Research, Legal Research Instruction | Tagged , , | 2 Comments

The Limit Does Not Exist

By Christine Anne George

Monday was Mean Girls Day (the date was chosen due to a particular scene in the movie). Among the many things Mean Girls has gifted us is a particular line that, from time to time, applies to reference work. Spoiler alert for those who have managed to dodge the incessant airings on TV. Towards the end of the movie, Cady is participating in a Mathletes competition and has to pull herself together to solve an equation. The solution comes to her like a lightning bolt. The limit does not exist! So too must the reference librarian occasionally break the news to their patron—the source does not exist. Only unlike the movie, for the reference librarian, the challenge doesn’t end there.

In my experience, there have been two main reasons to break that awful news to a researcher, or worse, an 11th hour cite checker. The first is that the source actually no longer exists due to limitations of our digital world, namely link rot. The second tends to be a mis-cite or perhaps a Bluebook faux pas on the part of the author. Both, though, provide opportunities to show the many and varied skills of reference librarians.

Based on a previous post on this blog and a presentation at the 2016 Cool Tools Café at AALL last summer, I have high hopes that law librarians are raising awareness about link rot and doing our best to, if not eradicate it, then to at least make it far less common. Until that day comes, though, there is the Wayback Machine. A link might not be really most sincerely dead. If one is lucky, it might have been preserved and available through the Internet Archive. Putting a dead link in the Wayback Machine and then pulling up the elusive document, much to the wonder of the researcher, is pretty much the closest I’ve felt like a magician pulling a rabbit out of a hat.

Sometimes there is no magic that can conjure up a source because it really doesn’t exist…at least not as it’s cited. Authors might misattribute a quote or flip a year or do any number of things that would make a source impossible to find. For example, earlier this week, a research assistant reached out to me to ask about Person X quoting Person Y saying Z. No amount of searching could turn up quote Z in any of Person Y’s writings. We concluded that the quote did not exist and that Person X paraphrased Person Y while claiming Z was a direct quote.

Forgive the cliché ending, but unlike Cady where discovering that the limit doesn’t exist leads to a letterman jacket, trip to prom, general rightness with the world, and happily ever after, finding out that the source doesn’t exist is only the start for reference librarians. The revelation that the source doesn’t exist means that it’s time to roll up our sleeves and get to work.

Posted in Legal Research, Reference Services | Tagged , , , | Leave a comment

RIPS-SIS Member News & Twitter

by Alyson Drake

The Executive Board continues to be impressed by all the amazing things RIPS-SIS members are doing. The Program Committee just finished reviewing eight excellent proposals for the Annual Meeting in Austin, making our task of choosing just two to send on to the AMPC very difficult! As always, RIPS will be well represented at the annual conference and we can’t wait to attend all programming put on by our members.

One of the Executive Board’s goals for this year is keeping in touch better with our members about all the great projects you’re working on and spreading that news on to the whole SIS. Please contact us to let us know about all the cool endeavors you’re working on.  Presenting at a conference? We want to know!  Just published a paper? We want to know! Working on an innovative project that you’d like to share with the RIPS membership? We want to know!

One way we’re hoping to facilitate this sharing is our new Twitter account. It’s a great place to follow all the RIPS-SIS happenings, from the newest RIPS blog posts to committee reports and member news. Start following RIPS-SIS today and send us your news so we can share it with the masses!

Posted in RIPS events, RIPS Member Achievements, RIPS-SIS Reports | Leave a comment

The Law Library & The Bar Exam

by Dean Duane Strojny

Because of another blog posting forwarded to me recently, I am moved to discuss the bar exam and the role libraries could, should, and must play in the preparation of it.

This is a critical time for law schools in this country. There are many experimental programs that have yet to prove their worth given the time and effort, not to mention the money, that students put into attending law school. During the interpretation of the latest bar exam results, it is easy for schools to forget that the next exam is less than five months away. New ideas can quickly be put on the back burner. Since there are finals and semester breaks between now and then, the time will quickly pass. It’s never too early to start thinking about bar prep programming for the next bar exam.

The vast majority of law school students intend to graduate and work with traditional types of legal questions whether they are in law firms, government, or corporate settings. The real question for librarians might be what is the purpose of law school and how can the library help?


Supporting academic programs is essential. Also, toss in assisting faculty in research. Finally, add a little alumni and general pro se assistance for good measure, and you likely have the basis for a solid library program. However, today there is something larger looming in our futures. Students walking across the stage are not completely done with us. There are bar exams and job interviews to secure. Bar preparation is not to be taken lightly. There are many schools with struggling students who deserve the opportunity to become licensed lawyers. These students often thrive with a little extra attention and support. By having some programming in place that all students can take advantage of, the library continues to serve the institution in new ways.

So, what can the library do to help students on the cusp of beginning new careers?

1.  Maintain a strong connection with your student support office. Constantly remind them that the library is a great place to study and can serve as a resource center for bar preparation.

2.  Profile a bar preparation collection. There are many print resources available to assist students. Make them readily available and make sure all the reference staff know where and what they are. This might be one area where buying print materials might be a wise choice.

3.  Create a research guide that directs students to both online and print resources. Those who would not think of stepping into the library after graduating might actually click on some links for assistance.

4.  Make space specifically for bar exam studying. From study rooms to an area of tables or carrels, it is easy to delegate a specific area for a few months before the exam. It makes the students feel more supported in this tough task that lies ahead.

5.  Partner with other law school departments to sponsor programs on bar prep, career services, and financial skills. All of these topics are important to upper level students as well as recent graduates. There is no limit to thinking outside of the box with programming ideas among your colleagues and many departments know how to successfully attract people to their events.

6.  Remind library staff about the bar exam dates. If librarians are assisting in appropriate outreach, they will be cognizant of the dates and be able to become well-wishers and supporters along the path to those important dates.

Remember that the mission of the library is to support the academic endeavors of the law school. With this in mind, outreach for bar preparation is of utmost importance and can pay great dividends especially if institution bar results improve and can be linked to any of the library’s efforts.

The following is a small compilation of bar prep research guides that can either be general or state specific:

Campbell School of Law Library

Georgetown University Edward Bennett Williams Law Library

New York Law School Library

University of Cincinnati Law Library

University of Missouri School of Law Library

Posted in Career, Customer Service, Library Collections, Patron Services | Tagged , | 1 Comment