The Library Literacy Survey started as a proposal submitted by Joanne Henning and Shelagh Mikulak to the Teaching Development Office. The project was later handed over to Helen Clarke, Maureen Hunter and Saundra Lipton.

The goal of the project was to develop a user focused understanding of library literacy and the role of library research in writing term papers. We deliberately avoided predefining what skills or knowledge students needed to be considered 'library literate'.

The methodology consisted of semi-structured interviews with students from Geography (one course), Religious Studies (two courses), and Psychology (one course). All courses required students to write a term paper and were at the 300 or 400 level. Students were volunteers and all identifying information was kept confidential following University Ethics guidelines. In all, 21 students were interviewed. The interviews were taped and transcribed. Nudist software was used to help with sorting and analyzing the data.

This report contains our initial findings concerning how students use the library and some recommendations that come out of those findings. We hope to be able to present more in-depth analysis of students' writing and reading practices and library research strategies later in 1996.

It is important to note that students may have used a strategy and have failed to report it in the interview.

Despite this proviso, the results show some striking trends. First, the four most frequently reported strategies were:

• using DOBIS
• searching the compact discs
• browsing
• referring to citations found in other documents

The number of strategies reported was also examined by grade and course:

This is not a quantitative study and these numbers should not be over interpreted. One conclusion that can be drawn is that students who use a very limited array of resources in finding material do poorly. Students who are engaged and search more strenuously for material do better. However, using more strategies does not lead linearly to better grades. Even for a simple term paper, research is a complex process and library skills are only part of what leads to complete success.

Recommendations:

We should emphasize to students the importance of being engaged in the entire research process and avoid implying that good marks are entirely dependent on the quality of their work in the library. Library skills can help students avoid getting very low marks but their final grade will depend on a variety of other actors. We are in danger of oversimplifying the process if we suggest that library skills are the ticket to success in research.

We often take the catalogue's foibles for granted. We must pay close attention to the design and content of the library catalogue since it is the most used single resource in the Library.

Most students will find research material without asking for assistance from Library staff. The information systems we design should support students' desire to work independently. We should also concentrate our energy on improving services students show a willingness to use.

"...because I don't think DOBIS is very helpful as far the titles of books can be very deceiving. And it is not exactly helpful for finding out if that particular book has what you need in it."

"I tend to find DOBIS first, because it is always easiest. And then, usually that doesn't help very much and you have to go out to the CD-ROM..."

"I often have trouble with that system actually. It is OK for generalizing areas, but for really narrowing down, it is not great."

"There wasn't really anything on DOBIS that would describe what (it) was actually about...But I was able to locate the general section where I wanted to go."

Findings:

The most frequently used strategy for finding material was searching DOBIS. The only student for which we had a complete interview and who did not use DOBIS for the assignment received a C+.

Most students (12) reported searching DOBIS by keyword for general topic areas. DOBIS was an important first step in students' attempts to find out more about a topic.

Another major use of DOBIS was to locate journals.

Students did express frustration with DOBIS. Some key problems they identified were:

• having to search title and subject separately.
• material located through DOBIS was dated.
• having to use DOBIS for journal locations instead of being able to find this out on the CD-ROMS.
• lack of boolean search capabilities.

Students also deliberately used DOBIS as a browsing aid. They would look for a book, find the call number and go directly to that area of the shelving, often without looking any further on DOBIS. Six students reported using this approach, and we can assume that at least some remaining students who reported using browsing as a strategy were also using DOBIS this way. The lack of descriptive information in DOBIS, ie., the lack of contents notes seems to have been one reason students preferred browsing.

Another factor contributing to the popularity of browsing was the students' inability to successfully use DOBIS to find materials on a specific topic. Part of this can be attributed to the general nature of monographic material, but part of the reason was that students did not understand how to use subject headings to create more specific search strategies. Only one student reported using the LCSH. One student used the subject headings she found in her records but most used a keyword approach, repeating the search in title and subject. Basically, students were using the same strategies for DOBIS that they used for CD-ROM searching.

Recommendations:

Clearly if the Library were not getting a new OPAC one of our first recommendations would be that it should. However, even with a sophisticated system like SIRSI some key factors should be kept in mind. Students do not have a specialist knowledge of the on-line catalogue and will not make use of relatively sophisticated searching methods such as using the LCSH. Students expect to be able to find the information through keyword searches. Our system design should take this into account and give priority to enhancements that will improve the effectiveness of this technique. One enhancement could be the inclusion of more descriptive information in the catalogue record.

Students did not report regularly asking for help if they were dissatisfied with the results of a DOBIS search. Instead they would fall back on other methods like browsing. Again, system design and instruction methodologies should take this basic characteristic into consideration. Besides designing as straightforward and simple a system as possible we should also put more emphasis on developing point of need instructional material. It is wrong to assume that students will ask for help when they are in trouble. Written guides posted and distributed both on-line through the catalogue and close to the terminals might be used by students who believe they should be independent users of the library.

"...what I did was went into the CD-ROM and looked at what they had under team building . . . Just look at the you know, the little blurb that you get. Just punch in a different term that I though had to relate to team building. And I saw, . . . different articles about different things. And then that symbolic one hit me . . . I went upstairs and I found it and I read through it and I thought it was interesting . . . And then after that, I went back to the CD-ROM again to find articles that related back to those specific areas within the symbolic aspect of team building."

"...the part that I find the most satisfying, maybe that is the best way to put it, is when I get on the ROM. And I just find the information . . . it is such a source of current stuff."

"How did you learn to do CD's?"

"Well, ______ (a librarian) gave us a little course . . .Which I didn't use at the time . . . I went upstairs and asked how you sign on and somebody showed me . .. And I just happened to hit the right buttons and it was easy. I mean it wasn't a difficult procedure."

"...one problem I can see right now though is CD-ROM. Once you get on there you usually only have one-half an hour...And a lot of times there isn't enough staff there...Like, if they are helping somebody else, a lot of time those tasks take quite a while. So by the time you wait around for them to ask and then you get the help, your time is up. So you basically have to muddle through on your own rather than sit there and wait."

"So I went over there on their (Management Resource Centre) CD-ROM because I didn't know it was different than ours. The one here in this library."

"How did you find that out?"

"Pure accident . . . I thought CD-ROM was the same all over campus . . . It was sheer luck that I had to go. And then I thought, 'Well, jeeze, they don't have Psychlit' and my husband says, 'Well, no.' He said, 'This is different than Mackimmie Library' and I said, 'No way'...my last year here and I have never been told that and I went to the library thing where they bring you through the library...and they didn't tell you that."

Findings:

Students expressed more enthusiasm for CD-ROMs than they did for DOBIS. They were pleased with the speed and ease of use of CDs. As with DOBIS, most students were using keyword searching and did not feel the CDs were difficult. Students did not express the same dissatisfaction with finding material on their topic or with the descriptive content of the record. The descriptive content of the records and the narrow focus of many articles probably helped make keyword searching a more successful strategy with CDs than it had been with DOBIS.

Students were dissatisfied with the booking system. Some students found the CD's intimidating and complicated to use, but these were in the minority.

Seven students reported on how they found out about the CDs. One learned from the information desk, two from library staff, two from instruction sessions, one from a friend and one from a professor. Instruction and contact with the library staff were important in making students aware of the CDs.

Nine students reported using the CDs for developing and focusing their topic.

Recommendations:

Just as with DOBIS we can expect that students will use the CD's independently, asking a minimum of questions. Again, we should give more emphasis to providing on-line and on site instructional materials that accomodate the students' desire to be independent.

Certain characteristics of CD indexes make them desirable, descriptive content, boolean searching, ease of use. We should keep these characteristics in mind when designing or evaluating information retrieval systems.

CDs are used as part of the recursive and often frustrating process students go through in deciding on topics and understanding the literature in a chosen area. We should avoid thinking of CDs as simply a tool students use to identify papers. They also play an important role in helping students understand the range and type of literature available in a subject area, and help them focus on the specific aspects of a topic discussed in the literature.

Findings

Few students mentioned using document delivery. The comments from those that did suggest that students are poorly informed about the mechanics and effectiveness of document delivery.

"Do you ever go to ILL to get material from other libraries out of the city?"

"I am not that well organized. And I know someone had told me about it before and that it usually takes a week or longer and I usually don't have that kind of time. If I were more organized, I would."

"And they will tell me that, oh, we don't have it, or you know you can send away for it or whatever, but usually I don't bother doing that because it takes like two, three weeks or whenever by the time it is due. So there is no point."

"...it was an article that I desperately needed to do one of my arguments. One of the first focuses I had . . . It was an article that one of the librarians thought was over in the education library. I got there and it wasn't there . . . I came back here to try to get it at Interlibrary Loans. Sometimes, you can get it in 3 or 4 days. If someone has an e-mail?...The U of A didn't have it . . . I didn't have enough time. If I had done that probably a month in advance. I could have done without it . . . But since I only had a couple of weeks I couldn't wait."

They may be aware that there is some way to get information from other libraries but they fail to pursue this option because they believe the process will take too long. Indeed, undergraduates rely heavily on the Library's collection and a delay of more than one week seems to invalidate document delivery as an option.

Recommendations:

Educate students about the process and effectiveness of document delivery.

Realize that students use materials in a variety of ways, often working within very short time lines. Document delivery may not be a suitable option for many undergraduates writing term papers.

Although all students had received some form of library instruction (tailored handouts for the Religion and Geography students and previous PsychLit instruction for the Psychology students), retention and application of information varied considerably.

Nine out of 15 students who received a handout as part of an instruction lecture mentioned its usefulness in reinforcing the information provided by the Librarian.

"...somebody from (the library) had come in. That is right. And given us a list of, there were a few books that were in the reference section that she thought would be good. And so I looked those up. And they were very, very helpful."

"How did you figure out that there probably was an encyclopedia like that that would give you the answer?"

"Oh well, because they were in the bibliography that she gave us. And because both (the librarian) and our professor said that those were good sources."

Some students preferred visual learning such as tours or viewing reference tools. Others found orientations useful for an overview of basic information and as an initial point of contact with library staff. It is critical to acknowledge the importance of differing learning styles by offering varying modes of library instruction.

Students appear to prefer independent learning that bypasses library staff. They usually have established some kind of routine for library research and only when it totally fails do they ask for help at the reference desk. When they do go to a reference desk, they seem to believe that they need to ask specific questions, eg. "What is the CD product for this subject?" rather than something more general, like "Help!".

(Student asking interviewer)"What kinds of questions would you ask at that desk? What would you think is suitable?

So you don't make it a habit of asking, either people in the library or outside the library for assistance?

"No. Not usually. Just unless I am stuck. If I couldn't find anything in the end I would probably go to library staff."

"If I can't find a particular book then I might go and I might ask them, "Do you have this?" Or, there some special code on DOBIS that I have never seen before, then I might go ask them. But usually, I find stuff on my own ."

Time was a significant factor; Many students said that they would have asked for help but it was too late. Instead of seeking assistance at the beginning of the process, they followed their established routine and, when faced with a dead end, try another strategy. By the time they had exhausted personal resources, time had run out.

"...You said you ran into some problems using the CD's. Did you ask at the reference desk for any help in using them?"

"Well, if I ran into a dead-end, I just tried something else, I didn't go and talk to (the librarian). I thought I might have to and if I had time I might have. But I was pummelled with time."

Friends played a critical role in library instruction. Eight students mentioned friends as sources of help. For four of the students, the only library instruction/help mentioned was that received from a friend.

"It was a friend who taught you (how to search CD-ROM's)?"

"...And now I have taught a friend."

"And I talked to the girl I sat beside. Probably she is the one who taught me how to use the CD-ROM."

Recommendations:

Revitalize the production of pathfinders. Printed pathfinders can address many of the issues that arise out of the reluctance of students to ask for help and their desire to be independent. Pathfinders can be placed at the point of need and in central areas of the library where students gather. They can be maintained and delivered electronically through the World Wide Web as well as in paper. Undoubtedly many more students could be reached through pathfinders than can ever be helped personally by Library staff.

Acknowledge the variety of learning styles employed by students and offer different modes of instruction; self-guided tours, pathfinders in print and on the Web, and targeted group instruction.

Emphasize the need for the reference interview to probe for broader needs that may underlie a specific question. A simple question can often mask a deeper problem the student is having finding information.

Make the variety of services being offered at reference points clearer to students. One way to do this might be better signage. The general atmosphere and impression created by a service point could also influence what questions at student thinks it is appropriate to ask.

Realize that students teach and confer with each other so that, when one is taught, the knowledge may be passed to many.

Findings:

The subject area seemed to dictate the range of tools and materials consulted by students. All but one (the least successful student) used a combination of reference material and CD- ROM searches to help them understand their general subject, fine-tune the topic and provide facts. Only three students chose a topic based solely on preliminary research. Students rarely employed a linear research process, most repeated steps, revisiting the Library several times, changing the emphasis on particular conclusions depending on availability of material, etc.

Analysis of research methods shows that even the most successful students did not follow a straight path based on an outline developed early in the process; they were much more fluid. If an outline was used, it was either mental or suggested by the arrangement of notes. The outline was often stated precisely only when the paper was in its final draft. Most students used a computer which simplifies this constant reworking of notes and ideas. Nearly half put notes directly into the computer, leaving only the necessary rearranging and polishing for the final draft.

The availability and comprehensiveness of primary access tools determine students' tendency to seek a greater variety of sources. For example, Psychology students, for whom PsychLit is such a complete source of information, were least likely to go far afield for other sources.

The students who exhibited the least amount of confusion and frustration in choosing and refining a topic were those in Geography, probably because their instructor had worked closely with the Librarian to ensure that the topics were clearly defined and that material was available. Students had freedom to choose within set parameters and could quickly choose a specific topic. About 30% of students did research on availability of materials before they committed themselves to a topic.

Recommendations:

Students and librarians need to recognize that research is not linear but is an elastic process; the most flexible and innovative students are often the most satisfied with their work. Pathfinders and instruction should emphasize the actual research processes and recognize the validity of the "scattershot" approach.

Students consult a variety of materials when trying to come to understand a subject area and determine a focus for their research. Much of this material is never used in the final paper, but its consultation was a basic part of the undergraduate research process. Because students are most often novices to the subject area they are researching they do not have the same ability to prejudge what problems or aspects of a topic are suitable for a paper or what materials will ultimately find their way into the paper. The availability of a strong undergraduate collection of reading and index tools is vital to support this aspect of undergraduate research.

We should increase awareness of the variety of tools available (through signage, on-screen help, etc.) to help students who cannot depend on a single comprehensive tool.

During the reference interview, encourage students to see each source as a possible lead to other sources.

Communicate with faculty to encourage detail and specificity in topic assignment. Discourage the "pick what you want and do it" method. Emphasize to students that topic selection and revision are a key element in a successful paper and should be expected to take a great amount of time.

This survey has been an intensive and time consuming experience. This first report has only focused on library specific aspects of the research, however, even from this we have been able to identify some significant findings.

First, students expect to be able to operate independently in the Library. System designs and support services that assume students will ask for help when they have a problem or will even be able to identify when they are having a problem will not reach this group. Personal assistance from a library staff member is a resource intensive, and in some cases unwelcome, form of service. Given staffing and resource constraints we should focus on activities that will reach the widest possible audience Our recommendation is that we produce more printed and electronic pathfinders that can be distributed to the point of need whether this is next to the library workstation, or at the student's desktop.

Second, students who do seek personal assistance in the Library often have no clear idea of what kind of service is offered. As a result, they will generally ask pointed questions concerning specific problems such as, location of an item, how to book time on CD-ROMs, or how to start a machine. As well, many students do not ask for help until they have reached a high level of frustration. It is important that we become better at communicating with students about what kinds of services we offer, and that we use even simple requests for help as an opportunity to find out about a student's real information needs.

Third, student research is a complex, recursive process that involves scanning, browsing, reading, acceptance and rejection. Unlike more experienced researchers students do not have the prior knowledge that would allow them to quickly identify a focus or evaluate the importance of material for their work. For undergraduates, in-house material is an irreplaceable resource and their needs should be the focus of our collection activities.

Fister Barbara, "The research processes of undergraduate students," Journal of Academic Librarianship 18 (July 1992): 163-169.

Fister Barbara, "Teaching the rhetorical dimensions of research," Research strategies 11 (Fall 1993): 211-219.

Nelson Jennie, "How the writing context shapes college students' strategies for writing from sources." Technical Report No. 16 (ERIC Document ED 297374, 1988).

Nelson Jennie, "This was an easy assignment: examining how students interpret academic writing tasks." Technical Report No. 43, (ERIC Document ED 334582, 1990).

Nelson Jennie, "Constructing a research paper: a study of students' goals and approaches." Technical Report No. 59, (ERIC Document ED 342019, 1992).