91´«Ã½

Sara J. Penhale, Nancy Taylor, and Thomas G. Kirk, Jr.
Lilly Library, Earlham College, Richmond Indiana

ABSTRACT

Bibliographic instruction at Earlham College has developed over the last thirty years into a program of course-integrated instruction extending throughout the college curriculum. We have developed a method for measuring the effectiveness of our program in reaching all students. This method is based on data available to most librarians at their home institutions: student transcripts for graduates, the college catalog, and library records of bibliographic instruction. We will illustrate how this data can be used to reveal patterns of bibliographic instruction received by members of a graduating class and how this type of analysis can suggest strategies for improving an instruction program.

Introduction

Earlham College, a small private liberal arts college, has an extensive bibliographic instruction program of course-integrated instruction extending throughout the college curriculum.(1) This program has developed gradually over the last thirty years through the efforts of the library staff who have worked closely with teaching faculty in developing course assignments requiring the use of the library.

All associated with the program -- librarians, faculty, and students -- have expressed satisfaction with the contribution that bibliographic instruction has made to the students' overall academic experience at the college. Formal assessment, however, has been limited. Nevertheless, two recent developments -- changing accreditation procedures and the college's decision to switch from a three term academic calendar to a semester system -- provided an impetus for the library staff to develop a measure of the overall reach of the program. We felt this would be a useful frame of reference from which to base the efforts we would make to ensure that course-integrated bibliographic instruction remained a major feature in the revised academic program of the college. It would also provide the foundation for an ongoing process of program assessment.

From our work with faculty, we already had a clear picture of which faculty and which departments relied heavily on bibliographic instruction for their students. This is one measure of the reach of the program. But we had never looked at the program from the point of view of the average student. How much bibliographic instruction does an individual Earlham student receive by the time he or she graduates? Is it guaranteed that all students will, in fact, receive instruction? If so, is it from instruction provided in general education courses, or in the courses for a major? And do all our majors have a similar experience? These are the sorts of questions that we wanted to answer.

We realized that we already had at hand the data that could answer these and many other questions. Our sources of data were threefold: student transcripts, the college catalog, and the library's own records for bibliographic instruction. Combining these sources of data would allow us to determine how much bibliographic instruction, and for which types of courses, our students received.

Since these sources of data are accessible to many librarians at their home institutions, we present our analysis as an example of one way that librarians can measure the reach of their own instruction programs.

Methods

Our analysis is based on the graduating class of 1993. The transcripts of each of the graduates were available from the Registrar's Office. These computerized records were transferred into FileMaker Pro, a database management program. The data for each graduate included the student's major and all of the course credits he or she received from Earlham College. Each course credit was distinguished by its course number, course title, professor, and term/year it was offered.

Under Earlham's three term calendar, students were expected to take three courses each term and each course was worth one credit. Thus, students needed to complete 36 credits for graduation. These included the required courses for a major, plus a set of general education courses: three language credits or the equivalent of one year of language acquisition, four humanities credits, four science credits, two social science credits, two religion or philosophy credits, and one fine arts credit. Students often took more than 36 credits.

The second source of information was the college catalog. This provided records of which courses fulfilled the various general education requirements, as well as the requirements for each major. Each course in the database was coded according to whether or not it fulfilled a general education requirement and according to whether or not it was required by a given major.

The third source of information was the library's records of bibliographic instruction offered during that time period. Our records indicated which of those courses received instruction.

Based on our reference service, however, we knew that faculty did not always request bibliographic instruction for each course which had a library-based assignment. Thus, in order to gain a sense of overall library use, we decided to supplement the library's records by surveying the teaching faculty. Each faculty member was sent a letter that explained the purposes of our study and included of list of courses they had offered to this class of graduates. They were asked to note, to the best of their memory, which courses received bibliographic instruction, which had a library assignment but no bibliographic instruction, and which courses had no library use at all. We combined faculty responses with our own records.

This information on library use was added to the database holding student transcripts. Each course taken by each student was coded for bibliographic instruction, for library use without bibliographic instruction, or for no library use at all.

We omitted from our analysis all courses that fit into the following categories: performance-based courses (physical education, music lessons, etc.), individually earned credits (independent studies, internships, etc.), and off-campus courses. These courses were omitted because they are ones for which no library instruction would reasonably be expected.

Not all of the 1993 graduates spent four years at Earlham College. Some student took longer than four years to graduate. These students were included in the analysis although transfer students who took less than 1/2 of their credits on campus were excluded. When analyzing data by departmental major, double majors were included in each department, but combined majors -- students who take only a portion of the requirements for several departments -- were excluded. Students majoring in interdisciplinary programs specific to Earlham College were also excluded from the departmental breakdowns presented here.

There were three parts to our analysis. We looked at the overall list of courses offered to these graduates and the distribution of BI and library use throughout. We looked at the experience of the average Earlham student, based on the transcripts of all of the graduates. Finally, we looked at the average experience of a student majoring in each specific discipline, based on the transcripts of those majors alone.

We used FileMaker Pro to sort the records and determine the average values of a number of variables. We also used Microsoft Excel, a spreadsheet program, to determine percentages for these variables. All averages and percentages were rounded to the nearest whole number.

Results

Library use is high at Earlham College [see Table 1]. Students used the library in 51% of the courses offered to the 1993 graduating class. The level of bibliographic instruction supporting this work is also high: 67 % of the courses requiring the use of the library also included bibliographic instruction. This instruction was spread throughout the curriculum [see Table 2]. In some departments, the amount of BI was remarkably high: 90% of the Political Science courses had BI, as did 82% of the History Department courses. At the other extreme were the departments of Math and Physics which each had only one course with BI.

Bibliographic instruction is well represented in the general education courses offered at the college [see Table 3]. Of the courses designated as fulfilling the general education requirement, 37% have bibliographic instruction. The Humanities and Social Science courses contribute a much larger percentage of general education courses with BI than do the Fine Arts, Religion and Philosophy, or Science courses.

The impact of this curricular pattern on the individual is significant [see Table 4]. The average graduate used the library in 54% of his or her courses and had bibliographic instruction in 12 courses, or 37% of all the courses they completed at the college. The range was considerable: the student with the most instruction had BI in 22 courses while the student with the least instruction had BI in four courses. A high proportion of the instruction, 49%, was in the general education courses taken by the average student.

The patterns for students majoring in different disciplines were quite variable [see Table 5]. First, there was a range among majors from different departments in the percentages of their overall course load that were courses with bibliographic instruction. The highest was the departments of History, where the average major had BI in 52% of all the courses he or she took at Earlham, followed by German majors with 50%, and Political Science majors with 48%. The bottom three departments were Math, where the average major had BI in only 21% of his or her classes, plus Physics majors with 20% and Geology majors with 18%.

Second, there were differences among majors in the average percentage of the courses taken in their major department that were courses with bibliographic instruction [see Table 6]. The highest average percentages were found for majors in the departments of History (90%), Political Science (80%), and English (73%). At the opposite end of the spectrum were students from the departments of Math (0%), Physics (0%), and Spanish (0%) where none of the majors had BI in their disciplinary courses.

Discussion

A central goal of the Earlham College libraries is to provide a program of course-integrated instruction and reference service that will help prepare students to be effective life-long learners. Objectives corresponding to that goal include a number that relate to the importance of integrating instruction into the curriculum. These objectives include the following:

1. that students have course-related library experiences through their Earlham career and that bibliographic instruction be provided where appropriate,
2. that students have exposure to the literature of their major discipline and instruction in accessing that literature,
3. that the amount and placement of instruction within the curriculum be consistent with the information needs of the student.

Since the data collected in the present analysis are a reflection of our efforts to achieve these objectives, we can use this analysis as one means of assessing our bibliographic instruction program. A careful examination of the data shows us whether each objective is being met, and it also provides a picture of how the objective is being met. Understanding the factors that contribute to fulfilling an objective is useful for generating ideas on how to improve those aspects of our program needing attention.

Because this paper is designed to illustrate the use of our methodology for measuring the reach of a bibliographic instruction program rather than to provide an exhaustive analysis of what we do at Earlham, we will limit our discussion to a small subset of the data and show how it can be used to assess our effectiveness in meeting the objectives noted above.

Exposure to the Library and Bibliographic Instruction

One way of assuring that all students have some exposure to the library and bibliographic instruction is to integrate it into a required part of the curriculum. For the 1993 graduates of Earlham College, there were three specific courses -- Humanities I, II, and III -- that were required of all graduates. Humanities I had BI and Humanities III had a library assignment without BI. A fourth course -- Humanities IV -- was also required of graduates, but there were multiple versions of this course taught by different professors and while most of these courses had BI, not all did. Thus, all graduates of the college have at least one exposure to instruction in the use of the library, and probably two, due to the Humanities requirement alone. Further, because Humanities I was taken in the fall semester of a student's first year, librarians could build upon this introduction when they provide instruction in subsequent courses.

As the college moved to the semester system, we worked with the Humanities faculty as they redesigned the Humanities sequence, hoping not to lose this crucial BI entry point. In fact, the revised sequence of three courses -- Humanities A, B, and C -- allows for somewhat more library experience since there are library assignments with bibiliographic instruction in Humanities A and C, and brief library assignment without instruction in Humanities B.

The amount of bibliographic instruction received by the graduates was considerably higher than that which would be guaranteed by the required Humanities courses. Two factors appear to be responsible: first, that Earlham has a substantial required general education component to its curriculum -- 36% of the total number of credits required for graduation must be credits from general education courses; and second, that a high proportion of these courses -- 37% -- have bibliographic instruction. This pattern is a particularly significant one for the objective of reaching all students because it means that all students learn to use the library, even if they major in disciplines at the college where very little library use is expected.

As we move into the semester system, we have been monitoring the frequency of bibliographic instruction in general educatioin fulfilling courses. At the end of our first year, 38% of the general education fulfilling courses taught included BI. Apparently, the reliance on BI in general education courses exhibited under the former term system has been preserved in our new curriculum.

Bibliographic Instruction in the Literature of a Student's Major Discipline

Our objective that students have training in the use of the literature of their disciplines has not been met for all the departments at the college. Those distinguishing themselves by their lack of library use are Mathematics and Physics. The faculty explain that the literature of these disciplines is intellectually inaccessible to undergraduates, and perhaps we should simply accept this explanation. On the other hand, the prevalence of library use everywhere else at the college, even in other science departments where faculty might make similar claims about the difficulty of their literature, suggests that we should continue to make occasional overtures to the Math and Physics faculty.

Level and Placement of Instruction Throughout the Curriculum

We are concerned about more than just inserting a bibliographic instruction experience into each student's major department. We also aim to provide instruction that is consistent with the information needs of students working in each discipline. Thus, it is important to look at those departments where there is a low percentage of courses with BI among those courses in which students are using the library. For instance, the Spanish Department only has BI in 11% of its courses where students use the library and the Music Department only has 7%. Here we need to consider whether these students are prepared to do the library work without additional instruction.

While we are used to thinking in terms of whether or not we are providing too little instruction in a given area, the question of whether or not there can be too much instruction also came to our minds as a result of this analysis. Do Political Science majors really need BI in 90% of the courses offered by the department? Do History majors really need BI in 82% of their departmental courses? A preliminary consideration suggests reasons for these high levels of BI. In the case of the Political Science Department, two of the three faculty members are particularly enthusiastic about library assignments -- one gives them in every one of the courses she teaches, large or small -- and these faculty continue to develop innovative new approaches, leading the students to use different types of literature. In the History Department, the high level of BI seems to relate more to the breadth of subject coverage in the discipline and the need to us a variety of different reference sources rather than to the tendencies of just onr or two faculty members. Nevertheless, if these patterns persist under the new semester system, it would be useful to approach faculty and students in the department for a consideration of whether some of the bibliographic instruction is redundant or overly repetitive.

Strengths and Weaknesses of this Measure of a BI Program

As we consider this analysis of student transcripts as a measure of the reach of a bibliographic instruction program, it is important to assess the strengths and weaknesses of this approach.

The drawbacks fall into three general domains. First, data collection and analysis. While students transcripts and records of bibliographic instruction are generally available, if one wishes to collect data on library use without instruction, it is necessary to survey faculty and rely on their memories. Then the data must be manipulated in order to create a reasonable picture of the reach of the BI program. For instance, we did not include independent study credits in our analysis because reference service, rather than BI, would be expected for those students. And we counted double majors in each department even though their overall curricula would certainly not be typical of the other students within each of those majors. Finally, learning to use the software package for the data analysis proved to be a real challenge for the library staff.

The second area of limitation relates to the accuracy and precision of this approach. Even though this method provides a quantitative analysis of a bibliographic instruction program, it is still an approximation rather than an exact measure. While we do not have reason to doubt the accuracy of the transcripts and the bibliographic instruction records, we are less certain about the data on library use. A few faculty did not respond to the survey and we have no way of ascertaining the accuracy of the memories of those who did. Also, the many decisions we made with regard to which data to include and how to analyse the data means that we have created a picture of the reach of our program, but it is certainly not the only picture that could have been created.

This second area of limitation affects the validity of using this approach to make comparative judgements. Given the differences among academic disciplines and the variety of types of courses in different departments, it would be unwise to rely on this data alone to make definitive comparisons among departments. Further, with the variability among different colleges and universities, it would be particularly unwise to make strict comparisons across different institutions.

Finally, this approach provides only a measure of the reach of a bibliographic instruction program. How much, and in which courses, is instruction is provided and how much do students receive. It does not assess the effectivenss of each particular bibliographic instruction session nor provide a measure of the long term impact of having received instruction.

In spite of these drawbacks, there are a number of factors which make this use of student transcripts an attractive approach for analyzing a BI program. The first relates again to data collection and analysis. If you restrict the analysis to BI rather than library use, all the data are probably already available to most librarians and it is not necessary to develop and administer surveys or other instruments.

Second, it can lead to insights that one may not have reached in other ways. For instance, we would not have predicted that among general education courses, religion and philosophy courses would have had as low a level of BI as did the science courses.

Finally, it provides a different frame of reference from which to develop new objectives for a BI program and new ways of approaching faculty. Following from the previous example, we might attempt to raise overall student exposure to BI by integrating more of it into religion and philosophy general education courses. We could also use the analysis as a means of opening a discussion with those departments. It would be a way of having a group of faculty collectively address an issue presented from an institutional perspective by asking -- for example -- do the Religion faculty think that students not majoring in their discipline should develop skills in finding information in that field so that they could continue to learn own their own later? If so, how could the department address this issue? This seems like more promising approach than pointing a finger at particular faculty members who have not asked for library instruction for their students before.

Conclusion

These examples illustrate how student transcripts can be used for measuring the reach of a bibliographic instruction program. This type of analysis clarifies whether all students are reached and reveals which components of a BI program have the biggest overall impact. It also can reveal areas in the curriculum where expansion may be needed and it provides a frame of reference from which to approach faculty.

Finally, measuring the reach of a BI program may be combined with other evaluation methods to provide a more comprehensive assessment of a bibliographic instruction program. At Earlham College, we intend to survey the 1993 graduates this summer -- at the five-year point after their graduation; asking questions about attitudes and behaviors with regard to ongoing library use and information access. We hope that this will allow us to gain a sense of the longer term impact of the bibliographic instruction experiences highlighted in the present study.

Acknowledgments

We would like to thank those who provided technical support for this project, Michael Bowden and Francesca Lane Rasmus; Shirley Smith, who patiently executed much of the data analysis; Christine Larson for data interpretation; and the Earlham College faculty who are our partners in bibliographic instruction.

NOTES

1. Hardesty, Larry, Jamie Hastreiter, and David Hendeson, eds. 1993. Bibliographic instruction in practice: A tribute to the legacy of Evan Ira Farber. Ann Arbor, Michigan: The Pierian Press.

BACK

BACK

BACK

BACK

BACK

BACK