Comparing statistical information

In the beginning of the project we collected and shared plenty of statistical information about our libraries and universities. The plan was to compare the activities and results. Areas were

  • Library areas, facilities and equipment
  • Services for the public, including loan, ILL and user training
  • Collection management, bibliographic records
  • Institutional repository
  • Library staff, both number and staff training
  • Financial data

Statistical data will give more value when comparing with others or with oneself, over time, but which statistics can be compared? One example is loans. Number of loans is easy to compare, and the numbers can be extracted from the library systems. For 2013 we have the following statistics for loans, visits to the libraries and size of collections. But we miss important data, e.g. on downloads of articles, and use of e-books.

bench-fig1

*) Include renewals, for NTNU is for example the number for first time loan is about 50%.

**) Apply to the whole institutions, not just the medical libraries. Medicine has focus on articles more than books, so these numbers are not valid for medicine.

How to compare?

We can observe that NTNU and UCL have quite similar number of loans, and almost twice as much as UEF, at the same time  NTNU and UCL have almost twice as many visits as UEF. And the inter library loan at NTNU is three times as high as at UCL. Can we see any correlation at all? It is also easy to compare interlibrary loans. Use of collections in medical libraries tend to have a predominance on articles, at the same time prices of electronic journals are increasing more then price of books. This mean that no library are able to have all the journals needed in their collection. So ILL can say something about the quality of the library collection – and also about the size of the media budget. An example: the NTNU Library (BMH) use about 2.5 % of the media budget on buying copies, a very small amount when more than 90% of the budget is used on electronic resources and journals.

Many element affect the statistics and the use of library services. Number of loans must be seen in relation to the size of the universities. The NTNU part of BMH serves 3000 students and 11000 staff members at NTNU and St. Olavs Hospital. UCL serves about 6000 medical students and 760 academic and research personnel, and the UEF Library serves a total of 3000 university staff members and 15 000 students (about 1400 medical and dental students). Other elements affecting statistics are size of the collection, how updated is the collection, acquisitions per year, number of users, the amount of e-books and e-journals, the number of printed books replaced by e-books and so on.

Should we be able to compare, we must use indicators; this will be discussed  in a separate blog post. Examples of other useful indicators could be

  • Number of e-journals / download of articles
  • Number of e-books / number of pages read
  • Loans from library collection / ILL
  • Loans from library collection / number of students

As an example we can calculate the relation between visits to the library and loans. NTNU have 0.42 loans per visit, while at UCL and UEF are respectively 0.17 and 0.1 loans per visit. If this is done for several years, one can get a picture of how user activity in the library develops over a period of time. And then it also gives more meaning to comparing libraries.

Follow the money

When we started collecting data, we did not know how to compare. We have lots of data, but not necessarily the most interesting or useful data. It became clear that though we all three are medical/health sciences libraries that serve both faculty and university hospital and also other users, we are neither organized nor financed in the same way. Due to these differences  it is difficult to compare economic data. Yet, it would be useful, and the library directors are keen to compare both financial and other data. While visiting the three libraries, we had discussions with the  library directors and got suggestions for further work on statistics and data. In Trondheim we were encouraged to measure the impact of the library, and to look at the connections between quantitative and qualitative indicators. In Louvain-la-Neuve we talked about library statistics and economics, and that it is important to seek out at least some comparable indicators nationally and internationally. In Kuopio we discussed statistics and other data as useful background information.

Next step for our project is to find indicators for library performance. More on this topic in another blog post.

Advertisements

Focus! — said the focus group

Ten delegates of EAHIL+ICAHIS+ICLC 2015 workshop participated in our focus group interview session in Edinburgh. Focus group is a qualitative method, the results cannot be generalized to a population (e.g. all participants at the EAHIL workshop), but can say something about trends and point out features and patterns that recur. In our session, we looked for perceptions, opinions, beliefs, and attitudes about some themes we have covered in our benchmarking project.

The focus group participants were Bente Aastad (Telemark University College, Porsgrunn), Elaine Garrett (Royal College of Obstetricians and Gynaecologists London), Imrana Ghumra (Cilip Health Libraries Group London), Hans Ket (University Medical Center Amsterdam),Marie Källberg (Karolinska University Hospital in Stockholm), Joëlle Longchamp Reuge (Haute école de travail social et de la santé in Lausanne), Emma-Lotta Säätelä (Biomedical- and Medical libraries at Uppsala University), and Guus van den Brekel (University Medical Center Groningen), and, as moderators, Karen J. Buset & Tuulevi Ovaska.

We talked about the challenges of data comparison, our site visits, marketing, library as a place, and — most importantly — how the project should proceed. We took notes but also asked the participants to put a word or two on post-it notes.

Post-it notes

Post-it notes


Preparation for the session 

We used a semi-structured interview guide to structures the discussion and secure that all topics were covered and that all of the participants had the opportunity to express their opinion. We made notes but also asked to the participants put a word or two on post-it notes. In the blog we asked the people attending our workshop session, to themselves very shortly (name, organization, main tasks) by commenting the blog post. Some of them also explain why they chose to attend this session:

  • … interested in focus groups as a way of identifying our user needs, and the benchmarking process because I have seen on recent visits to other libraries how much we can learn from each other
  • … it is important to do the right thing at the hospital, using benchmarking and focus groups are two different methods to learn what the library can do to keep updated and preparing for the future
  • … interested in hearing what you have learnt in your benchmarking … it will help our planning to learn what you found important during your project.
  • … hope to hear more about benchmarking in general …
  • … benchmarking to find out how to improve some of our services, for example : the marketing …
  • … interested in the question about site visits and the library as a physical space

The workshop started with a brief summary on benchmarking and focus group as methods, and we also explained to the group the intention of the workshop was not to teach them something about benchmarking, but to get input and ideas for our future work.

Topic 1: Data comparison

The discussion was mostly on library quality assurance in different countries, for example the Quality assurance framework in England, that provides tools for best practices, that we think we should check out. The focus group agreed that using indicators probably is the best way forward, but the big question is, which data to compare: number of books and e-books, number of loans…? They advised us to to go back to our data and start figuring out what we want to measure. They asked, if we should also compare our library staff numbers and competencies, and would it be possible to measure progress of services.

Post-It notes from the participants

Focus group tools

Focus group tools

Topic 2: Site visits

The group agreed that site visits are useful but take time from other activities, as well as money for traveling. They agreed that interviewing the students in all the three libraries has given us useful information but that we should also think how to get in touch with those who are not visiting the library. It was agreed that there are challenges in making visits and comparisons among libraries and their users in several countries, for example in our case three different languages (in addition to English as the common language) and the different (working) cultures of our countries. We think there is more information and value — and more to learn — in comparing with someone different from your own country.

Sidetracks from the discussion

  • EAHIL should engage in such projects
  • Libraries are not good at publishing and sharing results.
    • Could there be a performance parameter on “How good are the libraries in sharing information about their projects”?

Post-It notes from the participants

  • What about non-users?
  • Staff not always open to be interviewed
  • Language and cultural barriers
  • How do we share information?
  • Books and shelves
  • Furnishing
  • Number of study desks and PCs

Focus group

Focus group

Topic 3: Marketing

We talked about good examples like the Biblio-Jack project in Brussels. Libraries and librarians need skills and more knowledge on marketing; how to do it, what works out…. IT could be useful to collaborate and share marketing material, as an example we learned about the common hospital library week in Sweden. It was suggested that EAHIL should have a new special interest group for people interested in marketing so there could be more cooperation in this field. It was also asked if it is possible to measure or have indicators of the impact of marketing. The group thought marketing could be one of the main focuses of our benchmarking project.

Post-It notes from the participants

  • Targeting different user groups
  • Video Biblio-Jack
  • Core skills for librarians?
  • Share stories, examples, material, experiences e.g. promoting libraries in education
  • Small level marketing projects

Focus group

Focus group

Topic 4: Library as a place

Library as a place was discussed earlier in connection with the site visits and student/customer interviews.

Post-It notes from the participants

  • Find the people that do not use the library anymore, maybe clinicians and researchers or nurses
  • Re-zoning university libraries in UK — also health libraries

How should we proceed?

The group suggested that we figure out what we want to measure at this point, that we use indicators, that we should compare staff and — most importantly — that we must focus on fewer topics. We are grateful to the group for the discussion, the tips and the encouragement.