Home Evaluating Digital Libraries Content Evaluation

Content Evaluation

Content Evaluation

The content evaluation was processed at Graduate School of Library and Information Science (GSLIS) in the University of Illinois at Urbana-Champaign (UIUC) in 2010. The evaluation reflects many discussions and instructions of Professor McDonough, Dean Smith, and Dean Renear of GSLIS. Especially, the content evaluation was mainly done by Professor McDonough.

Purpose

The content evaluation was conducted to evaluate whether a digital library has qualified and uniquely specialized collections in a subject area. It was done emphasizing that Well-Designed Digital Libraries’ collections should be high qualified, since they will consist of IOPDL representing their countries. Professor McDonough, Dean Smith, Dean Renear and Boaz Sunyoung Jin chose carefully the following particular criteria to evaluate content collections of existing digital libraries. The content evaluation criteria include four sub-criteria.

Content Evaluation Criteria

  • Accuracy – whether collections have accurate information in the subject area that the users can trust;
  • Coverage – adequacy of the scope of the collection, considering both breadth and depth;
  • Authority – how authoritative the site appears to be, based on the reputation of the organization or sponsors; and
  • Satisfaction – experts’ overall response to the digital library’s collection.

Methodology

First of all, whether a digital library has a unique specialized collection in one of subject areas was investigated. Subject areas are drawn from the Library of Congress classification, listed below. Several similar subject areas are combined, and few subject areas are deleted from the Library of Congress Classification to simplify subject areas.

Classification based on Library of Congress classification

● Philosophy, psychology, religion
● World history and history of Europe, Asia, Africa, Australia, New Zealand, ETC
● History of the Americas
● Geography, Anthropology, Recreation
● Social sciences
● Political science, law
● Education
● Music and books on music
● Arts
● Language and literature
● Science
● Medicine
● Agriculture
● Technology
● Military science

Heuristic Method 

With the criteria, by heuristic method, Professor McDonough and Boaz Sunyoung Jin investigated whether a digital library of existing digital libraries is representative and have authority in their subject domain. In evaluating collections’ content of existing digital libraries, we put emphasis on whether each digital library satisfies accuracy, coverage, authority, and satisfaction criteria in their collections. Through the evaluation, three to seven digital libraries in each subject area of fifteen subject areas are recommended as candidates of well-designed digital libraries.

Results

Totally, sixty three digital libraries are recommended as candidates of well-designed digital libraries, as follows. One of them, ‘William Blake Archive,’ was duplicated in Language and literature, and in Art subject areas. It was evaluated in Art subject area. Thus, total recommended digital libraries were reduced as sixty two.

The recommended candidate digital libraries by Content Evaluation

● Philosophy, psychology, religion subject areas
(1) Chinese Philosophical Etext archive http://sangle.web.wesleyan.edu/etext/index.html
(2) Online Medieval & Classical Library http://omacl.org/
(3) Association of Religion Data Archives http://www.thearda.com/
(4) AhlulBayt Digital Islamic Library Project http://www.al-islam.org/index.php
(5) Philosophy Resources at Harvard http://hcl.harvard.edu/research/guides/philosophy/

● World History and history of Europe (Asia, Africa, Australia, New Zeal, and ETC)
(6) British Library Online Gallery http://www.bl.uk/onlinegallery/index.html
(7) EuroDocshttp://eudocs.lib.byu.edu/index.php/Main_Page
(8) Africa Focus: Sights and Sounds of a Continent University of Wisconsin Digital Collections http://digicoll.library.wisc.edu/AfricaFocus/

● History of the America
(9) Library of Congress: American History & Culture http://memory.loc.gov/ammem/index.html
(10) The National Archives Online Exhibits http://www.archives.gov/exhibits/index.html
(11) Documenting the American South http://docsouth.unc.edu/
(12) American Journeys http://www.americanjourneys.org/
(13) Library of Congress: Historic Newspapers http://chroniclingamerica.loc.gov/
(14) University of California Digital library- calisphere http://www.calisphere.universityofcalifornia.edu/
(15) Digital Past http://www.digitalpast.org/plugins/browse.php?type=collection

● Geography
(16) NASA’s Visible Earth http://visibleearth.nasa.gov/
(17) David Rumsey Map Collection http://www.davidrumsey.com/
(18) Census Atlas of the United States http://www.census.gov/population/www/cen2000/censusatlas/

● Social sciences
(19) Electronic Cultural Atlas Initiative http://www.ecai.org/
(20) The National Archives, Education Resources, UK http://www.nationalarchives.gov.uk/education/
(21) ICPSR http://www.icpsr.umich.edu/icpsrweb/ICPSR/access/index.jsp

● Political science, law 12
(22) THOMAS http://thomas.loc.gov/
(23) GPO Access http://www.gpoaccess.gov/index.html
(24) Civil Rights Digital Library http://crdl.usg.edu/?Welcome&Welcome
(25) Harvard Law School Library Digital Collections  http://www.law.harvard.edu/library/special/exhibits/digital/index.html

● Education
(26) The National Library of Education http://ies.ed.gov/ncee/projects/nat_ed_library.asp
(27) Exploratorium Digital Library http://www.exploratorium.edu/educate/dl.html
(28) HistoryMakershttp://www.thehistorymakers.com/
(29) SMETE digital library http://www.smete.org/smete/

● Music and books on music
(30) Database of Recorded American Music http://www.dramonline.org
(31) Sheet Music Consortium http://digital.library.ucla.edu/sheetmusic/
(32) California Sheet Music http://people.ischool.berkeley.edu/~mkduggan/neh.html
(33) E. AzaliaHackley Collection http://www.thehackley.org

● Arts
(34) NYPL Digital Gallery http://digitalgallery.nypl.org/nypldigital/index.cfm
(35) Making Sense of Modern Art http://www.sfmoma.org/pages/explore
(36) Art History Resources on the Web http://witcombe.sbc.edu/ARTHLinks4.html
(37) William Blake Archive http://www.blakearchive.org/blake/

● Language and literature
(38) ACLA (American Comparative Literature Association) http://www.acla.org/
(39) International children’s Digital Library http://en.childrenslibrary.org/
(40) Writers of the Purple Sage http://azmemory.lib.az.us/cdm4/index.php?CISOROOT=/pplwriters
(41) Southeast Asia Digital Library (SADL) http://sea.lib.niu.edu/reslanguage.html

● Science
(42) National science Digital Library http://nsdl.org/
(43) U.S. Department of Energy http://www.energy.gov/sciencetech/index.htm
(44) National Science Foundation http://www.nsf.gov/
(45) Science Photo Library http://www.sciencephoto.com/index.html
(46) Analytical sciences Digital Library (ASDL) http://www.asdlib.org/index.php
(47) Public Library of Science http://www.plosbiology.org/home.action

● Medicine
(48) National library of medicine http://www.nlm.nih.gov/
(49) AMA (American Medical Association) http://www.ama-assn.org/
(50) ecancermedicalsciencehttp://www.ecancermedicalscience.com/
(51) U.S. Department of Health & Human Services HHS.gov http://www.hhs.gov/
(52) Children’s Medical Center  http://www.childrens.com/HealthLibrary/HealthLibContent.cfm

● Agriculture
(53) National Agricultural Library Digital Repository http://naldr.nal.usda.gov/
(54) Core Historical Literature of Agriculture in Cornell University http://chla.library.cornell.edu/
(55) Western Waters Digital Library http://www.westernwaters.org/

● Technology
(56) National Institute of Standards and Technology http://www.nist.gov/index.html
(57) Dspace@MIT http://dspace.mit.edu/
(58) JSC Digital Image Collection http://images.jsc.nasa.gov/index.html
(59) IEEE computer society http://www.computer.org/portal/web/csdl13

● Military science
(60) Military History and Military Science of The Library of Congress http://www.loc.gov/rr/main/alcove9/military/military.html
(61) US Military Academy Digital Collections http://digital-library.usma.edu/collections/
(62) United States Department of Defense http://www.defense.gov/

Analyses and Discussion

Many existing digital libraries are investigated in fifteen subject areas, regarding to their collections’ accuracy, authority, scope, and overall satisfaction. Through the content evaluation by heuristic method, some important facts are discovered.

Overall, national digital libraries in the U.S. show very high quality and quantitative collections in accuracy, authority, breadth and depth scope, and satisfaction. Although many digital libraries exist, some digital libraries have not uniquely specialized collections in any subject areas. Some digital libraries cannot prove enough quality or quantitative collections in a subject area(s). Especially, private digital libraries seem not to have enough quantitative collections. Nevertheless, few private digital libraries (e.g., IEEE, etc.) show high quality and quantitative content collections in their domains. But, they are limited for evaluators to access by authority and fee limitations.

The biggest challenge in the content quality evaluation is limitations in accessing to digital libraries and their collections.

Effective Evaluation and Challenges

Generally, the content quality evaluation of existing digital libraries was efficient to discover high quality collections in fifteen subject areas for a short time. We found effectively and intensively the candidate digital libraries with four sub-criteria.

The four sub-criteria (accuracy, authority, scope, and overall satisfaction) are appropriate to evaluate overall quality of collections. Further, the content evaluation may present a certain way to evaluate content quality of existing national digital libraries all over the world.

But, there are few shortages in the content evaluation of this prototype. We may not find all quality digital libraries in the U.S. There must be better digital libraries in fifteen subject areas that were not included in the candidate digital libraries. Especially, private digital libraries that have small collections tend to be excluded in the recommended digital libraries, by the limitation of access.

Also, if more experts involved in evaluating content quality in their subject domains, more detailed investigations could have been done, and more undiscovered digital libraries might have been found.

For the future content evaluations, above all, evlauators should recognize how important the content evaluation is.

Impact

Finally, the evaluation results can demonstrate which digital libraries can be representative digital libraries in a subject domain(s). The results of the content evaluation may point out and address some problems of existing digital libraries in their content/collections. Through the content evaluation results, existing digital libraries can be encouraged to develop their unique specialized collections further. It will be good for the digital libraries to improve their quality generally, and for users to take the advantages from them.

Comments are closed.