Number of co-authors:37
Number of publications with 3 favourite co-authors:Susan L. Siegfried:Michael V. Sullivan:Dorothy Wippern:
Christine L. Borgman's 3 most productive colleagues in number of publications:W. Bruce Croft:124Susan Dumais:74Nicholas J. Belkin:45
go to course
Psychology of Interaction Design: The Ultimate Guide
Starts tomorrow LAST CALL!
go to course
User-Centred Design - Module 3
90% booked. Starts in 5 days
Marc Hassenzahl explains the fascinating concept of User Experience and Experience Design. Commentaries by Don Norman, Eric Reiss, Mark Blythe, and Whitney Hess
User Experience and Experience Design !
Our Latest Books
The Social Design of Technical Systems: Building technologies for communities. 2nd Edition
by Brian Whitworth and Adnan Ahmad
Gamification at Work: Designing Engaging Business Software
by Janaki Mythily Kumar and Mario Herger
The Social Design of Technical Systems: Building technologies for communities
by Brian Whitworth and Adnan Ahmad
The Encyclopedia of Human-Computer Interaction, 2nd Ed.
by Mads Soegaard and Rikke Friis Dam
Christine L. Borgman
Publications by Christine L. Borgman (bibliography)
Wynholds, Laura A., Wallis, Jillian C., Borgman, Christine L., Sands, Ashley and Traweek, Sharon (2012): Data, data use, and scientific inquiry: two case studies of data practices. In: JCDL12 Proceedings of the 2012 Joint International Conference on Digital Libraries 2012. pp. 19-22. http://dx.doi.org/10.1145/2232817.2232822
Data are proliferating far faster than they can be captured, managed, or stored. What types of data are most likely to be used and reused, by whom, and for what purposes? Answers to these questions will inform information policy and the design of digital libraries. We report findings from semi-structured interviews and field observations to investigate characteristics of data use and reuse and how those characteristics vary within and between scientific communities. The two communities studied are researchers at the Center for Embedded Network Sensing (CENS) and users of the Sloan Digital Sky Survey (SDSS) data. The data practices of CENS and SDSS researchers have implications for data curation, system evaluation, and policy. Some data that are important to the conduct of research are not viewed as sufficiently valuable to keep. Other data of great value may not be mentioned or cited, because those data serve only as background to a given investigation. Metrics to assess the value of documents do not map well to data.
© All rights reserved Wynholds et al. and/or ACM Press
Wynholds, Laura, Fearon, David S., Borgman, Christine L. and Traweek, Sharon (2011): When use cases are not useful: data practices, astronomy, and digital libraries. In: JCDL11 Proceedings of the 2010 Joint International Conference on Digital Libraries 2011. pp. 383-386. http://dx.doi.org/10.1145/1998076.1998146
As science becomes more dependent upon digital data, the need for data curation and for data digital libraries becomes more urgent. Questions remain about what researchers consider to be their data, their criteria for selecting and trusting data, and their orientation to data challenges. This paper reports findings from the first 18 months of research on astronomy data practices from the Data Conservancy. Initial findings suggest that issues for data production, use, preservation, and sharing revolve around factors that rarely are accommodated in use cases for digital library system design including trust in data, funding structures, communication channels, and perceptions of scientific value.
© All rights reserved Wynholds et al. and/or their publisher
Mayernik, Matthew S., Batcheller, Archer L. and Borgman, Christine L. (2011): How institutional factors influence the creation of scientific metadata. In: Proceedings of the 2011 iConference 2011. pp. 417-425. http://dx.doi.org/10.1145/1940761.1940818
Access to high volumes of digital data offer researchers in all disciplines the possibility to ask new kinds of questions using computational methods. Burgeoning digital data collections, however, challenge established data management and analysis methods. Data management is a multi-pronged institutionalized effort, spanning technology, policies, metadata, and everyday data practices. In this paper, we focus on the last two components: metadata and everyday data practices. We demonstrate how "frictions" arise in creating and managing metadata. These include standardization frictions, temporal frictions, data sharing frictions, and frictions related to the availability of human support. Through an illustration of these frictions in case studies of three large, distributed, collaborative science projects, we show how the degree of metadata institutionalization can strongly influence data management needs and practices.
© All rights reserved Mayernik et al. and/or ACM Press
Wynholds, Laura, Fearon, David, Borgman, Christine L. and Traweek, Sharon (2011): Awash in stardust: data practices in astronomy. In: Proceedings of the 2011 iConference 2011. pp. 802-804. http://dx.doi.org/10.1145/1940761.1940912
One of several major research initiatives into the grand challenge of data curation, the Data Conservancy (DC), funded by the National Science Foundation"s DataNet Initiative, is investigating data use, sharing, and preservation in multiple fields of science. Our group at the University of California, Los Angeles is conducting a deep case study of astronomy and astrophysics. DC partners at Cornell, Illinois, the National Center for Atmospheric Research, and the National Snow and Ice Data Center are studying data practices in several other science domains. The DC is a collaborative multi-sited research project that will offer new insights into data practices in an array of physical and life sciences. The mandate of the project is to "research, design, implement, deploy and sustain data curation infrastructure for cross-disciplinary discovery with an emphasis on observational data." . This poster will summarize findings from the first year of UCLA"s research on astronomers and astronomy data. Our approach to studying data practices is complementary to that of our DC project partners, most of whom are surveying a broader set of fields less deeply. The UCLA team is part of Data Conservancy information science and computer science (IS/CS) team, which will share methods and findings. Our overall goal is to compare comparative data practices and data curation requirements across a range of physical and life science fields. Astronomy is considered to be at the forefront of data-driven science. Hanisch and Quinn, in explaining the development of the Virtual Observatory, wrote, "Astronomy faces a data avalanche. Breakthroughs in telescope, detector, and computer technology allow astronomical instruments to produce terabytes of images and catalogs...These technological developments will fundamentally change the way astronomy is done. These changes will have dramatic effects on the sociology of astronomy itself." . Over the course of the last ten years, astronomy data projects have grown from terabyte scales to petabyte scales, and the data deluge has affected many more sciences, large and small. Long predicted by the science community , not only has Nature, a premier science journal, published feature sections on "big data"  so have Wired Magazine , and the Economist . However, significant tensions surrounding big data projects are present in the field, as expressed by two Nature editors: "Astronomy is in an era of unprecedented change...more and more astronomy papers are showing evidence that familiarity with the essential "dirtiness" of data and models is being lost. ...Worries that the centuries-old culture of astronomy is being eroded have been voiced in the community for several years, especially in cosmology where the big-science approach now dominates."  Data curation of these complex digital objects presents a significant challenge facing both scientific research and scholarly record keeping institutions. Bowker and Star  argued that of the problems of aggregating data within an information system are reflective of the sociotechnical systems that yielded the data. Following that argument, the quest to build repositories for data becomes largely a quest to fold the practices of an established community into evolving technological solutions. Thus it is essential to study the data practices of communities whose data is to be curated. Astronomy is a rich domain in which to study data practices, and the Data Conservancy offers a diverse environment in which to compare data curation challenges across the sciences. We approach astronomy data practices with three questions: 1. What are the data management, curation, and sharing practices of astronomers and astronomy data centers, and how have they developed? 2. Who uses what data when, with whom, and why? 3. What data are most important to curate, how, for whom, and for what purposes? The first question focuses on what people do, how they manage data, and what counts as relevant research data to generate, use, keep, and discard. The second question addresses the social contexts, networks, and communities within which these practices occur. The third question focuses on specific aspects of data curation, such as deciding what data will be of future use to others, assigning responsibilities for organizing and describing datasets for use, identifying incentives and disincentives for individuals or groups to curate their data, and developing tools and services necessary to exploit those data. At the core of our astronomy case study is an analysis of the large sky surveys, as these generate massive amounts of data that fuel both inquiry and the tensions outlined above. The first year of the project has been concerned with capturing a broad perspective of the empirical and theoretical research that can be accomplished with astronomical observations, comparing data activities associated with sky surveys to other types of inquiry. Our starting point has been the Sloan Digital Sky Survey (SDSS) , which began data capture in 2000 and recently completed its final data release of the SDSS-II project. This groundbreaking optical survey telescope and accompanying digital dataset provides distributed access to data for one quarter of the sky. We are studying the development, practices and challenges of data management and curation in the SDSS, as well as the project"s impact on astronomy. Our study of subsequent sky survey projects, such as PanSTARRS  and LSST , will offer insights to the role and value of synoptic surveys in physical science research. Our methods follow from our three research questions about data practices, social contexts, and curation requirements in these astronomy settings: 1. Examining data practices through qualitative ethnography, including in-depth interviews and site observations; and 2. Mapping the social context of projects by analyzing documents about projects and their history, and people"s networks of professional affiliations and research activities. Within the context of qualitative ethnographies, we are interviewing people who have worked in multiple roles in sky surveys and who use sky survey data in their own research. These interviewees include software developers, university faculty, postdocs, and other researchers using data from networked astrophysical instruments. We are comparing the range of curation requirements for managing large-scale archives and smaller collections of research data. We are examining the extensive documentation of the SDSS project, including an archived listserv discussion group of its builders and users. Our initial fieldwork on astronomy sites has found broad differences in curation practices and requirements between projects, data centers, academic collaborations, and domains of research. Identifying generalizable comparisons is a core challenge. We see historical and cultural changes at large and small levels, including the professionalization of data management and the role of informatics in astronomy. Adoption of computational approaches to knowledge discovery appears uneven across the astronomy community. Science-driven research has exhibited tensions with computer engineering approaches to data archives, according to some of our respondents. We are seeing considerable variation in the use of sky surveys, from scientific inquiry to calibration of other instruments. In conjunction with a considerable variation in use, we see significant diversity in what counts as data among those studying each wavelength, and between observational and theoretical approaches. Among the interviewed theoretical astrophysicists who rely on computational modeling, some archive the results of simulations, while others retain the algorithms but discard the data generated by simulations. Data archiving practices for sky surveys appear to vary widely by wavelength, partially due to differences in data volume, format and complexity. Similarly, astronomy data use may be further divided by practices of ground-based versus space-based instruments. Data practices and data curation requirements within astronomy are far less homogeneous than they may appear from the outside. Similarly, the computation- and data-intensive methods that characterize modern astronomical research are not embraced universally. Our poster will compare our initial results to those of our Data Conservancy partners' analyses of data practices in other science domains. We may see similar practices of data management and preservation practices among fields; however, early reports by DC partners at Illinois show "no field-wide norms" for sharing data among the researchers they interviewed, and diverse use of data repositories even within a research field.  Data practices appear to vary widely within disciplines in the physical and life sciences, and even more so between them.
© All rights reserved Wynholds et al. and/or ACM Press
Wallis, Jillian C., Mayernik, Matthew S., Borgman, Christine L. and Pepe, Alberto (2010): Digital libraries for scientific data discovery and reuse: from vision to practical reality. In: JCDL10 Proceedings of the 2010 Joint International Conference on Digital Libraries 2010. pp. 333-340. http://doi.acm.org/10.1145/1816123.1816173
Science and technology research is becoming not only more distributed and collaborative, but more highly instrumented. Digital libraries provide a means to capture, manage, and access the data deluge that results from these research enterprises. We have conducted research on data practices and participated in developing data management services for the Center for Embedded Networked Sensing since its founding in 2002 as a National Science Foundation Science and Technology Center. Over the course of eight years, our digital library strategy has shifted dramatically in response to changing technologies, practices, and policies. We report on the development of several DL systems and on the lessons learned, which include the difficulty of anticipating data requirements from nascent technologies, building systems for highly diverse work practices and data types, the need to bind together multiple single-purpose systems, the lack of incentives to manage and share data, the complementary nature of research and development in understanding practices, and sustainability.
© All rights reserved Wallis et al. and/or their publisher
Borgman, Christine L., Bowker, Geoffrey C., Finholt, Thomas A. and Wallis, Jillian C. (2009): Towards a virtual organization for data cyberinfrastructure. In: JCDL09 Proceedings of the 2009 Joint International Conference on Digital Libraries 2009. pp. 353-356. http://doi.acm.org/10.1145/1555400.1555459
We report on the exploratory stages of multi-university, multi-research-site, multi-year effort to investigate and compare data practices in multiple cyberinfrastructure projects and their emerging virtual organizations. Our long-term goal is to understand the data practices and data management requirements of virtual organizations and their implications for the design and development of data digital libraries. We have constructed our own virtual organization as a participant-observer approach to the research. Results to date suggest that collaborative technologies are emergent and that defining and scoping the data products of collaborations continues to be problematic.
© All rights reserved Borgman et al. and/or their publisher
Borgman, Christine L., Wallis, Jillian C., Mayernik, Matthew S. and Pepe, Alberto (2007): Drowning in data: digital library architecture to support scientific use of embedded sensor networks. In: JCDL07: Proceedings of the 7th ACM/IEEE-CS Joint Conference on Digital Libraries 2007. pp. 269-277. http://doi.acm.org/10.1145/1255175.1255228
New technologies for scientific research are producing a deluge of data that is overwhelming traditional tools for data capture, analysis, storage, and access. We report on a study of scientific practices associated with dynamic deployments of embedded sensor networks to identify requirements for data digital libraries. As part of continuing research on scientific data management, we interviewed 22 participants in 5 environmental science projects to identify data types and uses, stages in their data life cycle, and requirements for digital library architecture. We found that scientists need continuous access to their data from the time that field experiments are designed through final analysis and publication, thus reflecting a broader notion of "digital library." Six categories of requirements are discussed: the ability to obtain and maintain data in the field, verify data in the field, document data context for subsequent interpretation, integrate data from multiple sources, analyze data, and preserve data. Three digital library efforts currently underway within the Center for Embedded Networked Sensing are addressing these requirements, with the goal of a tightly coupled interoperable framework that, in turn, will be a component of cyberinfrastructure for science.
© All rights reserved Borgman et al. and/or ACM Press
Borgman, Christine L. (2006): What can Studies of e-Learning Teach us about Collaboration in e-Research? Some Findings from Digital Library Studies. In Computer Supported Cooperative Work, 15 (4) pp. 359-383. http://dx.doi.org/10.1007/s10606-006-9024-1
e-Research is intended to facilitate collaboration through distributed access to content, tools, and services. Lessons about collaboration are extracted from the findings of two large, long-term digital library research projects. Both the Alexandria Digital Earth Prototype Project (ADEPT) and the Center for Embedded Networked Sensing (CENS) project on data management leverage scientific research data for use in teaching. Two forms of collaboration were studied: (1) direct, in which faculty work together on research projects; and (2) indirect or serial, in which faculty use or contribute content to a common pool, such as teaching resources, concepts and relationships, or research data. Five aspects of collaboration in e-Research are discussed: (1) disciplinary factors, (2) incentives to adopt e-Learning and e-Research technologies, (3) user roles, (4) information sharing, and (5) technical requirements. Collaboration varied by research domain in both projects, and appears partly to be a function of the degree of instrumentation in data collection. Faculty members were more interested in tools to manage their own research data than in tools to facilitate teaching. They also were more reflective about their research than teaching activities. The availability of more content, tools, and services to incorporate primary data in teaching was only a minimal incentive to use these resources. Large investments in a knowledge base of scientific concepts and relationships for teaching did not result in re-use by other faculty during the course of the project. Metadata requirements for research and for teaching vary greatly, which further complicates the transfer of resources across applications. Personal digital libraries offer a middle ground between private control and public release of content, which is a promising direction for the design of digital libraries that will facilitate collaboration in e-Research.
© All rights reserved Borgman and/or Kluwer Academic Publishers
Borgman, Christine L., Smart, Laura J., Millwood, Kelli A., Finley, Jason R., Champeny, Leslie, Gilliland-Swetland, Anne J. and Leazer, Gregory H. (2005): Comparing faculty information seeking in teaching and research: Implications for the design of digital libraries. In JASIST - Journal of the American Society for Information Science and Technology, 56 (6) pp. 636-657. http://dx.doi.org/10.1002/asi.20154
Champeny, Leslie, Borgman, Christine L., Leazer, Gregory H., Gilliland-Swetland, Anne J., Millwood, Kelli A., D'Avolio, Leonard, Finley, Jason R., Smart, Laura J., Mautone, Patricia D., Mayer, Richard E. and Johnson, Richard A. (2004): Developing a digital learning environment: an evaluation of design and implementation processes. In: JCDL04: Proceedings of the 4th ACM/IEEE-CS Joint Conference on Digital Libraries 2004. pp. 37-46. http://doi.acm.org/10.1145/996350.996361
The Alexandria Digital Earth Prototype (ADEPT) Project (1999-2004) builds upon the Alexandria Digital Library Project (1994-1999) to add functions and services for undergraduate teaching to a digital library of geospatial resources. The 'Digital Learning Environment' (DLE) services are being developed and evaluated iteratively over the course of this research project. In the 2002-2003 academic year, the DLE was implemented during the fall and spring terms in undergraduate geography courses at the University of California, Santa Barbara (UCSB). Evaluation of the fall term implementation identified design issues of time and complexity for creating and organizing course domain knowledge. The spring term implementation added new services to integrate course content into class presentation formats. The implementation was evaluated via interviews with the course instructor, development staff, and students, and by observations (in person and videotaped) of the course. Results indicated that usability and functionality for the instructor had increased between the two course offerings Students found classroom presentations to be useful for understanding concepts, and Web access to the presentations useful for study and review. Assessments of student learning suggest modest improvements over time Developers are now applying lessons learned during these implementations to improve the system for subsequent implementation in the 2003-2004 academic year.
© All rights reserved Champeny et al. and/or ACM Press
Borgman, Christine L. (2001): Where is the librarian in the digital library?. In Communications of the ACM, 44 (5) pp. 66-68. http://doi.acm.org/10.1145/374308.374344
Leazer, Gregory H., Gilliland-Swetland, Anne J. and Borgman, Christine L. (2000): Evaluating the Use of a Geographic Digital Library in Undergraduate Classrooms: ADEPT. In: DL00: Proceedings of the 5th ACM International Conference on Digital Libraries 2000. pp. 248-249. http://www.acm.org/pubs/articles/proceedings/dl/336597/p248-leazer/p248-leazer.pdf
The evaluation plan for the Alexandria Digital Earth Prototype (ADEPT) centers on two investigations: a study of classroom use of the system by faculty and students and lab-based usability studies. The classroom-based study is primarily an investigation of the digital library's impact on student learning, using multiple research methods. The five-year work plan includes investigations of the use of ADEPT in non-geography classes.
© All rights reserved Leazer et al. and/or ACM Press
Borgman, Christine L. (2000): The Premise and Promise of a Global Information Infrastructure. In First Monday, 5 (8) . http://www.firstmonday.org/issues/issue5_8/borgman/index.html
Borgman, Christine L. (1999): The User's Mental Model of an Information Retrieval System: An Experiment on a Prototype Online Catalog. In International Journal of Human-Computer Studies, 51 (2) pp. 435-452.
An empirical study was performed to train naive subjects in the use of a prototype Boolean logic-based information retrieval system on a database of bibliographic records. The research was based on the mental models theory which proposes that people can be trained to develop a 'mental model' or a qualitative simulation of a system which will aid in generating methods for interacting with the system, debugging errors, and keeping track of one's place in the system. It follows that conceptual training based on a system model will be superior to procedural training based on the mechanics of the system. We performed a laboratory experiment with two training conditions (model and procedural), and with each condition split by sex. Forty-three subjects participated in the experiment, but only 32 were able to reach the minimum competency level required to complete the experiment. The data analysis incorporated time-stamped monitoring data, personal characteristics variables, affective variables, and interview data in which subjects described how they thought the system worked (an articulation of the model). As predicted, the model-based training had no effect on the ability to perform simple, procedural tasks, but subjects trained with a model performed better on complex tasks that required extrapolation from the basic operations of the system. A stochastic process analysis of search-state transitions reinforced this conclusion. Subjects had difficulty articulating a model of the system, and we found no
© All rights reserved Borgman and/or Academic Press
Borgman, Christine L. (1996): Social Aspects of Digital Libraries. In: DL96: Proceedings of the 1st ACM International Conference on Digital Libraries 1996. pp. 170-171. http://www.acm.org/pubs/articles/proceedings/dl/226931/p169-arms/p169-arms.pdf
In February 1996, UCLA and the National Science Foundation are holding a workshop on social aspects of digital libraries. This working session will present an outline of the issues raised at the workshop, and invite audience reaction and discussion. The research workshop plans to focus on the following topics: * Information needs: (a) Social context and culture -- to what extent can digital library components be generalized and to what extent must they be tailored to each environment? (b) Information needs and information seeking -- what is the relationship between information seeking and learning in digital libraries? (c) Linking user-learner needs and behavior to digital library design -- what design techniques are appropriate in applying user needs research to digital library design? * End user searching and filtering: (a) Organization, description and representation of information -- which methods of organization can be generalized for digital libraries? What new methods are needed? (b) Search capabilities for users -- how, if at all, should problem domain areas be divided? (c) Interface design for information retrieval -- what human-computer interaction principles can be applied to the information retrieval environment? Background materials for this session are at: http://www.dlib.org/social.html
© All rights reserved Borgman and/or ACM Press
Hancock-Beaulieu, Micheline and Borgman, Christine L. (1996): A New Era for OPAC Research: Introduction to Special Topic Issue on Current Research in Online Public Access Systems. In JASIST - Journal of the American Society for Information Science and Technology, 47 (7) pp. 491-492.
Borgman, Christine L., Hirsh, Sandra G. and Hiller, John (1996): Rethinking Online Monitoring Methods for Information Retrieval Systems: From Search Product to Search Process. In JASIST - Journal of the American Society for Information Science and Technology, 47 (7) pp. 568-583.
Borgman, Christine L. (1996): Why Are Online Catalogs Still Hard to Use?. In JASIST - Journal of the American Society for Information Science and Technology, 47 (7) pp. 493-503.
Hirsh, Sandra G. and Borgman, Christine L. (1995): Children's Browsing and Keyword Searching on the Science Library Catalog: The Effect of Domain Knowledge on Search Behavior. In: Proceedings of the Eighteenth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval 1995. pp. 371-372. http://www.acm.org/pubs/articles/proceedings/ir/215206/p369-liddy/p369-liddy.pdf
Research has shown that adults' subject domain knowledge influences the way they use information retrieval systems. However, the effect of domain knowledge on children's search behavior has not been investigated. This study examines children's search behavior on the Science Library Catalog, a hypertext-based automated library catalog for elementary school children. The Science Library Catalog provides two ways to search for information: a browsing-oriented search method which allows children to navigate through science knowledge hierarchies and a keyword search method which allows children to type in their search queries. We focus on the effect of science domain knowledge on children's search performance, search behavior, and learning as they look for science books on this system. Data were collected through one-on-one interviews, direct observation, and online monitoring of search sessions. We are using a pattern matching program to evaluate sequences of search moves in the monitoring logs and to help us understand how and when children use browsing and keyword search methods. This dissertation will contribute to our understanding of children's search behavior and the factors which influence their behavior. This research also has implications for information retrieval system evaluation and interface design.
© All rights reserved Hirsh and Borgman and/or ACM Press
Borgman, Christine L., Hirsh, Sandra G., Walter, Virginia A. and Gallagher, Andrea L. (1995): Children's Searching Behavior on Browsing and Keyword Online Catalogs: The Science Library Catalog Project. In JASIST - Journal of the American Society for Information Science and Technology, 46 (9) pp. 663-684.
Dumais, Susan, Belkin, Nicholas J., Borgman, Christine L. and Hancock-Beaulieu, Micheline (1994): Evaluating Interactive Retrieval Systems. In: Proceedings of the Seventeenth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval 1994. p. 361. http://www.acm.org/pubs/articles/proceedings/ir/188490/p361-dumais/p361-dumais.pdf
Most current information retrieval systems are highly interactive. Users ask queries, get immediate feedback, refine their queries, and so on. Methods for evaluating these dynamic systems have not kept pace with the rapid advances in system design. It is no longer enough to use the standard precision-recall measures to evaluate and to improve interactive retrieval systems. There is often no single final query to evaluate, with useful information being gathered from many different queries along the way. In addition, interfaces play a critical role in building effective retrieval systems. The best retrieval algorithm can be rendered functionally useless if the interface to it is unusable. Conversely, of course, the spiffiest new interface is not worth much without a good retrieval engine behind it. It would be easy if one could study interfaces and retrieval engines separately and take the best of both worlds. Unfortunately, there are important interactions that cannot be evaluated by studying components in isolation -- e.g., how do you incorporate ranking or relevance feedback for a Boolean retrieval engine, or how do you highlight matching terms if complex syntactic and semantic processing of queries is used? The design of effective interactive retrieval environments will require careful attention to the larger human - interface - retrieval-engine system. Systematic, generalizable evaluations of these larger interactive systems are possible both in the laboratory and in the field. The panelists will describe interactive retrieval experiments and experiences, focusing on: a) why it is important to study interactions b) how interactive retrieval performance should be measured, and c) how the methods for evaluation and findings generalize to other systems. Belkin will begin with an overview of some of the problems in evaluating interactive retrieval systems and will present a new framework characterizing IR as interaction with text. The remaining talks will describe end-user experiments involving highly interactive retrieval systems. The focus of these talks will be on the approaches, methods and instruments used to evaluate retrieval effectiveness and ease of use as well as the relationship between system functionality and the interface. Dumais will describe the importance of interfaces in retrieval, and will present examples of successful iterative interface design with the SuperBook and X-LSI systems. Hancock-Beaulieu will review a series of experiments on the Okapi system to systematically examine the effectiveness of different retrieval aids. Bergman will describe the multiple evaluation methods employed to study children's information-seeking behavior using the Science Library Catalog, a graphical browsing system supplemented by keyword searching tailored to children's skills, and attempts to generalize these evaluation methods to other IR environments.
© All rights reserved Dumais et al. and/or ACM Press
Borgman, Christine L. (1992): Cultural Diversity in Interface Design. In ACM SIGCHI Bulletin, 24 (4) p. 31.
Borgman, Christine L. and Rice, Ronald E. (1992): The Convergence of Information Science and Communication: A Bibliometric Analysis. In JASIST - Journal of the American Society for Information Science and Technology, 43 (6) pp. 397-411.
Borgman, Christine L. and Siegfried, Susan L. (1992): Getty's Synoname and Its Cousins: A Survey of Applications of Personal Name-Matching Algorithms. In JASIST - Journal of the American Society for Information Science and Technology, 43 (7) pp. 459-476.
Borgman, Christine L., Walter, Virginia A., Rosenberg, Jason B. and Gallagher, Andrea L. (1991): Children's Use of a Direct Manipulation Library Catalog. In ACM SIGCHI Bulletin, 23 (4) pp. 69-70.
Sullivan, Michael V., Borgman, Christine L. and Wippern, Dorothy (1990): End-users, mediated searches, and front-end assistance programs on Dialog: A comparison of learning, performance, and satisfaction. In JASIST - Journal of the American Society for Information Science and Technology, 41 (1) pp. 27-42.
Borgman, Christine L., Belkin, Nicholas J., Croft, W. Bruce, Lesk, Michael E. and Landauer, Thomas K. (1988): Retrieval Systems for the Information Seeker: Can the Role of the Intermediary be Automated?. In: Soloway, Elliot, Frye, Douglas and Sheppard, Sylvia B. (eds.) Proceedings of the ACM CHI 88 Human Factors in Computing Systems Conference June 15-19, 1988, Washington, DC, USA. pp. 51-53.
The introduction of automated information retrieval (IR) systems was met with great enthusiasm and predictions that manual literature searching soon would be replaced. Three decades later, IR systems have not progressed to the stage where any but the dedicated few can operate them without a highly skilled human intermediary acting as interface between user and system. In the interim, we have learned that the retrieval process is extremely complex both in terms of understanding people and their communication and in terms of understanding scientific information and technical vocabulary. Experiments with new techniques suggest to many the possibility of eliminating the human intermediary, either in large part or altogether; others would argue that the retrieval problems are too complex to be resolved for more than highly restricted domains. The possibility of eliminating the human intermediary is of current research interest to the several disciplines that are represented on this panel.
© All rights reserved Borgman et al. and/or ACM Press
Borgman, Christine L. (1987): Individual Differences in the Use of Information Retrieval Systems: Some Issues and Some Data. In: Proceedings of the Tenth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval 1987. pp. 61-71.
The population using information retrieval systems is becoming increasingly diverse. We find a wide range of skills in ability to use these systems; this diverse population must be accommodated by the next generation of systems. This paper reports on a study to identify variables related to information retrieval aptitude, based on results from earlier studies of searchers and programmers. A sample of undergraduate subjects from English, psychology, and engineering majors was given a series of psychometric tests and compared to known populations. We find that engineering majors exhibit academic background and personality characteristics most like those of skilled searchers and programmers, with contrasting patterns or no discernible patterns in English and psychology majors. The strength of most associations increases when restricted to subjects who have either stayed in one major or who have changed major only within one disciplinary area. About half the variance in choice of major can be explained by scores on the tests administered, and a comparable amount of variance in test scores can be explained by the academic background variables.
© All rights reserved Borgman and/or ACM Press
Borgman, Christine L. (1986): The User's Mental Model of an Information Retrieval System: An Experiment on a Prototype Online Catalog. In International Journal of Man-Machine Studies, 24 (1) pp. 47-64.
An empirical study was performed to train naive subjects in the use of a prototype Boolean logic-based information retrieval system on a database of bibliographic records. The research was based on the mental models theory which proposes that people can be trained to develop a "mental model" or a qualitative simulation of a system which will aid in generating methods for interacting with the system, debugging errors, and keeping track of one's place in the system. It follows that conceptual training based on a system model will be superior to procedural training based on the mechanics of the system. We performed a laboratory experiment with two training conditions (model and procedural), and with each condition split by sex. Forty-three subjects participated in the experiment, but only 32 were able to reach the minimum competency level required to complete the experiment. The data analysis incorporated time-stamped monitoring data, personal characteristics variables, affective variables, and interview data in which subjects described how they thought the system worked (an articulation of the model). As predicted, the model-based training had no effect on the ability to perform simple, procedural tasks, but subjects trained with a model performed better on complex tasks that required extrapolation from the basic operations of the system. A stochastic process analysis of search-state transitions reinforced this conclusion. Subjects had difficulty articulating a model of the system, and we found no differences in articulation by condition. The high number of subjects (26%) who were unable to pass the benchmark test indicates that the retrieval tasks were inherently difficult. More interestingly, those who dropped out were significantly more likely to be humanities or social science majors than science or engineering majors, suggesting important individual differences and equity issues. The sex-related differences were slight, although significant, and suggest future research questions.
© All rights reserved Borgman and/or Academic Press
Join our community and advance:
Page maintainer: The Editorial Team