SIGIR is the major international forum for the presentation of new research results and the demonstration of new systems and techniques in the field of information retrieval.
The following articles are from "Proceedings of the Eighteenth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval":
“Complex documents are used in many environments, e.g., information retrieval (IR). Such documents contain subdocuments, which may contain further subdocuments, etc. Powerful tools are needed to facilitate their retrieval, restructuring, and analysis. Existing IR systems are poor in complex document restructuring and data aggregation. However, in practice, IR system users would often want to obtain aggregation information on subdocuments of complex documents. In this paper we address this problem and provide a truly declarative and powerful interface for the users. Our interface is based on the non-first-normal-form (NF{squared}) relational model. It allows intuitive and systematic modeling of complex documents.”
“This paper presents an efficient signature file approach for fast and intelligent retrieval of large Chinese full-text document databases. The proposed approach is an integrated and efficient text access method, which performs well both in exact match searching of Boolean queries and best match searching (ranking) of quasi-natural language queries. Using this approach, the inherent difficulties of Chinese word segmentation and proper noun identification can be effectively reduced, queries can be expressed with non-controlled vocabulary, and the ranking function can be easily implemented neither demanding extra space overhead nor affecting the retrieval efficiency. The experimental results show that the proposed approach achieves good performance in many ways, especially in the reduction of false drops and space overhead, the speedup of retrieval time, and the capability of best match searching using quasi-natural language queries. In conclusion, the proposed approach is capable of retrieving gigabytes of Chinese texts very efficiently and intelligently.”
“We find that dissemination of collection wide information (CWI) in a distributed collection of documents is needed to achieve retrieval effectiveness comparable to a centralized collection. Complete dissemination is unnecessary. The required dissemination level depends upon how documents are allocated among sites. Low dissemination is needed for random document allocation, but higher levels are needed when documents are allocated based on content. We define parameters to control dissemination and document allocation and present results from four test collections. We define the notion of iso-knowledge lines with respect to the number of sites and level of dissemination in the distributed archive, and show empirically that iso-knowledge lines are also iso-effectiveness lines when documents are randomly allocated.”
“A character based indexing is preferable for Japanese IR systems since Japanese words are not segmented. This paper proposes a new character indexing method to enhance our previous method which divided character pair index entries into disjoint groups based on character classes. Since frequency data is used to determine hashed entries for character pairs and to establish a special string index, both search speed and precision are improved. Moreover, bit strings are managed using small and large blocks, so registration and retrieval are accelerated. Experiments using patent abstracts showed these proposals are quite effective.”
“Most information retrieval systems use stopword lists and stemming algorithms. However, we have found that recognizing singular and plural nouns, verb forms, negation, and prepositions can produce dramatically different text classification results. We present results from text classification experiments that compare relevancy signatures, which use local linguistic context, with corresponding indexing terms that do not. In two different domains, relevancy signatures produced better results than the simple indexing terms. These experiments suggest that stopword lists and stemming algorithms may remove or conflate many words that could be used to create more effective indexing terms.”
“Evaluation is a major force in research, development and applications related to information retrieval (IR). This paper is a critical and historical analysis of evaluations of IR systems and processes. Strengths and shortcomings of evaluation efforts and approaches are discussed, together with major challenges and questions. A limited comparison is made with evaluation in experts systems and Online Public Access Catalogs (OPACs). Evaluation is further analyzed in relation to the broad context and specific problems addressed. Levels of evaluation are identified and contrasted; most IR evaluations were concerned with the processing level, but others were conducted at the output, users and use, and social levels. A major problem is the isolation of evaluations at a given level. Issues related to systems under evaluation, and evaluation criteria, measures, measuring instruments, and methodologies are examined. A general point is also considered: IR is increasingly imbedded into many other applications, such as the Internet or digital libraries. Little evaluation in the traditional IR sense is undertaken in relation to these applications. The challenges are to integrate IR evaluations from different levels and to incorporate evaluation in new applications.”
“Differences between the most and least consistent searchers are considered. Attention is payed both to term-consistency and concept-consistency. The paper is based on an empirical study where 32 searchers formulated query statements from 12 search requests. The searchers were also interviewed to obtain information about their experience. There was a statistically significant dependence between term-consistency and the terminological styles of searchers on the one hand and between concept-consistency and searchers' search strategies on the other hand. There were also clear differences in the experience of most and least consistent searchers both in information storage and information retrieval.”
“We report findings from an exploratory study whose overall goal was to design an online document surrogate for journal articles, customized for use in clinical problem solving. We describe two aspects of literature-based medical decision making. First, there are interaction effects among citations in a search output (or among articles in a stack of articles) that affect the physician's judgment of clinical applicability. Second, physicians select among different information processing strategies when attempting to use literature for finding an answer a clinical question.”
“All of the methods currently used to evaluate information retrieval (IR) systems have limitations in their ability to measure how well users are able to acquire information. We utilized an approach to assessing information obtained based on the user's ability to answer questions from a short-answer test. Senior medical students look the ten-question test and then searched one or two IR systems on the five questions for which they were least certain of their answer. Our results showed that pre-searching scores on the test were low but that searching yielded a high proportion of answers with both systems. These methods are able to measure information obtained, and will be used in subsequent studies to assess differences among IR systems.”
“Collection fusion is a data fusion problem in which the results of retrieval runs on separate, autonomous document collections must be merged to produce a single, effective result. This paper explores two collection fusion techniques that learn the number of documents to retrieve from each collection using only the ranked lists of documents returned in response to past queries and those documents' relevance judgements. Retrieval experiments using the TREC test collection demonstrate that the effectiveness of the fusion techniques is within 10% of the effectiveness of a run in which the entire set of documents is treated as a single collection.”
“It has been known that using different representations of either queries or documents, or different retrieval techniques retrieves different sets of documents. Recent work suggests that significant improvements in retrieval performance can be achieved by combining multiple representations or multiple retrieval techniques. In this paper we propose a simple method for retrieving different documents within a single query representation, a single document representation and a single retrieval technique. We classify the types of documents, and describe the properties of weighting schemes. Then, we explain that different properties of weighting schemes may retrieve different types of documents. Experimental results show that significant improvements can be obtained by combining the retrieval results from different properties of weighting schemes.”
“In this paper, we consider vague queries in text and fact databases. A vague query can be formulated as a combination of vague criteria. A single database object can meet a vague criterion to a certain degree. We confine ourselves to queries for which the answer can be computed efficiently by (perhaps repetitive) combination of rankings to new rankings. Since users usually will inspect some of the best answer objects only, the corresponding rankings need to be computed just as far as necessary to generate these first answer objects. In this contribution we describe an approach for estimating the number of elements needed from the basic rankings to compute a given number of elements of the resulting ranking. Experiments with a large text database prove the applicability of our approach.”
“An experimental term selection strategy for document visualization is described. Strong discriminators with few co-occurrences increase the clustering tendency of low-dimensional document browsing spaces. Clustering tendency is tested with diagnostic measures adapted from the field of cluster analysis, and confirmed using the VIBE visualization tool. This method supports browsing in high recall, low precision document retrieval and classification tasks.”
“Computerised information spaces evolved using the Boolean logic paradigm for retrieval of their stored information. While many studies have looked at ways to improve training to enable users to create queries appropriate to their information need, little attention has been paid to users' cognitive models of the information. This research shows that when people can illustrate their queries graphically, they have at least two quite different mental models. These incorporate the information space they are working with and the particular approach applied to their search. This paper presents the findings to date and indicates the additional avenues which we believe warrant investigation.”
“The use of information retrieval systems in networked environments raises a new set of issues that have received little attention. These issues include ranking document collections for relevance to a query, selecting the best set of collections from a ranked list, and merging the document rankings that are returned from a set of collections. This paper describes methods of addressing each issue in the inference network model, discusses their implementation in the INQUERY system, and presents experimental results demonstrating their effectiveness.”
“This paper touches upon the problems arising in connection with indexing and retrieval for effective searching of digitized images. An empirical study, based on 13 newspaper archives, demonstrates that rules for indexing images can be formulated and that a user group typology can be established. An image user model is suggested based on word clusters. The empirical analysis demonstrates how the results of the word association tests can be used as the foundation for a semantic model.”
“Results of cognitive psychology research are analysed to explain why it is difficult for retrieval system users to bring to mind alternative search terms. A human memory model is modified in such a way that it produces additional search terms instead of human associations. A small experiment shows that such a spreading activation network can find alternative terms -- with a performance similar to the normally used similarity measures.”
“In this paper, we compare learning techniques based on statistical classification to traditional methods of relevance feedback for the document routing problem. We consider three classification techniques which have decision rules that are derived via explicit error minimization: linear discriminant analysis, logistic regression, and neural networks. We demonstrate that the classifiers perform 10-15% better than relevance feedback via Rocchio expansion for the TREC-2 and TREC-3 routing tasks. Error minimization is difficult in high-dimensional feature spaces because the convergence process is slow and the models are prone to overfitting. We use two different strategies, latent semantic indexing and optimal term selection, to reduce the number of features. Our results indicate that features based on latent semantic indexing are more effective for techniques such as linear discriminant analysis and logistic regression, which have no way to protect against overfitting. Neural networks perform equally well with either set of features and can take advantage of the additional information available when both feature sets are used as input.”
“We have built a hybrid Case-Based Reasoning (CBR) and Information Retrieval (IR) system that generates a query to the IR system by using information derived from CBR analysis of a problem situation. The query is automatically formed by submitting in text form a set of highly relevant cases, based on a CBR analysis, to a modified version of INQUERY's relevance feedback module. This approach extends the reach of CBR, for retrieval purposes to much larger corpora and injects knowledge-based techniques into traditional IR.”
“Text retrieval systems typically produce a ranking of documents and let a user decide how far down that ranking to go. In contrast, programs that filter text streams, software that categorizes documents, agents which alert users, and many other IR systems must make decisions without human input or supervision. It is important to define what constitutes good effectiveness for these autonomous systems, tune the systems to achieve the highest possible effectiveness, and estimate how the effectiveness changes as new data is processed. We show how to do this for binary text classification systems, emphasizing that different goals for the system lead to different optimal behaviors. Optimizing and estimating effectiveness is greatly aided if classifiers that explicitly estimate the probability of class membership are used.”
“This paper studies noise reduction for computational efficiency improvements in a statistical learning method for text categorization, the Linear Least Squares Fit (LLSF) mapping. Multiple noise reduction strategies are proposed and evaluated, including: an aggressive removal of "non-informative words" from texts before training; the use of a truncated singular value decomposition to cut off noisy "latent semantic structures" during training; the elimination of non-influential components in the LLSF solution (a word-concept association matrix) after training. Text collections in different domains were used for evaluation. Significant improvements in computational efficiency without losing categorization accuracy were evident in the testing results.”
“This paper describes a novel paradigm for representing many types of information about documents in a manner particularly suited to text categorization by a trivial empirical rule induction system. It also has potential application to full-text retrieval paradigms. The paradigm allows many different types of document predicates to be combined together with logical dependencies being controlled for. This is shown to be justified by any reasonable model of descriptor inference, and the effect of increasing representation sophistication is shown for two corpora.”
“Text categorization can be viewed as a process of category search, in which one or more categories for a test document are searched for by using given training documents with known categories. In this paper a cluster-based search with a probabilistic clustering algorithm is proposed and evaluated on two data sets. The efficiency, effectiveness, and noise tolerance of this search strategy were confirmed to be better than those of a full search, a category-based search, and a cluster-based search with nonprobabilistic clustering.”
“In the logical approach to information retrieval, retrieval is considered as uncertain inference. Here we present a new, powerful inference method for this purpose which combines Datalog with probability theory on the basis of intensional semantics. We describe syntax and semantics of probabilistic Datalog and also present an evaluation method and a prototype implementation. This approach allows for easy formulation of specific retrieval models for arbitrary applications, and classical probabilistic IR models can be implemented by specifying the appropriate rules. In comparison to other approaches, the possibility of recursive rules allows for more powerful inferences. Finally, probabilistic Datalog can be used as a query language for integrated information retrieval and database systems.”
“In this paper we discuss the dynamics of probabilistic term weights in different IR retrieval models. We present four different models based on different notions of retrieval. Two of these models are classical probabilistic models long in use in IR, the two others are based on a logical technique of evaluating the probability of a conditional called Imaging, one is a generalisation of the other. We analyse the transfer of probabilities occurring in the representation space at retrieval time for these four models, compare their retrieval performance using classical test collections, and discuss the results.”
“Information retrieval systems are being challenged to manage larger and larger document collections. In an effort to provide better retrieval performance on large collections, more sophisticated retrieval techniques have been developed that support rich, structured queries. Structured queries are not amenable to previously proposed optimization techniques. Optimizing execution, however, is even more important in the context of large document collections. We present a new structured query optimization technique which we have implemented in an inference network-based information retrieval system. Experimental results show that query evaluation time can be reduced by more than half with little impact on retrieval effectiveness.”
“Images are a communication medium, hence objects of a linguistic nature having a form and a content. The form of an image is the image appearance and is understood as depicting a scene, the image content. The relationship between the form of an image and its content is established through a process of interpretation, capturing the meaning of the image form. Any information need on images can, and indeed has to, be seen as addressing either the image form, or its content, or the relationship between them. Consequently, any general, domain independent image retrieval facility should be based on a model supporting all these aspects of images. An image retrieval model, based on classical logic, is proposed which fulfills this basic requirement.”
“How effective is stemming? Text normalization? Stemming experiments test two hypotheses: one term (+stemmer) or two (-stemmer). The truth lies somewhere in between. The correlations, {rho}, between a word and its variants (e.g., +s, +ly, +uppercase) tend to be small (refuting the one term hypothesis), but non-negligible (refuting the two term hypothesis). Moreover, {rho} varies systematically depending on the words involved; it is relatively large for a good keyword, {rho}(hostage, hostages){approx}0.5, and small for pairs with little content, {rho}(anytime, Anytime){approx}0, or conflicting content, {rho}(continental, Continental){approx}0.”
“Information Retrieval Systems typically distinguish between content bearing words and terms on a stop list. But "content-bearing" is relative to a collection. For optimal retrieval efficiency, it is desirable to have automated methods for custom building a stop list. This paper defines the notion of serial clustering of words in text, and explores the value of such clustering as an indicator of a word bearing content. The numerical measures we propose may also be of value in assigning weights to terms in requests. Experimental support is obtained from natural text databases in three different languages.”
“We report on a probabilistic weighting approach to indexing the scanned images of very short documents. This fully automatic process copes with short and very noisy texts (67% word accuracy) derived from the images by Optical Character Recognition (OCR). The probabilistic term weighting approach is based on a theoretical proof explaining how the retrieval effectiveness is affected by recognition errors. We have evaluated our probabilistic weighting approach on a sample of index cards from an alphabetic library catalogue where, on the average, a card contains only 23 terms. We have demonstrated over 30% improvement in retrieval effectiveness over a conventional weighted retrieval method where the recognition errors are not taken into account. We also show how we can take advantage of the ordering information of the alphabetic library catalogue.”
“Modern text collections often contain large documents that span several subject areas. Such documents are problematic for relevance feedback since inappropriate terms can easily be chosen. This study explores the highly effective approach of feeding back passages of large documents. A less-expensive method that discards long documents is also reviewed and found to be effective if there are enough relevant documents. A hybrid approach that feeds back short documents and passages of long documents may be the best compromise.”
“Information Retrieval (IR) systems exploit user feedback by generating an optimal query with respect to a particular information need. Since obtaining an optimal query is an expensive process, the need for mechanisms to save and reuse past optimal queries for future queries is obvious. In this article, we propose the use of a query base, a set of persistent past optimal queries, and investigate similarity measures between queries. The query base can be used either to answer user queries or to formulate optimal queries. We justify the former case analytically and the latter case by experiment.”
“Methods for learning weights of terms using relevance information from a learning set of document has been studied for decades in information retrieval research. The approach used here, Dynamic Feedback Optimization, starts with a good weighting scheme based on Rocchio feedback, and then improves those weights in a dynamic fashion by testing possible changes of query weights on the learning set documents. The resulting optimized query performs 10-15% better that the original when evaluated on the test set. We discuss this constant tension between describing what a relevant document should contain, and describing what the known relevant documents do contain.”
“The SIGIR Education Committee has recently been formed based on the model of the SIGCHI committee which completed its final report in 1992. The committee is charged with developing curriculum recommendations for IR-related education to serve the computer science, library science, and information science communities. It will be soliciting input from information retrieval educators and the consumers of IR education (students and employers), with a view of determining the current status of IR education, the marketplace, and future direction. The committee is also interested in: * clearinghouses for IR courseware and training materials * electronic as well as traditional courses * demonstrations for online access to state of the art systems * other innovative efforts The purpose of this panel is to report briefly on the activities of the Education Committee and to stimulate discussion on the state of information retrieval education. The panel will consist of three IR educators from different communities (computer science, library science, information science) who will give brief (about 10 minute) presentations on their view of the purpose and content of IR curricula; a representative from the government sector will report on the role of IR in government agencies; and a participant from the industrial sector will consider the role of IR education in industry.”
“CeDAR -- The Centre for Database Access Research, School of Computing and Mathematics, University of Huddersfield, UK, has pioneered the use of view-based techniques to improve the effectiveness of user-interfaces to both bibliographic and corporate databases. Two systems are presented: VUSE for INSPEC: This front-ending software searches the 5 million record INSPEC database and is a by-product of a research project launched on 1st Sept. 1991. The project has been funded by the University of Huddersfield in collaboration with the Institution of Electrical Engineers, Marconi Research Laboratories and STN-International (FIZ-Karlsruhe). The VUSE (View-based User Search Engine) system removes the need for the user to appreciate explicit Boolean statements by introducing a search strategy of successive refinement through the use of filtering views. These techniques are described in "Peek-a-Boo revived -- End-user searching of bibliographic databases using filtering views." by A Steven Pollitt, Martin P Smith and Geoffrey P Ellis, Online 94, 18th International Online Information Meeting, London, December 1994 pp 63-72. This PC-resident software has been used in the investigation of ranking and relevance feedback extensions to VUSE, the subject of research to PhD being undertaken by Martin P Smith. EPOQUE for Windows: Presented in collaboration with the Directorate of Informatics and Telecommunications at the European Parliament in Luxembourg. CeDAR is responsible for specifying the thesaurus interface that provides the new guided search mode to EPOQUE (European Parliament Online QUEry system). EPOQUE for Windows, made available in April 1995, is designed to facilitate querying of the European Parliament's main documentary database through the incorporation of VUSE techniques. EPOQUE documents are indexed by the multilingual EUROVOC thesaurus which provides a significant demonstration of the suitability of view-based techniques for multilingual retrieval. An example of how this approach has been demonstrated on the Apple Macintosh can be found in: "Using the thesaurus to view and filter environmental databases: An example using EUROVOC to search EPOQUE -- the European Parliament Online Query System." by A Steven Pollitt, Geoffrey P Ellis and Martin P Smith, The First European ISKO Conference on Environmental Knowledge Organisation and Information Management, 14-16 Sept. 1994, Bratislava, Slovakia. in Stancikova P and Dahlberg I (Eds) Knowledge Organization in Subject Areas, Vol. 1 (1994) Supplement pp 21-32 Pub: INDEKS VERLAG Frankfurt/Main.”
“An interactive program, "dotplot," has been developed for browsing millions of lines of text and source code, using an approach borrowed from biology for studying homology (self-similarity) in DNA sequences. With conventional browsing tools such as a screen editor, it is difficult to identify structures that are too big to fit on the screen. In contrast, with dotplots we find that many of these structures show up as diagonals, squares, textures and other visually salient features, as will be illustrated in examples selected from biology and two new application domains: text (AP news, Canadian Hansards) and source code (5ESS(RG)).”
“The system to be demonstrated provides head-coupled stereoscopic viewing of 3-d visual representations for document collections, associative term thesauri, and individual documents. This style of interface has been called "fish tank VR" and shown to be relatively effective for 3-d viewing and interaction tasks. The system also provides mechanisms to integrate query formulation across the visual representations. Supplying interaction techniques to access the system's several visual representations is one of the system's goals. The demonstration allows users to experiment with 3-d interaction in the highly interactive, iterative process of information retrieval.”
“The Eclair class library is a set of extensible C++ classes for implementing best-match IR applications. Developers use Eclair to either add IR functionality to an existing application or to develop new applications from scratch. Using a loosely-coupled user interface (written in Tcl/Tk), we demonstrate a variety of IR application features and discuss how the code abstractions offered by Eclair were employed. A traditional best-match model as well as a recent probabilistic inference approach (IJdens, Bruza & Harper, submitted) are used in the demonstration. Also discussed is a MultiMedia application where pictures are represented by complex indexing features, designed for effective retrieval. We explain how the implementation of IR applications is supported by Eclair.”
“The LyberWorld system introduces a prototypical application of information visualization components for IR user interfaces. The prototype implements visualizations of an abstract information space -- fulltext. It demonstrates a visual user interface for the probabilistic fulltext retrieval system INQUERY. Visualizations are used to communicate information search and browsing activities in a natural way by applying metaphors of spatial navigation and attraction in abstract information spaces. Visualization tools for exploring textual information spaces and judging relevance of information items are introduced and example sessions are provided. The presence of a spatial model in the user's mind and interaction with a system's corresponding display methods is regarded as an essential contribution towards natural interaction and reduction of cognitive costs during e.g. query construction, orientation within the database content, relevance judgement and orientation within the retrieval context.”
“Our demonstration, Automatic Building of Hypertext Links in Digital Libraries, seeks to reduce the cost of authoring quality hypertext documents by taking advantage of promising information retrieval techniques. A set of tools will be presented that assist document authors in dynamically creating hypertext documents. The ability of the hypertext engine to semi-automatically and automatically create and remove bi-directional links will be demonstrated. The links will be generated based on similarity between documents and document components that reside in the collection. A World-Wide Web browser will be used to demonstrate the results of the hypertext linking tools.”
“PIRCS (Probabilistic Indexing and Retrieval -- Components -- System) is a highly effective, probability and network-based IR system designed for large scale heterogeneous collections. Factors contributing to its effectiveness include representation enhancements, sophisticated term weighting, learning network capability, and combining multiple retrieval strategies. PIRCS does not need a full inverted file but maintains a direct file that allows a network for retrieval and learning to be created dynamically. Documents are organized into subcollections served by a common master lexicon with cumulative statistics. Documents are then mixed and ranked for retrieval as if from a single collection. PIRCS currently runs on a SparcStation with 128MB and 7 GB of disk space.”
“In most text retrieval systems, similarity between words and documents are captured in large similarity matrices which derive their coefficients from various similarity measures. To a typical human observer, the similarity matrix represents abstract mathematical relations where extraction of meaningful relationships is virtually impossible. We will demonstrate an interactive system which maps these relations into virtual semantic spaces whereby Euclidean distances between objects are inversely proportional to their similarity. Furthermore, we provide navigational tools that enable one to travel inside this virtual semantic space as one might explore a physical space. The interactivity allows one to exploit human experience rather than technological prowess to comprehend the semantic relations.”
“In an earlier SIGIR paper (Lin, et al. 1991), a method was proposed to construct a semantic map for information retrieval by a self-organizing algorithm, Kohonen's feature map. An important feature of the semantic map is to help the user visualize contents of underlying documents. This demonstration shows a prototype that implements such a semantic map as a graphical interface for retrieval systems. Using documents from SIGIR 86-93 as a test base, the prototype shows several IR literature maps based on different indexing methods such as title indexing, title-abstract indexing, and fulltext indexing. Comparing these maps has led to the exploration of relationships between document visualization and document indexing. Some preliminary results of the exploration will be illustrated during interactive demonstrations of the prototype.”
“Promenade is an image-document retrieval system which integrates free-text and attribute-value queries in an object-oriented database query language (OSQL from Ontos). The query language provides a protocol upon which we were able to build an HTML interface to make the stored collections available to the World Wide Web through standard Web browsers (Mosaic, Netscape, MacWeb). Promenade currently hosts two image databases for the National Agricultural Library on the World Wide Web: a collection of botanical prints from Curtis Botanical Magazine (1797-1827), and a collection of plant pest and disease photographs from the Michigan State University Cooperative Extension Service.”
“Envision is a multimedia digital library of computer science literature, with full-text searching and full-content retrieval capabilities, serving computer science researchers, teachers, and students at all levels of expertise. The most unusual feature of Envision is its Graphic View window, which provides powerful information visualization facilities that enable users to explore patterns in the literature. Envision's Graphic View window displays search results as a matrix of icons that represent documents. Users have control over the semantics of six graphical devices: icon position along the x-axis and y-axis, the alphanumeric icon label, icon size, icon color, and icon shape. These graphical devices may represent a number of document attributes: probable relevance to query, publication year, document type (e.g., text, video, hypermedia), document size, number of sources, author names, and index terms. Working in tandem with the Graphic View, the Item Summary window presents bibliographic information for icons selected by the user. Document content is presented on demand using Mosaic and a suite of related viewers. Recent studies show strong user interest and satisfaction, and minor changes they have suggested are being incorporated into newer versions of the interface software. Implementation efforts have led to an X Motif version of the Envision interface, which will be shown with a sample of the overall digital library collection.”
“The EUROSPIDER system is a full fledged Information Retrieval (IR) system to search large and complex data collections for relevant objects. Depending on the configuration, the EUROSPIDER system can be used as a standalone IR system, it can be added to a World-Wide Web server to make a data collection accessible through a network, or it can be added to a commercial database (DB) system to provide access to a possibly very dynamic and structured data collection. In the last case, the integration of the EUROSPIDER system and a DB system provides both IR functionality (relevance ranking, feedback searches, document analysis) and DB functionality (data model, query language, transaction processing, access control). An advanced integration of the EUROSPIDER system with a DB system is achieved by using a probabilistic retrieval model which takes into account the DB scheme. The EUROSPIDER system is the commercial version of the IR system SPIDER developed at the Swiss Federal Institute of Technology (ETH) Zurich, Switzerland. Demonstrations are available at [http://www-ir.inf.ethz.ch].”
“We demonstrate a graphical interface to a library catalog information retrieval system. This ranked output IR system interface has combined a novel set of features to help the end-user in a wide range of information gathering situations. The system supports the following: * Navigational features such as browsing table of contents and browsing list of articles written by a specific author. * An integrated online thesaurus from which end-users can pick words and phrases to expand their original query. * A visualization scheme that helps the user in understanding how the query result ranking was computed. * Simple drag-and-drop operations of objects into positive and negative areas on the screen for providing relevance feedback information. The above features help the user's interactive and iterative nature of the information seeking process.”
“New retrieval capabilities for Russian texts based on the language processor Russicon are introduced: - "word-changing" search, which means that all word-changing forms of a given word will be found; - "paradigm" search, which means that all words -- members of a given word paradigm (or the part of paradigm) will be found; - word search with given grammatical characteristics (part of speech, changeability, animation, case, number, gender, person, aspect, tense, transition, mood, form, reflexive (verb), length of word-building and wordchanging stem etc.); - word search with given word-building (root) and word changing stem, prefixes, suffixes and endings -- "linguistic wild cards" etc.; - word search enriched by synonyms from thesaurus; - usage for forming natural query from Russian sentence. These capabilities are achieved by using Russian language processor RUSSICON inside of retrieval systems. Technical specifications of the system. Russian language processor is realised as a C-library of such functions: morphological analyzer, normalizer, syntactic analyzer and semantic analyzer. The processor is designed as C++ library of mentioned functions which allow quick generation of different C-based retrieval systems on multiple platforms (DOS/WINDOWS).”
“Advances in software engineering are making it possible to design systems that are totally open and can also be tailored to the specific needs of an information provision centre. This demonstration will show how we have utilised the concepts: USER-DEFINED ENTITIES, FREE-TEXT RETRIEVAL, THESAURUS, SDI, OPEN SYSTEM and USBC, in order to build the integrated package called LIBRETTO for the total control of a modern library or information centre.”
“Information filtering refers to the task of selecting objects of interest from an incoming stream of information. As part of NIST/ARPA's TREC Workshop, we used Latent Semantic Indexing (LSI) for filtering 336k documents from diverse sources (newswires, patents, newspapers, technical abstracts) for 50 topics of interest. We developed representations of user interests using two sources of information. A Word Filter used just the words in the topic statements. A RelDocs Filter used just relevant training documents and ignored the topic statement (a variant of relevance feedback). The RelDocs filter vector was 30% more effective than the detailed natural language description of interests. Combining these two vectors provided small additional improvements in filtering. On average, 7 of the top 10 documents are relevant using the combined vector method. Performance can further be improved by continually incorporating relevant test documents into the filter vector. Data combination of the Word and RelDocs retrieved sets was not generally successful in improving performance compared to the best individual method, although we believe it might be if additional sources are used. Both query and data combination methods are quite general and applicable to a variety of filtering applications.”
“Over the last decade, a certain degree of progress has been achieved in the morphological analysis of Turkish. However, this work has not been used, thus far, to improve the effectiveness of information retrieval systems. This poster considers the development and evaluation of conflation techniques necessary for the implementation of a document retrieval system for Turkish text databases. We have evaluated stemming and n-gram matching for searching six dictionaries of Turkish words. Our results indicate that stemming can bring about substantial reductions in the number of word variants that must be processed in a Turkish free-text retrieval system. Thus, the six Turkish”
“The Category Discrimination Method (CDM) is a new learning algorithm designed for text categorization. The motivation is there are statistical problems associated with natural language text when it is applied as input to existing machine learning algorithms (too much noise, too many features, skewed distribution). The bases of the CDM are research results about the way that humans learn categories and concepts vis-a-vis contrasting concepts. The essential formula is cue validity borrowed from cognitive psychology, and used to select from all possible single word-based features the 'best' predictors of a given category. Using a precategorized test collection of text documents, for each category: * Determine the 'best' predictors (i.e. features) for the category by computing the cue validity of all single word-based features from all training documents and selecting those exceeding a threshold. * Conduct a multi-stage search over a limited search space to learn how these features might best be organized into a logical structure suitable for use as a text categorizer. * Evaluate the performance of best categorizer by running on the test documents. The hypothesis that CDM's performance will exceed two non-domain specific algorithms, Bayesian classification and decision tree learners is empirically tested.”
“This study was designed to identify the degrees of fit between or appropriateness of the knowledge provided via enduser searching of bibliographic information systems with the information seeking practices of scientists involved in pharmaceutical research and development. The critical incident techniques was utilized to extract information regarding the impact of information retrieved from online bibliographic information resources on its users. Scientists (n=10) actively involved in some aspect of pharmaceutical research and development were interviewed. The intent was to determine how online bibliographic database systems are used, how well they serve the needs of their users as well as the nature of their impact on these users. Content analysis of interview transcripts show that users in this context are engaged in publication/report writing or research design when motivated to use online bibliographic databases. The impact of having information retrieved includes increased: publication rates, credibility in report/ information generation, credibility in provision of information to the Medical Community and timesavings. Respondents stated having the retrieved information positively impacted their research activities. Evidence suggests that enduser searching has an impact on publication productivity as well as other aspects of productivity.”
“Research has shown that adults' subject domain knowledge influences the way they use information retrieval systems. However, the effect of domain knowledge on children's search behavior has not been investigated. This study examines children's search behavior on the Science Library Catalog, a hypertext-based automated library catalog for elementary school children. The Science Library Catalog provides two ways to search for information: a browsing-oriented search method which allows children to navigate through science knowledge hierarchies and a keyword search method which allows children to type in their search queries. We focus on the effect of science domain knowledge on children's search performance, search behavior, and learning as they look for science books on this system. Data were collected through one-on-one interviews, direct observation, and online monitoring of search sessions. We are using a pattern matching program to evaluate sequences of search moves in the monitoring logs and to help us understand how and when children use browsing and keyword search methods. This dissertation will contribute to our understanding of children's search behavior and the factors which influence their behavior. This research also has implications for information retrieval system evaluation and interface design.”
“I present two experiments that investigate the interactive searching behavior of two groups of people using a best-match, ranked-output retrieval engine (INQUERY) to search a large, full-text document collection. The group for the first experiment consisted of ten users experienced in the use of traditional, boolean online retrieval systems who were novices in the use of best-match, ranked output systems. I describe their behavior and retrieval performance for five searches each in the context of the TREC-3 routing task with a special focus on their use of relevance feedback. The second experiment has been designed to analyze the contribution of relevance feedback more closely: a baseline system without relevance feedback is contrasted with three versions of relevance feedback systems that systematically vary user knowledge and user control with regard to relevance feedback but otherwise maintain the same interface, the same retrieval engine, the same full-text document collection (75000 Wall Street Journal articles from the TIPSTER collection), and the same search topics. I present an initial analysis of behavioral data and retrieval performance data gathered from 60 end-users with no training in information retrieval who each performed searches on the baseline system and one of the relevance feedback systems.”
“This article outlines a method for the structuring of expository texts which are not explicitly structured -- no sections and subsections are present. We first perform a quantitative segmentation consisting in finding the topic boundaries. This stage uses the td.idf coherence measure, a standard measure in Information Retrieval. Quantitative segmentation allows one to isolate sets of paragraphs which are "topically coherent", that is to say that this process divides the text into several distinct developments or parts. We then perform a qualitative segmentation by establishing the nature of the relations, such as cause, illustration, conclusion, explanation, which are present between thematic blocks and between sentences inside each block. To achieve this, we developed a linguistic analysis based upon clue words detection and making use of the thematic boundaries previously obtained. These lexical clues can be connectors (e.g. thus), variable expressions (e.g. anteposed prepositional phrases) or invariable ones (e.g. in conclusion), punctuation (e.g. interrogation), verbal tenses and mood (e.g. conditional), verb (e.g. to introduce) This text structuring method is the first component of a system for the automatic generation of abstracts.”
“This work focuses on the navigation of hypertexts for Passage Retrieval (PR). In particular hypertexts that are automatically constructed from a large and heterogeneous collection of full-text documents have been considered to extract node-passages relevant to user's informative requirements. Full-text documents summarize different subjects and then they are containers of ambiguous words. In retrieving passages we have to select those excerpts that match narrow queries. For doing that a PR technique has to disambiguate the sense of words occurring in full-text documents. Most approaches to PR do not consider user's query because passages are often defined before, and independently of query. We are studying a navigation-based technique for PR from collections of large documents. The proposed technique is based on a methodology and a prototype for the automatic construction of hypertexts for IR. Users start to navigate the automatically constructed hypertext to retrieve the passages that are as close to his requirements as possible. In retrieving passages the navigation provides some useful information to disambiguate the sense of the passage terms because each passage term belongs to a semantically meaningful context, namely a passage, and it is related to the terms previously visited.”
“We propose a probabilistic approach to document classification, and experiment it on an application where a new article is automatically assigned to a Usenet newsgroup. Each newsgroup is represented by a probabilistic language model (based on unigrams). A Maximum A Posteriori rule is used to decide which newsgroup generated the article with the highest probability. We experiment this approach on newsgroups dealing with various facets of Artificial Intelligence, and we try to guess from the body of an article the precise group it was posted to. First, a set of efficient keywords is automatically extracted from training data using a maximum precision criterion. A keyword-based approach is compared with the probabilistic approach and evaluated on the same test data, both for recall and precision rates, using various sizes for the vocabulary. Experiments indicate that the probabilistic approach is more efficient that the keyword-based approach. In the keyword case, increasing the number of keywords always increases the number of documents selected, and thus the recall rate. This is not so in the probabilistic case, because considering more words provides more information to the decision rule, so that the size of the vocabulary has to be chosen carefully for a maximum efficiency.”
“In textual information retrieval, the vector space retrieval model has proven its robustness in manipulating large collections of unrestricted natural language text. In our approach, we try to improve the retrieval effectiveness of this model by introducing the notion of distributional semantics. The content of retrievable units or text excerpts, as well as user queries, are represented in a unified way as projections in a vector space of pertinent terms. These projections are derived from co-occurrence matrices computed on large reference text corpora collecting the distributional semantic information. Different measures of similarity may be used to characterize the proximity between user queries and related texts. In our first experiments, we use the cosine similarity measure.”
“HNC Software Inc. has developed a system called CONVECTIS for automatically indexing free text documents. CONVECTIS uses HNC's context vector representation of text, which encodes similarity of meaning at the word level, and is learned automatically from free text examples. The key new feature of CONVECTIS is its use of supervised learning based on relevance feedback to tune the system's indexing behavior. This approach to indexing is also directly applicable to the document routing task. CONVECTIS has been demonstrated on datasets of gigabyte size and is currently being used by a large newswire company. The learning procedure typically achieves close to 100% precision and recall on training documents. For test documents not trained on, preliminary results indicate that performance in the 80-90% range for both recall and precision is commonly obtained. CONVECTIS is implemented in a client-server configuration and indexes over 10,000 documents per hour on a dual-CPU Sun Sparc20 (based on 3KB/doc with markup, 5000 index term context vectors).”
“Signature file access method is efficient for processing of content-based retrieval of multimedia database. In large multimedia database server, parallel device is utilized to achieve concurrent access. Efficient allocation of signature file on parallel device minimizes the query response time and is important in the design of large multimedia databases. In this paper, we propose a new dynamic allocation technique to distribute the signature file on parallel devices. It is an improvement of previous approach, Fragmented Signature File. While Fragmented Signature File distributes the partitioned frame signature file by using Quick Filter, the proposed Parallel Signature File distributes by using a disk allocation technique. The proposed Parallel Signature File has some advantages. First, the qualified blocks are distributed more uniformly than Fragmented Signature File. Second, it can be used in dynamic environment. Third, the blocks allocated in each processing unit can be clustered to reduce the disk random access time. Performance analysis shows that the proposed approach outperforms that of Fragmented Signature File and is not far from theoretical optimal response time. Besides, the performance of Parallel Signature File has significant improvement than that of Fragmented Signature File, especially in the application domain of multimedia.”
“In information retrieval, one can describe documents by the terms they contain. To improve the effectiveness of retrieval, the expansion method extends terms with related terms. It includes two steps: the selection and the weighting of terms to be added. This experiment involves comparing different systems for weighting the new terms of the expansion for automatic document classification, a problem somewhat similar to information retrieval. Document expansion occurs automatically and uses a thesaurus. The majority of weighting methods assign to new terms a weight equal to the expanded term multipied by a constant. This model has the disadvantage of modifying the similarity between documents when the information provided by the expansion should not interfere. In this specific instance, the model proposed here meets the invariance of similarity constraint and substantially improves document classification. When the similarity is the cosine, the space of solutions of this constraint applied to the expansion of a term t in q terms, is the hypersphere of radius, the weight w of t. The expansion function is then linear. One solution experimented here, is to assign to the q terms, w divided by the square root of q.”
“A visual information retrieval interface (VIRI) is defined as one that uses graphic elements in addition to text to aid the solution of a problem related to information storage and retrieval. More than twenty such interfaces already exist, with different retrieval models, graphical metaphors, and user interactions. Furthermore, the interfaces have different strengths, for example, retrieval, browsing, and document classification. The focus of the workshop is to exchange information, and to begin development of a method for comparing these interfaces. Researchers and practitioners who are actively working on VIRI projects are particularly invited to participate. Some effort will be put into developing a classification scheme for VIRIs and identifying major research issues related to visual interfaces. Following this the discussion will center on identifying test collections and developing experimental tasks and measures that will provide a sound basis for comparing and evaluating the interfaces.”
“The Z39.50 Computer-to-computer retrieval protocol is an increasingly mature US national standard (version 3 is currently in the ballot process as of early 1995); it is widely implemented both in the US and, increasingly, also seeing use internationally, particularly in Europe. Z39.50 is potentially of great importance to the IR research community for several reasons: * Because Z39.50 provides a means of separating a user interface from a retrieval system, it allows research in clients and user interfaces to proceed independently from research in back-end retrieval engines, and, of particular importance, allows new user interfaces to be tested against very large production databases. It also allows new experimental retrieval systems to be offered to large user communities through familiar interfaces. * Z39.50 can form the linkage between a number of large-scale research projects that involve the IR community, such as the various Digital Library efforts. * Z39.50 raises and provides a concrete framework to explore a number of important research issues in its own right about the design of interoperable clients and servers for information retrieval, the representation and exchange of metadata about information servers, and related matters. The workshop has several goals: * To introduce the broad IR community to Z39.50, including its history, its current status, its function, and implementation progress; * To highlight several IR research projects that are exploiting Z39.50 today; * To sketch some of the research issues that are raised by Z39.50. After an introduction delineating the history of Z39.50 and the current status of implementations, a short tutorial will explain the operation of the protocol. The second part of the workshop will include two panels: one about the use of Z39.50 to support IR research, and another about research issue in information retrieval protocols. Attendees will be invited to contribute to the discussion.”
“The integration of database management systems and information retrieval systems is of great practical interests. There are, however, hard research problems that remain to be solved. The workshop aim is to assist the information retrieval community in understanding the integration problems and to set up a research agenda. The workshop will include short presentations on the following topics: Architecture: loosely coupled, tightly coupled, total integration; does the DBMS control the IRS or vice versa; support for distributed computing. Retrieval Model and Query Language: reconciling classical DB retrieval and classical (weighted) IR retrieval; retrieval models taking advantage of DB schema; treating DB attributes in an IR way, e.g. in a probabilistic way; integration query languages for IR/DB systems; query processing/optimization. Concurrence Control and Transaction Management: concurrence control on the IR index; is ACID enough or is ACID too much for IR; new transaction models (nested transactions); long lived transactions (for indexing). Performance: new access structures; new buffering schemes (caches); retrieval performance on dynamic data; insertion, deletion, modification performance; scalability (parallel architectures); identify bottlenecks. After the presentations, attendees will participate in round table discussions about each topic. To allow this to proceed in a workshop atmosphere, the workshop is restricted to 30 participants.”
“In this one-day workshop, Doris Lidtke and Michael Mulder will report on their extensive experience in the development of new curricula in computer information science, emphasizing preparation of students to deal with large scale information systems AND new paradigms of learning/teaching. Topics to be covered by the workshop leaders include: (1) involvement of the stakeholders -- employers, faculty, and instructional/curriculum designers; (2) determining content -- both depth and breadth; (3) validation by the stakeholders; (4) packaging -- knowledge units vs. courses; (5) special delivery mechanisms, and (6) essential/desired infrastructure to support the new/revised curriculum. These topics will provide a framework for discussion of curriculum development in information retrieval. Individuals and groups representing various points of view (library and information science, computing science, MIS, information systems, business, government and academia) will be invited to prepare submissions and act as group leaders. An opportunity will be provided for attendees to participate in working groups developing an IR curriculum in their area of interest.”
“The workshop will address IR methods and tools that can be used in the automatic construction of a hypermedia base to produce an informative hypertext collection of documents that can be searched and browsed by content. Passage retrieval is one of the methods that can be used in the segmentation of documents in a collection of flat documents for hypermedia information retrieval design. This method, as well as other methods for automatic authoring of hypermedia bases will be presented and discussed in the workshop. Both techniques that construct a hypertext from an unlinked set of data and those that can be applied to an existing hypertext/media permitting augmention of its set of links are relevant to the workshop. Typing of links in the resulting hypertext needs to be addressed as well as having both static and dynamic links in the resulting hypertext. The workshop also will address evaluation of the quality of hypertext collections and their construction. After the presentations of a few position papers, the participants will discuss specific methods or other topics of interest. The workshop will conclude with the approval of a short working paper presenting all the methods that the participants deem useful for automatic construction of hypermedia.”
“An efficient variant of an optimal algorithm is presented, which, in the context of a large dynamic full-text information retrieval system, reorganizes data that has been compressed by an on-the-fly compression method based on LZ77, into a more compact form, without changing the decoding procedure. The algorithm accelerates a known technique based on a reduction to a graph-theoretic problem, by reducing the size of the graph, without affecting the optimality of the solution. The new method can thus effectively improve any dictionary compression scheme using a static encoding method.”
“This paper describes the architecture, implementation and evaluation of NetSerf, a program for finding information archives on the Internet using natural language queries. NetSerf's query processor extracts structured, disambiguated representations from the queries. The query representations are matched to hand-coded representations of the archives using semantic knowledge from WordNet (a semantic thesaurus) and an on-line Webster's dictionary. NetSerf has been tested using a set of questions and answers developed independently for a game called Internet Hunt. The paper presents results comparing the performance of NetSerf and the standard IR system SMART on this set of queries.”
“In this paper, we describe the design of a reusable IR framework, called FIRE, that is being implemented to facilitate the development of IR systems. In addition, FIRE is designed to support the experimental evaluation of both indexing and retrieval techniques. First, we discuss the development of reusable software in the IR domain and derive essential criteria for the design of an IR framework. Next, we sketch the object model developed for FIRE. We present the basic concepts and their modeling and show how the components interact when performing indexing and retrieval tasks.”
“This paper discusses the efficiency of a parallel text retrieval system that is based on the Vector Space Model. Specifically, we describe a general parallel retrieval algorithm for use with this model, the application of the algorithm in the FIRE system [1], and its implementation on the high performance GCel3/512 Parsytec parallel machine [2]. The use of this machine's two-dimensional grid of processors provides an efficient basis for the virtual tree that lies at the heart of our retrieval algorithm. Analytical and experimental evidence is presented to demonstrate the efficiency of the algorithm.”
“* To summarize is to reduce in complexity, and hence in length, while retaining some of the essential qualities of the original. * This paper focusses on document extracts, a particular kind of computed document summary. * Document extracts consisting of roughly 20% of the original can be as informative as the full text of a document, which suggests that even shorter extracts may be useful indicative summaries. * The trends in our results are in agreement with those of Edmundson who used a subjectively weighted combination of features as opposed to training the feature weights using a corpus. * We have developed a trainable summarization program that is grounded in a sound statistical framework.”
“The full integration of information retrieval (IR) features into a database management system (DBMS) has long been recognized as both a significant goal and a challenging undertaking. By full integration we mean: i) support for document storage, indexing, retrieval, and update, ii) transaction semantics, thus all database operations on documents have the ACID properties of atomicity, consistency, isolation, and durability, iii) concurrent addition, update, and retrieval of documents, and iv) database query language extensions to provide ranking for document retrieval operations. It is also necessary for the integrated offering to exhibit scaleable performance for document indexing and retrieval processes. To identify the implementation requirements imposed by the desired level of integration, we layered a representative IR application on Oracle Rdb and then conducted a number of database load and document retrieval experiments. The results of these experiments suggest that infrastructural extensions are necessary to obtain both the desired level of IR integration and scaleable performance. With the insight gained from our initial experiments, we developed an approach, called cooperative indexing, that provides a framework to achieve both scaleability and full integration of IR and RDBMS technology. Prototype implementations of system-level extensions to support cooperative indexing were evaluated with a modified version of Oracle Rdb. Our experimental findings validate the cooperative indexing scheme and suggest alternatives to further improve performance.”
“We present a model for querying textual databases by both the structure and contents of the text. Our goal is to obtain a query language which is expressive enough in practice while being efficiently implementable, features not present at the same time in previous work. We evaluate our model regarding expressivity and efficiency. The key idea of the model is that a set-oriented query language based on operations on nearby structure elements of one or more hierarchies is quite expressive and efficiently implementable, being a good tradeoff between both goals.”
Give us your opinion! Do you have any comments/additions that you would like other visitors to see?
Yousay:
Mar 19th, 2010
#1
Be the first to add a thoughtful note to this page !
Changes to this page (conference)
24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited
24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was edited 24 Jun 2007: Conference Proceedings was added to the bibliography
Page maintainer: The Editorial Team How to cite/reference this page
URL: http://www.interaction-design.org/references/conferences/proceedings_of_the_eighteenth_annual_international_acm_sigir_conference_on_research_and_development_in_information_retrieval.html
You say:
Mar 19th, 2010
