RALPH » DUSI: A Prototype System for Rich User Interactions with Diagrams in Online Scientific Research Papers

Presenter: Brian Guthrie
Sponsor: Prof. Bob Futrelle
Downloads: Slides
Presentation date: Nov. 17th, 2005


A substantial amount of the information in scientific literature is presented in the form of diagrams and figures. Traditional natural-language analysis, while useful, cannot address this information, which may represent up to 40% of the content of a given paper in the field of biology. To be able to address this problem, a new generation of diagram analysis and interaction tools must be invented, with the ultimate goal of changing the way people read and interact with scientific literature. A preliminary step in this goal is the creation of a tool that allows the developer of a diagram parser to examine the results of a parse, as well as allowing some end user to examine the parsed diagram side-by-side with the relevant body text and diagram caption text. The Diagram Understanding System Inspector ("DUSI") is an application, written in Java, by the author, and is designed to fulfill both of these needs.

The DUSI is provided with a collection of graphic objects that have been parsed out of a vector diagram containing nothing but primitives, such as those found in PDF documents. The parser assigns semantics to these primitive graphic objects and groups them into larger collections, e.g. X-Axis or GeneSegment. The DUSI allows users to inspect the results of this parse using a tightly-coupled graphic canvas and parse tree interface. The tree displays the hierarchy of the parse, with varying levels of complexity and semantics, and the canvas displays the diagram itself. Clicking on a particular object in the tree highlights the appropriate object in the canvas, and vice versa. In this way, particular image groups, such as a coordinate in a bar graph or a plasmid segment in a gene diagram, can be examined by highlighting them in the parse tree. Viewed alongside the corresponding caption for the given diagram, or even a paper's body text, particular image groups can be mapped to text, and a user who clicks the hyperlinked text "the gene segment labeled FA0" can see the appropriate image objects highlighted.

The DUSI work was undertaken for Professor Futrelle's Biological Knowledge Laboratory ("BKL") as part of their efforts to help biologists derive greater meaning from their literature, of which one major part is diagram parsing. It is manipulable using an external API, for which a simple command-line interface has been provided as a proof-of-concept. The DUSI prototype has been presented and described in two recent proposals to the National Science Foundation. The parse trees in the current prototype were built by hand in the spirit of earlier parser development, and greater integration between DUSI and the working parser is expected as work progresses in that area. The diagram and body text present in the prototype are from an Open Access paper published by BioMed Central and are located at http://www.biomedcentral.com/1471-2199/6/2/. The current prototype represents one year of intermittent work by the author.

The presentation will include slides on the system itself, its API, screenshots, and background material on diagram parsing. The slides will also cover goals for the continuing evolution of the system. There will be a live demo of the current working version of the system, with two different diagrams: one a fictional proof-of-concept diagram which has been fully parsed and provides an illustration of parser-DUSI integration, and another which has been reconstructed from a diagram located at the URL above and provides an illustration of the end-user experience. Possible topics also include open-access publishing, Java/Swing development, and API design.

More information on DUSI can be found at http://www.ccs.neu.edu/home/bguthrie/dusi.