Duration (expected): 2018-03-30 – 2020-09-31
Award amount: $175,000
Award title: CRII: III: Visualization of Event Sequences for Decision Making
PI/point of contact: Cody Dunne, Assistant Professor, Khoury College of Computer Sciences, Northeastern University
Students:
Date of Last Update: 2021-03-15
Many datasets of interest to scientists, analysts, clinicians, and patients include events that take place over time. Events can come from electronic monitoring devices or manual data entry, and all together form an event sequence. By analyzing event sequences, we can make inferences about complex behaviors over time. Interactive visualization tools are an important tool for human decision makers to use for exploring and understanding data. However, the existing visualization techniques need to be improved to support interpretation of data in applications with a large number of events that vary in frequency, accuracy, and timing.
This project will create new visualization encoding and interaction design techniques that will advance the state of the art for understanding long streams of event sequences. Techniques will be validated for general uses as well as in a case study on type 1 diabetes treatment decision support. This research will afford more effective data exploration and decision-making tools for analyzing temporal data, applicable to many domains. The resulting visualization techniques will benefit clinicians and patients performing intensive insulin management for type 1 diabetes by enabling them to reduce the burden of care and improve outcomes. In addition, there are immediate applications to the treatment of chronic heart failure as well as non-health domains such as data centers monitoring cyber security. The outreach efforts in this research will disseminate the findings to the type 1 diabetes community, including both caregivers and patients, as well as encourage teenage girls to pursue careers in STEM.
This research will advance the state of the art in visualization and visual analytics for long streams of temporal event sequences with multidimensional, interrelated data. It will create methodologies for: (1) folding and reconfiguring long event sequences to align by dual non-periodic sentinel events; (2) scaling time axes between dual aligned events to show time distributions; (3) dual-event alignment of overlapping units of folded and reconfigured event sequences with data duplication; (4) compositing data from multiple event sequence sources to enable temporal inference with uncertain, erroneous, and missing data; and (5) displaying rapidly changing time series scalar values composited with point and interval event sequences at different time granularities and at different aggregation levels. The domain case study problem will be characterized and refined through semi-structured interviews with certified diabetes educators (CDEs). The visual encodings and interaction designs will be iteratively designed and evaluated through formative usability studies, visualization expert review, controlled task-based experiments, and qualitative studies with CDEs. The project website will include resulting materials, including relevant publications, visualization software, data analysis tools, documentation, deidentified data from user studies, and deidentified domain data for reproducibility and demonstration.
As a whole, our contributions will help future researchers or practitioners interested in temporal event sequence visualization to develop and employ more effective techniques.
We developed a novel form of task abstraction for visualization design called hierarchical task abstraction (HTA). HTA marries hierarchical task analysis with task abstraction.
We developed a data abstraction and hierarchical task abstraction for clinicians performing type 1 diabetes data analysis for treatment recommendation. These were based on semi-structured interviews with 6 clinicians, and we used inductive qualitative analysis to understand their responses. We also developed summative evaluation results from these interviews.
Using the data and task abstractions we contributed the design and implementation of IDMVis, an open source interactive visualization tool to support clinicians adjusting intensive diabetes management treatment plans. IDMVis includes techniques for folding and reconfiguring temporal event sequences; aligning by one or two sentinel events of interest and scaling time axes between two aligned events; and for superimposing data from multiple sources to enable temporal inference with uncertain data.
We conducted a summative evaluation of our abstractions and IDMVis with clinicians which validates our approach and elucidates how visualization can support clinicians make treatment decisions.
We designed a series of tasks that are tailored for superimposed time-series and temporal event-sequence visualization.
We conducted a controlled experiment to evaluate different temporal event alignment techniques, including no alignment, single-event alignment, and dual-event alignment.
Based on our experimental results, we make design recommendations for future researchers and practitioners.
We developed a set of generalizable tasks for reading timeline visualizations, based on a task analysis we conducted and previous work.
Using these tasks, we performed a crowdsourced experiment to compare readability across 4 common timeline shapes and 3 types of data.
Based on our experimental results, we provide design recommendations for researchers and practitioners interested in visually presenting temporal event sequence data.
We developed a novel network visualization technique for providing overviews of temporal event sequences and attributes.
The technique uses a new n-layer network layout algorithm with grouping and group ordering constraints to put similar attribute values together and reduce edge intersections.
A summative evaluation of the design in comparison with IDMVis validated our approach. Based on this evaluation, we provide design recommendations for future researchers and practitioners.
Yixuan Zhang, Kartik Chanana, and Cody Dunne, “IDMVis: Temporal Event Sequence Visualization for Type 1 Diabetes Treatment Decision Support”, IEEE Transactions on Visualization and Computer Graphics (Proc. Information Visualization 2018), vol. 25, no. 1, 2019-01. doi: 10.1109/TVCG.2018.2865076.
Yixuan Zhang, Sara Di Bartolomeo, Fangfang Sheng, and Cody Dunne “Evaluating Alignment Approaches in Superimposed Time-Series and Temporal Event Sequence Visualizations”, Proc. IEEE Visualization Conference (VIS), 1 – 5, 2019-10. doi: 10.1109/VISUAL.2019.8933584. Open paper and all materials doi: 10.31219/osf.io/q764s.
Sara Di Bartolomeo, Aditeya Pandey, Aristotelis Leventidis, David Saffo, Uzma Haque Syeda, Elin Carstensdottir, Magy El-Nasr, Michelle Borkin, and Cody Dunne, “Evaluating the Effect of Timeline Shape on Visualization Task Performance”, Proc. CHI Conference on Human Factors in Computing Systems (CHI ’20), 1 – 12, 2020-04. doi: 10.1145/3313831.3376237. Open paper and all materials doi: 10.31219/osf.io/2kdb9. Consumable writeup on Medium.
Sara Di Bartolomeo, Yixuan Zhang, Fangfang Sheng, and Cody Dunne “Sequence Braiding: Visual Overviews of Temporal Event Sequences and Attributes”, IEEE Transactions on Visualization and Computer Graphics (Proc. VIS 2020), vol. 27, no. 2, 2021-02. doi: 10.1109/TVCG.2020.3030442. Open paper and all materials doi: 10.31219/osf.io/mq2wt.
Sara Di Bartolomeo, “Evaluating the Effect of Timeline Shape on Visualization Task Performance”, Northeastern HIMS alumni and faculty, 2020-05.
Sara Di Bartolomeo, “Evaluating the Effect of Timeline Shape on Visualization Task Performance” [recording], CambridgeCHI, 2020-05.
Cody Dunne, “Interactive network and time series visualizations for reasoning and communicating about data”, BostonCHI, 2018-03.
Yixuan Zhang, Kartik Chanana, and Cody Dunne, “IDMVis: Temporal Event Sequence Visualization for Type 1 Diabetes Treatment Decision Support” [slides] [recording], Information Visualization, 2018-10.
Cody Dunne, “Temporal event sequence visualization for type 1 diabetes treatment decision support”, Open Insights Seminar at Harvard Medical School Department of Biomedical Informatics, 2019-03.
IDMVis is an open source interactive visualization tool for showing type 1 diabetes patient data. It is designed to help clinicians perform temporal inference tasks: specifically for recommending adjustments to patient insulin protocol, diet, and behavior. A live demo, videos, source code, sample data, and documentation are available on the IDMVis website and associated GitHub repo.
Sequence Braiding is an open source interactive visualization tool for providing visual overviews of temporal event sequences. A live demo, source code, sample data, and documentation are available on the Sequence Braiding documentation site, associated GitHub repo, and on OSF at osf.io/mq2wt.
Additional materials for our other papers are available on OSF: * Alignment evaluation: osf.io/q764s * Diversiform timelines: osf.io/2kdb9
This project serves as an example for people with diabetes, the clinicians treating them, and developers building for them of how data visualization can assist in the treatment of diabetes and in particular in conjunction with intensive diabetes management. Moreover, the generalizable temporal event sequence and timeline visualization recommendations we make will help design practitioners to deliver more effective and consumable visualizations for the general public.
We presented and discussed our work with 8 clinicians, in a lecture at the Harvard Department of Biostatistics, with a group of Health Informatics Master’s Degree alumni and faculty who are working in industry, and presented at BostonCHI and CambridgeCHI. Many more informal discussions have stemmed from this work, including with an endocrinologist and two CDEs.
We included our VIS 2018 paper results in two iterations of an undergraduate data science course DS 4200 and leveraged by three students conducting a service learning project titled “Visualizing the Efficacy of Metformin for Blood Glucose Control”. Two MS students have participated in the research, one funded and the other not. One is a co-author of the VIS 2018 paper and the other a co-author on the VIS 2020 paper.
The project has funded and/or facilitated the training of three female graduate students and one male graduate student. This project marks the first paper on visualization submitted by any of these students.
This material is based upon work supported by the National Science Foundation under Grant No. 1755901. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.