Repeatability, Reproducibility and Rigor

  • Date
    February 26, 2014
  • Time
  • Location
    366 WVH


Computer systems research spans sub-disciplines that include embedded systems, programming languages, networking, and operating systems. In this talk my contention is that a number of structural factors inhibit quality systems research. Symptoms of the problem include unrepeatable and unreproduced results as well as results that are either devoid of meaning or that measure the wrong thing. I will illustrate the impact of these issues on our research output with examples from the development and empirical evaluation of the Schism real-time garbage collection algorithm. Solutions that focus on fostering repeatability are being adopted by a number of ACM conference under the name of Artifact Evaluation Committees or AECs. These AECs provide positive feedback to authors that take the time to create repeatable research.

Brief Biography

Jan Vitek is a Professor of Computer Science and University Faculty Scholar. He is the Chair of the ACM Special Interest Group on Programming Languages (SIGPLAN), the vice chair of AITO and of the IFIP WG 2.4, and is Chief Scientist at Fiji Systems. He holds a PhD from the University of Geneva and a MSc from the University of Victoria. He works on various aspects of programming languages including virtual machines, compilers, software engineering, real-time and embedded computing, concurrency and information security. Prof. Vitek led the Ovm project which resulted in the first succefull flight test of real-time Java virtual machine. With Noble and Potter, Vitek proposed the notion of ownership for alias control, which became known as ownership types. He chaired PLDI, ISMM and LCTES and was program chair of ESOP, ECOOP, VEE, Coordination, and TOOLS. Vitek has started a number of successful workshop series, including MOS on Mobile Objects, IWACO, on aliasing and confinement, and TRANSACT on transactional memory. He is on the steering committees of ECOOP, JTRES, TRANSACT, ICFP, OOPSLA, POPL, PLDI and LCTES. He is a member of the JSR-302 Safety Critical Java expert group.

Host: Matthias Felleisen