Date and Location: 10/10/2016 Monday, 2:00pm. CS Conference Room, 206

Speaker: Dong Dai

Title: Adaptive, Lightweight Provenance Service for High Performance Systems

Abstract:
Provenance describes detailed information about the history of a piece of data, containing the relationships to elements like users, processes, jobs, and workflows, that contribute to the existence of data. It is a key to support many data management functionalities that are increasingly critical in nowadays. Despite its importance, provenance support is largely missing in current large-scale distributed systems due to its demanding requirements on the provenance service: it needs to be scalable and lightweight to be always on, and, at the same time, to be comprehensive enough to support various data management usages. In this research, we introduce an adaptive, lightweight provenance service (ALPS), to manage provenance transparently and efficiently in high-performance computing systems. The core of ALPS is an adaptive provenance model and its efficient implementations to automatically balance the overheads and comprehensiveness of provenance in an online manner. Multiple optimizations are also introduced. Extensive evaluations have confirmed its efficiency and usability. We believe that ALPS can be integrated into future large-scale distributed systems as a default service to support various advanced data management needs.