Data Provenance and Financial Systemic Risk

We describe the needs for data provenance in a large-scale analytic environment to support financial systemic risk analysis. Government financial regulators need to make sense of the outputs of thousands to tens of thousands of simulation runs invoked by a large analytic staff; automatic capture of data provenance (dataset sources and processing steps) supports analysts without adding to their workloads. We present an architecture for automated provenance capture from both simulations and data transformation tools. Finally, we describe a prototype implementation and next steps.