Sensor Data & Analysis Framework (SDAF) Data Warehouse

The Sensor Data & Analysis Framework (SDAF) Data Warehouse is part of the SDAF project. As more and more sensors are producing volumes of data regarding objects that change with respect to location and time, evaluating this stream of information in a timely fashion requires the integration of current and historical data. The SDAF research project seeks to investigate and understand various approaches to integrating streamed and historical sensor data to support spatio-temporal queries. The SDAF Data Warehouse (SDAF DW) effort experiments with different techniques to store and organize the historical sensor data efficiently in a persistent data store. In this paper, we will discuss the finding that to support spatial queries of objects relating to location and time, it is best to partition the data by date/time. There are two advantages to the approach. First, partition elimination keeps the number of partitions to search for a query to a minimum. Second, as the size of the database grows, the number of partitions to search for the same query remains the same, thereby keeping the response time relatively constant.