Cube Data Model for Multilevel Statistics Computation of Live Execution Traces

Problem and motivation

Live trace streams are potentially unbounded, but analysts need multidimensional statistics across recent fine-grained and older coarse-grained time ranges without retaining every event or detailed value (pp. 2-5).

Method and contribution

The architecture combines a trace reader, circular-buffer processing, dimension/metric trees, interval-form history storage, separate cubes per tilted time frame, and a query engine. It supports sum/count/average, point/stabbing, range, sliding-window, top-k, drill-down, roll-up, slice, and dice queries. Minimal, partial, and full cubing trade memory for response time (pp. 4-12).

Findings and evidence

Java/Eclipse experiments use LTTng on Linux kernel 2.6.38.6, a Core i7 2.80 GHz/6 GB machine, generated activity from recursive grep -r, wget -r -l, and ls -R, 1,000 measures, and three time levels: last 5 minutes, 24 hours, and 12 days. A one-day trace is used for the three-level memory study; maximum memory is approximately 35 MB. Partial cubing with selected non-leaf measures gives the best reported response-time/memory trade-off, and range-query time is largely independent of interval length in the tested cases (pp. 12-16).

Limitations and future directions

Limitations: Materialization choices control memory/latency; the partial-cubing selection is static in the evaluation; memory grows with metric count/frequency; and the workload is a selected generated LTTng trace rather than a broad production benchmark. Very busy streams may require buffering, delayed processing, or dropping events (pp. 5, 12-16).

Future work: Dynamically choose non-leaf cuboids or switch between minimal/partial materialization using user feedback/query history, and extend the system to problem detection and data-mining analyses (p. 16, conclusion).

Sources and identifiers

• Published version published
• Institutional full text · PDF public_full_text

When to cite this paper

Cite this paper when your work uses or compares tilted-time-frame stream cubes that retain detailed recent history and compact older history.

• Tilted-time-frame stream cubes that retain detailed recent history and compact older history.
• Interval-form cube summaries enabling arbitrary time-range queries through endpoint subtraction.
• Minimal/partial/full cuboid materialization as a memory-versus-query-response trade-off.
• The 1,000-measure LTTng live-trace evaluation with approximately 35 MB peak memory and range-query duration independence in the tested setup.

Citation

@article{ezzatiJivan2015cubedata,
  author = {Naser Ezzati-Jivan and Michel R. Dagenais},
  title = {Cube Data Model for Multilevel Statistics Computation of Live Execution Traces},
  year = {2015},
  journal = {Concurrency and Computation: Practice and Experience},
  volume = {27},
  number = {5},
  pages = {1069-1091},
  publisher = {Wiley},
  issn = {1532-0626, 1532-0634},
  doi = {10.1002/cpe.3272},
  url = {https://doi.org/10.1002/cpe.3272}
}

Ezzati-Jivan, N., & Dagenais, M. R. (2015). Cube Data Model for Multilevel Statistics Computation of Live Execution Traces. Concurrency and Computation: Practice and Experience, 27(5), 1069-1091. https://doi.org/10.1002/cpe.3272

N. Ezzati-Jivan and M. R. Dagenais, "Cube Data Model for Multilevel Statistics Computation of Live Execution Traces," Concurrency and Computation: Practice and Experience, vol. 27, no. 5, pp. 1069-1091, 2015, doi: 10.1002/cpe.3272