2023 · IEEE International Conference on Program Comprehension (ICPC)
Towards a Classification of Log Parsing Errors
Evidence basis: full-text-reviewed · Review status: catalog-reviewed; paper-author approval pending
observability trace-analysis anomaly-detection performance-analysis
log parsing parser errors Drain AEL Spell ULP LogHub tokenization logging practices dynamic tokens static tokens delimiter-aware logging parser error taxonomy
Core contribution: The paper classifies recurring log-parsing errors and connects them to token and event characteristics that can guide parser design and logging practice.
Problem and motivation
Log parsers vary substantially across datasets, but recurring error causes lack a common classification that can guide parser design and logging practice (p. 1).
Method and contribution
Drain, AEL, Spell, and ULP are compared against manual ground truth from four LogHub datasets: OpenStack, Linux, HealthApp, and Apache. Each contributes 2,000 manually parsed events; parser errors are manually inspected and grouped into nine categories. Static-token correctness/order and dynamic-token presence/order define the comparison; repeated instances of the same error are counted once (pp. 2-4).
Findings and evidence
The study reports 523 errors: Linux 231, OpenStack 151, HealthApp 130, and Apache 11. C1/C2/C3/C7 tokenization-related categories account for 65.51% of errors; regex preprocessing accounts for 28.5% of captured errors. OpenStack C2 ranges from 29.63% to 68.75% across parsers (51.31% average); C6 averages 20.05% versus C2's 20.95%; C7 reaches 67.75% of HealthApp errors (pp. 2-4).
Limitations and future directions
Limitations: Preliminary scope: four datasets, four parsers, and 8,000 manually labelled events. Industrial generalization and system-type effects are untested; parser versions/configurations are unknown.
Future work: Larger/cross-system datasets, improved parser algorithms and preprocessing, delimiter-aware/consistent logging practices, and further study of system-type effects (p. 4).
Resources
Sources and identifiers
- Published version published
- Public full text · PDF public_full_text
When to cite this paper
Cite this paper when your work uses or compares the nine-category taxonomy of recurring log parsing failures involving token format, demarcation, static/dynamic confusion, and slight structural variation.
- For the nine-category taxonomy of recurring log parsing failures involving token format, demarcation, static/dynamic confusion, and slight structural variation.
- For the four-parser/four-LogHub comparison and its 523 manually examined error instances.
- For evidence that tokenization-related categories dominate the observed errors and that regular-expression preprocessing is a distinct error source.
- For delimiter-aware and more consistent logging as design responses to parser failures.
Citation
@inproceedings{ezzatiJivan2023towardsa,
author = {Issam Sedki and Abdelwahab Hamou-Lhadj and Otmane Ait Mohamed and Naser Ezzati-Jivan},
title = {Towards a Classification of Log Parsing Errors},
year = {2023},
booktitle = {IEEE International Conference on Program Comprehension (ICPC)},
pages = {84-88},
publisher = {IEEE},
doi = {10.1109/ICPC58990.2023.00023},
url = {https://doi.org/10.1109/ICPC58990.2023.00023}
}Other citation formats for Word and reference managers
Sedki, I., Hamou-Lhadj, A., Mohamed, O. A., & Ezzati-Jivan, N. (2023). Towards a Classification of Log Parsing Errors. In IEEE International Conference on Program Comprehension (ICPC) (pp. 84-88). https://doi.org/10.1109/ICPC58990.2023.00023I. Sedki, A. Hamou-Lhadj, O. A. Mohamed, and N. Ezzati-Jivan, "Towards a Classification of Log Parsing Errors," in IEEE International Conference on Program Comprehension (ICPC), pp. 84-88, 2023, doi: 10.1109/ICPC58990.2023.00023