-
Views
-
Cite
Cite
Sandra Sampaio, Mashael Aljubairah, Hapsoro Adi Permana, Pedro Sampaio, A Conceptual Approach for Supporting Traffic Data Wrangling Tasks, The Computer Journal, Volume 62, Issue 3, March 2019, Pages 461–480, https://doi.org/10.1093/comjnl/bxy113
- Share Icon Share
Abstract
Data wrangling (DW) is the subject of growing interest given its potential to improve data quality. DW applies interactive and iterative data profiling, cleaning, transformation, integration and visualization operations to improve the quality of data. Several domain-independent DW tools have been developed to tackle data quality issues across domains. Using generic DW tools requires a time-consuming and costly DW process, often involving advanced IT knowledge beyond the skills set of traffic analysts. In this paper, we propose a conceptual approach to DW for traffic data by creating a domain-specific language for specifying traffic DW tasks and an abstract set of wrangling operators that serve as the target conceptual construct for mapping domain-specific wrangling tasks. The conceptual approach discussed in this paper is tool-independent and platform agnostic and can be mapped into specific implementations of DW functions available in existing scripting languages and tools such as R, Python, Trifacta. Our aim is to enable a typical traffic analyst without expert Data Science knowledge to be able to perform basic DW tasks relevant to his domain.