“To ingest several terabytes of data per day and the underlying metadata requires a management platform. You either buy it or build it. If you choose to buy it then Arena is the industry standard.” Senior Vice President, du
du, one of the largest telecom operators in the Middle East, wanted to use its data for advanced analytics to continuously improve their customer experience while ensuring data security. This is why they chose to implement a telecom industry-compliant data lake.
Challenge: du was unable to ingest network, business, and other data sources, including geo-location data into an enterprise-wide data platform, while meeting stringent security requirements. Also, du could no longer verify its service quality reports in its Teradata systems, as the volume of the data sets was too great for the system to ingest it cost-effectively.
Solution: The data lake provided data governance capabilities (metadata, schemas), processing workflows, processing engines, and role-based access to the data and the tools in a highly secure multi-tenant environment. It also provided business end-users with visualization tools such as QLIKView, which enabled them to interactively query the data with drill-downs and roll-ups. Solution features included:
Ingested data in a file-based batch mode from various sources. Leveraged Apache Sqoop to transfer data to/from Teradata in a highly parallel manner.
Applied data management and governance best practices from the Data Management Association (DAMA). Ensured that no data with PII information was allowed to be accessed by any team. Highly secure cryptographic algorithms like SHA2 and other format-preserving encryption (FPE) were used to secure this data.
Geo-location data feed
Consumed relevant data sources (CDRs, LBS, and DPI), enhanced with data consumed from DWH, and extract and store location information for all subscribers. Encrypted location data was made query-able across several dimensions (e.g., subscriber, location, service, time, device, applications)
Results: The customer was able to measure its mobile network quality of service at the subscriber level, which allowed them to optimize the value of the network and continue to improve its customer experience. Additionally, Zaloni’s Arena DataOps platform enabled them to monetize their data by using it to develop new customer offerings.