With the rapid development of mobile Internet technology, data is growing explosively. Its importance has been raised to an unprecedented height by enterprises, and it has become a strategic asset for enterprise development. The generation of massive data brings huge potential value to enterprises, but also makes enterprises face greater challenges in data storage and processing capacity.
Before the emergence of mobile Internet, operator data was mainly generated by man-machine conversation, mainly structured data. With the development of Internet, especially mobile Internet, a large number of unstructured and semi-structured data are generated. The traditional database has obvious shortcomings in both technical and functional aspects to cope with such application requirements, while the birth of Hadoop and other technologies provides new research ideas and solutions for unstructured mass data processing.
In terms of technology, ZTE Soft Innovation through comprehensive comparison of traditional data warehouse technology and Hadoop and other new technologies, combined with years of operator service experience, put forward the construction of a new generation of operator data center solutions. Zte Soft Innovation chooses the traditional data warehouse (such as Oracle) and Hadoop to mix and match, drawing on the rapid response capability of relational data warehouse in traditional application support and complex query and analysis, as well as the unstructured data processing capability of Hadoop, so as to realize efficient data calculation and low-energy storage. Meet the daily strategic needs of operators.
Figure 1 Technical architecture diagram of enterprise data center
In terms of function, the new generation of carrier data center includes three capability centers: data production center, data service center and data management center.
Figure 2 Functional architecture diagram of enterprise data centers
As the core of the data center, the data production center is mainly responsible for the collection and processing of enterprise data to ensure the timeliness, high quality and stability of the data. The ability of computing, storage, job submission, task scheduling, data query, authentication and authorization to build the foundation for data production, testing and mining based on big data technology; Meanwhile, the real-time performance and efficiency of the platform are further improved by introducing Spark, Spark Streaming, impala and other systems.
In the aspect of interface data collection, real-time incremental collection is adopted to improve the efficiency of data storage and reduce the pressure of data extraction in the production system. The integration layer improves real-time and timeliness through real-time and incremental integration modes; In the middle layer, the base width table is calculated in the way of baseline + increment to further optimize the efficiency of the base width table calculation. The summary layer modularizes labels and counters to implement a flexible and efficient label and counter library.
The data service center is responsible for encapsulating the capabilities of the enterprise data center, providing external services through API, query, push, resources and other means, and exploring the way of data realization. Through the system catalog and portal website, the enterprise data capability can be visible and searchable; Sharing data, resources, and messages through standardized subscription, approval, and service delivery processes; Through customized encryption and desensitization, refined permission control, and detailed usage log recording and audit, secure data sharing is finally realized.
Data management center is based on data assets as the core to build data quality, data security, data management and production process management; At the same time, whole-process scheduling is taken as the center to connect management and production, improving the system's end-to-end collaborative data production efficiency and safety quality.
In 2016, ZTE Softron cooperated with Hunan Branch of China Telecom and took the lead in assisting Hunan Telecom to complete the on-line of the new generation enterprise data center (EDC) project. The newly created enterprise data center has become the connector of Hunan Telecom's big data and various businesses. With comprehensive, timely, high-quality, agile and secure capabilities, it supports the internal and external data needs of enterprises and lays a solid foundation for practicing the development strategy of Hunan Telecom and even China Telecom Group.
