From a few servers in the initial stage to a huge data center, operators have been unable to meet the requirements of technology, business, management and other aspects by relying on manpower in the face of massive data. Standardization, automation, architecture optimization, process optimization, and reduction of OPEX have become the urgent demands of operators in the intelligent transformation of IT.
In the information development work of Chongqing Telecom, the scale of server has reached 500+ hosts, and the scale of data processing tasks has reached 10,000 +(100+ real-time tasks, 4000+ daily tasks, 6000+ monthly tasks). As industry standards get higher and higher, the requirements for data timeliness (real-time, daily, monthly), quality (execution plan, process management) and security (audit, audit) are also getting higher and higher. The means and modes of operation and maintenance in the past can not keep up with the development of the information age, and the continuous increase of operation and maintenance costs has brought a great burden to enterprises.
Combined with years of operational support experience in the field of operators, whale Technology proposed intelligent scheduling solutions. Through the intelligent scheduling platform, tasks can be managed in the whole life cycle, time scenarios can be covered, process monitoring, audit and security audit can be implemented, intelligent operation and maintenance of operators can be enabled, data processing efficiency of operators can be effectively improved, data quality of operators can be improved, and data asset security can be guaranteed.
Automatic task lifecycle management
The intelligent scheduling platform can flexibly manage the whole life cycle of tasks, realize the full coverage of automatic tasks, and automatically and intelligently transform various states of tasks.
Figure 1 Lifecycle management of intelligent scheduling tasks
Real-time task management and full coverage
The intelligent scheduling platform provides full coverage of task timeliness, which can flexibly support tasks no matter in real time or in hours/days/months/seasons/years, fully realizing data timeliness management.
Figure 2 Full coverage of real-time tasks in intelligent scheduling
Visual execution plan
Intelligent scheduling platform can realize the pre-construction of data node instances, input data node specifications, generate node instances and their account period value, execution time, account period value corresponding to the above link, node instance mark, trigger strategy, parameter value and other core information. Provide step-by-step, actionable plan guidance for scheduling; The pre-generation plan avoids repeated and unordered generation during each scheduling execution.
Figure 3 Execution plan of intelligent scheduling
360-degree data analysis
In the scheduling execution process, transparent and visual scheduling processing flow is provided, providing monitoring and analysis view, exception handling mechanism, and consanguinity analysis view. Based on data kinship, a 360-degree data view covering data standards/quality requirements/security policies, data production schedule, quality and data value is constructed.
Figure 4 360 monitoring of intelligent scheduling
All-round data quality assurance
Based on data asset relationships, computing, auditing and security auditing tasks are integrated to ensure data quality and security.
Figure 5 Intelligent scheduling security audit
In 2018, the intelligent dispatching platform was deeply applied in Chongqing Telecom. At present, it has undertaken 10000+ data processing tasks of Chongqing Telecom Enterprise Information Department. The time of data uploading to China Telecom Group has been increased from 5 to 2, and the coverage of data quality has been increased to 98%, saving 60% of operation and maintenance human resources. Intelligent scheduling has become a tool of the Information Department of Chongqing Telecom. It supports the internal and external data needs of Chongqing Telecom with high efficiency, high quality, agility and security, and achieves the demands of the industry for intelligent operation.
