On December 13, Jia Yangqing, head of Alibaba Cloud computing platform, pointed out at the Flink Forward Asia 2020 Summit that embracing "integration" is the new direction of big data evolution, the integration of big data and artificial intelligence is an inevitable trend, and artificial intelligence is the ultimate export of big data business development. At the same time, he discussed the open source technology of Flink from different perspectives, as well as the development and future thinking of open source technology in the cloud era.
Flink Open Source Community 'Getting Better'
From the perspective of an ordinary open source technology enthusiast, Jia Yangqing defines the development of Flink in 2020 as "gradually entering a good state", which is mainly reflected in three aspects:
● Technology. This year, Flink has made two new breakthroughs in technology: first, Flink flow and batch integrated technology was successfully applied in the scale of Alibaba Double 11 core data business scenarios; Second, Flink real-time computing peak for the first time exceeded 4 billion PCS/second. Compared to last year, the computing performance has doubled without increasing the cluster size.
● Application. Flink is no longer a high technology, but has become the de facto standard of open source streaming processing, flying into the "ordinary home". For example, China Information and Communication Academy uses Flink as a technical blueprint to design streaming processing standards, and cloud computing companies including Ali Cloud all use Flink as the underlying engine of streaming processing.
● Community. The Flink open source community is growing rapidly, topping Apache's most active mailing list; The Flink project is one of the most submitted Apache top-level projects on Github. Last year,Flink Forward Asia had 2,000 attendees, and the Flink Geek Challenge attracted more than 4,000 developers. This year, the conference will be held both online and offline. The awards of the Flink Geek Challenge show more and more developers' enthusiasm for open source technology and the spirit of challenge.
Open source standardizes the cloud
Jia Yangqing believes that open source makes the cloud more standardized. He notes that cloud computing is becoming more standardized, which is good for its own growth. The vast array of development technologies in the cloud is becoming more consistent, which means that it's easier for cloud developers to become part of the cloud as they become familiar with the system.
Open source helps Aliyun break its own technology system and form a standard, open and public technology system on the basis of fully respecting users' original usage habits. It is very difficult for a single company to set standards.
What does Aliyun mean for open source? Jia Yangqing mentioned the following three points:
● From the perspective of organization, last year Alibaba Group set up the Open source Technology Committee, from the group level to unify and promote the implementation of Alibaba's open source technology strategy.
● In terms of engagement, Alibaba Group has used open source technology extensively to build its technology system from the beginning. Take the big data products of Alibaba's cloud computing platform division as an example. For example, real-time computing Flink uses Hadoop/K8S ecological technology extensively. Machine learning PAI also uses open source deep learning frameworks like TensorFlow.
● In terms of contribution, Alibaba Group continues to contribute to the open source community. Take the Flink technology community for example. Last year alone, Alibaba Cloud contributed millions of lines of code to the Apache Flink community. At present, the group has more than 1,000 open source projects, covering big data, AI, cloud native, database, middleware, hardware and other fields. More than 700,000 developers around the world light GitHub Star for Ali, and thousands of enthusiasts participate in the project contribution.
Embracing "convergence" is the new direction of big data evolution
Jia Yangqing said that the three development trends of cloud are mainly the cloud of IT infrastructure, the Internet of core technology, and the data and intelligence of applications, which is also the core value of cloud.
After the enterprise cloud on IT and business Internet, its application will change from process driven to data intelligent driven to realize data and intelligence. Cloud computing and low-level IaaS compete for infrastructure maturity, stability, and price, while upper-level PaaS compete for data capabilities and intelligence capabilities.
Currently, big data, including databases and IOT, is still the most basic way to use big data, and more big data and AI capabilities need to be integrated.
Therefore, embracing "integration" is the new direction of big data evolution, the integration of big data and artificial intelligence is the inevitable direction of big data development, and artificial intelligence is the ultimate export of big data business development.
Only with AI can big data bring real business value into play. The reason why Ali Cloud Big Data has core competitiveness is that big data is combined with AI ability and is the big data of intelligent upgrading. For example,Flink +Alink,Flink can not only provide the processing power of stream and batch, but also realize the integration of big data and artificial intelligence of Flink with the help of Alink machine learning algorithm library, which has been open source.
In the future, the social economy will be more integrated with the Internet, and the business of enterprises will be fully transformed into the Internet business. IT is not enough for an enterprise to only have IT infrastructure. The enterprise application will also realize the transformation from process driven to data intelligent driven. Ali cloud computing platform will output the full stack big data technology capability and provide the world's top AI technology to realize the data and intelligence together.
