Building large AI models best suited to industry needs

In 2024, the Chinese government proposed the AI+ action plan in its work report, regarding AI as an engine for developing new productive forces. Over the past two years, the field of AI has seen a sudden rush of wind in its sails. The emergence of ChatGPT was like a tsunami that made large AI models a buzzword in the tech industry. After ChatGPT, many other large models sprang up worldwide. Many large model companies are targeting traditional industries and exploring ways to leverage large model technologies to help industries go digital and intelligent.

Challenges to the deployment of large models in traditional industries

In 2021, Huawei established an Oil, Gas & Mining Business Unit, with the aim of using AI to enable manufacturing industries. Today, AI is widely applied in coal mining, smelting, oil and gas, and the chemical industry. However, there are five key challenges that hinder the smooth application of AI in the industrial sector.

First, the accuracy of small models and expert models (also known as traditional models) is low. In model training and inference phases, the accuracy of many small models is relatively low, since past computing power could not meet their requirements. In addition, the knowledge storage and learning structure on which algorithms depend is simple, and the number of parameters is small. As a result, overfitting may occur when an algorithm fits too closely or even exactly to its training data, resulting in a model that can't make accurate predictions or conclusions from any data other than the training data. For example, when learning a new skill, the model forgets the old skill it has already learned. It may have learned the skill of subtraction, but then it will forget the skill of addition.

Second, traditional large models are highly customized, difficult to generalize. If a model is developed and trained for a specific scenario and then applied to other similar ones, the model needs to be re-developed or optimized. It is no small feat to smoothly migrate and replicate the model cross scenarios. This high level of customization leads to poor generalization and high costs. In one example, a building material company invested heavily to build a smart factory model. However, due to poor generalization, the model's success could not be replicated across the other 140 factories. In another example, a large smart coal mine model was verified in more than 40 scenarios, but it remained in the demonstration phase due to poor generalization. It was nearly impossible to integrate the model with real production services.

Third, negative samples (abnormal cases) cannot cover all scenarios or cases. Although the basic logic of AI is to solve a large number of problems in the production process, comprehensive negative samples are often difficult to obtain because new, unknown problems will keep emerging. It is not enough to rely merely on historical problem samples for learning and prediction.

Fourth, data security concerns abound. Data security concerns arise from concerns over the public cloud environment, especially in fields involving sensitive service data and core technologies. When production data is trained outside a company, data security issues may arise. Large enterprises attach great importance to data security. They tend to deploy AI training centers on the private cloud rather than public cloud.

Fifth, there is a lack of talent reserves. AI development has a high threshold on personnel skills. Enterprises focus on production and often lack sufficient IT talent to support AI deployment and application.

Unlike in the Internet field, the integration of large models in industrial fields is relatively slow. Large models have great potential to optimize production processes, improve product quality, and reduce operation costs, but the effect of their application in industrial fields is not ideal owing to complex environments, diverse types of data, and a high technical threshold.

Help traditional energy industry to tackle these challenges

Huawei is committed to developing innovative technologies and products to help customers tackle these challenges. To overcome the issues surrounding AI implementation in the industrial field, Huawei has launched a new, two-tier architecture of cloud-edge synergy for central training and edge inference.

The gist of this architecture is that the training center deployed on the group side and the inference mechanism on the edge side can work collaboratively. On the group side, normal data and known negative samples are used for training and development, and the trained model is pushed to the edge side for inference. During the process of inference, the system accurately judges known negative samples and also identifies and captures abnormal, unknown data. Abnormal data will then be marked and sent back to the group side periodically or quantitatively for further learning and analysis.

Through the "learning through using" cycle featuring abnormal data capture at the edge, learning and optimization at the group, and application at the edge, the model can continuously adapt to new production environments and exceptions. This effectively solves otherwise hidden issues that cannot be addressed in traditional architecture. This also improves its generalization as well as its capability to cope with new problems, negating the need for customization. The success rate of direct deployment in new scenarios exceeds 20%.

Many large models have emerged in China. However, few vendors can deploy AI training centers on private clouds, because this requires both good private cloud products and good AI training platform products. Huawei is one of the few vendors that can provide different solutions for different customers with different requirements.

It is worth mentioning that Huawei has worked to lower the skill threshold of AI development through large model workflows and deployment architecture optimization. This facilitates the quick implementation of AI in enterprises.

Pangu Model 5.0 — the model of choice for traditional industries

On June 21, 2024, Huawei officially released Pangu Model 5.0 at the Huawei Developer Conference. The model contains different parameters and a diverse range of functions, including the visual model, prediction model, Pangu natural language model, multi-style model, and scientific computing model. Huawei Pangu Model 5.0 has three prime features. It fits a diverse range of business environments, is a multi-style model, and is capable of powerful thinking. Rich innovative applications and implementation practices are designed to continuously solve industry challenges. Therefore, different models are optimized for specific application scenarios and requirements.

In terms of the large vision model, we designed a foundation model with 1 billion parameters and pre-trained the model on a dataset consisting of more than 100 million unlabeled images, achieving high classification precision on ImageNet benchmark. Take the mining industry as an example. Through AI-based video analysis, exceptions can be accurately detected in real time, improving security and efficiency. By continuously improving the performance of the large vision model, we can deploy it in more application scenarios and create new opportunities for computer vision applications.

The predictive model is oriented to structured data. It provides accurate prediction capabilities through model recommendation and convergence technologies. It can be widely used in areas like meteorological prediction, pharmaceuticals, new materials, coal blending/washing, and gasification furnaces. The prediction large model may further establish an association model between data and production results by learning and analyzing historical data, so that it can predict results corresponding to new production data. This capability helps enterprises detect potential problems in advance, optimize production processes, and improve production efficiency and device reliability. In the coal mine field, we expanded the CV large model and prediction large model according to customer requirements, achieving many desired outcomes.

In addition, through model generalization, the Pangu Model 5.0 solves the problem of large-scale industrial application that is typical of the traditional AI workshop development mode. It supports multiple natural language processing tasks, including text generation, text classification, and Q&A system.

Long-Time Coexistence of Large and Small Models

The prevalence of large models doesn't mean we should abandon small and mechanism models altogether. It's like having a car and a bike at the same time. Small and mechanism models will always have a role to play. In industrial scenarios where the system runs properly with stable working conditions, small models should continue to be used. Large models are essentially different from small models. A large model requires a large number of samples and high-quality data input. Without sample and data input, the efficacy of large models will be compromised. During scientific research or the development of new equipment and processes, there may be no available data support. In this case, mechanism, physical, and chemical knowledge must be used to describe the model, which means mechanism models are indispensable. Large models cannot completely replace small models. They will coexist in the long run to jointly drive the development of traditional industries.

Especially in the coal mining industry, Huawei provides a relatively universal IT and CT platform. Coal mining customers can innovate and develop applications on their own on Huawei's platform. The intelligent transformation of the coal mining industry is a multi-modal process that requires participation from the entire industry chain. Huawei's role is to build and provide the platform, rather than directly participating in the specific operations of coal mines.

Huawei is deepening the application of AI in the industrial field. In the steel industry, Huawei's AI applications are enabling the prediction and optimization of blast furnaces and continuous casters. In the coal industry, they have been deployed in key links such as coal washing. In all these cases, AI has demonstrated great potential and value in solving industrial problems. Industrial problems are often complex and difficult to solve. Huawei has ventured down the most difficult but also the most valuable path. With AI, Huawei is gradually surmounting these challenges and driving industry development with new productive forces.