Identify what road – Practice AR technology based business processes to achieve full digitization of South Union aviation bright field of view

Editor’s Recommended Language:

Strengthening the physical world to the virtual real control, real-time connectivity to form a space network, which is the essence of the universe yuan. China Southern Airlines and bright vision together, define the era’s leading interactive space subversive gestures include standard hardware and software, data and interactive paradigm paradigm and other general-purpose base specification, AR new ways of working will also be based around aviation yuan universe.

The following article comes from aviation maintenance and engineering, author Wang Jinshen, Frank Leung, Wu Fei, Zhou laser

In practice, the digital transformation of all walks of life in full swing, China Southern Locomotive system can be described at the forefront of the industry. This article on how to broaden the extension of digital systems, new technologies and scientific means endless choices, and even a certain norms and standards, so that the typical problems of digital systems "good fake to things," a simple discussion.

5G led the advent of the era of continuous progress all interconnected. The current digital technology has become the people’s work and life indispensable element, the Internet, big data and artificial intelligence and the real economy is becoming the depth of integration continued to enhance industrial competitiveness of traditional industries technical basis points. For "Fourteen five", one of four new situation in the Civil Aviation Authority proposed the idea of ??the overall work of the new era, "the wisdom of building civil aviation breakthroughs." The larger environment, the digital transition has been successively expanded in all fields of aviation maintenance. Among them, China Southern Airlines Engineering proposes to build a digital platform configuration management as the core, aircraft, personnel and resources to the virtual world of "dynamic digital portrait", effectively reflect the real changes in the physical world.

Aircraft maintenance is a typical discrete manufacturing, a lot of changes and uncertainties in the production process, many of the factors of production between long-term in a state of weak coupling. To make configuration management penetrated into the production end, so that "all traffic is digitized," China Southern Airlines joint augmented reality leader in the field of vision technology company shiny through research and exploration, found the augmented reality (AR) technology as a digital "eye "based on AR by creating work, and strive to achieve full coverage of the digital business processes.

1 digital platform with configuration-based

Configuration management is the real-time status of the physical object systematic control method "complete description" and digital output or delivery. The basic elements of configuration management is to ensure the integrity of the target configuration, accuracy and timeliness. Therefore, the need to connect all the relevant factors of production and people, equipment, processes, etc. online, and can sense changes in second-class status. Based on state-aware configuration management is the Civil Aviation Precision engineering staff unremitting pursuit of the ideal. The past five years, relying on the progress of native cloud technology, large front-end and real-time data such as engine technology, this ideal to gradually into reality.

Updating the configuration data from the business end of each link. Based on the characteristics of the civil aviation maintenance industry, the difficulty of producing such data collection ends mainly in the following areas:

1) the content of the work is not fixed and can not use the machine or standardized collection alternative embodiment;

2) in the workplace is not fixed and can not use fixed-point acquisition mode;

3) does not allow frequent interruptions of work, can not get workers ready to fill in the data;

4) factors involved in many aspects of the human-machine batch method requires a complete loop recording.

How real-time data acquisition, and faithfully reflected in the system, it is necessary to solve the problem. Further, based on the business of the whole process real-time acquisition of data base, to build a more accurate digital models, simulation and intelligent decision-making, and ultimately back to the physical site, enabling field staff to strengthen the physical world to the virtual reality control form real-time connectivity of the network space, which is the essence of the universe yuan. Combined with increasingly sophisticated 5G, cloud computing, digital twin, AR / VR and other new technologies, applications and experience the universe yuan in recent years, ushered in the explosive growth. In order to let more people into the universe yuan, based on the interaction of AR technology space, it has become the key.

The introduction of technology changes –AR 2 Human-Computer Interaction

People + AR model is a revolutionary change in the history of maintenance industry, because it opened up the last meter man and machine connections. Compared to other common information terminals, such as law enforcement recorder, intelligent flashlight, tablets and mobile phones, the advantages of AR technology is obvious. Lack of intelligence and law enforcement records flashlight that can only meet the functional single video image recording; a cell phone or tablet at the time of data entry, although more convenient, but because of the complexity of the field work and the environment, often took out his mobile and tablet also seriously reduced efficiency, increased security risks; at the same time using a variety of equipment will also have the problem of data specifications and compatibility.

As a new generation of interactions after PC, mobile phones, mobile phones, with technology development, Nantasay believes that AR glasses can meet the actual environment of the machine’s site work, become the final person and digital world One meter best tool. Workers will do not need to re-hold workers, walkie-talkies, signature pens, flashlights, and truly liberate their hands and first perspective interventions, greatly enhance the convenience of work. At the same time, along with the depth fusion of the Internet of Things (IoT), Knowledge, 5G edge calculation, AR technology can efficiently empower one-line staff at any time.

At present, most of the AR industry scenarios focus on remote engineering collaboration with the interaction function of the AR glasses, enhance the work efficiency of the first line. As of the end of 2021, South Airlines had completed the deployment of 28 major bases and external AR remote engineering at home and abroad. On this basis, China Southern is committed to building all levels such as hardware and hardware and interactive methods since 2020, which is suitable for different functional fields in the aviation maintenance industry, based on the complete solution of the AR base, and uses this as a configuration management as The epitaxial architecture of the basic digital system, the extension of the actual power of the platform.

3 to build the cornerstone of the aviation universe –Ar

3.1 Introduction of the AR base

The AR base constructs an open source ecological AR operating system, integrated enhanced reality, IOT, digital twin, AI capabilities and automation implementation, etc., has a variety of advanced productive solutions such as automation, with smooth integration interaction, natural cloud original capacity and powerful awareness Ability, gives a new work and data vision of field workers and managers. The base defines the basic specification including universal types such as hardware and software standards, data paradigms and interactive paradigms. Subsequent AR applications and terminal outputs are built on the standard and specifications of the base, and adaptive transformation and customization.

Schematic diagram of the AR base

AR base application architecture

3.2 Hardware Standard

Aviation industry has many different working environment and work scenes, there are many different needs in hardware. After deep investigation and use of existing equipment in the market, China Southern proposes its own demand for Ar hardware, and enhances realistic glasses. It is best expected.


In Ar enhanced display of the world, the core is the technique of displaying and related technologies, the main indicators are as follows.

Transmit rate. The transmittance will directly affect the wearer’s safe use in the cloudy, nights and indoors. With optical waveguide technology, the transmittance can exceed 80%, which is almost the same as the daily lenses;

b. Perspective deviation. Distortion caused by perspective deviation affects the real world that the viewer can see will bring safety hazards. Currently, the light waveguide is minimal in the optical plan on the market.

c. No light leakage, no mirror. That is, others can’t see the information you see by the wearer, and protect the work content;

d. Brightness and contrast. Both of these decides whether it can adapt to more environments, without shading, you can see more information, and make the display more integrated into reality;

e. Resolution and display of viewing angles. They determine whether sufficient content can be displayed through the head movement and coming.

Structural design specification

Structure and design, security, comfort and scalability are indicators that need to be considered.

a. Foldable integrated design is easier to carry. Professionals mechanic design and light weight can protect the wearer’s safety, comfort, and long working stability.

b. Since smart glasses have very high design integration, there are also high power components such as display, camera, so there is a need for good heat dissipation performance. And there are three anti-design to adapt to a complicated harsh environment of mechanical work. c. Requires high scalability, quickly connect, such as ranging modules, radar modules, temperature measurement modules, portable probes, etc., adapt to different maintenance inspection environments.

d. Requires Hyper-Connect super connection technology, can make the equipment independently work, can also connect to mobile phones such as mobile phones through Bluetooth and wired, and smart glasses will be in terms of performance, battery transmission, data transmission, etc. by Bluetooth and wired connection. Get jumping.

Power management specification

Smart glasses are like a mobile phone, relatively stagnation is a battery module when all of the modules are stepped. The volume of the smart glasses is destined to obtain long-term battery life through the violent stacking battery. It can only be optimized by optimizing the chip, display, etc. to improve the battery ability, ensuring that a certain amount of work can be completed; at the same time, it is also possible to ensure that the information can only be woke up for a long time when it is only needed. Reduce standby consumption.

Audio and video capacity specifications

a. Acoustic, noise suppression mode, on-site sound reducing ability for many noisy air maintenance sites and need to hear the on-site sound to do diagnosis;

b. In terms of imaging ability, the need for full depth imaging, enhancement and recognition accuracy, but also has a fast deformation response time, ultra low static power consumption and smaller volume control;

c. The headset can produce a large angular jitter, which is unacceptable for the background. It is necessary to solve the problem of anti-shake, anti-dizziness, and reduce the blizzain. At the same time, it also needs to be established on a prerequisite for lightweight, low power consumption; D. Range ability, human signs of parameters perceived ability, and even environmentally perceived ability, while the AR glasses enhances the perception, with a more powerful active sense of intelligence.

3.3 Interaction Specification

How does the information of the three-dimensional world accurately present the wearer’s field of view, by adaptive budding parallax, the left and right eyes can output different images, reach the effect of stereoscopic visual, and then pass the algorithm to different people’s refractive, clutch distance Different and display content for small range adjustments to achieve the best display effect.

Visual simultaneous localization and mapping (vSLAM) technology through virtual space image positioning technology, in addition to the information required superimposed superimposed onto a more stable foreign space, but also to perceive the spatial position of the wearer is located, information collaboration .

TOUCH multimodal fusion technology combines speech recognition, gesture recognition, body gesture recognition, spatial orientation, AI algorithms and other interactive tools, allowing the wearer to better interact with the enhanced information space.

4 Based on AR works

AR processes to work card, for example, AR is not a simple job card card electronic chemicals, but from the beginning to write, to perform, record, feedback and a series of process reengineering. China Southern Airlines Boeing 737NG main model as a pilot on the primary site has already begun work to develop and implement landing AR job card.

Written work 4.1 AR card

At present, many companies are beginning to repair the structure of the work of the cards, in short, is to previously only people who can read the cards, the cards into a computer can read and understand. The AR card is a work based on the chemical structure of this card on the steps of the cards transition to object-oriented, through the property of the object tag for the step can, such as support for three-dimensional animation drawings focus on components, dynamic risk Fu support alerts support, AI pattern recognition support, manual information, APS information support, support aids, and even the status of implementation of past records, etc., just follow these norms enhanced label, can be presented directly on the AR device in the future.

Also added to the visual editing process, the need for improving the feedback of steps, and a selection operation judgment criteria. Research performed in conjunction with feedback on the site, for editing work AR cards for continuous adjustment and upgrading.

Perform work 4.2 AR card

By reconstruction of the job card information, and optimize human-computer interaction process, the implementation of AR job cards to workers is to bring a new work experience.

1) the scene of the steps: workers no longer face the monotony of writing tasks, but with the working process and the gradual expansion of the scenes, scenes of workers to facilitate the focus of specific tasks, to fully understand the task at the same time eliminating invalid interference information; and clear the cards process route, visual guidance, the steps described multidimensional, but also to further avoid ambiguity text may bring, while clearly grasp the current progress of the situation; provide gesture signed, voice signature and interaction a variety of convenient new way to type information is automatically signed records, more records at the same time to achieve the signing of the contents of the signing of the burden borne by workers but greatly reduced.

2) into equation work guidelines: in the scene, workers have the perspective of God-oriented, enhanced signing, auxiliary camera, automatic measurement aid can wave that is coming; Handbook of information, experience documents, APS guidelines and other means of knowledge map technology easy to touch up; backstage experts can call at any time, to provide more support. Access to knowledge, decision support workers is a problem frequently encountered in their daily work, the traditional approach is to stop work to look up manuals, consulting engineers, usually from work may also need to field and office many times. With AR way, workers can facilitate access to a variety of support required, and without leaving the site.

3) immersive work experience: the establishment of the works of AR is a resource to find someone, find someone new experience knowledge, workers can easily obtain a valid field presence to support the knowledge, tools, aircraft material. Although the "man, armed with lightweight", but at any time in the "heavy artillery" support beneath, creating a "workplace" human-centered, workers can concentrate on the greatest degree of worker processes to avoid frequent interruptions.

4.3 System Upgrade

Based on lifting AR works to bring the system is also very significant.

1) safe lifting AR work cards can provide the following security methods:

. A Risk Warning: When the flow advances to related node, the risk of triggering points are generated warning;

. B identify key parts AI: The AI ??in key position to identify the state of exception is prohibited signed or implemented the next step; c multidimensional security image retention and comprehensive report: no sense of automatic working process image retention, integrated multi-dimensional integrated reporting, you can provide review and correction system upgrade;

Environment d. 5G private network can be achieved not park workers card data, network security risk of data leakage, which do not exist.

2) enhance site management

Execution information site, the implementation of real-time video, will return back the control center. Background can instantly grasp the overall situation of the site, including the overall progress and implementation of local or even individual progress, it can also perform retrieval of relevant historical data at any time, when the tasks changed, you can also distribute tasks in real-time or background to intervene.

3) objective real-time data acquisition

Based on AR manner, data collection may extend to the end of the production of full-service links, perceptually relevant real impact on the work configuration and configuration data synchronization changes, to ensure real-time configuration data is valid.

Objective collection of field data, in addition to providing security and self-analysis than the record, but also greatly enriched the maintenance records of the data, the number of data acquisition and objectivity are unmatched by traditional means, more conducive to subsequent data processing, data analysis and knowledge to feed.

4.4 explore the underlying logic

Manual workers involved in complex cross-linked card inquiries, multiplayer interactive, not only need to study how the glasses more suitable for the cards, will also consider changing the original model of the cards. The use of eye tracker data collected eye movement indicators, workers attention allocation method of analysis in this process, the conventional way to write optimizing job card; different workers simultaneously identify and correct existing problems in the allocation of attention for the improve the quality and efficiency of maintenance has obvious effect; further, correlation analysis data will be extended to the manual, put forward the demand for improved manual.

At the same time, use AR smart glasses, eye movement, collecting route, check, and other special work video processes, eye movement, pulse wave, skin current, skin temperature, heart rate, etc., not only "portrait" Provided, it will also provide valuable data support for interactive interface design, personnel working pressure, on-site work, efficiency, safety supervision, etc.

5G digital base

The 4G network is mainly focused on public service, and there is a small network on the network. There are also certain security issues in the network architecture. The 5G network focus industry application uses a variety of technical means to increase the uplink bandwidth, reduce the transmission delay, and the network architecture also improves security.

For Ar, complex scene understanding, massive three-dimensional information, real-time virtual interaction, requires high-grade power rendering, low time extension code, and high bandwidth low-delay capacity. Only 5G custom travelers can fully meet these requirements of AR development.

In the fast moving scene, 5G makes the stable millisecond real-time data synchronization possible, and also guarantees the smoothness of the AR terminal; when the virtual information is superimposed on the three-dimensional scene based on 3D graphics, the 5G large broadband role is reflected in the improvement The real-time load speed of multimedia content such as 3D model does not need to be downloaded to the local, it needs to be loaded directly from the cloud, reducing the operational burden of the terminal device; in a real sense, with 5G low-delay attribute, and the power of the cloud server Performance, you can put the rendering task to the server side, then pass back on the local AR device in the form of a video stream so that high quality 3D rendering can be obtained, allowing the AR device of the mobile chip to display top-level paintings. Quality; in the edge computing scenario that requires rapid acquisition, 5G’s core NE UPF can be used to sink to the network edge, data processing, reduce transmission delays, can quickly perform oversized data volume , Complete the initialization of the scene, closed-loop detection, etc., quickly pass the calculation result to the user. At the same time, 5G can also significantly increase the AR transmission data capacity: 5G networks can provide: QoS exclusive custom service, MIMO technology, multi-spectral capabilities, etc., can provide a more abundant network capacity for enterprise production network.

6 Hui Eye

Based on the AR-based work mode by grinding the AR base of the industrial scene, bonding high-speed generally, security and credible 5G custom network, breaking through the traditional working paradigm, forming the digitalization of AR equipment as nodes, coupling people and digital world " ". However, we must recognize that while introducing new technologies, new tools are digitally transformed, and it is inevitably experienced, and the scientific inspection standards and iterative methods can be experienced in this process. The direction of technology development, accelerating new technologies and new tools in industry applications, and form a new generation of industry norms and standards. China Southern Airlines Engineering Technology and Lightning Technical Team gradually explored the "Taoix" (taoix) of a digital technology application test in exploration and practice, which is reflected in the following five aspects.

1) Nothing and micro-intraveniting (Touchless): The use of systems, methods and tools can give more links to the system automatic completion, reduce artificial participation. This is not only for improving efficiency, it is also an effective guarantee from the source to reduce human errors and data errors; while systems, methods, and tools are used and wearing should maximize interference on normal natural behavior and perception of users;

2) Adaptive: System, method and tools require certain perception and adjustment of adaptive capabilities, including system and people, system and environment, and system and system and system and systematic, adaptation, and automatic adjustment capabilities;

3) Continuous optimization (Optimized): The use of systems, methods, and tools can continue to optimize and improve process, steps, etc. Form a new industry practice;

4) Immersive: The use of systems, methods, and tools enhances the immersive feeling of work, so that the work process is not interrupted, changing the traditional people to find knowledge, people find resources, forming knowledge, Resources find some intelligent immersive working paradigm;

5) Extensible: As business needs continue to extend, the extension of the related applications has scalable, and the full-term segment of the relevant business is passed, each end, or even the upstream of the related industrial chain.

7 Conclusion

In summary, although China Southern Air Engineering Technology has found a "眼" that AR technology as digital is found, it is striving to create an AR-based work, but it is also very hoped to continue with many partners. Relevant explorations and research makes the AR base to provide more common, open, modular services to the industry to achieve fast deployment and multiplexing. More scene applications are the direction of our continuous efforts. We believe that with the continuous expansion of the application, the power of the AR base will continue to be verified and upgraded, and the new way of working based on Arch will lead the arrival of the era of space with a subversive attitude.

About the Author :

Wang Jinthen, China Southern Aviation Engineering Technology Branch (Matient Engineering Department), deputy general manager.

Liang Weiming, China Telecom Guangzhou Baiyun Branch, General Manager.

Wu Fei, Beijing Bright Vision Technology Co., Ltd., Chairman and Founder.

Zhou Laser, China Southern Aviation Engineering Technology Branch (Machinery Engineering Department), senior engineer.