Next generation tumor immunization technology: Single-cell analysis and artificial intelligence from immunohistogenesis


The huge progress of immunotherapy has changed the model of cancer treatment, however, in view of only a few patients have responding to the blocked the immunogenic examination and other immunotherapy strategies, more new technologies need to be used to decipher the micro-micro-microenvironment of tumor cells and tumors (TME The complex interaction between the ingredients.

Tumor immunougueology refers to a comprehensive study of TME using multi-synthetic data using immunologically, immunological proteomics, immunogenic information, etc. reflects tumor immunization state, depending on the rapid development of next-generation sequencing technology. The high-throughput genome and transcriptum data can be used to calculate the abundance of immunocytes and predict tumor antigens, however, due to batch sequencing represents the average characteristics of heterogeneous cell populations, different cell subtypes cannot be distinguished. Single-cell-based technology can better analyze TME through accurate immune cell subpopulations and spatial structures. In addition, the depth learning model based on radiographics and digital pathology has a large extent to the study of tumor immunity, which is well shown in predicting immunotherapy response. The progress and breakthroughs of these new technologies have far-reaching significance for cancer treatment.

Tumor immunization micro environment

In the past few years, the research progress of tumor immunity has made us fundamentally changed the understanding of tumors. The definition of tumors also evolved from a simple tumor cell to a complex organ pattern structure, consisting of tumor cells, immunocytes, fibroblasts, vascular endothelial cells, and other matrix cells. Various cells and components near tumors, such as structural, such as immunofuncing cells, blood vessels, extracellular matrices, etc., also known as tumor immunochemical, and have become one of the topics of tumors. TME has been proven to play a decisive role in cancer, tumor progression, metastasis and recurrence.

TME includes extremely diverse immune cells, including T lymphocytes, B lymphocytes, natural killing (NK) cells, macrophages, dendritic cells (DC), granulocytes and mycinoid inhibitory cells (MDSCs) Wait. Typically, T cells, B cells, NK cells and macrophages help inhibit tumor growth, while MDSC and regulatory T cells (Treg) tend to inhibit anti-tumor immunity. However, existing studies have confirmed that in view of the complex interaction with tumor cells, the specific role of immune cells may have changed, and even completely opposite.

In summary, various immune cell types, even different functional conditions of specific immune cell types may have an opposite effect against tumor immunity. Therefore, this requires the most advanced bioinformatics techniques to minimize the immunological characteristics of tumors to the greatest extent, and provide more information to enhance our understanding of tumor immunity.

Immune genomics in NGS era

Over the past twenty years, NGS, including all genome sequencing (WGS), full exon group sequencing (WES), and RNA sequencing (RNA SEQ) has been successfully developed and applied to acquire human genome information. NGS produces high-throughput genome and transcriptional data to lay the foundation for the study of multi-step immune responses.

Immune cells in TME

TME consists of a variety of immunocytes, for quantification of tumor immunocytocyte components, conventional methods, such as flow cytometry and immunohistochemistry (IHC), due to its high cost and low tissue availability, is not applicable to large scale analysis. With the rapid development of NGS, we can estimate the abundance of dozens of immune cell types through NGS data, which are also proven to be reliable. These analyzes are mainly DNA and RNA sequencing, especially the latter. Regarding RNA sequence data, the principle of calculation methods is mainly divided into gene set enrichment analysis (GSEA) and inverse volume.

Typically, the representative algorithm based on GSEA includes Estimate, XCell, and MCP counts. A common feature of a GSEA-based approach is to establish specific genomic sets for each of the immune cell subsets of interest. The inverse volume of the cellular component is the reverse process of cell subtype convolution in a body tissue based on gene expression characteristics. Tools based on reverse volume include Decornaseq, Pert, Cibersort, Timer, EPIC, QuantiseQ and Deconf.

Identification of tumor antigens

Body cell DNA mutation, including mononucleotide variation (SNV) and insertion and deletion (INDEL), is the main source of abnormal antigen. Currently, the Genome Analysis Toolkit (GATK) is the industry standard of SNV and Indel by analyzing WES, WGS and RNA sequence data. It is also expanded to encompass copy number variation (CNVS) and structural variation (SVS).

Furthermore, the abnormal peptide needs to be combined with HLA to assist the T cell receptor (TCR) to initiate an immune response. The prediction of HLA is essential to identify tumor antigens. HLA Miner and SEQ2HLA are two early tools for HLA typing from NGS data, Four, six and eight-bit resolutions in PHLAT, HLAREPORTER, SNP2HLA, HLA-HD, Optype, and HLA-VBSEQ in different cancers. The performance is quite good. In these tools, Polysolver is one of the recognized standard tools currently using low coverage WES data.

In addition to identifying an abnormal peptide and HLA type, antigen MHC binding pro and is the next focus of tumor antigen prediction. Many peptide-MHC-I (PMHC-I) affinity prediction tools are based on artificial neural network (ANN) training methods and position specific score matrices (PSSM), as currently widely used tools NetMHC and NetMHCPAN. Due to the diversity of MHCII binding peptide and "openness" of the binding zone, predicting PMHC II affinity is more challenging, the number of PMHC II affinity prediction methods is far less than PMHC-I.

Single cell age immunohistology

Although the study of tumor immunity is used to study NGS technology to greatly promote the development of tumors, batch sequencing may cause signals to be diluted below the detection limit and mask the reaction of individual cells. This may hide many important biological phenomena. Until recently, the technical breakthrough of single cell-related methods thoroughly changed our understanding of tumor immunity, and transitioned from regional levels from regional levels to a single cell level.

Multi-color flow cytometry

Multi-parameter analysis is functional and physically distinguished by different immune cell subsets. In addition, with the advancement of technology, the instrument design of more parameters can be measured, such as 30 parameters and 50 parameter flow cytometry. However, due to the lower parameter accuracy, or higher precision measuring parameters, especially due to overlap between fluorescent dye emission spectroscopy, these disadvantages limit the application of multi-color flow cytometry to a certain extent And further development.

Matrix flow cytometry

The mass spectrometry is a new innovation in the field, also known as flight time (CYTOF), combined with streaming cytometry with mass spectrometry. Compared to conventional flow cytometry, mass spectrometry is not a fluorom antibody with metal isotope, and then quantifies the signal using the flight time detector, the detector detects at least 40 parameters, and avoids spectral overlap. Cytof has been confirmed to be a precise tumor tissue high-dimensional analysis method for explorating immunoassays and biomarkers found.

Although in theory, mass spectrometry flow cytometry allows us to detect up to 100 parameters per cell, but the processing speed and flux are limited by ion flight. After atomization and ionization, the cells are completely destroyed during the pretreatment, resulting in the end of subsequent cell classification. In addition, Cytof may not be appropriate for the measurement of certain low expression molecular features.

Spectral flow cytometry

Spectrum flow cytometry is another latest technological advancement that promotes traditional flow cytometry. Unlike the mass spectrometer, spectral flow cytometry is still labeled antibodies with fluorescent dye, but a new detector with a dispersion optical and measuring full emission spectrum replaces conventional optical and detectors. Based on the same principle, traditional flow cytometry and spectral flow cytometry maintain considerable compatibility, particularly in terms of commercial antibodies, but can better eliminate confusion factors, such as spectral overlap to improve efficiency . With the development of compensation technology, spectral flow cytometry may replace multi-color flow cytometry.

Single cell RNA sequencing

Based on streaming cytometry combines a specific label with the corresponding cell subgroup and identifies the label, indicating that the target must be determined before the sample collection, and the initial goal limits the information obtained from these technologies, only these Technologies find "known unknown".

The emergence of single cell sequencing technology pushes a single cell sequence to a new height. Single cells can be sequenced using a standard NGS protocol for predetermined targets such as streaming cytometry.

At present, SCRNA-SEQ applications are more mature than other methods, and the re-tumor immunotherapeutic field provides us with a lot of very valuable discovery and revelation. However, the technical noise produced by trace substance is still the most significant challenge. How to separate a single cell and maintain its bioactivity, how to solve the enlarged huge technical noise and improve sensitivity, how to get the highest number of measurable genes at the lowest price, how to analyze data more effectively, these are greatly improved single cells The threshold for sequencing limits its wide application.

Immunohistology and artificial intelligence

Artificial intelligence in tumor immunization studies mainly involve the following aspects: (1) Alleviation of immunological infiltration on artificial identification of pathological sections; (2) Provide an alternative technology to identify immune cells that are difficult to identify in naked eye The subpopulation and spatial structure; (3) Provides a non-invasive method to predict the TME characteristics of a particular patient and the reaction of immunotherapy.

Tumor antigen prediction based on deep learning method

The first step in deciphering the tumor antigen is to predict the abnormal peptide. In addition to identifying a variety of algorithms of SNV, the recently designed CN learning tool is also designed to detect CNV and exhibit good performance. About HLA profiles, Bulik et al. Generated a large integrated data set, including various types of cancer tissue HLA types and HLA peptides, which announced data that can be used to train intact mass spectrum depth learning model EDGEs, which is already non-small Verification is obtained in patients with cell lung cancer (NSCLC). In addition, two very promising depth learning methods Maria and MixMHC2PRED have recently been developed, which greatly increases MHC-II prediction accuracy.

Application of radiology in tumor immunity

With the development of artificial intelligence in medical imaging, the image is not just a picture, but a large-scale digital data, the process of analyzing the imaging data using AI technology is the radiographic. Radiological techniques applied to tumor immunization are mainly used to identify biomarkers reflecting immunoff and predict the therapeutic response of ICB treated patients.

Calculation of tumor immunity

AI, or so-called digital pathology, by calculating analysis, providing new insights for exploring the interaction between immune cells and tumor cells and the key behavior of cancer biology.

Similar to radiology, digital patients combine deep learning from the image to excavate the invisible information, so that we can understand TME in cell or molecular level. Digital pathology may be a promising method for studying the relationship between TME structure and cancer biology and treatment.

Application of immunohistology in tumor immunotherapy

Identifying ICB biomarkers for patient layers

As a target of ICB, the PD-L1 expression level detected by IHC is the first predictive biomarker, but some clinical trials have shown that ICB has only slight effect on some PD-L1 high expression patients, and ICB also Will respond to PD-L1 low expression patients. Therefore, other biomarkers are urgently needed to fill this gap.

In 2014, the researchers were linked to the clinical survival rate of tumor mutation (TMB) with patients who were treated with CTLA-4 inhibitors by WES. Subsequently, other retrospective studies also prove that high TMB is related to lasting clinical benefits. Regarding the method used to evaluate TMB, due to the high cost and complexity of WES, the FDA approves two alternative NGS platforms, which is FoundationOndongOne CDX (F1CDX) and MSKCC operable cancer target integrated mutant (MSK-IMPACT), and passed Prospective study of multiple cancers has been verified.

On the other hand, immune cell infiltration, especially TIL, a key role in immune response. In order to find more desirable treatment and prognosis biomarkers, single-cell sequencing is used to identify more immune cells. TCF7 + memory T cells have been found to be related to clinical improvement in patients with melanoma after anti-PD1 treatment, while stem cell-like TCF1 + PD1 + T cells have been confirmed, which contributes to tumor control in ICB treatment. Thorbing T cell subpopulations and functional conditions associated with treatment and prognosis are determined by single-cell sequencing techniques.

Prediction of new antigen in ACT treatment

Overpounding Cell Therapy (ACT) is re-transferred to the patient by transgene or amplified autologous or allogeneic T cells to enhance anti-tumor immunity. Immunoatology is mainly used to identify the ideal tumor antigen in ACT treatment.

At present, the new antigen-specific TCR-T cell has not yet entered the clinical application. However, it is gratifying that some case reports show the neat colorectal cancer, breast cancer and cholangiocarcinoma patients with immunohistology, T cell recognition tumor new antigen Effectiveness. TRAN et al. WGS on the sample of metastatic cholangiocarcinoma patients, determined 26 species of cell mutation. The series microchrome composed of mutant genes was transcribed and transfected into autologous APC, and then the new antigen was presented with TIL of APC and the patient’s source of TIL, and finally identified antigen-specific CD4 + VB22 + T cell clones and induced epithelial cancer.

The conventional autologous APC and T cell co-cultured new antigen selection is limited due to its low flux, high cost and time consuming characteristics. In order to eliminate these obstacles, more high-throughput immunogenic new antigen detection techniques were developed. Li et al. Established a Trogocytosis-based platform where the TCR and PMHC were combined, the surface marker protein was transferred from the APC to T cells. Therefore, the ideal new antigen can be identified by analyzing labeling positive cells. In the future, these emerging immunologies will achieve high-throughput antigen selection.

Choose a new antigen for individual tumor vaccines

The immunohistological method has been widely used in vaccine development in clinical studies. In general, a new antigen for generating a personalized vaccine is identified by analyzing a tumor and normal tissue WES and RNA sequences, and is identified by an algorithm (such as NetMHCPAN).

Similar to ACT, key parameters for tumor vaccine development are ideal for new antigen identification. In order to improve the accuracy of the new antigen prediction and the new epitope selection of the immunogenicity, immunohistology technology has made unremitting efforts in these respects. In a recent study, Wells et al. Compiled all new antigen predictions and selection methods and provided a new candidate measurement pipe, including 14 immunogenic characteristics of MHC presence and T cell recognition. This study laid a solid foundation for improving the efficacy of tumor vaccine and the adverse cell therapy.


In recent years, with the huge leap of emerging technologies in the field of immunohma, we can now analyze tumor immunity before.

In the bulk sequencing era, we can better explore the individual infiltration model of tumor immunocytes, and the prediction of abnormal peptides, HLA-based and tumor antigen MHC binding pro-and predictions, using immunogenics, predicting tumor antigens have been in front of it. It proves its reliable effect in clinical research.

In addition, with the development of single-cell immune related technologies, from multi-color flow cytometry to CYTOF, single-cell tumor immunization maps help us to classify the immune cell subtracitions to decide the TME component. The emergence of artificial intelligence also provides a new direction for the development of immunohistology.

With the booming of immunohistology, sustainable development needs several issues. First, although many methods of quality control and improvement algorithm have been implemented, the effectiveness of these technologies remains to be improved. Especially in tumor antigen prediction, single cell sequencing and space resolving transcriptional groups, technical noise and mixing factors have hindered subsequent analysis. Second, we look forward to the more cost-effective, easier to obtain and more automated techniques, thoroughly change the development of the discipline. Third, we also expect researchers to make full use of prior art to explore tumor immunity and promote clinical transformation.

Although there is still a lot of work to do, the immunohistology is likely to dominate the future in the future, and its clinical value will undoubtedly promote the development of the subject in the field of immunohistology, single cells and artificial intelligence.


1.Technological Advances in Cancer Immunity: from Immunogenomics to Single-Cell Analysis and Artificial Intelligence. SignalTransduct Target Her. 2021; 6: 312.

Identify what road – Practice AR technology based business processes to achieve full digitization of South Union aviation bright field of view

Editor’s Recommended Language:

Strengthening the physical world to the virtual real control, real-time connectivity to form a space network, which is the essence of the universe yuan. China Southern Airlines and bright vision together, define the era’s leading interactive space subversive gestures include standard hardware and software, data and interactive paradigm paradigm and other general-purpose base specification, AR new ways of working will also be based around aviation yuan universe.

The following article comes from aviation maintenance and engineering, author Wang Jinshen, Frank Leung, Wu Fei, Zhou laser

In practice, the digital transformation of all walks of life in full swing, China Southern Locomotive system can be described at the forefront of the industry. This article on how to broaden the extension of digital systems, new technologies and scientific means endless choices, and even a certain norms and standards, so that the typical problems of digital systems "good fake to things," a simple discussion.

5G led the advent of the era of continuous progress all interconnected. The current digital technology has become the people’s work and life indispensable element, the Internet, big data and artificial intelligence and the real economy is becoming the depth of integration continued to enhance industrial competitiveness of traditional industries technical basis points. For "Fourteen five", one of four new situation in the Civil Aviation Authority proposed the idea of ??the overall work of the new era, "the wisdom of building civil aviation breakthroughs." The larger environment, the digital transition has been successively expanded in all fields of aviation maintenance. Among them, China Southern Airlines Engineering proposes to build a digital platform configuration management as the core, aircraft, personnel and resources to the virtual world of "dynamic digital portrait", effectively reflect the real changes in the physical world.

Aircraft maintenance is a typical discrete manufacturing, a lot of changes and uncertainties in the production process, many of the factors of production between long-term in a state of weak coupling. To make configuration management penetrated into the production end, so that "all traffic is digitized," China Southern Airlines joint augmented reality leader in the field of vision technology company shiny through research and exploration, found the augmented reality (AR) technology as a digital "eye "based on AR by creating work, and strive to achieve full coverage of the digital business processes.

1 digital platform with configuration-based

Configuration management is the real-time status of the physical object systematic control method "complete description" and digital output or delivery. The basic elements of configuration management is to ensure the integrity of the target configuration, accuracy and timeliness. Therefore, the need to connect all the relevant factors of production and people, equipment, processes, etc. online, and can sense changes in second-class status. Based on state-aware configuration management is the Civil Aviation Precision engineering staff unremitting pursuit of the ideal. The past five years, relying on the progress of native cloud technology, large front-end and real-time data such as engine technology, this ideal to gradually into reality.

Updating the configuration data from the business end of each link. Based on the characteristics of the civil aviation maintenance industry, the difficulty of producing such data collection ends mainly in the following areas:

1) the content of the work is not fixed and can not use the machine or standardized collection alternative embodiment;

2) in the workplace is not fixed and can not use fixed-point acquisition mode;

3) does not allow frequent interruptions of work, can not get workers ready to fill in the data;

4) factors involved in many aspects of the human-machine batch method requires a complete loop recording.

How real-time data acquisition, and faithfully reflected in the system, it is necessary to solve the problem. Further, based on the business of the whole process real-time acquisition of data base, to build a more accurate digital models, simulation and intelligent decision-making, and ultimately back to the physical site, enabling field staff to strengthen the physical world to the virtual reality control form real-time connectivity of the network space, which is the essence of the universe yuan. Combined with increasingly sophisticated 5G, cloud computing, digital twin, AR / VR and other new technologies, applications and experience the universe yuan in recent years, ushered in the explosive growth. In order to let more people into the universe yuan, based on the interaction of AR technology space, it has become the key.

The introduction of technology changes –AR 2 Human-Computer Interaction

People + AR model is a revolutionary change in the history of maintenance industry, because it opened up the last meter man and machine connections. Compared to other common information terminals, such as law enforcement recorder, intelligent flashlight, tablets and mobile phones, the advantages of AR technology is obvious. Lack of intelligence and law enforcement records flashlight that can only meet the functional single video image recording; a cell phone or tablet at the time of data entry, although more convenient, but because of the complexity of the field work and the environment, often took out his mobile and tablet also seriously reduced efficiency, increased security risks; at the same time using a variety of equipment will also have the problem of data specifications and compatibility.

As a new generation of interactions after PC, mobile phones, mobile phones, with technology development, Nantasay believes that AR glasses can meet the actual environment of the machine’s site work, become the final person and digital world One meter best tool. Workers will do not need to re-hold workers, walkie-talkies, signature pens, flashlights, and truly liberate their hands and first perspective interventions, greatly enhance the convenience of work. At the same time, along with the depth fusion of the Internet of Things (IoT), Knowledge, 5G edge calculation, AR technology can efficiently empower one-line staff at any time.

At present, most of the AR industry scenarios focus on remote engineering collaboration with the interaction function of the AR glasses, enhance the work efficiency of the first line. As of the end of 2021, South Airlines had completed the deployment of 28 major bases and external AR remote engineering at home and abroad. On this basis, China Southern is committed to building all levels such as hardware and hardware and interactive methods since 2020, which is suitable for different functional fields in the aviation maintenance industry, based on the complete solution of the AR base, and uses this as a configuration management as The epitaxial architecture of the basic digital system, the extension of the actual power of the platform.

3 to build the cornerstone of the aviation universe –Ar

3.1 Introduction of the AR base

The AR base constructs an open source ecological AR operating system, integrated enhanced reality, IOT, digital twin, AI capabilities and automation implementation, etc., has a variety of advanced productive solutions such as automation, with smooth integration interaction, natural cloud original capacity and powerful awareness Ability, gives a new work and data vision of field workers and managers. The base defines the basic specification including universal types such as hardware and software standards, data paradigms and interactive paradigms. Subsequent AR applications and terminal outputs are built on the standard and specifications of the base, and adaptive transformation and customization.

Schematic diagram of the AR base

AR base application architecture

3.2 Hardware Standard

Aviation industry has many different working environment and work scenes, there are many different needs in hardware. After deep investigation and use of existing equipment in the market, China Southern proposes its own demand for Ar hardware, and enhances realistic glasses. It is best expected.


In Ar enhanced display of the world, the core is the technique of displaying and related technologies, the main indicators are as follows.

Transmit rate. The transmittance will directly affect the wearer’s safe use in the cloudy, nights and indoors. With optical waveguide technology, the transmittance can exceed 80%, which is almost the same as the daily lenses;

b. Perspective deviation. Distortion caused by perspective deviation affects the real world that the viewer can see will bring safety hazards. Currently, the light waveguide is minimal in the optical plan on the market.

c. No light leakage, no mirror. That is, others can’t see the information you see by the wearer, and protect the work content;

d. Brightness and contrast. Both of these decides whether it can adapt to more environments, without shading, you can see more information, and make the display more integrated into reality;

e. Resolution and display of viewing angles. They determine whether sufficient content can be displayed through the head movement and coming.

Structural design specification

Structure and design, security, comfort and scalability are indicators that need to be considered.

a. Foldable integrated design is easier to carry. Professionals mechanic design and light weight can protect the wearer’s safety, comfort, and long working stability.

b. Since smart glasses have very high design integration, there are also high power components such as display, camera, so there is a need for good heat dissipation performance. And there are three anti-design to adapt to a complicated harsh environment of mechanical work. c. Requires high scalability, quickly connect, such as ranging modules, radar modules, temperature measurement modules, portable probes, etc., adapt to different maintenance inspection environments.

d. Requires Hyper-Connect super connection technology, can make the equipment independently work, can also connect to mobile phones such as mobile phones through Bluetooth and wired, and smart glasses will be in terms of performance, battery transmission, data transmission, etc. by Bluetooth and wired connection. Get jumping.

Power management specification

Smart glasses are like a mobile phone, relatively stagnation is a battery module when all of the modules are stepped. The volume of the smart glasses is destined to obtain long-term battery life through the violent stacking battery. It can only be optimized by optimizing the chip, display, etc. to improve the battery ability, ensuring that a certain amount of work can be completed; at the same time, it is also possible to ensure that the information can only be woke up for a long time when it is only needed. Reduce standby consumption.

Audio and video capacity specifications

a. Acoustic, noise suppression mode, on-site sound reducing ability for many noisy air maintenance sites and need to hear the on-site sound to do diagnosis;

b. In terms of imaging ability, the need for full depth imaging, enhancement and recognition accuracy, but also has a fast deformation response time, ultra low static power consumption and smaller volume control;

c. The headset can produce a large angular jitter, which is unacceptable for the background. It is necessary to solve the problem of anti-shake, anti-dizziness, and reduce the blizzain. At the same time, it also needs to be established on a prerequisite for lightweight, low power consumption; D. Range ability, human signs of parameters perceived ability, and even environmentally perceived ability, while the AR glasses enhances the perception, with a more powerful active sense of intelligence.

3.3 Interaction Specification

How does the information of the three-dimensional world accurately present the wearer’s field of view, by adaptive budding parallax, the left and right eyes can output different images, reach the effect of stereoscopic visual, and then pass the algorithm to different people’s refractive, clutch distance Different and display content for small range adjustments to achieve the best display effect.

Visual simultaneous localization and mapping (vSLAM) technology through virtual space image positioning technology, in addition to the information required superimposed superimposed onto a more stable foreign space, but also to perceive the spatial position of the wearer is located, information collaboration .

TOUCH multimodal fusion technology combines speech recognition, gesture recognition, body gesture recognition, spatial orientation, AI algorithms and other interactive tools, allowing the wearer to better interact with the enhanced information space.

4 Based on AR works

AR processes to work card, for example, AR is not a simple job card card electronic chemicals, but from the beginning to write, to perform, record, feedback and a series of process reengineering. China Southern Airlines Boeing 737NG main model as a pilot on the primary site has already begun work to develop and implement landing AR job card.

Written work 4.1 AR card

At present, many companies are beginning to repair the structure of the work of the cards, in short, is to previously only people who can read the cards, the cards into a computer can read and understand. The AR card is a work based on the chemical structure of this card on the steps of the cards transition to object-oriented, through the property of the object tag for the step can, such as support for three-dimensional animation drawings focus on components, dynamic risk Fu support alerts support, AI pattern recognition support, manual information, APS information support, support aids, and even the status of implementation of past records, etc., just follow these norms enhanced label, can be presented directly on the AR device in the future.

Also added to the visual editing process, the need for improving the feedback of steps, and a selection operation judgment criteria. Research performed in conjunction with feedback on the site, for editing work AR cards for continuous adjustment and upgrading.

Perform work 4.2 AR card

By reconstruction of the job card information, and optimize human-computer interaction process, the implementation of AR job cards to workers is to bring a new work experience.

1) the scene of the steps: workers no longer face the monotony of writing tasks, but with the working process and the gradual expansion of the scenes, scenes of workers to facilitate the focus of specific tasks, to fully understand the task at the same time eliminating invalid interference information; and clear the cards process route, visual guidance, the steps described multidimensional, but also to further avoid ambiguity text may bring, while clearly grasp the current progress of the situation; provide gesture signed, voice signature and interaction a variety of convenient new way to type information is automatically signed records, more records at the same time to achieve the signing of the contents of the signing of the burden borne by workers but greatly reduced.

2) into equation work guidelines: in the scene, workers have the perspective of God-oriented, enhanced signing, auxiliary camera, automatic measurement aid can wave that is coming; Handbook of information, experience documents, APS guidelines and other means of knowledge map technology easy to touch up; backstage experts can call at any time, to provide more support. Access to knowledge, decision support workers is a problem frequently encountered in their daily work, the traditional approach is to stop work to look up manuals, consulting engineers, usually from work may also need to field and office many times. With AR way, workers can facilitate access to a variety of support required, and without leaving the site.

3) immersive work experience: the establishment of the works of AR is a resource to find someone, find someone new experience knowledge, workers can easily obtain a valid field presence to support the knowledge, tools, aircraft material. Although the "man, armed with lightweight", but at any time in the "heavy artillery" support beneath, creating a "workplace" human-centered, workers can concentrate on the greatest degree of worker processes to avoid frequent interruptions.

4.3 System Upgrade

Based on lifting AR works to bring the system is also very significant.

1) safe lifting AR work cards can provide the following security methods:

. A Risk Warning: When the flow advances to related node, the risk of triggering points are generated warning;

. B identify key parts AI: The AI ??in key position to identify the state of exception is prohibited signed or implemented the next step; c multidimensional security image retention and comprehensive report: no sense of automatic working process image retention, integrated multi-dimensional integrated reporting, you can provide review and correction system upgrade;

Environment d. 5G private network can be achieved not park workers card data, network security risk of data leakage, which do not exist.

2) enhance site management

Execution information site, the implementation of real-time video, will return back the control center. Background can instantly grasp the overall situation of the site, including the overall progress and implementation of local or even individual progress, it can also perform retrieval of relevant historical data at any time, when the tasks changed, you can also distribute tasks in real-time or background to intervene.

3) objective real-time data acquisition

Based on AR manner, data collection may extend to the end of the production of full-service links, perceptually relevant real impact on the work configuration and configuration data synchronization changes, to ensure real-time configuration data is valid.

Objective collection of field data, in addition to providing security and self-analysis than the record, but also greatly enriched the maintenance records of the data, the number of data acquisition and objectivity are unmatched by traditional means, more conducive to subsequent data processing, data analysis and knowledge to feed.

4.4 explore the underlying logic

Manual workers involved in complex cross-linked card inquiries, multiplayer interactive, not only need to study how the glasses more suitable for the cards, will also consider changing the original model of the cards. The use of eye tracker data collected eye movement indicators, workers attention allocation method of analysis in this process, the conventional way to write optimizing job card; different workers simultaneously identify and correct existing problems in the allocation of attention for the improve the quality and efficiency of maintenance has obvious effect; further, correlation analysis data will be extended to the manual, put forward the demand for improved manual.

At the same time, use AR smart glasses, eye movement, collecting route, check, and other special work video processes, eye movement, pulse wave, skin current, skin temperature, heart rate, etc., not only "portrait" Provided, it will also provide valuable data support for interactive interface design, personnel working pressure, on-site work, efficiency, safety supervision, etc.

5G digital base

The 4G network is mainly focused on public service, and there is a small network on the network. There are also certain security issues in the network architecture. The 5G network focus industry application uses a variety of technical means to increase the uplink bandwidth, reduce the transmission delay, and the network architecture also improves security.

For Ar, complex scene understanding, massive three-dimensional information, real-time virtual interaction, requires high-grade power rendering, low time extension code, and high bandwidth low-delay capacity. Only 5G custom travelers can fully meet these requirements of AR development.

In the fast moving scene, 5G makes the stable millisecond real-time data synchronization possible, and also guarantees the smoothness of the AR terminal; when the virtual information is superimposed on the three-dimensional scene based on 3D graphics, the 5G large broadband role is reflected in the improvement The real-time load speed of multimedia content such as 3D model does not need to be downloaded to the local, it needs to be loaded directly from the cloud, reducing the operational burden of the terminal device; in a real sense, with 5G low-delay attribute, and the power of the cloud server Performance, you can put the rendering task to the server side, then pass back on the local AR device in the form of a video stream so that high quality 3D rendering can be obtained, allowing the AR device of the mobile chip to display top-level paintings. Quality; in the edge computing scenario that requires rapid acquisition, 5G’s core NE UPF can be used to sink to the network edge, data processing, reduce transmission delays, can quickly perform oversized data volume , Complete the initialization of the scene, closed-loop detection, etc., quickly pass the calculation result to the user. At the same time, 5G can also significantly increase the AR transmission data capacity: 5G networks can provide: QoS exclusive custom service, MIMO technology, multi-spectral capabilities, etc., can provide a more abundant network capacity for enterprise production network.

6 Hui Eye

Based on the AR-based work mode by grinding the AR base of the industrial scene, bonding high-speed generally, security and credible 5G custom network, breaking through the traditional working paradigm, forming the digitalization of AR equipment as nodes, coupling people and digital world " ". However, we must recognize that while introducing new technologies, new tools are digitally transformed, and it is inevitably experienced, and the scientific inspection standards and iterative methods can be experienced in this process. The direction of technology development, accelerating new technologies and new tools in industry applications, and form a new generation of industry norms and standards. China Southern Airlines Engineering Technology and Lightning Technical Team gradually explored the "Taoix" (taoix) of a digital technology application test in exploration and practice, which is reflected in the following five aspects.

1) Nothing and micro-intraveniting (Touchless): The use of systems, methods and tools can give more links to the system automatic completion, reduce artificial participation. This is not only for improving efficiency, it is also an effective guarantee from the source to reduce human errors and data errors; while systems, methods, and tools are used and wearing should maximize interference on normal natural behavior and perception of users;

2) Adaptive: System, method and tools require certain perception and adjustment of adaptive capabilities, including system and people, system and environment, and system and system and system and systematic, adaptation, and automatic adjustment capabilities;

3) Continuous optimization (Optimized): The use of systems, methods, and tools can continue to optimize and improve process, steps, etc. Form a new industry practice;

4) Immersive: The use of systems, methods, and tools enhances the immersive feeling of work, so that the work process is not interrupted, changing the traditional people to find knowledge, people find resources, forming knowledge, Resources find some intelligent immersive working paradigm;

5) Extensible: As business needs continue to extend, the extension of the related applications has scalable, and the full-term segment of the relevant business is passed, each end, or even the upstream of the related industrial chain.

7 Conclusion

In summary, although China Southern Air Engineering Technology has found a "眼" that AR technology as digital is found, it is striving to create an AR-based work, but it is also very hoped to continue with many partners. Relevant explorations and research makes the AR base to provide more common, open, modular services to the industry to achieve fast deployment and multiplexing. More scene applications are the direction of our continuous efforts. We believe that with the continuous expansion of the application, the power of the AR base will continue to be verified and upgraded, and the new way of working based on Arch will lead the arrival of the era of space with a subversive attitude.

About the Author :

Wang Jinthen, China Southern Aviation Engineering Technology Branch (Matient Engineering Department), deputy general manager.

Liang Weiming, China Telecom Guangzhou Baiyun Branch, General Manager.

Wu Fei, Beijing Bright Vision Technology Co., Ltd., Chairman and Founder.

Zhou Laser, China Southern Aviation Engineering Technology Branch (Machinery Engineering Department), senior engineer.