WO2024153239A1 - Prediction model training method, gene expression data correction method, and downstream task execution method - Google Patents

Abstract

A prediction model training method, a gene expression data correction method, a downstream task execution method, an electronic device, and a medium. The training method comprises: acquiring a plurality of samples, wherein each of the plurality of samples comprises first gene expression data and masked gene expression data, the first gene expression data comprises respective counts of different genes in a single cell, the masked gene expression data is obtained by processing the counts of some genes in the first gene expression data, and the processing for obtaining the masked gene expression data comprises masking; for each of the plurality of samples: processing the masked gene expression data in the sample by using a prediction model to be trained, so as to obtain predicted values of the counts of some genes corresponding to the sample; according to the predicted values, and the counts corresponding to some genes in the first gene expression data, determining a loss value corresponding to the sample; and updating said prediction model according to the loss value corresponding to each of the plurality of samples.

Description

Methods for training prediction models, correcting gene expression data, and performing downstream tasks

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to application CN202310097546.2 filed on January 19, 2023 and application CN202310630156.7 filed on May 30, 2023. The prior application is deemed to be part of the disclosure of this application and is incorporated into this application as a whole.

Technical Field

The present disclosure relates to the field of artificial intelligence, and in particular to a prediction model training method, a gene expression data correction method, a downstream task execution method, an electronic device, a computer-readable storage medium, and a computer program product.

Background technique

Single-cell sequencing technology is a technology that performs sequencing analysis on the genome, transcriptome, and epigenome at the level of a single cell. Due to the interference of technical noise, there are certain differences in the sequencing results of similar cells at the same sequencing depth. This leads to certain difficulties in the results of downstream applications when the single-cell sequencing results are directly used. Therefore, the results of single-cell sequencing need to be processed in a certain way to better serve downstream applications.

The methods described in this section are not necessarily methods that have been previously conceived or employed. Unless otherwise indicated, it should not be assumed that any method described in this section is considered to be prior art simply because it is included in this section. Similarly, unless otherwise indicated, the issues mentioned in this section should not be considered to have been recognized in any prior art.

Summary of the invention

According to a first aspect of the present disclosure, a method for training a prediction model is provided, comprising: obtaining a plurality of samples, wherein each of the plurality of samples comprises first gene expression data and masked gene expression data, the first gene expression data comprises counts of different genes in a single cell, the masked gene expression data is obtained by processing counts of some genes in the first gene expression data, and the processing for obtaining the masked gene expression data comprises masking; for each of the plurality of samples: processing the masked gene expression data in the sample using the prediction model to be trained to obtain a predicted value of the counts of some genes corresponding to the sample; and calculating a predicted value based on the predicted value and the counts of some genes in the first gene expression data corresponding to the sample. The corresponding count determines the loss value corresponding to the sample; and updates the prediction model to be trained according to the loss value corresponding to each sample in the multiple samples.

According to a second aspect of the present disclosure, a method for correcting gene expression data is provided, comprising: obtaining current gene expression data, wherein the current gene expression data includes respective counts of different genes measured at an actual sequencing depth; and processing the current gene expression data using at least part of the network layers in a prediction model trained by the training method of the first aspect of the present disclosure to obtain a correction value of the current gene expression data or an intermediate processing result of the correction value.

According to a third aspect of the present disclosure, a downstream task execution method is provided, comprising: obtaining input data, wherein the input data includes i) a correction value obtained by the method according to the second aspect of the present disclosure; or ii) an intermediate processing result of the correction value according to the second aspect of the present disclosure; or iii) a preprocessing result obtained by preprocessing the correction value obtained by the method according to the second aspect of the present disclosure; or iv) a preprocessing result obtained by preprocessing the intermediate processing result of the correction value obtained by the method according to the second aspect of the present disclosure; and processing the input data using a downstream task algorithm to obtain a downstream task result, wherein the downstream task includes a cell classification task, a perturbation prediction task or a drug response prediction task.

According to another aspect of the present disclosure, an electronic device is provided, including: a processor; and a memory, wherein the memory stores instructions executable by the processor, and when the instructions are executed by the processor, the processor executes any one of the above methods.

According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing instructions is provided. When the instructions are executed by a processor, the processor executes any of the above methods.

According to another aspect of the present disclosure, a computer program product is provided, comprising: instructions, wherein when the instructions are executed by a processor, the processor is caused to execute any of the above methods.

These and other aspects of the disclosure will be apparent from and elucidated with reference to the embodiments described hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

FIG1 is a flow chart of a method for training a prediction model according to an embodiment of the present disclosure;

FIG1A is a schematic diagram of a model structure of a prediction model according to an embodiment of the present disclosure;

FIG1B is a flowchart of a method for training a prediction model according to another embodiment of the present disclosure;

FIG1C is a flowchart of a method for training a prediction model according to another embodiment of the present disclosure;

FIG2 is a flow chart of a method for correcting gene expression data according to an embodiment of the present disclosure;

FIG2A is a flow chart of a method for correcting gene expression data according to another embodiment of the present disclosure;

2B is a flow chart of a method for correcting gene expression data according to another embodiment of the present disclosure;

FIG3 is a flow chart of a downstream task execution method according to an embodiment of the present disclosure;

FIG. 4 is a block diagram of an exemplary electronic device that can be used to implement an embodiment of the present disclosure.

Detailed ways

In the present disclosure, unless otherwise specified, the use of the terms "first", "second", etc. to describe various elements is not intended to limit the positional relationship, timing relationship, or importance relationship of these elements, and such terms are only used to distinguish one element from another element. In some examples, the first element and the second element may refer to the same example of the element, and in some cases, based on the description of the context, they may also refer to different examples.

The terms used in the description of various examples described in this disclosure are only for the purpose of describing specific examples and are not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element can be one or more. As used herein, the term "plurality" means two or more, and the term "based on" should be interpreted as "based at least in part on". In addition, the terms "and/or" and "at least one of..." cover any one of the listed items and all possible combinations.

In the field of genomics, genes have expression levels in cells, and each cell has its true gene expression level. By sequencing cells, the number of reads mapped to each gene at a certain sequencing depth can be obtained. The number of reads mapped to a gene can be called the count of the gene, and the count can be understood as the gene expression level observed at the sequencing depth.

Due to the existence of technical noise, the accuracy of the relative relationship between the counts of each gene measured does not meet the requirements. Therefore, it is hoped that based on the relative relationship between the counts of each gene observed at the actual sequencing depth, the corrected relative relationship or the intermediate representation of the corrected relative relationship can be predicted through the prediction model, and the corrected relative relationship can be used as the true relative relationship, or the intermediate representation of the corrected relative relationship can be used as the intermediate representation of the true relative relationship for various downstream tasks.

Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present disclosure belongs. It is understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and/or this specification, and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

Fig. 1 is a flow chart of a prediction model training method 100 according to an embodiment of the present disclosure. As shown in Fig. 1 , the method 100 includes steps 110 to 130.

In step 110, a plurality of samples are obtained. Each of the plurality of samples includes first gene expression data and masked gene expression data, the first gene expression data includes counts of different genes in a single cell, the masked gene expression data is obtained by processing counts of some genes in the first gene expression data, and the processing for obtaining the masked gene expression data includes masking.

A single cell refers to a single cell of a human body, an animal body or other organisms. The single cells in multiple samples can be different types of cells, such as T cells, B cells, etc. A single cell expresses multiple genes. The first gene expression data is the count of each of the different genes in a single cell measured at a certain sequencing depth. Table 1 is an example of the first gene expression data. Exemplarily, the first gene expression data may include counts of N (about 20,000) genes, and the first gene expression data of each sample are composed of counts corresponding to the same gene. Exemplarily, the genes in the first gene expression data include both highly variable genes (highly variable genes are genes that are more likely to be regulated by other genes, and here, being regulated by other genes may refer to being activated or inhibited by other genes, etc.), and non-highly variable genes. Exemplarily, the first gene expression data includes genes of the entire genome or genes of the entire transcriptome.

Table 1

There are usually about 500 non-zero gene counts in the single-cell sequencing results. For example, the single-cell sequencing results whose non-zero gene counts exceed a non-zero count threshold (such as 200) can be screened out from the collected single-cell sequencing results as the first gene expression data to filter out extremely low-quality or damaged single-cell sequencing results.

Exemplarily, obtaining the first gene expression data included in the plurality of samples may be obtaining an initial gene expression matrix, where each row or column of the initial gene expression matrix corresponds to the first gene expression data in one sample. The matrix size may be C*N (C is greater than or equal to 1, which is the number of samples), with the cell identifier (or sample identifier) of a single cell and the gene identifier of each gene as the first dimension and the second dimension of the matrix respectively, and the expression amount of each gene in each single cell (hereinafter also referred to as the gene count) as is the value of the corresponding element in the matrix. For example, the cell identifiers of the single cells corresponding to the first gene expression data in the three samples are Cell 1, Cell 2, and Cell 3, respectively. The gene identifiers of the six genes in these three single cells are Gene1, Gene2, Gene3, Gene4, Gene5, and Gene6, respectively. The initial gene expression matrix is shown in Table 2. Taking the value of 2.7 in the last cell of the first row in Table 2 as an example, it means that the count of gene Gene6 in cell Cell 1 is 2.7.

Table 2

In order to enable the prediction model to learn to capture the relationship between gene expression levels in a single cell, the prediction model is trained using the MAE (Masked Autoencoders) method, that is, the prediction model is used to predict the masked gene counts based on the partially masked gene counts (the unmasked counts are used as the context of the masked counts). Therefore, when constructing the training data, the first gene expression data obtained by actual sequencing is used as the true value, and the masked gene expression data obtained by masking the counts of some genes in the first gene expression data is used as the input of the prediction model.

The masking (hereinafter also referred to as "masking") operation can be implemented by replacing the counts of the masked part of the genes in the first gene expression data with mask symbols. For example, the masked gene data can be [4, 1, M, M, 0, M, ..., M], wherein the gene counts of the masked part are replaced with the mask symbol "M", while the unmasked gene counts retain the original counts. In the obtained masked gene expression data, the gene ID corresponding to the masked gene count is known, and the gene count is unknown.

The number of masked gene counts in the masked gene expression data can be controlled by setting the mask ratio. The larger the mask ratio, the more masked gene counts.

In the following, the masked counts are also referred to as the first type of elements selected from the initial gene expression matrix, and the unmasked counts (i.e., the elements in the initial gene expression matrix other than the first type of elements) are referred to as the second type of elements selected from the initial gene expression matrix. Exemplarily, multiple elements in the initial gene expression matrix can be randomly selected as the first type of elements, and the elements in the gene expression matrix other than the first type of elements can be used as the second type of elements. Exemplarily, when selecting the masked counts, the mask ratio, i.e., the number of masked gene counts, can be determined first. For example, the masked gene counts can be set to The total number of counts is less than 70% of the total number of gene counts. Taking Table 2 as an example, the total number of gene counts in Table 2 is 18. When the total number of masked gene counts is set to be less than 70% of the total number of gene counts, the number of masked gene counts can be determined to be 12. Then, the number of masked gene counts is randomly selected as masked gene counts. For example, 12 counts are randomly selected from the first gene expression data as masked counts, and the remaining 6 elements are used as unmasked counts. Among them, in the process of selecting 12 masked counts, the selection should be carried out according to the principle of uniform randomness, for example: generate multiple random numbers, these random numbers correspond to each count respectively, and these random numbers obey a uniform distribution with a mean of 0 and a variance of 1, and select the first 12 random numbers with the largest values from these random numbers as the random numbers corresponding to the masked counts, and determine the counts corresponding to the first 12 random numbers as the masked counts.

It should be noted that the mask ratio of each sample may be different. The specific mask ratio and mask mode may be set according to actual needs and algorithm design. The above is only an example.

Exemplarily, the first gene expression data may be original sequencing data, or may be sequencing data after normalization of the original sequencing data. It is understandable that if the first gene expression data is normalized, the influence of factors such as sequencing depth on the absolute value of counts can be eliminated, so that the model focuses on the relative relationship between gene counts.

In some training tasks, the original sequencing data is not needed, and only the normalized sequencing data is needed. In this case, the first gene expression data in the sample is normalized, the masked gene expression data that is masked is also normalized, and the predicted value of each gene count predicted by the prediction model through learning is also normalized; in some training tasks, the normalized sequencing data is not needed, and only the original sequencing data is needed. In this case, the first gene expression data in the sample is not normalized, the masked gene expression data that is masked is also not normalized, and the predicted value of each gene count predicted by the prediction model through learning is also not normalized; in some training tasks, both the original sequencing data and the normalized sequencing data are needed. For example, the sample includes the normalized first gene expression data, the normalized first gene expression data corresponds to the first gene expression data (i.e., the unnormalized sequencing data), and the masked gene expression data is obtained by processing the first gene expression data.

The following uses the transcript sequencing results as an example to illustrate how the first gene expression data is collected. The scRNA-seq results of cells are stored in databases such as the Gene Expression Comprehensive Database, Human Cell Atlas, and EMBL-EBI. scRNA-seq is manually collected from these databases. seq data, and delete the data set with duplicate id. Some of the sequencing data in the database are original sequencing data, and some are normalized. In the normalized sequencing data, some of the normalized sequencing data can be restored to the original sequencing data, and some of the normalized data cannot be restored to the original sequencing data. Exemplary, in order to facilitate the unified processing of the collected data in subsequent operations, it can be collected as the original sequencing data when the data is collected. In fact, most data sets provide raw count matrices (i.e., raw sequencing data). For data sets with standardized expression profiles ("standardization" is "normalization", "expression profile" is "sequencing data"), it is converted back to the original count form: the smallest non-zero value in the original count matrix is regarded as the original count value 1, all the remaining non-zero values are divided by the minimum value, and the integer part is taken (this process restores the normalized sequencing data to the original sequencing data). For data sets with TPM or FKPM expression profiles that cannot be converted back to raw counts, keep them unchanged. In this way, transcriptome sequencing results of more than 50 million single cells were collected in multiple organs (such as heart, kidney, brain) and tissues (i.e. connective tissue, epithelial tissue, muscle tissue and neural tissue).

In step 120, for each sample in the plurality of samples, the operations of step 121 and step 122 are performed. It can be understood that step 121 and step 122 can be sub-steps of step 120.

In step 121, the masked gene expression data in the sample is processed using the prediction model to be trained to obtain the predicted value of the partial gene count corresponding to the sample.

In this step, the prediction model to be trained (hereinafter also referred to as the gene regulation relationship model) may refer to an untrained model. Exemplarily, the partial gene counts refer to the masked partial gene counts.

In one example, the architecture of the prediction model to be trained can adopt an encoder network-decoder network, a decoder network. The prediction model may include more than 1 million parameters, for example, 3 million, 10 million, or 100 million parameters. The model parameters can be increased by increasing the number of layers, the number of latent vectors in each layer, and/or the dimension of the latent vectors. Exemplarily, the decoder network or the encoder network-decoder network outputs an intermediate representation of the relative relationship between the counts of each gene after correction (hereinafter also referred to as "gene regulatory relationship representation"). Exemplarily, the prediction model to be trained also includes a network layer (for example, a multi-layer perceptron MLP) located after the decoder network or the encoder network-decoder network, which is used to project the intermediate representation of the relative relationship between the counts of each gene after correction into a predicted value of the gene count. It is understandable that all genes can be projected to obtain the predicted value of each gene count; it is also possible to project only some of the masked genes to obtain the predicted value of the masked part of the genes, and the latter can reduce the amount of calculation. In other words That is, the predicted value includes the predicted values of at least some genes (i.e., some genes that are masked), and may also include the predicted values of all genes. The predicted values mentioned in the embodiments of the present disclosure can be understood in this way. Exemplarily, the first gene expression data included in the sample is the initial gene expression matrix, and the masked gene expression data obtained by masking the first gene expression data and the predicted values of the gene counts corresponding to the sample are matrices of the same size as the initial gene expression matrix.

In step 122, a loss value corresponding to the sample is determined according to the predicted value and the counts corresponding to the part of the genes in the first gene expression data;

As mentioned above, the predicted value may include only the predicted values of the masked partial genes, or may include the predicted values of all genes. When the predicted values of all genes are included, the masked partial genes may be selected therefrom, and the loss value corresponding to the sample may be determined based on the predicted values of the masked partial genes and the counts corresponding to the masked partial genes in the first gene expression data.

In step 130, the prediction model to be trained is updated according to the loss value corresponding to each sample in the plurality of samples.

In one example, in batch training, a batch of training examples is fed into the model, the batch loss is calculated, and then the parameters are updated through backpropagation. The updates can be done one by one or simultaneously.

In one example, the preset training end condition may include any of the following: the loss value is less than the loss value threshold, the number of training times reaches a preset number of times, and the training time reaches a preset duration.

Method 100 uses the first gene expression data and the masked gene expression data obtained by masking the first gene expression data as samples to train a prediction model. By predicting the masked counts as a training task, the prediction model learns to capture the relationship between gene counts in a single cell, so that the gene expression data input into the prediction model can be corrected in the inference stage to minimize technical noise interference in sequencing, thereby improving the accuracy of the output results of downstream tasks.

In a specific embodiment, in step 120, the operation of step 123 is also performed, and step 123 may be a sub-step of step 120. In step 123, the feature tensor corresponding to the masked gene expression data is determined. It is understandable that step 123 is performed before step 121, and accordingly, after step 123, step 121 is step 121A, and the feature tensor corresponding to the masked gene expression data in the sample is processed using the prediction model to be trained.

It is understandable that before entering the prediction model to be trained, the masked gene expression data needs to be converted into its corresponding feature tensor. Exemplarily, step 123 includes: determining the masked gene expression data The feature tensor of each count in the data (hereinafter also referred to as "embedding vector"). Exemplarily, the masked gene expression data includes three types of counts: unmasked and non-zero counts, zero and unmasked counts (this case is also referred to as zero-value counts in this article), and masked counts (the feature vectors corresponding to these three types of counts are also referred to as second gene features, zero-value gene features, and first gene features in the following text, respectively). The feature tensor of each count can be determined according to the gene embedding vector representing the gene ID (also referred to as "gene identification feature" in this article) and the count embedding vector representing the gene count (also referred to as "gene expression feature" in this article), for example, by adding the two. These three types of counts all have gene IDs (also referred to as gene identifiers), and their corresponding gene embedding vectors can be determined according to the gene ID (for example, N different gene embedding vectors corresponding to N gene IDs are set one by one, and the gene embedding vector corresponding to the gene ID can be uniquely determined according to the gene ID). Exemplarily, the count embedding vector is related to the size of the count, and the count embedding vectors corresponding to non-zero counts of the same size are the same. For counts that are 0 and not masked, their count embedding vectors can be randomly generated or pre-specified (i.e., a uniform count embedding vector is specified for all counts that are 0). For masked counts, whose counts are unknown, their count embedding vectors can be pre-specified (i.e., a uniform count embedding vector is specified for all masked counts). For counts that are not masked and are not 0, the count embedding vector can be determined by a lookup table. For example, the lookup table can record the correspondence between the count value range and the count embedding vector, and count values in different ranges correspond to different count embedding vectors. For example, when the gene expression value is a decimal, the corresponding count embedding vector can be determined after rounding or classification of the gene expression value. For example, if the gene expression value is rounded to 1, it corresponds to a count embedding vector of 1. Alternatively, if the gene expression value is 1-1.99 and is classified into category 1, it corresponds to a count embedding vector of 1. However, this pre-rounding and classification rule is not flexible enough and will limit the model's ability to learn mapping. For example, the model will not be able to see the difference between gene expression values of 1.1 and 1.9, because they both correspond to gene expression values of 1 after rounding/classification. To solve this problem, illustratively, the method for determining the count embedding vector corresponding to the unmasked and non-zero count in step 123 can be: for the unmasked and non-zero count, it can be input into the module for determining the embedding vector (also called "gene expression feature extraction model") to obtain the count embedding vector corresponding to the unmasked and non-zero count. The module for determining the embedding vector includes learnable parameters, which can be updated according to the loss value corresponding to each sample in a plurality of samples, and can be trained together with the prediction model. Exemplarily, the operations performed in the module for determining the embedding vector include: randomly initializing a lookup table, which includes 100 1*768 vectors. For an unmasked and non-zero count x, first pass through a linear layer and a leaky ReLU layer to obtain an intermediate vector v1=leaky ReLU(x*w1), where w1 is Learnable parameters, size is 1*100. Afterwards, the intermediate vector v1 is processed with another linear layer (parameter is w2) and the scaling factor α to obtain the intermediate vector v2. v2＝v1*w2+α*v1. Among them, the size of w2 is 100*100, w2 and α are learnable parameters. Afterwards, v2 is normalized with SoftMax to obtain the weight vector v3, the size of v3 is 1*100. Taking v3 as the weight of each of the 100 vectors in the lookup table, the weighted sum of the 100 vectors in the lookup table is performed to obtain the count vector Ex. In this way, through adaptive discretization, the model can learn that gene expression values 1.1 and 1.2 are closer expression values, and gene expression values 1.1 and 1.9 are more different expression values, so that gene features can better characterize the relationship between different gene expression values and different count embedding vectors.

In a specific embodiment, step 121A includes: inputting the feature tensor corresponding to the unmasked and non-zero counts in the masked gene expression data into the first network layer of the prediction model to be trained; inputting the feature tensor corresponding to the masked counts in the masked gene expression data and the feature tensor corresponding to the counts of 0 in the masked gene expression data into a network layer of the prediction model to be trained that is different from the first network layer. In other words, the feature tensor corresponding to the unmasked and non-zero counts in the masked gene expression data is input into the first network layer of the prediction model to be trained to obtain the processing result of the first network layer, and then the processing result of the first network layer and the feature tensor corresponding to the masked counts in the masked gene expression data and the feature tensor corresponding to the counts of 0 in the masked gene expression data are spliced and input into the second network layer of the prediction model to be trained, wherein the second network layer is located downstream of the first network layer. In this way, the first network layer only needs to process unmasked and non-zero counts, which greatly reduces the number of parameters required for the first network layer. The zero-value counts are input into the second network layer, so that the model takes into account the influence of the zero-value counts when learning the relative relationship between genes, allowing the model to learn the potential gene regulatory relationship at the whole genome level.

In one embodiment, the prediction model to be trained includes an encoder network and a decoder network. Step 121A includes:

Step 1211, using the encoder network to encode the input of the encoder network to obtain the output of the encoder network; the input of the encoder network includes a feature tensor corresponding to the unmasked and non-zero counts in the masked gene expression data.

The input of the encoder network is also referred to as input features in this article, the feature tensor corresponding to the unmasked and non-zero counts in the masked gene expression data is also referred to as the first tensor in this article, and the output of the encoder network is also referred to as the initial encoded feature tensor or encoded tensor in this article.

Exemplarily, although the feature tensors corresponding to the three types of counts in the masked gene expression data are obtained in step 123, the feature tensors corresponding to the counts that are 0 and not masked and the feature tensors corresponding to the masked counts are not input into the encoder network. That is to say, the feature tensors corresponding to the counts that are 0 and not masked and the feature tensors corresponding to the masked counts are filtered out before being input into the encoder network, which can significantly reduce the amount of data input into the encoder network.

Exemplarily, in the input of the encoder network, after filtering out the feature tensors corresponding to the counts with a value of 0 and not masked and the feature tensors corresponding to the counts with a masked value, the feature tensors corresponding to the counts with a value of 0 and not masked are arranged according to the positions of the counts with a value of not masked and not 0 in the first gene expression data. For example, the counts corresponding to genes 2, 5, etc. of single cell 1 (or sample 1) are filtered out because they are 0 or masked, then in the part of the input of the encoder network corresponding to single cell 1, the counts corresponding to genes 1, 3, 4, etc., which are not 0 and not masked, are arranged in the order of gene numbers, as shown in Table 3.

It is understandable that the number of masked counts can be determined based on the model learning effect (it is understandable that when the proportion of masked counts is too high, accurate prediction cannot be made, and when the proportion of masked counts is too low, the prediction task is too simple), the sparsity of the first gene expression data (i.e., the proportion of 0-value counts) and the desired model scale. The smaller the desired model scale, the higher the number of masked counts can be set. The sparser the first gene expression data, the more counts are filtered out because they are 0 values, and the number of masked counts can be appropriately reduced. Exemplarily, different mask ratios can be set for 0-value counts and non-0-value counts in the first gene data to achieve a suitable total number of masked counts, while retaining more information on non-0-value counts.

table 3

It is understandable that in order to batch process multiple samples, the input of the encoder network usually contains feature tensors corresponding to multiple samples. The encoder network can batch process them only when the feature tensors corresponding to each sample have the same size. However, the number of unmasked and non-zero counts included in the masked gene expression data in different samples may be different, which requires padding them to the same size to make each sample The feature tensor size corresponding to the samples is the same. For example, the number of non-zero and unmasked gene counts included in the masked gene expression data in each sample fluctuates between 300 and 1000, and can be uniformly padded to 1000 (so that the length of the input feature tensor after padding is on the order of 10% of the total number of genes), and the padding result is shown in Table 3. In Table 3, in the feature vector corresponding to sample 2 in the input of the encoder network, the 1-999th elements of the 1000 elements correspond to non-zero and unmasked counts (corresponding to gene 11, gene 32, gene 42, gene 54, .. gene 18900, respectively), and the 1000th element is the padded element. Each element in Table 3 (including the elements corresponding to the counts and the padded elements) corresponds to a feature tensor, and these feature tensors constitute the input of the encoder network with a size of C*L*D (where C is the number of samples, i.e., the number of single cells, L is the size after uniform padding, and D is the length of the feature tensor corresponding to the element). That is to say, the input of the encoder network includes not only the feature tensor corresponding to the unmasked and non-zero counts in the masked gene expression data, but also the feature vector corresponding to the filling element. Exemplarily, a filling vector can be specified for the filling element as its corresponding feature tensor (that is, the feature tensors corresponding to all filling elements are the same), and there is no need to distinguish between the gene embedding vector and the count embedding vector. In the encoder network, the feature tensor corresponding to the filling element may not participate in the calculation and only serve as a placeholder. Exemplarily, the values of the feature tensors specified by elements of different types are different from each other, for example, the feature tensors specified for the filling elements and the count embedding vectors specified for the masked counts are different from each other.

Step 1212, obtaining the input of the decoder network according to the output of the encoder network, the feature tensor corresponding to the masked counts in the masked gene expression data, and the feature tensor corresponding to the counts of zero values.

In order to establish gene regulation relationships within the whole genome (or transcriptome), the influence of zero-valued genes must also be considered when restoring masked counts. Therefore, the information of zero-valued genes needs to be input into the decoder network.

Exemplarily, according to the position of each count in the first gene expression data, the feature tensor corresponding to the unmasked count of 0, the feature tensor corresponding to the masked count, and the output of the encoder network (understandably, the feature tensor corresponding to the filling elements in the output of the encoder network needs to be removed at this time, otherwise the size of the tensors corresponding to each sample in the input of the decoder network will be inconsistent) are merged to obtain the input of the decoder network.

Exemplarily, the feature tensor corresponding to the unmasked and non-zero counts in the masked gene expression data is replaced with the encoding tensor to obtain the input of the decoder network.

Exemplarily, the size of the input of the decoder network can be C*N*D, where C is the number of samples, i.e., the number of single cells, N is the number of genes in the first gene expression data, and D is the length of the feature tensor corresponding to each count.

Step 1213, using the decoder network to decode the input of the decoder network.

The input to the decoder network is also referred to in this article as the target encoded feature tensor or the encoded input feature vector.

Exemplarily, the output of the decoder network is an intermediate representation of the relative relationship between the counts of each gene after correction, and the size of the output of the decoder network can be C*N*F, where C is the number of samples, i.e., the number of single cells, N is the number of genes in the first gene expression data, and F is the length of the feature tensor corresponding to a single count in the decoder output.

Exemplarily, step 121A also includes step 1214, using a multilayer perceptron to project the output of the decoder network into a predicted value of the count of a portion of the genes (i.e., the masked portion of the genes). The output of the decoder network is processed by the multilayer perceptron in the prediction model to obtain the predicted value of the count of each gene corresponding to the sample. The size of the predicted value is C*N. Exemplarily, the multilayer perceptron can be trained together with other parts in the prediction model.

Exemplarily, the encoder network is used to determine the expression information and regulatory relationship information of the unmasked and non-zero counts, and the decoder network is used to restore the expression information and regulatory relationship information to the expression information of the masked counts. It is understandable that in the training stage, the model including the encoder network and the decoder network learns and mines the correlation and interaction between gene expression. In the legend stage, the output of the encoder network and the output of the decoder network are both intermediate representations of the relative relationship between the counts of each gene after correction, which contains a large amount of information that can characterize the regulatory relationship of cellular genes. The output of the encoder network and the output of the decoder network can be used for downstream tasks, or the output of the encoder network and the output of the decoder network can be preprocessed and used for downstream tasks.

In the model architecture of the prediction model, the computational complexity of the model is exponentially the length of the input data. When the amount of input data increases, the number of model parameters required also increases, which increases the computational complexity and amount of the model, making model training difficult. In practice, the amount of input data can be reduced by only selecting the expression of highly variable genes in cells as input, but ignoring non-highly variable genes will cause the regulatory relationship map learned by the model to have systematic omissions.

In the disclosed embodiment, the encoder network focuses on the feature tensors corresponding to the non-zero counts in the masked gene data, while the decoder network receives the feature tensors corresponding to all counts (i.e., the feature tensors that are not masked and are 0). The encoder network integrates information from all positions by processing the feature tensor corresponding to the counts of the unmasked and masked counts and the feature tensor corresponding to the unmasked and non-zero counts processed by the encoder network. After the zero-valued counts and the masked counts are filtered, the length of the input sequence input to the encoder network is approximately 10% of the length of the whole genome (or transcriptome). This design greatly reduces the required computing resources, allowing the encoder network to use a series of ordinary Transformer blocks to capture gene dependencies, greatly improving training efficiency and training effects. Since the encoder network module only processes unmasked and non-zero counts, the model can effectively focus on the most informative non-zero expressed genes, while allowing zero-valued genes to participate in model training in the decoder network stage, so that the model can make more comprehensive and accurate predictions based on genes with zero-valued and non-zero-valued counts. At the same time, the encoder network only processes unmasked and non-zero counts, so that the gene regulatory relationship model can have a smaller scale. This allows the whole genome expression of the cell to be used as input during training, so that the gene regulatory relationship model to be trained can learn the regulatory relationship of the whole genome (or transcriptome). In this way, the problem of the difficulty of training the regulatory relationship model and the systematic omission of the regulatory relationship map learned by the model can be solved.

In a specific embodiment, the encoder network includes M layers of encoding units. The decoder network includes N layers of decoding units. The value of M is greater than the value of N.

The encoder network often requires more layers to extract high-order features of the data, so as to better represent the original data. The decoder network can use fewer layers to complete the decoding task, because it only needs to restore the low-order feature representation to the original data, and does not need to perform feature extraction and abstraction like the encoder network. Therefore, when setting the number of layers of the encoder network and the decoder network, a deeper encoder network and a shallower decoder network can be selected. While retaining the original data information, the number of parameters of the model can be reduced, and the training efficiency and generalization ability of the model can be improved. Exemplarily, M+N=8 can be set, when M=5, N=3; when M=6, N=2. Exemplarily, the encoder network and the decoder network can adopt the structure in the existing Transformer model. Exemplarily, the encoder network adopts the structure of the encoder in the existing Transformer model, and the decoder network can adopt the architecture of Performer.

In a specific embodiment, each layer of encoding units of the encoder network includes at least one multi-head attention unit and at least one forward propagation unit. Each layer of decoding units of the decoder network includes at least one forward propagation unit and at least one linear attention unit or sparse attention unit.

Exemplarily, each layer of encoding units includes multiple units, such as a multi-head attention unit and a feed forward unit. Each layer of decoding units includes multiple units. Units, such as feed forward and linear attention, where the linear attention unit can also be replaced by a sparse attention unit.

The input of the encoder network is input into the multi-head attention unit of the encoder network, and then the residual connection and layer normalization operations, forward propagation unit, residual connection and layer normalization operations are performed in the encoder network in sequence to obtain the output of the encoder network.

Using multi-head attention units and forward propagation units as encoding units in each layer of the encoder network can help the model better capture the dependencies and semantic information between different genes, further improving the performance of the model. First, gene expression data is usually high-dimensional and sparse, with high noise and complexity, so the multi-head attention units of the encoder network can help the model better mine the associations and interactions between genes.

In the decoder network, the use of forward propagation units and linear attention units or sparse attention units can help the model better predict the first gene expression data gene by gene. Linear attention units can help the model better control the distribution of attention weights and improve the performance and stability of the model. Sparse attention units can further reduce the amount of calculation of the model and improve the efficiency of the model. It can be seen that the decoder uses lightweight units, whose parameter volume and computational complexity are lower than those of the encoder. Compared with multi-head attention units, linear attention units or sparse attention units can further reduce the algorithm time and space complexity, so that the model parameter volume can be increased to the billion level.

In general, the encoder network-decoder network model using multi-head attention units, forward propagation units, and attention mechanisms can achieve better performance and results in predicting gene expression data. This model can automatically learn complex patterns and associations in the data and accurately predict and analyze the first gene expression data.

In a specific embodiment, the masked gene expression data is normalized; the first gene expression data included in each sample of the multiple samples is the normalized first gene expression data; step 121 includes: using the prediction model to be trained to process the masked gene expression data in the sample to obtain the predicted value of the normalized value of each gene count corresponding to the sample; step 122 includes step 1220: determining the loss value corresponding to the sample based on the predicted value and the counts corresponding to some genes in the normalized first gene expression data.

In this embodiment, the method 100 includes:

Step 110, obtaining a plurality of samples, wherein each of the plurality of samples comprises normalized first gene expression data and masked gene expression data, and the normalized first gene expression data comprises a single The counts of different genes in the cell are respectively obtained by processing the counts of some genes in the first gene expression data, and the processing for obtaining the masked gene expression data includes masking.

It is understandable that both the first gene expression data (unnormalized) and the normalized first gene expression data include the counts of different genes in a single cell that are actually measured, except that the normalized first gene expression data is normalized.

It is understandable that, exemplarily, the masked gene expression data is obtained by processing the counts of some genes in the normalized first gene expression data, and the processing for obtaining the masked gene expression data includes masking; exemplarily, the masked gene expression data is obtained by processing the counts of some genes in the first gene expression data (unnormalized original sequencing data), and the processing for obtaining the masked gene expression data includes normalization and masking. In either case, the masked gene expression data is normalized.

Step 120, for each sample in the plurality of samples, execute steps 121 and 122:

Step 121, using the prediction model to be trained to process the masked gene expression data in the sample to obtain the predicted value of the normalized value of the count of the partial gene (i.e., the masked partial gene) corresponding to the sample;

Step 122, determining the loss value corresponding to the sample according to the predicted value and the counts corresponding to the part of the genes in the normalized first gene expression data (ie, step 1220);

Step 130: update the prediction model to be trained according to the loss value corresponding to each sample in the multiple samples.

Making the sequencing data used as the true value in the sample the normalized first gene expression data can eliminate the influence of factors such as sequencing depth on the absolute value of the count, and make the model focus on the relative relationship between gene counts. At this time, the predicted value output by the prediction model is also the normalized predicted value.

In a specific embodiment, the masked gene expression data is normalized, and each sample in the multiple samples includes normalized first gene expression data and masked gene expression data, the first gene expression data (i.e., the original gene expression data before normalization corresponding to the normalized first gene expression data) includes the counts of different genes in a single cell measured at a first sequencing depth, and the masked gene expression data is obtained by processing the counts of some genes in the first gene expression data, and the processing for obtaining the masked gene expression data includes downsampling, normalization, and masking; the downsampled first gene expression data simulates the counts measured at a second sequencing depth lower than the first sequencing depth. The method further comprises the following steps: obtaining the counts of different genes in a single cell; each sample in the multiple samples further comprises auxiliary information, the auxiliary information comprising a first total count and a second total count, the first total count being the sum of the counts of each gene in the first gene expression data; the second total count being the sum of the counts of each gene in the downsampled first gene expression data; step 121 comprises step 121B: processing the masked gene expression data and the auxiliary information in the sample using the prediction model to be trained to obtain a predicted value of the normalized value of the counts of some genes (i.e., some masked genes) at the first sequencing depth corresponding to the sample.

In this embodiment, the method 100 includes:

Step 110, obtaining multiple samples, wherein each of the multiple samples includes normalized first gene expression data, masked gene expression data and auxiliary information, the first gene expression data includes counts of different genes in a single cell measured at a first sequencing depth, the masked gene expression data is obtained by processing the counts of some genes in the first gene expression data, the processing for obtaining the masked gene expression data includes downsampling, normalization, and masking, the masked gene expression data is normalized, the downsampled first gene expression data simulates the counts of different genes in a single cell measured at a second sequencing depth lower than the first sequencing depth, the auxiliary information includes a first total count and a second total count, the first total count is the sum of the counts of each gene in the first gene expression data; the second total count is the sum of the counts of each gene in the downsampled first gene expression data.

It is understandable that in this embodiment, the first gene expression data (original sequencing data) needs to be used when calculating the first total count and when downsampling, and the normalized first gene expression data needs to be used as the true value corresponding to the masked count, that is, in the training task of this embodiment, both the original sequencing data and the normalized sequencing data need to be used. As mentioned above, if the original sequencing data is obtained from the database, it can be normalized to obtain the normalized sequencing data; if the normalized sequencing data is obtained from the database, it can be restored (the inverse of the normalization process) to obtain the original sequencing data.

It is understandable that the first sequencing depth refers to the higher sequencing depth actually used when measuring the first gene expression data, and the second sequencing depth refers to the lower sequencing depth simulated by the downsampled first gene expression data. The first sequencing depth and the second sequencing depth are used to indicate the sequencing depth corresponding to the first gene expression data and the mask expression data in the sample, respectively, and do not limit the first sequencing depth or the second sequencing depth of all samples to be the same.

The relative relationship between the counts of each gene measured under different sequencing methods and sequencing depths may be different. It is understandable that the higher the sequencing depth, the greater the relative relationship between the observed counts of each gene. A higher probability is closer to the relative relationship between the actual gene expression of each gene in the cell. For example, when the sequencing depth is too low, the observed counts of some genes are 0, but when the sequencing depth is high enough, the observed counts are no longer 0. However, due to various restrictions, sometimes only a low sequencing depth is used in actual sequencing without using the expected high sequencing depth. At the same time, due to the existence of technical noise, the accuracy of the relative relationship between the measured counts of each gene does not meet the requirements. Therefore, a prediction model is required to correctly predict the gene counts at a high sequencing depth based on the gene counts at a low sequencing depth. Since a single cell will be damaged after one sequencing, it is impossible to repeat sequencing of the same cell to obtain gene expression data at high and low sequencing depths, and thus it is impossible to construct gene expression data at low sequencing depth for each cell by actual sequencing methods - gene expression data at high sequencing depth as training samples. The only way is to simulate the gene expression data of the cell at low sequencing depth by algorithms based on the gene expression data at high sequencing depth, i.e., the first gene expression data after downsampling, so as to construct the first gene expression data (equivalent to the gene expression data at low sequencing depth) - the first gene expression data (i.e., the gene expression data at high sequencing depth) after downsampling for each cell for training. During actual training, the two can be normalized, and then the gene expression data at low sequencing depth can be masked, and the masked gene expression data - the normalized first gene expression data can be used as training samples. That is, the normalized first gene expression data is used as the true value of the gene count at high sequencing depth, and the model is trained by MAE, so that the prediction model can learn to capture the relationship between the gene expression of similar cells at different sequencing depths.

In order to make it easier for the prediction model to predict the gene count at a high sequencing depth based on the gene count at a low sequencing depth, thereby reducing the learning cost and accelerating the model convergence, the prediction model can be explicitly informed of the current input low sequencing depth and the high sequencing depth that is expected to be predicted, so that the sample includes auxiliary information reflecting the low sequencing depth and the high sequencing depth, allowing the prediction model to process the masked gene expression data and the auxiliary information.

Although the first sequencing depth can be obtained through actual sequencing and is known, the second sequencing depth is unknown. The higher the sequencing depth, the greater the sum of the counts of each gene in the sequencing result. Therefore, the sequencing depth can be characterized by the sum of the counts of each gene at the sequencing depth, so that the first sequencing depth and the second sequencing depth can be characterized by the same measure. Therefore, the auxiliary information can include a first total count T and a second total count S, the first total count being the sum of the counts of each gene in the first gene expression data, which is used to characterize a high sequencing depth, and the second total count being the sum of the counts of each gene in the downsampled first gene expression data, which is used to characterize a low sequencing depth. When the first gene is obtained by transcript sequencing (such as RNA-Seq technology), the first gene expression data is obtained by transcript sequencing (such as RNA-Seq technology). When the first gene expression data is obtained, the first total count may also be referred to as a first total transcript count, that is, the sum of the transcript counts of each gene in the first gene expression data.

In one example, masked gene expression data can be obtained from the first gene expression data in the following manner: first, downsampling the first gene expression data; second, normalizing the downsampled first gene expression data to obtain normalized first gene expression data; and third, masking the counts of some genes in the normalized first expression data to obtain masked gene expression data.

The downsampling method is described below.

It is understandable that if the first gene expression data is measured at a sufficiently high sequencing depth, downsampling it can obtain a more credible gene count at a low sequencing depth. However, if the first gene expression data is originally measured at a low sequencing depth, the gene count at a low sequencing depth obtained by downsampling it is less credible. Therefore, in one example, only when the sum of the counts of each gene in the first gene expression data is greater than a preset threshold (e.g., 1000) (as described above, the sequencing depth can be characterized by the sum of the counts of each gene at the sequencing depth, so it is considered that the sequencing first gene expression data with the sum of the counts of each gene greater than the threshold is measured at a sufficiently high sequencing depth) is downsampled; for the case where the sum of the counts of each gene in the first gene expression data is not greater than the preset threshold, the normalized first gene expression data can be directly masked to obtain masked gene expression data without downsampling. By including samples of masked gene expression data obtained without downsampling, the model can learn to capture the relationship between genes in a single cell, but cannot learn the association between gene expression amounts at different sequencing depths.

In one example, for the first gene expression data whose sum of counts of each gene is greater than a preset threshold, a downsampling probability may be set, that is, the first gene expression data may be downsampled with a certain probability. For example, for the first gene expression data whose sum of counts of each gene is greater than a preset threshold, a 50% probability of downsampling the data may be set, and a 50% probability of not downsampling the data may be set.

In one example, for the first gene expression data that is determined to be downsampled, a sampling ratio (i.e., sampling factor) can be set when downsampling. The sampling ratio represents the degree of downsampling of the first gene expression data. By setting a variety of different sampling ratios, the first gene expression data that has been downsampled to different degrees can be obtained from the same first gene expression data, so that the gene counts at different low sequencing depths can be simulated with the gene counts at the same high sequencing depth, thereby expanding the training data and achieving the augmentation of the training data. Exemplarily, the sampling ratio can be set by using Poisson Distribution, Bayesian Distribution, etc. The sampling ratio of the first gene expression data is determined by a statistical sampling algorithm such as Beta-Binomial Distribution or Binomial Distribution.

The normalization method is described below.

The normalization method can be applied to the process of obtaining normalized first gene expression data from first gene expression data, and can also be applied to the process of normalizing downsampled first gene expression data.

In one example, the sequencing data to be normalized is normalized by dividing each count in the sequencing data to be normalized (such as each count in the first gene expression data) by the sum of each count in the sequencing data to be normalized (such as the sum of each gene count in the first gene expression data).

In one example, a normalization method such as TPM or RPKM may be used to normalize the first gene expression data or the downsampled first gene expression data.

In one example, the log(TPM+1) normalization method can also be used to normalize the first gene expression data and the downsampled first gene expression data. For example, the first gene expression data is a vector X1, and the normalized first gene expression data is log[X1/sum(X1)*10000+1]. The downsampled first gene expression data is a vector X2, and the downsampled and normalized first gene expression data is log[X2/sum(X2)*10000+1]. The log(TPM+1) normalization method performs a logarithmic transformation based on TPM. Logarithmic transformation can reduce the skewness in the original data and improve the comparability between variables. In this method, a constant 1 is added to the TPM of each first gene expression data, and then the logarithm is taken. The final result is a real number, representing the TPM value after logarithmic transformation. In short, the log(TPM+1) normalization method is a variant method of logarithmic transformation of the standard TPM normalization method, which can eliminate skewness and improve the comparability between variables.

The masking method has been explained in the previous article and will not be repeated here.

Step 120, for each sample in the plurality of samples, execute steps 121 and 122:

Step 121, use the prediction model to be trained to process the masked gene expression data and auxiliary information in the sample to obtain the predicted value of the normalized value of the count of some genes (i.e., the masked part of the genes) at the first sequencing depth corresponding to the sample (i.e., step 121B).

Exemplarily, in step 121B, the feature tensor corresponding to the masked gene expression data and the feature tensor corresponding to the auxiliary information in the sample may be processed using the prediction model to be trained.

Step 122, determining the loss value corresponding to the sample according to the predicted value and the counts corresponding to the part of the genes in the normalized first gene expression data (ie, step 1220);

Step 130: update the prediction model to be trained according to the loss value corresponding to each sample in the multiple samples.

In a specific embodiment, the prediction model to be trained includes an encoder network and a decoder network; step 121B includes:

Step A: Encode the input of the encoder network using the encoder network to obtain the output of the encoder network; the input of the encoder network includes a feature tensor corresponding to the unmasked and non-zero counts in the masked gene expression data, and the input of the encoder network also includes a feature tensor corresponding to the auxiliary information.

It is understandable that, before step A, method 100 further includes step 123 for determining a feature tensor corresponding to the masked gene expression data and step 125 for determining a feature tensor corresponding to the auxiliary information.

Exemplarily, the feature tensor corresponding to the auxiliary information and the feature tensor corresponding to the unmasked and non-zero counts in the masked gene expression data are used as the input of the encoder network together, and the two are input to the prediction model from the first network layer of the prediction model. For example, the feature tensor corresponding to the auxiliary information can be spliced after the feature tensor corresponding to the unmasked and non-zero counts. At this time, the input of the encoder network is [the feature tensor corresponding to gene 1, the feature tensor corresponding to gene 3, the feature tensor corresponding to gene 4, ... the feature tensor corresponding to the filler element, the feature tensor corresponding to the first total count T, and the feature tensor corresponding to the second total count S].

Step B: obtaining an input to a decoder network based on the output of the encoder network, the feature tensor corresponding to the masked counts in the masked gene expression data, and the feature tensor corresponding to the counts of 0 values in the masked gene expression data; and

Step C: Decode the input of the decoder network using the decoder network.

Similar to step 121A, step 121B may further include step D: using a multilayer perceptron to project the output of the decoder network into a predicted value of the normalized value of each gene count at the first sequencing depth.

For the description of steps B to D, please refer to the description of steps 1212 to 1214 and will not be repeated here.

A specific example corresponding to this specific implementation is shown in FIG1A. In this specific example, the prediction model includes an encoder network and a decoder network, the first gene expression data is subjected to Bayesian downsampling to obtain downsampled first gene expression data, the count sum of the first gene expression data is calculated as the first total count T, the count sum of the downsampled first gene expression data is calculated as the second total count S, the downsampled first gene expression data is normalized (not shown in FIG1A), masked to obtain masked gene expression data, and the 0-value count and the masked value count in the feature tensor corresponding to the masked gene expression data are removed. The feature tensor corresponding to the count is spliced with the feature tensor corresponding to T and S (the feature tensor used is not shown in FIG1A), and the spliced result is input into the encoder network (Encoder in FIG1A) to obtain the output of the encoder network, and the output of the encoder network is combined with the feature tensor corresponding to the masked count and the feature tensor corresponding to the 0-value count to obtain the input of the decoder network, which is processed by the decoder network to obtain the output of the decoder network, and the output of the decoder network enters the MLP to obtain the predicted value of each gene count at the first sequencing depth, and the predicted value and the normalized first gene expression data (the difference between the normalized first gene expression data and the unnormalized first gene expression data is not shown in FIG1A) calculate the loss value (reconstruction loss in FIG1A). Pooling the output of the encoder network can obtain a cell representation (cellular embedding in FIG1A).

In a specific embodiment, the prediction model to be trained includes an input layer, an output layer, and a plurality of intermediate layers between the input layer and the output layer; using the prediction model to be trained to process the masked gene expression data and auxiliary information in the sample includes:

Step 1: inputting the feature tensor corresponding to the masked gene expression data in the sample into the input layer and the first predetermined layer of the plurality of intermediate layers to obtain an intermediate feature tensor, wherein the number of the first predetermined layers is greater than or equal to 0;

It is understandable that, before step 1, method 100 further includes step 123 for determining a feature tensor corresponding to the masked gene expression data.

Step 2: Concatenate the feature tensor corresponding to the auxiliary information with the intermediate feature tensor to obtain a concatenated intermediate feature tensor;

It is understandable that, before step 2, method 100 further includes step 125 for determining a feature tensor corresponding to the auxiliary information.

Step 3: Input the concatenated intermediate feature tensor into a second predetermined layer among the multiple intermediate layers and an output layer, where the second predetermined layer is different from the first predetermined layer.

Similar to step 121A, step 121B may also include step 4: using a multilayer perceptron to project the output of the output layer into a predicted value of the normalized value of each gene count at the first sequencing depth.

In this embodiment, the masked gene expression data can be input from the first network layer of the prediction model, and the auxiliary information can be input from a network layer of the prediction model that is different from the first network layer. For example, when the prediction model includes a decoder network, the feature tensor corresponding to the masked gene expression data is input to the first layer of the decoder network, and the feature tensor corresponding to the auxiliary information is input before the last several layers of the decoder network; for another example, when the prediction model includes an encoder network and a decoder network, the masked gene expression data is input to the first layer of the encoder network. Due to the feature tensor corresponding to the expression data, the feature tensor corresponding to the auxiliary information is input after a certain network layer of the encoder network, or the feature tensor corresponding to the auxiliary information is input before the last several layers of the decoder network. Exemplarily, the specific input method of the feature tensor corresponding to the auxiliary information can be to splice the feature tensor corresponding to the auxiliary information with the intermediate feature tensor obtained by processing the feature tensor corresponding to the masked gene expression data by the previous network layer (the first network layer + the first predetermined layer in multiple intermediate layers) (for example, splicing the feature tensor corresponding to the auxiliary information at the tail of the intermediate feature tensor), and input the spliced intermediate feature tensor into the subsequent network layer (the second predetermined layer in multiple intermediate layers + the output layer). In this way, the model can use auxiliary information to extract more meaningful features or make more accurate predictions in the process of processing input data.

In a specific implementation, step 125 specifically includes:

Step 1251, inputting the auxiliary information into a module for determining an embedding vector to obtain a count embedding vector corresponding to the auxiliary information; wherein the module for determining the embedding vector includes learnable parameters;

Exemplarily, the first total count T and the second total count S are regarded as ordinary counts, and a module for determining embedding vectors is used to determine a count embedding vector corresponding to the first total count and a count embedding vector corresponding to the second total count.

Step 1252, obtaining the gene embedding vector corresponding to the auxiliary information.

Exemplarily, the first total count T and the second total count S are assigned symbols that are different from each other and from other gene IDs, and the corresponding gene embedding vectors are determined using the symbols.

Step 1253, obtaining a feature tensor corresponding to the auxiliary information according to the count embedding vector corresponding to the auxiliary information and the gene embedding vector corresponding to the auxiliary information.

Exemplarily, the count embedding vector corresponding to the auxiliary information and the gene embedding vector corresponding to the auxiliary information are added in place to obtain a feature tensor corresponding to the auxiliary information.

Exemplarily, the module for determining the embedding vector in step 125 and the module for determining the embedding vector in step 123 may be the same module and have the same parameters.

In a specific embodiment of the present disclosure, as shown in FIG. 1B , the present disclosure embodiment further provides a gene regulation relationship model generation method 100B, which includes steps 110B to 150B.

Step 110B: Obtaining an initial gene expression matrix corresponding to the target cells, wherein the initial gene expression matrix includes the expression levels of the highly variable genes and the expression levels of the non-hypervariable genes in the cells.

In this step, the target cell may refer to a cell in a human or animal body, and the target cell may contain multiple genes. The target cell may include multiple different types of cells, such as T cells, B cells, Etc. The initial gene expression matrix may refer to a matrix used to characterize the expression of different genes in different cells. The initial gene expression matrix includes the expression of highly variable genes and non-hypervariable genes in a single cell, that is, the expression of the whole genome of a single cell. Among them, highly variable genes refer to genes whose expression varies greatly in different cells. For example, if the expression of a gene is extremely large in some cells and extremely small in other cells, then the gene is a highly variable gene. A highly variable gene is a gene that is more likely to be regulated by other genes. Here, being regulated by other genes may refer to being activated or inhibited by other genes. The non-zero proportion of gene expression in cells is about 10%, so the initial gene expression matrix is a very sparse matrix. The more elements with a value of 0 in the matrix, the sparser the matrix is, and the fewer elements with a value of 0 in the matrix, the denser the matrix is.

Exemplarily, step 110B includes taking the cell identifier of the target cell and the gene identifier of each gene in the target cell as the first dimension and the second dimension of the matrix respectively, and taking the gene expression value of each gene as the value of the corresponding element in the matrix to construct an initial gene expression matrix; wherein the gene expression value of each gene is obtained by gene sequencing.

Step 120B: randomly selecting multiple elements in the initial gene expression matrix as first-category elements, and taking elements in the gene expression matrix other than the first-category elements as second-category elements.

In this step, the first type of elements may refer to the elements in the initial gene expression matrix that are filtered out for the first time, and the role of the first type of elements is to reduce the amount of input data of the gene regulation relationship model to be trained at the whole genome level. The second type of elements may refer to the elements in the initial gene expression matrix that are not filtered after the first filtering. Exemplarily, the selection method of the first type of elements is as described in the above method of selecting masked counts.

Step 130B: Determine input features according to the positions of the elements in the initial gene expression matrix, the input features including second gene features corresponding to elements of the second category whose values are non-zero, and the input features do not include first gene features corresponding to the first category elements and zero-value gene features corresponding to elements of the second category whose values are zero, the second gene features being determined according to the expression of the gene and the gene identifier corresponding to the gene;

Exemplarily, step 130B includes: generating a second gene expression feature according to the element value of the element whose element value is not 0 in the second category of elements; generating a second gene identification feature according to the gene identification corresponding to the element whose element value is not 0 in the second category of elements; determining a second gene feature according to the second gene expression feature and the second gene identification feature (specifically, the element value of the element whose element value is not 0 in the second category of elements is input into the gene expression feature extraction model to generate the second gene expression feature, The parameter values of the gene expression feature extraction model are adjusted according to the loss value of the following step 150B). The method 100B also includes: determining the first gene feature according to the first gene expression feature corresponding to the first-category element and the first gene identification feature corresponding to the first-category element; wherein the first gene expression features corresponding to all the first-category elements are the same, and the first gene expression feature is different from the second gene expression feature; determining the zero-value gene feature according to the third gene expression feature corresponding to the element with a value of 0 in the second-category element and the third gene identification feature corresponding to the element with a value of 0 in the second-category element.

In this step, the input features include the second gene features corresponding to the elements in the second category whose values are not 0, and the input features do not include the first gene features corresponding to the first category elements and the zero-value gene features corresponding to the elements in the second category whose values are 0, and the second gene features are determined based on the expression level of the gene and the gene identifier corresponding to the gene.

In this step, the input feature may refer to the gene feature corresponding to the remaining elements after the first filtering and the second filtering. Among them, the second filtering may refer to the filtering of the elements with a value of 0 in the second class elements. The second gene feature is determined according to the gene expression feature and the gene identification feature, and the second gene features corresponding to the elements with different gene expression amounts and/or different gene identifications are different. The first class elements, the second class elements with a value of 0, and the non-zero elements in the second class elements all correspond to gene features, and the corresponding gene features are respectively called the first gene feature, the second gene feature, and the zero-value gene feature.

Exemplarily, in this step, the input feature can be a tensor of C×L×D, where C is the number of cells, L is the number of genes with the maximum non-zero expression expected in a single cell, and D is the length of the gene feature. If the number of unmasked and non-zero genes of a cell is less than L, it is filled to L. The fill-in elements correspond to fill-in gene features, for example, a unified fill-in gene feature is specified for all fill-in elements. Exemplarily, the input feature can be a tensor of the number of non-zero elements of the second category × D, where the second category non-zero elements in the input feature are arranged in the order of row number and column number (for example, first in order from small to large according to the row number of the element, and then in order from small to large according to the column number).

Exemplarily, gene features can be determined by gene expression features and gene identification features. Gene expression features are related to the gene expression levels corresponding to the elements. If the gene expression levels corresponding to the elements are the same, then the corresponding gene expression features are the same. Gene identification features are related to the gene identifications corresponding to the elements. If the gene identifications corresponding to the elements are the same, then the corresponding gene identification features are the same. Gene expression features and gene identification features can be determined by looking up a mapping relationship table, or by other methods. For example, a mapping relationship between a gene identifier and a gene identification feature is preset (for example, gene identifier 1 corresponds to gene identification feature 1...), so that the gene identification feature can be determined based on the gene identifier. Pre-set A mapping relationship between gene expression and gene expression features is set so that the gene expression features can be determined based on the gene expression. It is understandable that the gene expression is usually a decimal, and the gene expression can be rounded or classified to determine the corresponding gene expression features. For example, if the gene expression is rounded to 1, it corresponds to gene expression feature 1. Alternatively, if the gene expression is 1-1.99 and is classified into category 1, it corresponds to gene expression feature 1. The gene feature corresponding to the Pad element can be a specified value of the gene feature of different other elements.

It is understandable that the gene expression corresponding to the first type of element is unknown, so a unified first gene expression feature can be assigned to the first type of element. The position of the first type of element is known, so the gene identification feature corresponding to the first element can be determined. The gene expression feature of the zero-value element in the second type of element is the same, and the gene identification feature can be determined according to the gene identification corresponding to its position.

Exemplarily, determining the second gene feature according to the second gene expression feature and the second gene identification feature may be element-wise adding the second gene expression feature and the second gene identification feature to obtain the second gene expression feature. The first gene feature and the zero-value gene feature may also be determined in this manner.

Exemplarily, the first gene expression feature may be different from any second gene expression feature. For example, the second gene expression features corresponding to gene expression levels of categories 1-10 are gene expression features 1-10, respectively, and the first gene expression feature may be gene expression feature 11. The third gene expression feature corresponding to the element with a value of 0 in the second category element may correspond to gene expression feature 0.

Exemplarily, the element value of the element whose element value is not 0 in the second class of elements generates the second gene expression quantity feature, including: inputting the element value of the element whose element value is not 0 in the second class of elements into the gene expression quantity feature extraction model to generate the second gene expression quantity feature. As mentioned above, the mapping relationship between gene expression quantity and gene expression quantity feature can be preset, and the gene expression quantity feature is determined according to the gene expression quantity. However, this pre-setting is not flexible enough and will limit the ability of the model to learn mapping. For example, the model will not be able to see the difference between gene expression quantities 1.1 and 1.9, because they both correspond to gene expression quantity 1 after rounding/classification. Therefore, a gene expression quantity feature extraction model can be set, and the element value of the element whose element value is not 0 in the second class of elements is input into the gene expression quantity feature extraction model to generate the second gene expression quantity feature. This model can be updated together with the gene regulation relationship model to be trained, that is, when the training termination condition is not met, the model parameters of the gene regulation relationship model and the gene expression feature extraction model are adjusted according to the difference in loss value determined by the difference between the value of each element in the predicted gene expression matrix and the value of the element at the corresponding position in the initial gene expression matrix. In this way, the model can learn the gene expression Gene expression levels 1.1 and 1.2 are closer, and gene expression levels 1.1 and 1.9 are more different, so that gene signatures can better characterize the relationship between different gene expression levels and different genes.

Step 140B: inputting the input features into the gene regulation relationship model to be trained to obtain a gene regulation relationship representation, converting the gene regulation relationship representation to generate a predicted gene expression matrix, wherein the elements in the predicted gene expression matrix correspond to the elements in the initial gene expression matrix;

Exemplarily, the gene regulation relationship model to be trained includes an encoder and a decoder; step 140B includes: inputting the input feature to the encoder to obtain an initial coding feature tensor; according to the position of the element in the initial gene expression matrix, the first gene feature corresponding to the first type of element and the zero-value gene feature corresponding to the element with a value of 0 in the second type of element are merged with the initial coding feature tensor to obtain a target coding feature tensor; the target coding feature tensor is input to the decoder to obtain a gene regulation relationship representation. Exemplarily, the encoder includes M layers of encoding units, and the decoder includes N layers of decoding units, and the value of M is greater than the value of N. Exemplarily, each layer of encoding units of the encoder includes a multi-head attention unit and a forward propagation unit; each layer of decoding units of the decoder includes a forward propagation unit, and also includes a linear attention unit or a sparse attention unit.

Exemplarily, the gene regulatory relationship representation may refer to the expression tensor of the regulatory relationship between each gene. The gene regulatory relationship representation is a three-dimensional tensor, which includes not only the regulatory relationship between each gene, but also the expression level information corresponding to each gene in the cell.

Exemplarily, the gene regulation relationship is represented as a three-dimensional tensor of C×G×D, where C represents the number of cells, taking the above example, C=3; G represents the number of genes, taking the above example, G=6; D represents the feature dimension, and usually the value of D is 2 to the power of n, for example, 128.

Exemplarily, after obtaining the gene regulatory relationship representation, the gene regulatory relationship representation is transformed through a layer of network to obtain a predicted gene expression matrix, which is a prediction matrix for predicting the values of the first type of elements in the initial gene expression matrix.

Exemplarily, the elements in the predicted gene expression matrix correspond to the elements in the initial gene expression matrix, that is, if the initial gene expression matrix is A×B dimensional, then the predicted gene expression matrix is also A×B dimensional.

Exemplarily, the encoder is used to determine feature information of the second type of element, wherein the feature information includes expression information and regulatory relationship information. The decoder is used to restore the expression information of the first type of element based on the expression information and the regulatory relationship information.

Exemplarily, the gene regulation relationship model to be trained consists of two parts, namely an encoder and a decoder. First, the input feature is input into the encoder to obtain an initial coding feature tensor (for example, a tensor of C×L×D), which includes the expression information and regulatory relationship information of the second-class elements. Then, according to the position of all elements (first-class elements and second-class elements with 0 values and non-0 values) in the initial gene expression matrix, the first gene feature corresponding to the first-class element and the zero-value gene feature corresponding to the element with a value of 0 in the second-class element are merged with the initial coding feature tensor to obtain a target coding feature tensor (for example, a tensor of C×G×D). Finally, the target coding feature tensor is input into the decoder to obtain a gene regulation relationship representation (for example, a tensor of C×G×D) to predict the updated first gene feature corresponding to each first-class element.

Step 150B: updating the model parameters in the gene regulation relationship model to be trained based on the initial gene expression matrix and the predicted gene expression matrix to generate a trained gene regulation relationship model.

Exemplarily, step 150B includes: calculating the difference between the value of each element in the predicted gene expression matrix and the value of the element at the corresponding position in the initial gene expression matrix; and determining the loss value according to the difference in the values.

If the training termination condition is not met, the values of the model parameters are adjusted according to the loss value, and the process returns to the step of randomly selecting multiple elements in the initial gene expression matrix as the first type of elements.

Exemplarily, the model parameters may include parameters of an encoder and parameters of a decoder, and may also include parameters of a network used to convert a gene regulatory relationship representation into a predicted gene expression matrix.

In this specific implementation, the first-class elements and elements with a value of 0 are removed from the initial gene expression matrix including the expression of highly variable genes and non-hypervariable genes, and only the input features corresponding to the elements with non-0 values in the second-class elements are used as the input of the gene regulation relationship model to be trained. There is no need to calculate a large amount of data, which reduces the computational complexity and amount, improves the training efficiency, and enables the model to learn the potential gene regulation relationship at the whole genome level. Compared with the gene regulation relationship model generation method in the prior art, it solves the problem of difficulty in training the regulation relationship model and systematic missing of the regulation relationship map learned by the model.

In a specific implementation of the present disclosure, as shown in FIG. 1C , the present disclosure embodiment further provides a prediction model training method 100C, and the method 100C includes steps 110C to 130C.

Step 110C, obtaining a plurality of samples, wherein each of the plurality of samples comprises first gene expression data, mask gene expression data, and auxiliary information, wherein the first gene expression data comprises counts of different genes in a single cell measured at a first sequencing depth, and the mask gene expression data is obtained by The first gene expression data is downsampled and counts of some genes in the downsampled first gene expression data are masked, the downsampled first gene expression data simulates respective counts of different genes in a single cell measured at a second sequencing depth lower than the first sequencing depth, and the auxiliary information includes a first total count, which is the sum of the counts of each gene in the first gene expression data;

The explanation of each term, as well as the description of downsampling and masking methods, can be found in the previous text and will not be repeated here.

When training the prediction model, the first gene expression data obtained by actual sequencing is used as the true value of the gene count at a high sequencing depth, the first gene expression data after downsampling is used to simulate the gene count at a low sequencing depth, the model is used to predict the gene count at a high sequencing depth based on the gene count at a low sequencing depth, and the loss is calculated based on the predicted value and the true value of the gene count at a high sequencing depth, and the model is updated with the loss value. In this way, the prediction model can learn to capture the relationship between the gene expressions of similar cells at different sequencing depths. In order to further improve the training effect, the MAE method is used for training, that is, the first gene expression data after downsampling is used as the gene count at a low sequencing depth, and partially masked, and the model is used to predict the gene count at a high sequencing depth based on the partially masked gene count at a low sequencing depth, and the loss is calculated based on the predicted gene count at a high sequencing depth and the corresponding part of the masked elements in the true value of the gene count at a high sequencing depth, and the model is updated with the loss value. Therefore, when constructing the training data, it is necessary to use the first gene expression data obtained by actual sequencing, downsample the first gene expression data, and mask the counts of some genes in the downsampled first gene expression data to obtain the masked gene expression data.

In order for the prediction model to correctly predict the gene count at a high sequencing depth based on the gene count at a low sequencing depth, the prediction model needs to understand what the low sequencing depth and high sequencing depth are. It is understandable that the higher the sequencing depth, the greater the sum of the counts of each gene in the sequencing result, so the sequencing depth can be characterized by the sum of the counts of each gene at the sequencing depth.

In one example, the auxiliary information includes the first total count, but does not necessarily include the sum of the gene counts at the low sequencing depth, that is, the sum of the counts of each gene in the downsampled first gene expression data. Because even if the auxiliary information does not include the sum of the counts of each gene in the downsampled first gene expression data, the model can guess the gene counts of the masked part based on the masked gene expression data in the sample, and then estimate the sum of the counts of each gene in the downsampled first gene expression data. Of course, the auxiliary information may also include the sum of the counts of each gene in the downsampled first gene expression data, so that the model can be explicitly informed of the low sequencing depth, thereby reducing the learning cost and accelerating model convergence.

Step 120C, for each sample in the plurality of samples, executing step PB1 and step PB2, wherein step PB1 and step PB2 are sub-steps of step 120C;

Step PB1, using the prediction model to be trained to process the masked gene expression data and auxiliary information in the sample to obtain the predicted value of each gene count at the first sequencing depth corresponding to the sample;

Exemplarily, when the prediction model to be trained is used to process the masked gene expression data and auxiliary information in the sample, the masked gene expression data and the auxiliary information can both be input from the first network layer of the prediction model. The masked gene expression data in the sample and the auxiliary information are concatenated as the input feature tensor corresponding to the sample; or the feature tensor corresponding to the masked gene expression data in the sample and the feature tensor corresponding to the auxiliary information are concatenated as the input feature tensor corresponding to the sample; and the input feature tensor is processed using the prediction model to be trained.

Exemplarily, the masked gene expression data can also be input from the first network layer of the prediction model, and the auxiliary information can be input from the network layer behind the prediction model. The prediction model to be trained includes an input layer, an output layer, and multiple intermediate layers between the input layer and the output layer. At this time, step PB1 includes: using the masked gene expression data in the sample as an input feature tensor or the feature tensor corresponding to the masked gene expression data in the sample as an input feature tensor, inputting the input layer and the first predetermined layer in the multiple intermediate layers to obtain an intermediate feature tensor, and the number of the first predetermined layers is greater than or equal to 0; splicing the auxiliary information with the intermediate feature tensor to obtain the spliced intermediate feature tensor; and inputting the spliced intermediate feature tensor into the second predetermined layer in the multiple intermediate layers and the output layer, and the second predetermined layer is different from the first predetermined layer. For example, when the prediction model includes a decoder network, the auxiliary information is input before the last several layers of the decoder network; for another example, when the prediction model includes an encoder network and a decoder network, the auxiliary information can be input before a certain layer of the encoder network or before the last several layers of the decoder network. In this way, the model can use the auxiliary information to extract more meaningful features or make more accurate predictions in the process of processing input data.

Exemplarily, the input to the prediction model may be the masked gene expression data and auxiliary information themselves, or the masked gene expression data and auxiliary information may be converted into corresponding embedding vectors and then input into the prediction model; the input to the first network layer of the prediction model may be the counts corresponding to all genes in the masked gene expression data, or the counts corresponding to some genes (for example, the counts corresponding to genes whose counts are not 0 and are not masked).

The masked gene expression data, the feature tensor corresponding to the masked gene expression data, the concatenation result of the masked gene expression data and the auxiliary information, the feature tensor corresponding to the masked gene expression data and the auxiliary information are concatenated. The concatenation result of the feature tensors corresponding to the information can be interpreted broadly as the input feature tensor, which refers not only to the first network layer of the input prediction model, but also to any network layer of the input prediction model. For example, part of the tensors in the input feature tensor are used as the input of the first network layer of the prediction model, and part of the tensors are used as the input of the middle network layer of the prediction model. In addition, the input feature tensor may include the part corresponding to all elements of the masked gene expression data, or only the part corresponding to some elements of the masked gene expression data (such as non-zero and unmasked parts), and also include the part corresponding to elements outside the masked gene expression data (such as filled elements, which will be explained later).

It should be noted that the size of the object to be processed of the network layer of the input prediction model is certain, for example, 2000*768. When the counts corresponding to the genes with non-zero and unmasked counts in the masked gene expression data are input into the network layer of the prediction model, the different numbers of non-zero gene counts and the different proportions of masked elements included in different masked gene expression data may lead to different sizes of the objects to be processed of the network layer of the input prediction model. In this case, they can be padded to the same size to ensure that the sizes of the objects to be processed of the network layer of the input prediction model are the same. For example, the number of non-zero and unmasked gene counts included in the masked gene expression data in each sample fluctuates between 300-1000, and the size of the input feature tensor can be padded to 1000. For example, the size of the input feature tensor can be padded to 10% of the total number of genes.

The method of converting the masked gene expression data into its corresponding embedding vector is described above and will not be repeated here.

Step PB2, determining a loss value corresponding to the sample according to the predicted value and the counts corresponding to the part of the genes in the first gene expression data; and

Step 130C: update the prediction model to be trained according to the loss value corresponding to each sample in the multiple samples.

This specific implementation method downsamples the sequencing results at an actual high sequencing depth, simulates the sequencing results at a low sequencing depth, uses the sequencing results at the high sequencing depth and the sequencing results at the low sequencing depth as samples to train the prediction model, and explicitly introduces auxiliary information that characterizes the expected sequencing depth, so that the prediction model learns to capture the relationship between gene counts of similar cells at different sequencing depths and the relationship between gene counts in a single cell, thereby minimizing technical noise interference in sequencing, and increasing the sequencing depth by computational means, thereby improving the accuracy of downstream task output results.

Exemplarily, each sample among the multiple samples includes normalized first gene expression data, normalized masked gene expression data and auxiliary information, the normalized first gene expression data is obtained by normalizing the first gene expression data, the normalized masked gene expression data is obtained by downsampling the first gene expression data, normalizing the downsampled first gene expression data, and masking the counts of some genes in the normalized result; the auxiliary information also includes a second total count, which is the sum of the counts of each gene in the downsampled first gene expression data; step PB1, including: using the prediction model to be trained to process the normalized masked gene expression data and the auxiliary information in the sample to obtain a predicted value of the normalized value of the counts of each gene at the first sequencing depth corresponding to the sample, and step PB2, including: determining the loss value corresponding to the sample based on the predicted value and the counts corresponding to the part of the genes in the normalized first gene expression data.

Compared to the absolute counts of genes, the relative relationship between the counts of different genes is more important for downstream tasks. In order to eliminate the differences in gene counts caused by sequencing depth and make the sequencing results of different samples and different sequencing depths comparable, the sequencing results need to be standardized and normalized to obtain the samples used for training. In other words, at this time, the normalized first gene expression data and the normalized masked gene expression data can be used to replace the first gene expression data and the masked gene expression data in step 110, step 120, step 121, and step 122.

The sequencing depth is represented by the total counts. In order to eliminate the influence of the sequencing depth, the total counts in the gene expression data can be used for normalization. The specific normalization method is described above.

The downsampled first gene expression data simulates the counts of different genes in a single cell measured at a second sequencing depth lower than the first sequencing depth, and the second total count can be used as a representation of the second sequencing depth. Although the first sequencing depth can be obtained through actual sequencing and is known, the second sequencing depth is unknown. In order to characterize the first sequencing depth and the second sequencing depth with the same measure, the first total count and the second total count are used to characterize the first sequencing depth and the second sequencing depth.

Exemplarily, the first total count T and the second total count S can both determine their corresponding embedding vectors in the manner described above, and the sign of the first total count is different from the sign of the second total count and is also different from other gene IDs.

In the example, since the sample only includes the normalized masked gene expression data but not the downsampled first gene expression data before normalization, the model cannot estimate the sum of the counts of each gene in the downsampled but not normalized first gene expression data, that is, it cannot estimate the normalized masked gene expression data. How low is the lower sequencing depth corresponding to the gene expression data. Therefore, the second total count must be included in the auxiliary information. In this way, the prediction model can explicitly know how low the lower sequencing depth corresponding to the normalized masked gene expression data is, and how high the higher sequencing depth corresponding to the first gene expression data is based on the second total count, and then can predict the gene count at the higher sequencing depth based on the normalized masked gene expression data.

It is understandable that, since the normalized first gene expression data is used as the true value, the predicted value of each gene count at the first sequencing depth output by the model is also normalized. The predicted value can be restored to the predicted value of the gene count that has not been normalized by multiplying the predicted value by the first total count, thereby remapping the normalized predicted value to the original count space.

In one example, the normalized masked gene expression data can be obtained from the first gene expression data in the following manner: Step 1, downsampling the first gene expression data. Step 2, normalizing the downsampled first gene expression data. Step 3, masking the counts of some genes in the result of Step 2 to obtain the masked gene expression data.

Exemplarily, the prediction model to be trained includes an encoder network and a decoder network, wherein processing the input feature tensor using the prediction model to be trained includes: encoding the first tensor corresponding to the unmasked and non-zero counts in the input feature tensor using the encoder network to obtain an encoded tensor; replacing the first tensor in the input feature tensor with the encoded tensor to obtain an encoded input feature tensor; and decoding the encoded input feature tensor using the decoder network.

In one example, step PB1 includes:

Step PB11, using the encoder network to encode the first tensor corresponding to the unmasked and non-zero counts in the input feature tensor to obtain an encoded tensor. Exemplarily, the size of the first tensor can be 2000*768, and the size of the encoded tensor can be 2000*768. The 1st, 2nd, 4th, 10th, 30th, etc. elements of the 2000 elements are non-zero and unmasked elements, and the 1561st to 2000th elements are filled elements.

Step PB12, replace the first tensor in the input feature tensor with the encoding tensor to obtain the encoded input feature tensor. Replace the 1st, 2nd, 4th, 10th, 30th, etc. elements in N*768 with the corresponding tensors in the encoding tensor.

Step PB13, using a decoder network to decode the encoded input feature tensor to obtain a decoded tensor. The size of the decoded tensor can be N*768.

Exemplarily, the prediction model further includes a multilayer perceptron. After step PB13, step PB1 further includes step PB14: using the multilayer perceptron in the prediction model to process the decoded tensor to obtain the predicted value of each gene count at the first sequencing depth corresponding to the sample. The size of the predicted value is N*1. It is understandable that the multilayer perceptron is trained together with other parts in the prediction model.

Exemplarily, before step PB11, the masked gene expression data and auxiliary information can be input into a module that determines count embedding vectors based on counts using the method described above to obtain an input feature tensor. For example, the size of the masked gene expression data is N*1, the auxiliary information includes T and S, and the size of the input feature tensor is (N+2)*768.

In this specific embodiment, the encoder part focuses on the embedding vector corresponding to the non-zero count in the input feature tensor, while the decoder part receives the embedding vectors of all genes (i.e., the embedding processed by the encoder and the embedding of other genes), integrating the information of all positions. After the zero-value positions and masked positions are filtered before being fed to the encoder, the input sequence length of the encoder input is approximately 10% of the full gene length. This design greatly reduces the required computing resources, allowing the encoder to use a series of ordinary Transformer blocks to capture gene dependencies, greatly improving training efficiency and training effects. Since the encoder module only processes non-zero, unmasked counts, the model can effectively focus on the most informative non-zero expressed genes, while allowing zero-value genes to participate in model training in the decoder stage.

Exemplarily, the prediction model to be trained is one of an encoder-decoder network, a decoder network, and a multi-layer perceptron.

Exemplarily, the downsampled first gene expression data is obtained by downsampling the first gene expression data using a statistical sampling algorithm. For details, see the above description of downsampling.

According to another aspect of the present disclosure, a method for correcting gene expression data is provided. FIG. 2 is a flow chart of a method for correcting gene expression data according to an embodiment of the present disclosure. In method 100, a prediction model has been trained, and the trained prediction model can correct gene expression data. In method 200, the trained prediction model is used for reasoning. As shown in FIG. 2, the method 200 includes steps 210 and 220.

In the present disclosure, the method for correcting gene expression data is also referred to as a method for determining gene regulatory relationships.

Step 210, obtaining current gene expression data, wherein the current gene expression data includes respective counts of different genes measured at an actual sequencing depth.

It is understandable that the prediction model trained in method 100 is used by method 200, and the current gene expression data should be in the same form as the masked gene expression data, except that no mask is required. If the masked gene expression data in the training sample is normalized, the current gene expression data should also be normalized; if the masked gene expression data in the training sample is not normalized, the current gene expression data should also be not normalized.

It is understandable that if method 200 uses a prediction model that provides auxiliary information during training, then the auxiliary information corresponding to the current gene expression data should also be obtained in step 210; if method 200 uses a prediction model that does not provide auxiliary information during training, then there is no need to obtain the auxiliary information corresponding to the current gene expression data in step 210.

It is understandable that the current gene expression data may be single-cell gene expression data or gene expression data obtained by sequencing a large number of cells (bulk).

Exemplarily, the current gene expression data may be a gene expression matrix, and the gene expression matrix may be obtained through single-cell sequencing. If the current gene expression data is derived from a single cell, the gene expression matrix may be a vector; if the current gene expression data is derived from multiple single cells, the gene expression matrix may be a matrix, and each row or column of the matrix corresponds to the gene expression level of a single cell.

It is understandable that the genes in the current gene expression data are a subset of the genes contained in the first gene expression data in the sample used when training the prediction model. Otherwise, the expression level relationship between the genes in the current gene expression data is not learned by the prediction model, and the corresponding gene expression level relationship cannot be predicted by the prediction model.

Step 220 , using at least part of the network layers in the prediction model to process the current gene expression data to obtain a correction value of the current gene expression data or an intermediate processing result of the correction value. The prediction model may be trained according to method 100 .

Exemplarily, in step 220, all network layers in the prediction model may be used to process the current gene expression data, and the corrected value of the current gene expression data is obtained at this time; or only some network layers in the prediction model (e.g., the first M layers, e.g., the network layers in the prediction model except for the network layer used to project the intermediate representation of the relative relationship between the counts of each gene after correction as the predicted value of the gene count) may be used to process the current gene expression data, and the intermediate processing result of the corrected value of the current gene expression data is obtained at this time. It depends on the needs of the downstream tasks.

Exemplarily, the correction value is the output of the multilayer perceptron in FIG. 1A , and the intermediate processing result of the correction value refers to the intermediate processing result obtained to obtain the correction value, and does not mean that the correction value is finally obtained. The encoding vector output by the encoder network and the decoding vector output by the decoder network in Figure 1A are all intermediate processing results.

Exemplarily, continuing to refer to Figure 1A, at least some network layers in the prediction model used in method 220 include an encoder network and may also include a decoder network. Features characterizing cells can be obtained based on the output of the encoder network, and features representing genes can be obtained based on the output of the decoder network. When the downstream task needs to use features characterizing cells, in step 220, only the encoder network can be used to process the current gene expression data, and in subsequent steps, the features characterizing cells obtained from the output of the encoder network are used for the downstream task; when the downstream task needs to use features characterizing genes, in step 220, the encoder network and the decoder network can be used to process the current gene expression data, and in subsequent steps, the features characterizing genes obtained from the output of the decoder network are used for the downstream task.

Exemplarily, when the prediction model includes an encoder network, the method of obtaining the characteristics of the cell from the output of the encoder network may be that the output of the encoder network passes through a pooling layer (e.g., a maximum pooling layer) to obtain the characteristics of the cell. Exemplarily, when the prediction model includes an encoder network and a decoder network, the method of obtaining the characteristics of the gene from the output of the decoder network may be that the output of the decoder network passes through a multi-layer perceptron to obtain the characteristics of the gene.

It is understandable that if method 200 uses a prediction model that provides auxiliary information during training, then step 220 also uses at least part of the network layer in the prediction model to process the auxiliary information of the current gene expression data. In this case, what is obtained in step 220 is the corrected value of the current gene expression data at the expected sequencing depth or the intermediate processing result of the corrected value; if method 200 uses a prediction model that does not provide auxiliary information during training, in this case, what is obtained in step 220 is the corrected value of the current gene expression data or the intermediate processing result of the corrected value at a sequencing depth where the current gene expression data remains substantially unchanged. In this case, there is no need to process the auxiliary information corresponding to the current gene expression data in step 220.

Exemplarily, method 200 further includes step 230, determining a feature tensor corresponding to the current gene expression data, and step 220 includes processing a feature vector corresponding to the current gene expression data using at least part of the network layer in the prediction model. It is understandable that, since the current gene expression data only includes unmasked counts (0 values and non-0 values), step 230 can refer to the method for determining the feature tensors corresponding to unmasked and non-0 counts and unmasked and 0 counts in step 123.

Exemplarily, at least part of the network layers of the prediction model include an encoder network and may also include a decoder network. Step 220 includes: Step 2201, encoding the input of the encoder network using the encoder network to obtain the output of the encoder network; the input of the encoder network includes the current gene expression The feature tensor corresponding to the non-zero counts in the data (the counts that are not masked in the current gene expression data); exemplarily, the input of the encoder network also includes the feature vector corresponding to the filler element. Optionally, after step 2201, step 220 includes step 2202, according to the output of the encoder network and the feature tensor corresponding to the counts of 0 values in the current gene expression data (exemplarily, the two are spliced according to the position of the counts), the input of the decoder network is obtained; the input of the decoder network is decoded by the decoder network to obtain the output of the decoder network. Optionally, after step 2202, step 220 includes 2203, projecting the output of the decoder network as a correction value of the current gene expression data.

It is understandable that when the downstream task needs to use features that characterize cells, step 2201 can be executed to obtain the output of the encoder network, and then the output of the encoder network (which can be the feature tensor of the position corresponding to the filler element removed from the output of the encoder network) is input into a pooling layer to obtain features that characterize cells, and the features are used for downstream tasks. When the downstream task needs to use features that characterize genes, steps 2201 and 2202 can be executed to obtain the output of the decoder network, and then the output of the decoder network is input into a multilayer perceptron to obtain features that characterize genes, and the features are used for downstream tasks. When the downstream task needs to use the correction value of the current gene expression data, steps 2201, 2202 and 2203 can be executed to use the correction value for the downstream task.

Method 200 can improve the accuracy of the relative relationship between each gene count in the current gene expression data through the prediction model, obtain the correction value of the current gene expression data that can be used for different downstream tasks or the intermediate processing result of the correction value, and apply it to the downstream task to improve the accuracy of the downstream task. It is understandable that the downstream task in the disclosed embodiment can be an existing downstream task algorithm or model, such as a cell classification model, a disturbance prediction model, etc., and these algorithms or models use the characteristics of characterizing cells or the characteristics of characterizing genes as input. In the disclosed embodiment, the characteristics of characterizing cells and the characteristics of characterizing genes provided by the disclosed embodiment replace the characteristics of characterizing cells and the characteristics of characterizing genes used in the downstream task algorithm or model of the prior art. That is, the characteristics of characterizing cells or the characteristics of characterizing genes obtained according to method 200 can replace the characteristics of characterizing cells or the characteristics of characterizing genes originally used in the downstream task.

In a specific embodiment, step 210 includes: obtaining normalized current gene expression data and current auxiliary information; wherein the current gene expression data includes the counts of different genes measured at the actual sequencing depth, and the normalized current gene expression data is obtained by normalizing the current gene expression data; the current auxiliary information includes an expected first total count and a current second total count; the current second total count is the sum of the counts of each gene in the current gene expression data; the expected first total count is used for Characterizing the expected sequencing depth, the expected first total count is greater than or equal to the sum of the counts of each gene in the current gene expression data.

In some embodiments of method 100, the sample includes normalized first gene expression data, masked gene expression data, and auxiliary information, so that the prediction model learns to capture the relationship between gene counts of similar cells at different sequencing depths and the relationship between gene counts in a single cell, and is thus able to predict gene counts at a high sequencing depth based on gene counts at a low sequencing depth; accordingly, when using the prediction model trained using the sample, the input of the prediction model includes normalized current gene expression data and current auxiliary information characterizing low sequencing depth and high sequencing depth, so that the prediction model is able to predict the expected gene counts at a high sequencing depth based on the gene counts at a low sequencing depth.

It is expected that the first total count may be greater than or equal to the current second total count. When the expected first total count is greater than the sum of the counts of each gene in the current gene expression data, it is expected that the prediction model predicts the expected gene count at a higher sequencing depth based on the gene counts actually measured at a lower sequencing depth. When the expected first total count is equal to the sum of the counts of each gene in the current gene expression data, it is expected that the prediction model predicts the corrected gene count under the condition that the sequencing depth remains substantially unchanged based on the gene counts actually measured.

Since the space for increasing the sequencing depth is limited, and in order to ensure the data accuracy of the predicted value of each gene count in the target single cell at the expected sequencing depth, it is expected that the first total count cannot be much larger than the sum of the counts of each gene in the current gene expression data. Exemplarily, the expected sequencing depth can be set according to the sequencing method. For example, the sequencing method is 10X Genomics Chromium sequencing technology (10X Genomics Chromium sequencing technology, referred to as "10X sequencing technology"), and the sum of the counts of each gene in the current gene expression data is about 1000. At this time, the expected sequencing depth can be set to 10000.

Exemplarily, after obtaining current gene expression data at a low sequencing depth (the current gene expression data is not normalized), on the one hand, the sum of the counts of each gene in the current gene expression data is calculated as the current second total count, and on the other hand, the current gene expression data is normalized to obtain normalized current gene expression data, so that at least part of the network layer in the prediction model can be used to process the normalized current gene expression data later. It is understandable that if the gene expression data obtained at a low sequencing depth is normalized, it can be restored to the unnormalized current gene expression data first.

Step 220 includes: using at least part of the network layer in the prediction model to process the normalized current gene expression data and the current auxiliary information to obtain a corrected value of the normalized value of the current gene expression data at the expected sequencing depth or an intermediate processing result of the corrected value of the normalized value.

Exemplarily, at least part of the network layer in the prediction model is used to process the normalized feature tensor corresponding to the current gene expression data and the feature tensor corresponding to the current auxiliary information. Prior to this, the normalized feature tensor corresponding to the current gene expression data and the feature tensor corresponding to the current auxiliary information need to be determined. The method for determining the feature tensor corresponding to the normalized current gene expression data is described in the description of step 230, and the method for determining the feature tensor corresponding to the current auxiliary information is described in the description of step 125.

Exemplarily, which part of the network layer of the prediction model is used to process the normalized current gene expression data and the current auxiliary information is determined according to the downstream task requirements, as described above.

Exemplarily, the current auxiliary information can be input into the first network layer along with the current gene expression data, or only the current gene expression can be input into the first network layer, and the auxiliary information can be input into a network layer different from the first network layer. The specific input method and input form must be consistent with the training stage and will not be repeated here.

In a specific embodiment of the present disclosure, as shown in FIG2A , a method 200A for determining a gene regulation relationship is provided, including: step 210A, obtaining a gene expression matrix corresponding to the cells to be treated; wherein the genes expressed by the cells to be treated are a subset of the genes contained in the initial gene expression matrix; step 220A, inputting the gene expression matrix into the gene regulation relationship model, obtaining a gene regulation relationship representation, and the gene regulation relationship representation is a three-dimensional tensor; wherein the gene regulation relationship model is generated according to the gene regulation relationship model generation method of method 100B. The gene expression matrix can be obtained, for example, by single-cell sequencing. When the cell to be treated is a single cell, the gene expression matrix can be a vector, and when the cell to be treated is a plurality of cells, the gene expression matrix can be a matrix, and each row or column of the matrix corresponds to the gene expression of a cell. The obtained gene expression matrix is input into the trained gene regulation relationship model, and the gene regulation relationship model outputs the gene regulation relationship representation corresponding to the cells to be treated. Among them, the gene regulation relationship is expressed as a three-dimensional tensor (for example, a three-dimensional tensor of C×G×D), and the gene regulation relationship corresponding to each cell to be treated is characterized by the matrix corresponding to the cell to be treated in the three-dimensional tensor. It is understandable that when C is 1, the gene regulation relationship model is generated according to the gene regulation relationship model generation method. It is understandable that when C is 1, the three-dimensional tensor of C×G×D becomes 1×G×D. It is understandable that the genes expressed by the cells to be treated are a subset of the genes contained in the initial gene expression matrix used when training the gene regulation relationship model. If the genes expressed by the cells to be treated are not in the genes contained in the initial gene expression matrix, it means that the regulatory relationship between its genes has not been learned by the gene regulation relationship model, and its corresponding gene regulation relationship cannot be predicted by the gene regulation relationship model.

In a specific embodiment of the present disclosure, a method 200B for correcting gene expression data is provided, as shown in FIG2B , and the method 200B includes: step 210B, obtaining current gene expression data and current auxiliary information, wherein the current gene expression data includes the counts of different genes measured at the actual sequencing depth, and the current auxiliary information includes the expected first total count, the expected first total count is used to characterize the expected sequencing depth, and the expected first total count is greater than or equal to the sum of the counts of each gene in the current gene expression data; step 220B, using at least part of the network layer in the prediction model trained according to method 100C to process the current gene expression data and the current auxiliary information, so as to obtain the correction value of the current gene expression data at the expected sequencing depth or the intermediate processing result of the correction value. As described above, in the training stage, the auxiliary information may only include the first total count but not the second total count, and accordingly, in the inference stage, the current auxiliary information may only include the expected first total count but not the current second total count. At this time, the first gene expression data in the training sample in the training stage and the current gene expression data in the inference stage are both unnormalized. For the explanation of terms and step description of this specific embodiment, please refer to the above.

FIG3 is a flow chart of a downstream task execution method according to an embodiment of the present disclosure. As shown in FIG3 , the method 300 includes step 310 and step 320 .

In step 310, input data is obtained, wherein the input data includes i) a correction value obtained according to method 200; or ii) an intermediate processing result of the correction value obtained according to method 200; or iii) a preprocessing result obtained by preprocessing the correction value obtained by method 200; or iv) a preprocessing result obtained by preprocessing the intermediate processing result of the correction value obtained by method 200.

Exemplarily, preprocessing can be pooling, processing using a linear layer, processing using a specific model, etc. For example, preprocessing can be the method described above for obtaining features characterizing cells from the output of the encoder, or it can be the method described above for obtaining features characterizing genes from the output of the decoder.

In step 320, the input data is processed using a downstream task algorithm to obtain a downstream task result, wherein the downstream task includes a cell classification task, a disturbance prediction task or a drug response prediction task.

Exemplarily, downstream tasks include cell classification tasks (further subdivided into cell classification tasks, cell clustering tasks), perturbation prediction tasks or drug response prediction tasks. Exemplarily, features characterizing cells are used for cell classification tasks, cell clustering tasks, and drug response prediction tasks, and features characterizing genes are used for perturbation prediction tasks.

For example, when the downstream task is a cell classification task, the features characterizing the cells are used in the cell classification model to obtain the cell classification result. In the example, the classification model can use, for example, support vector Machine (Support Vector Machine, SVM), Multilayer Perceptron (Multilayer Perceptron, MLP) or Decision Tree (Decision Tree), etc.

For example, when the downstream task is a cell clustering task, a clustering algorithm can be used to cluster the features representing the cells to obtain a clustering result. The clustering algorithm can be K-means clustering or the like.

The cell classification results may be, for example, classification results of cancer cells and normal cells, clustering results of different types of immune cells, or classification results of cell types in different organs.

When the downstream task is a drug response prediction task, graph neural networks can be used to extract drug features, and prediction models can be used to determine features that characterize cells. The drug features and features that characterize cells are input into the drug response prediction model to obtain a prediction result on whether the cells are sensitive to the drug.

When the downstream task is a perturbation prediction task, the features characterizing the genes determined by the prediction model can be used as the gene features corresponding to the gene nodes of the gene co-expression graph neural network. The perturbation features are applied to the gene co-expression graph neural network to predict the expression of the perturbed genes.

Exemplarily, models other than the prediction model used in downstream tasks can be trained together with the prediction model.

According to one aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and at least one memory communicatively connected to the at least one processor, wherein the at least one memory stores instructions, and when the instructions are executed by the at least one processor, the at least one processor executes the above method.

According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing instructions is provided. When the instructions are executed by at least one processor of a computer, the computer executes the above method.

According to another aspect of the present disclosure, a computer program product is provided, including a computer program, and the computer program implements the above method when executed by a processor.

FIG. 4 is a block diagram of an exemplary electronic device that can be used to implement an embodiment of the present disclosure.

The electronic device 400 can be a variety of different types of devices. Examples of the electronic device 400 include, but are not limited to, desktop computers, server computers, laptop or netbook computers, mobile devices (e.g., tablet computers, cellular or other wireless phones (e.g., smart phones), notepad computers, mobile stations), wearable devices (e.g., glasses, watches), entertainment devices (e.g., entertainment appliances, set-top boxes communicatively coupled to a display device, game consoles), televisions or other display devices, automotive computers, and the like.

The electronic device 400 may include at least one processor 402, memory 404, communication interface(s) 406, a display device 408, other input/output (I/O) devices 410, and one or more mass storage devices 412 that can communicate with each other, such as via a system bus 414 or other appropriate connection.

Processor 402 may be a single processing unit or multiple processing units, all of which may include a single or multiple computing units or multiple cores. Processor 402 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and/or any device that manipulates signals based on operating instructions. Among other capabilities, processor 402 may be configured to obtain and execute computer-readable instructions stored in memory 404, mass storage device 412, or other computer-readable media, such as program code for operating system 416, program code for application program 418, program code for other programs 420, and the like.

The memory 404 and the mass storage device 412 are examples of computer-readable storage media for storing instructions that are executed by the processor 402 to implement the various functions described above. For example, the memory 404 may generally include both volatile memory and non-volatile memory (e.g., RAM, ROM, etc.). In addition, the mass storage device 412 may generally include a hard drive, a solid-state drive, a removable medium, including external and removable drives, a memory card, a flash memory, a floppy disk, an optical disk (e.g., a CD, a DVD), a storage array, a network attached storage, a storage area network, etc. The memory 404 and the mass storage device 412 may all be collectively referred to herein as memory or computer-readable storage media, and may be a non-transitory medium capable of storing computer-readable, processor-executable program instructions as computer program code, which may be executed by the processor 402 as a specific machine configured to implement the operations and functions described in the examples herein.

A plurality of programs may be stored on the mass storage device 412. These programs include an operating system 416, one or more application programs 418, other programs 420, and program data 422, and they may be loaded into the memory 404 for execution. Examples of such applications or program modules may include, for example, computer program logic (e.g., computer program code or instructions) for implementing the following components/functions: method 100 (including any suitable steps of method 100), method 100B (including any suitable steps of method 100B), method 100C (including any suitable steps of method 100C), method 200 (including any suitable steps of method 200), method 200A (including any suitable steps of method 200A), method 200B (including any suitable steps of method 200B), method 300 (including any suitable steps of method 300), and/or other embodiments described herein.

4 as being stored in the memory 404 of the electronic device 400, the modules 416, 418, 420, and 422, or portions thereof, may be implemented using any form of computer-readable media accessible by the electronic device 400. As used herein, "computer-readable media" includes at least two types of computer-readable media, namely, computer-readable storage media and communication media.

Computer-readable storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information, such as computer-readable instructions, data structures, program modules or other data. Computer-readable storage media include but are not limited to RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD), or other optical storage device, magnetic cassette, magnetic tape, magnetic disk storage device or other magnetic storage device, or any other non-transmission medium that can be used to store information for access by electronic devices. In contrast, communication media can embody computer-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transmission mechanism. Computer-readable storage media defined herein do not include communication media.

One or more communication interfaces 406 are used to exchange data with other devices, such as through a network, direct connection, etc. Such communication interfaces can be one or more of the following: any type of network interface (e.g., a network interface card (NIC)), a wired or wireless (such as IEEE 802.11 wireless LAN (WLAN)) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a BluetoothTM interface, a Near Field Communication (NFC) interface, etc. The communication interface 406 can facilitate communication within a variety of network and protocol types, including wired networks (e.g., LAN, cable, etc.) and wireless networks (e.g., WLAN, cellular, satellite, etc.), the Internet, etc. The communication interface 406 can also provide communication with external storage devices (not shown) such as storage arrays, network attached storage, storage area networks, etc.

In some examples, a display device 408 such as a monitor may be included for displaying information and images to the user. Other I/O devices 410 may be devices that receive various inputs from the user and provide various outputs to the user, and may include touch input devices, gesture input devices, cameras, keyboards, remote controls, mice, printers, audio input/output devices, and the like.

The techniques described herein may be supported by these various configurations of electronic device 400 and are not limited to the specific examples of the techniques described herein. For example, the functionality may also be implemented in whole or in part on a "cloud" using a distributed system. The cloud includes and/or represents a platform for resources. The platform abstracts the underlying functionality of the hardware (e.g., servers) and software resources of the cloud. Resources may include resources that are located remotely from the computer. Applications and/or data that can be used when performing computing processing on the server of the sub-device 400. Resources can also include services provided through the Internet and/or through a subscriber network such as a cellular or Wi-Fi network. The platform can abstract resources and functions to connect the electronic device 400 with other electronic devices. Therefore, the implementation of the functions described herein can be distributed throughout the cloud. For example, the functions can be implemented partially on the electronic device 400 and partially through a platform that abstracts the functions of the cloud.

Although the present disclosure has been illustrated and described in detail in the drawings and the foregoing description, such illustration and description should be considered illustrative and exemplary rather than restrictive; the present disclosure is not limited to the disclosed embodiments. By studying the drawings, the disclosure and the appended claims, those skilled in the art will be able to understand and implement variations to the disclosed embodiments when practicing the claimed subject matter. In the claims, the word "comprising" does not exclude other elements or steps that are not listed, the indefinite article "a" or "an" does not exclude a plurality, and the term "plurality" means two or more. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims (18)

A prediction model training method, comprising: Acquire multiple samples, wherein each sample in the multiple samples includes first gene expression data and masked gene expression data, the first gene expression data includes counts of different genes in a single cell, the masked gene expression data is obtained by processing counts of some genes in the first gene expression data, and the processing for obtaining the masked gene expression data includes masking; For each sample in the plurality of samples: Processing the masked gene expression data in the sample using the prediction model to be trained to obtain a predicted value of the partial gene count corresponding to the sample; Determining a loss value corresponding to the sample according to the predicted value and the counts corresponding to the part of genes in the first gene expression data; and The prediction model to be trained is updated according to the loss value corresponding to each sample in the multiple samples. The method according to claim 1, wherein the method further comprises: determining a feature tensor corresponding to the masked gene expression data; The using the prediction model to be trained to process the masked gene expression data in the sample includes: using the prediction model to be trained to process the feature tensor corresponding to the masked gene expression data in the sample. The method according to claim 2, wherein the step of processing the feature tensor corresponding to the masked gene expression data in the sample using the prediction model to be trained comprises: Inputting the feature tensor corresponding to the unmasked and non-zero counts in the masked gene expression data into the first network layer of the prediction model to be trained; The feature tensor corresponding to the masked counts in the masked gene expression data and the feature tensor corresponding to the counts of 0 values in the masked gene expression data are input into a network layer of the prediction model to be trained that is different from the first network layer. The method according to claim 3, wherein the prediction model to be trained comprises an encoder network and a decoder network, The feature tensor corresponding to the masked gene expression data in the sample is processed using the prediction model to be trained, including: Encoding the input of the encoder network using the encoder network to obtain the output of the encoder network; the input of the encoder network includes a feature tensor corresponding to the unmasked and non-zero counts in the masked gene expression data; obtaining an input to a decoder network based on the output of the encoder network, the feature tensor corresponding to the masked counts in the masked gene expression data, and the feature tensor corresponding to the counts of 0 values in the masked gene expression data; and An input to the decoder network is decoded using the decoder network. The method according to claim 4 is characterized in that the encoder network includes M layers of encoding units, the decoder network includes N layers of decoding units, and the value of M is greater than the value of N. The method according to claim 4 or 5 is characterized in that each layer of encoding units of the encoder network includes at least one multi-head attention unit and at least one forward propagation unit, and each layer of decoding units of the decoder network includes at least one forward propagation unit and also includes at least one linear attention unit or sparse attention unit. According to the method according to any one of claims 2 to 6, determining the feature tensor corresponding to the masked gene expression data comprises: Inputting the unmasked and non-zero counts in the masked gene expression data into a module for determining an embedding vector, and obtaining a count embedding vector corresponding to the unmasked and non-zero counts; wherein the module for determining the embedding vector includes learnable parameters; Determining, according to gene identifiers corresponding to unmasked and non-zero counts in the masked gene expression data, gene embedding vectors corresponding to the unmasked and non-zero counts; Obtaining a feature tensor corresponding to the unmasked and non-zero count according to the count embedding vector corresponding to the unmasked and non-zero count and the gene embedding vector corresponding to the unmasked and non-zero count; The method also includes: updating the learnable parameters according to the loss value corresponding to each sample in the multiple samples. The method according to any one of claims 1 to 7, wherein the masked gene expression data is normalized; the first gene expression data included in each of the plurality of samples is the normalized first gene expression data; The using the prediction model to be trained to process the masked gene expression data in the sample to obtain the predicted value of the partial gene count corresponding to the sample includes: Processing the masked gene expression data in the sample using the prediction model to be trained to obtain a predicted value of the normalized value of the partial gene count corresponding to the sample, The determining the loss value corresponding to the sample according to the predicted value and the counts corresponding to the part of genes in the first gene expression data includes: The loss value corresponding to the sample is determined according to the predicted value of the normalized value and the counts corresponding to the part of genes in the normalized first gene expression data. The method according to claim 8, wherein the masked gene expression data is obtained by processing the counts of some genes in the first gene expression data; the processing for obtaining the masked gene expression data includes downsampling, normalization, and masking; the first gene expression data includes the counts of different genes in a single cell measured at a first sequencing depth, and the downsampled first gene expression data simulates the counts of different genes in a single cell measured at a second sequencing depth lower than the first sequencing depth; each sample in the multiple samples also includes auxiliary information, and the auxiliary information includes a first total count and a second total count, the first total count is the sum of the counts of each gene in the first gene expression data; the second total count is the sum of the counts of each gene in the downsampled first gene expression data; Processing the masked gene expression data in the sample using the prediction model to be trained to obtain the predicted value of the partial gene count corresponding to the sample, including: The masked gene expression data in the sample and the auxiliary information are processed using the prediction model to be trained to obtain a predicted value of the normalized value of each gene count at the first sequencing depth corresponding to the sample. The method according to claim 9, wherein the prediction model to be trained includes an encoder network and a decoder network; the method further comprises: determining a feature tensor corresponding to the auxiliary information; Processing the masked gene expression data in the sample and the auxiliary information using the prediction model to be trained includes: Encoding the input of the encoder network using the encoder network to obtain the output of the encoder network; the input of the encoder network includes a feature tensor corresponding to the unmasked and non-zero counts in the masked gene expression data, and the input of the encoder network also includes a feature tensor corresponding to the auxiliary information; obtaining an input to a decoder network based on the output of the encoder network, the feature tensor corresponding to the masked counts in the masked gene expression data, and the feature tensor corresponding to the counts of 0 values in the masked gene expression data; and An input to the decoder network is decoded using the decoder network. The method according to claim 9, wherein the prediction model to be trained comprises an input layer, an output layer, and a plurality of intermediate layers between the input layer and the output layer; the method further comprises: determining a feature tensor corresponding to the auxiliary information; The method of processing the masked gene expression data and auxiliary information in the sample using the prediction model to be trained includes: Inputting the feature tensor corresponding to the masked gene expression data in the sample into the input layer and a first predetermined layer among the plurality of intermediate layers to obtain an intermediate feature tensor, wherein the number of the first predetermined layers is greater than or equal to 0; splicing the feature tensor corresponding to the auxiliary information with the intermediate feature tensor to obtain a spliced intermediate feature tensor; and The concatenated intermediate feature tensor is input into a second predetermined layer among the multiple intermediate layers and the output layer, where the second predetermined layer is different from the first predetermined layer. The method according to claim 10 or 11, wherein determining the feature tensor corresponding to the auxiliary information comprises: Inputting the auxiliary information into a module for determining an embedding vector to obtain a count embedding vector corresponding to the auxiliary information; wherein the module for determining the embedding vector includes learnable parameters; Obtaining a gene embedding vector corresponding to the auxiliary information; Obtaining a feature tensor corresponding to the auxiliary information according to the count embedding vector corresponding to the auxiliary information and the gene embedding vector corresponding to the auxiliary information; The learnable parameters are updated according to the loss value corresponding to each sample in the multiple samples. A method for correcting gene expression data, comprising: Acquiring current gene expression data, wherein the current gene expression data includes counts of different genes measured at an actual sequencing depth; and The current gene expression data is processed using at least part of the network layers in the prediction model trained according to the training method according to any one of claims 1 to 12 to obtain a corrected value of the current gene expression data or an intermediate processing result of the corrected value. According to the method of claim 13, wherein the obtaining of the current gene expression data comprises: obtaining normalized current gene expression data and current auxiliary information; wherein the normalized current gene expression data is obtained by normalizing the current gene expression data; the current auxiliary information comprises an expected first total count and a current second total count; the current second total count is the sum of the counts of each gene in the current gene expression data; the expected first total count is used to characterize the expected sequencing depth, and the expected first total count is greater than or equal to the sum of the counts of each gene in the current gene expression data; and The processing of the current gene expression data by at least part of the network layers in the prediction model trained by the training method according to any one of claims 1 to 12 to obtain the corrected value of the current gene expression data or the intermediate processing result of the corrected value comprises: At least part of the network layers in the prediction model trained by the training method described in any one of claims 9 to 12 processes the normalized current gene expression data and the current auxiliary information to obtain a corrected value of the normalized value of the current gene expression data at the expected sequencing depth or an intermediate processing result of the corrected value of the normalized value. A downstream task execution method, comprising: Acquire input data, wherein the input data comprises i) a correction value obtained by the method according to claim 13 or 14; or ii) an intermediate processing result of the correction value in the method according to claim 13 or 14; or iii) a preprocessing result obtained by preprocessing the correction value obtained by the method according to claim 13 or 14; or iv) a preprocessing result obtained by preprocessing the intermediate processing result of the correction value obtained by the method according to claim 13 or 14; and The input data is processed using a downstream task algorithm to obtain a downstream task result, wherein the downstream task includes a cell classification task, a disturbance prediction task or a drug response prediction task. A computing device comprising: a memory, a processor and a computer program stored on the memory, The processor is configured to execute the computer program to implement the steps of the method according to any one of claims 1 to 15. A non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 15. A computer program product comprises computer instructions, wherein when the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 15 are implemented.