WO2024230826A1 - Protein solubility prediction method - Google Patents

Abstract

A protein solubility prediction method. Features, which are extracted on the basis of a protein pre-training model, are combined with primary structural features of a protein, an automatic learning framework is combined with an automatic hyper-parameter screening algorithm, and an MCC is used as a model optimization index, so as to construct a prediction model for both a classification problem and a regression problem, such that whether the protein is soluble can be predicted, and the solubility probabilities of different proteins can also be predicted, thus facilitating the screening of a high-potential variant in advance.

Description

Protein solubility prediction method

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to Chinese patent application No. 202310533179.6 filed on May 11, 2023, the entire contents of which are incorporated herein by reference.

Technical Field

The present disclosure relates to protein research, and more particularly to methods for predicting protein solubility.

Background Art

Improving protein solubility plays a very important role in increasing the yield of recombinant proteins, conducting protein function research (including structural proteomics and functional proteomics research) and completing enzyme engineering precursor screening, which is directly related to the use function and production cost of recombinant proteins. For example, in the process of protein engineering, if the solubility of different protein variants can be accurately predicted in advance based on the amino acid sequence of the protein, this important evaluation index can be used to improve the screening efficiency, find high-potential proteins, and effectively reduce the scale of downstream experimental verification. There are many ways to improve protein solubility in the production process. For example, in the Escherichia coli expression system, by using weak promoters, using specific protein tags or changing signal peptides, strong denaturants, denaturation and renaturation, low temperature and long-term expression, codon optimization and expression of molecular chaperones, etc., the inclusion bodies of recombinant proteins are reduced, and the effective yield of proteins (ie, soluble proteins) is increased. However, accurately predicting protein solubility in advance is still a very important issue, which is also the reason why models and tools for predicting protein solubility have been continuously updated in recent years (as shown in Table 1).

The primary sequence of a protein determines its higher-order structure after folding, and the higher-order structure of a protein determines its solubility. Therefore, both sequence features based on the primary structure and features based on the higher-order structure have been used to construct protein solubility prediction models, including protein sequence length, amino acid residue composition (such as the number of charged amino acid residues, the number of hydrophobic/hydrophilic amino acid residues, k-mer features), protein secondary structure (such as the ratio of alpha helix and beta fold), protein tertiary structure (such as the turn between the alpha carbon atoms of two amino acids), etc. (For detailed feature descriptions, see the papers in Table 1). The statistical methods, machine learning, and deep learning methods currently used to predict protein solubility include: Gaussian discriminant analysis model, linear regression, support vector machine, logistic regression, linear programming, random Machine forest, gradient boosting machine, naive Bayes, neural network, convolutional neural network, dual-channel convolutional neural network, bidirectional threshold recurrent unit neural network, graph neural network, etc. (For detailed model usage, see the text of each paper in Table 1).

Table 1: Currently known methods for predicting protein solubility

However, these current protein solubility prediction models are not compatible with classification and regression problems, and their accuracy needs to be improved.

Summary of the invention

In view of the fact that it is difficult for the protein solubility classification prediction model in the prior art to simultaneously ensure high accuracy of classification and regression, the present disclosure proposes a protein solubility prediction method, which combines the features extracted based on the protein pre-training model with the protein primary structure features, and uses an automatic machine learning framework to build and train a model that is compatible with classification problems (predicting whether a protein is soluble) and regression problems (predicting the probability of different proteins being soluble, predicting the effect of amino acid mutations on solubility, and helping to screen high-potential variants in advance) for predicting protein solubility. Furthermore, the hyperparameter combination that optimizes model performance can be obtained through automatic hyperparameter screening in the automatic machine learning framework.

According to the first aspect of the present disclosure, a method for predicting protein solubility is provided. The method may include: extracting features based on a protein sequence using a protein pre-training model; extracting primary structural features of the protein based on the protein sequence; concatenating the features extracted by the protein pre-training model with the primary structural features of the protein to obtain a feature vector of the protein; inputting the feature vector of the protein into a protein solubility prediction model to obtain a prediction result of the protein solubility.

In the method according to the first aspect of the present disclosure, preferably, the protein pre-training model can be implemented by a combination of one or more of the following models: ESM-1b, UniRep, ProteinBert, TAPE, ProtGPT2, ProtTXL, ProtBert, ProtXLNet, ProtAlbert, ProtElectra, ProtT5-XL-BFD, ProtT5-XL-UniRef50 and ProtT5-XXL; more preferably, the protein pre-training model is ProtT5-XL-BFD or ProtT5-XL-UniRef50; more preferably, the protein pre-training model is ProtT5-XL-UniRef50.

In the method according to the first aspect of the present disclosure, preferably, the protein sequence can be used as input, and the ProtT5-XL-UniRef50 pre-trained model can be used to extract the embedding layer vector output by the encoder, wherein each amino acid corresponds to a 1024-dimensional vector, and each sequence corresponds to an L×1024-dimensional feature matrix, wherein L is the length of the amino acid sequence of the protein; the above-mentioned feature matrix is averaged by column to obtain a 1024-dimensional feature vector.

Preferably, the Sentence-T5 Encoder-only mean method can be used to average the values of each dimension of the embedding layer feature values of all amino acids output by the encoder by column to obtain a 1024-dimensional feature vector.

In the method according to the first aspect of the present disclosure, preferably, the protein sequence can be used as input to extract the primary structural features of the protein using the iFeature tool.

Preferably, the primary structural features of the protein can be selected from AAC (Amino Acid Composition), DPC (Di-Peptide Composition), DDE (Dipeptide Deviation from Expected Mean), TPC (Tri-Peptide Composition), CKSAAP (Composition of k-spaced Amino Acid Pairs), EAAC (Enhanced Amino Acid Composition), GAAC (Grouped Amino Acid Composition), ition (grouped amino acid composition), CKSAAGP (Composition of k-Spaced Amino Acid Group Pairs), GDPC (Grouped Di-Peptide Composition), GTPC (Grouped Tri-Peptide Composition), Moran (Moran correlation), Geary (Geary correlation), NMBroto (Normalized Moreau-Broto Autocorrelation), CTDC (Compositio n, Transition and Distribution: Composition, Composition, Transition and Distribution: Composition), CTDT (Composition, Transition and Distribution: Transition, Composition, Transition and Distribution: Transition), CTDD (Composition, Transition and Distribution: Distribution, Composition, Transition and Distribution: Distribution), CTriad (Conjoint Triad, Conjugate Triad), KSCTriad (k-Spaced Conjoint Triad, k-Spaced Conjoint Triad), SOCNumber (Sequence-Order-Coupling Number, Sequence-Order-Coupling Number), The code may be one or more of the following: QSOrder (Quasi-sequence-order), PAAC (Pseudo-Amino Acid Composition), APAAC (Amphiphilic Pseudo-Amino Acid Composition), KNNprotein (K-Nearest Neighbor for proteins), PSSM (PSSM profile, position-specific scoring matrix), AAINDEX (AAindex) and BLOSUM62 (blocks substitution matrix, amino acid substitution scoring matrix used for sequence comparison in bioinformatics).

More preferably, the primary structural features of the protein extracted using the iFeature tool include 11 categories, namely AAC, GAAC, Moran, Geary, NMBroto, CTDC, CTDT, CTDD, CTriad, PAAC and APAAC.

In the method according to the first aspect of the present disclosure, preferably, the protein solubility prediction model can be constructed by the following steps: obtaining a data set, the data set comprising a plurality of training samples, each of the training samples comprising The invention discloses a method for predicting protein solubility of a protein by using a protein pre-training model and a sample protein sequence and its solubility data; extracting features based on the sample protein sequence using a protein pre-training model; extracting primary structural features of the protein based on the sample protein sequence; splicing the features extracted by the protein pre-training model with the primary structural features of the protein to obtain a feature vector of the sample protein; using the feature vector of the sample protein as input data and the solubility data of the sample protein as output data, and obtaining a final protein solubility prediction model by training a machine learning model.

In the method according to the first aspect of the present disclosure, preferably, the protein solubility prediction model can be constructed and trained based on an automatic machine learning framework, using one or more machine learning models as candidate models. In some embodiments, the machine learning model can be a deep learning model. In some embodiments, the automatic machine learning framework can be selected from AutoGluon (https://auto.gluon.ai/stable/index.html), Auto-Sklearn (https://automl.github.io/auto-sklearn/master/), TPOT (http://epistasislab.github.io/tpot/), Hyperopt Sklearn (https://github.com/hyperopt/hyperopt-sklearn), Auto-Keras (https://autokeras.com/), H2O AutoML (https://docs.h2o.ai/h2o/latest-stable/h2o-docs/automl.html), TransmogrifAI (https://transmogrif.ai/). In some specific embodiments, the automatic learning framework can be AutoGluon.

In some embodiments, the machine learning model is a deep learning model, which can be selected from NN_Torch, FASTAI, LGBModel, RFModel, KNNModel, etc. In some embodiments, the deep learning model can be one or more. In a preferred embodiment, the deep learning model can be NN_Torch and/or FASTAI. In a specific embodiment, the deep learning model can be NN_Torch and FASTAI.

In the method according to the first aspect of the present disclosure, the machine learning model can be automatically selected by an automatic learning framework, or a list and screening range of machine learning models can be manually set and then selected by the automatic learning framework. Preferably, AutoGluon can be used as an automatic machine learning framework, and deep learning models NN_Torch and FASTAI can be selected as candidate models to construct and train the protein solubility prediction model.

In the method according to the first aspect of the present disclosure, preferably, other features of the protein can also be extracted based on the protein sequence; the features extracted by the protein pre-training model are spliced with the primary structure features of the protein and other features of the protein to obtain a feature vector of the spliced protein.

Preferably, the other characteristics of the protein are selected from the group consisting of molecular weight, aromaticity, instability index, flexibility, isoelectric point, molar absorption coefficient, grand average of hydrophilicity, Hydropathy), the proportion of helix/turn/sheet in the secondary structure of the protein, SSEC (Secondary Structure Elements Content), SSEB (Secondary Structure Elements Binary), Disorder, DisorderC, DisorderB (Disorder Binary), ASA (Accessible Solvent Accessibility) and TA (Torsion angle), sequence length, ZSCALE (Z-scale) and 48 PseKRAAC (48 pseudo K-tuple reduced amino acids composition); more preferably, the other characteristics of the protein include molecular weight, aromaticity, instability coefficient, elasticity, isoelectric point, molar absorptivity, total average hydrophilicity and the proportion of helix/turn/sheet in the secondary structure of the protein.

According to the second aspect of the present disclosure, a method for constructing a protein solubility prediction model is provided. The construction method may include: obtaining a data set, the data set includes multiple training samples, each of the training samples includes a sample protein sequence and its solubility data; based on the sample protein sequence, extracting features using a protein pre-training model; based on the sample protein sequence, extracting the primary structural features of the protein; splicing the features extracted by the protein pre-training model with the primary structural features of the protein to obtain a feature vector of the sample protein; using the feature vector of the sample protein as input data, using the solubility data of the sample protein as output data, and training the machine learning model to obtain a final protein solubility prediction model.

The method for constructing a protein solubility prediction model in the first aspect of the present disclosure is suitable for the second aspect of the present disclosure.

According to the third aspect of the present disclosure, a system for predicting protein solubility is provided. The system may include: an acquisition module for acquiring a sample to be predicted, wherein the sample to be predicted includes a protein sequence of the sample to be predicted; a processing module for obtaining a prediction result of the protein solubility of the sample to be predicted by inputting the protein sequence of the sample to be predicted, wherein the processing module further includes the following submodules: a first feature extraction submodule for extracting features based on the protein sequence of the sample to be predicted using a protein pre-training model; a second feature extraction submodule for extracting the primary structural features of the protein based on the protein sequence of the sample to be predicted; a feature splicing submodule for splicing the features extracted by the protein pre-training model with the primary structural features of the protein to obtain a feature vector of the protein of the sample to be predicted; and a model prediction submodule for inputting the feature vector of the protein of the sample to be predicted into a protein solubility prediction model to obtain a prediction result of the protein solubility of the sample to be predicted.

The system for predicting protein solubility according to the third aspect of the present disclosure can be implemented by a computer. Preferably, the system can be implemented by executing a computer program to implement the following operations: obtaining a sample to be predicted, wherein the sample to be predicted includes The protein sequence of the sample to be predicted is input; by inputting the protein sequence of the sample to be predicted, a prediction result of the protein solubility of the sample to be predicted is obtained, and the operation further includes the following sub-operations: based on the protein sequence of the sample to be predicted, a protein pre-training model is used to extract features; based on the protein sequence of the sample to be predicted, the primary structure features of the protein are extracted; the features extracted by the protein pre-training model are spliced with the primary structure features of the protein to obtain a feature vector of the protein of the sample to be predicted; the feature vector of the protein of the sample to be predicted is input into the protein solubility prediction model to obtain a prediction result of the protein solubility of the sample to be predicted.

According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided for storing a computer program. The computer program includes instructions. When the instructions are executed by a processor of an electronic device, the electronic device implements the protein solubility prediction method according to the first aspect of the present disclosure or the method for constructing a protein solubility prediction model according to the second aspect of the present disclosure.

According to a fifth aspect of the present disclosure, a computer system is provided. The computer system comprises: a processor, a memory, and a computer program. The computer program is stored in the memory and is configured to be executed by the processor. The computer program comprises instructions for implementing the protein solubility prediction method according to the first aspect of the present disclosure or the method for constructing a protein solubility prediction model according to the second aspect of the present disclosure.

According to a sixth aspect of the present disclosure, a protein solubility prediction model constructed by the method of the second aspect of the present disclosure is provided. Further provided is the application of the protein solubility prediction model in predicting protein solubility. For example, the protein solubility prediction model can be applied to predict the solubility of any protein sample of known sequence (such as a mutant of a known protein), and the predicted result can be the probability of solubility or the result of solubility yes or no.

The comprehensive performance of the method disclosed in the present invention in predicting protein solubility exceeds the NetSolP method which is currently ranked first in classification performance.

In addition, the method disclosed in the present invention has a high accuracy rate in predicting the solubility of proteins, and is applicable to both the classification prediction problem of predicting whether a protein is soluble and the regression problem of predicting the effect of amino acid mutations on solubility. It can be used to predict the solubility of recombinant protein expression and to screen enzyme engineering mutants based on solubility potential.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure will be more fully understood from the following detailed description taken in conjunction with the accompanying drawings, in which like elements are numbered in a similar manner, and in which:

FIG1 is a flow chart of a method for predicting protein solubility according to an embodiment of the present disclosure.

FIG. 2 is a schematic diagram showing the association between a protein solubility prediction method and a prediction model construction method according to an embodiment of the present disclosure.

FIG. 3 is a more detailed flowchart of a protein sequence-based feature extraction method according to an embodiment of the present disclosure.

FIG. 4 is a more detailed flowchart of the method for constructing a protein solubility prediction model used in the present disclosure.

FIG5 shows a schematic block diagram of a system for predicting protein solubility according to the present disclosure.

DETAILED DESCRIPTION

Unless otherwise defined, technical and scientific terms used in the present disclosure have the same meaning as commonly understood by one of ordinary skill in the art to which the present disclosure belongs.

The technical scheme of the present disclosure is further described in detail below by way of examples and in conjunction with the accompanying drawings. Unless otherwise specified, the methods and materials of the embodiments described below are conventional products that can be purchased from the market. It will be understood by those skilled in the art that the present disclosure belongs to that the methods and materials described below are merely exemplary and should not be construed as limiting the scope of the present disclosure.

Predicting protein solubility can be divided into two categories. One is a binary classification problem or a multi-classification problem of predicting whether a protein is soluble or insoluble or whether it is easy or difficult to dissolve; the other is a regression problem of predicting the probability of protein solubility. Regardless of the type of problem, the prediction cannot be separated from the collection of solubility data based on expression experiments. The commonly used protein solubility experimental data sets are as follows (see Table 2).

Table 2: List of protein solubility experimental datasets

Since the classification cannot be predicted simply based on the regression problem, it is difficult for the existing protein solubility classification prediction model to ensure high accuracy of both classification and regression at the same time. The existing protein solubility classification prediction method has low compatibility when used for regression prediction. For example, the NetSolP method has an accuracy of 72.8% on the independent test set NESG dataset for classification problems, but only 66.1% on the CamSol independent test set for regression problems.

The present disclosure provides a method for predicting protein solubility. By constructing a prediction model that is compatible with classification problems and regression problems, it is possible to predict whether a protein is soluble and the probability of different proteins being soluble, which is helpful for screening high-potential variants in advance.

From the overall perspective of the prediction method, FIG1 shows a flow chart of a protein solubility prediction method according to an embodiment of the present disclosure.

As shown in FIG. 1 , the protein solubility prediction method 100 according to an embodiment of the present disclosure starts at step S110 , in which features are extracted based on the protein sequence using a protein pre-training model.

Next, in step S120, still based on the protein sequence, the primary structural features of the protein are extracted.

In step S130, the features extracted by the protein pre-training model in step S110 are concatenated with the primary structure features of the protein extracted in step S120 to obtain a feature vector of the protein.

Finally, in step S140, the feature vector of the protein obtained in step S130 is input into a protein solubility prediction model to obtain a prediction result of protein solubility.

In a preferred embodiment according to the present disclosure, the prediction result of protein solubility is: the protein is classified as soluble or insoluble; in another preferred embodiment according to the present disclosure, the prediction result of protein solubility is the probability that the protein is soluble.

As mentioned above, the method disclosed in the present invention combines the features extracted based on the protein pre-training model with the primary structure features of the protein, and uses an automatic learning framework to construct a prediction model that is compatible with classification problems (predicting whether a protein is soluble) and regression problems (predicting the probability that different proteins are soluble).

Figure 2 shows a schematic diagram 200 of the association between the protein solubility prediction method and the prediction model construction method according to an embodiment of the present disclosure. In the schematic diagram 200 shown in Figure 2, the left side of the dotted line shows the model construction method, and the right side of the dotted line shows the method of using the model for prediction.

The process on the right side of the dotted line in FIG. 2 is actually the protein solubility prediction method according to the embodiment of the present disclosure shown in FIG. 1. The process on the left side of the dotted line in FIG. 2 will be described in detail below (e.g., FIG. 4 and its corresponding text description). As can be seen from FIG. 2, both the model construction and the actual prediction process need to use the protein feature extraction method, which will be described in detail below (e.g., FIG. 3 and its corresponding text description). For the training sample, it includes both the protein sequence of the training sample and the protein solubility data of the training sample. As described later, the protein solubility data may include protein soluble or insoluble. Therefore, in the process of model construction, the protein sequence in the training sample is used to extract features. By using the extracted protein features as input data and the protein solubility data in the training sample as output data, the model based on the automatic machine learning framework (e.g., using the AutoGluon framework) is fully trained to obtain the final protein solubility prediction model based on the automatic machine learning framework. For the constructed and trained prediction model, it can be put into the actual work of predicting the protein solubility of the predicted sample, that is, the process on the right side of the dotted line in FIG. 2.

The following is a detailed description of the protein feature extraction method according to the embodiments of the present disclosure and the more detailed process of the method for constructing a protein solubility prediction model based on an automatic machine learning framework.

FIG. 3 is a more detailed flowchart of a protein feature extraction method 300 according to an embodiment of the present disclosure.

As shown in step S110 of the prediction method of FIG. 1 , in the protein feature extraction step, first, based on the protein sequence, a protein pre-training model is used to extract features.

The protein pre-training model described here can be implemented by a combination of one or more of the following models: ESM-1b, UniReP, ProteinBert, TAPE, ProtGPT2, ProtTXL, ProtBert, ProtXLNet, ProtAlbert, ProtElectra, ProtT5-XL-BFD, ProtT5-XL-UniRef50 and ProtT5-XXL.

Preferably, ProtT5-XL-BFD or ProtT5-XL-UniRef50 can be used as the protein pre-training model.

In a preferred embodiment of the present disclosure, the protein sequence pre-training model can be a ProtT5-XL-UniRef50 pre-training model. ProtT5-XL-UniRef50 is a protein sequence pre-training model using a masked language modeling (MLM) objective. Reference may be made to A. Elnaggar et al., “ProtTrans: Toward Understanding the Language of Life Through Self-Supervised Learning”, in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 10, pp. 7112-7127, 1Oct. 2022, doi: 10.1109/TPAMI.2021.3095381. By reference herein, the entire contents of the above-mentioned document are incorporated into the present disclosure and become part of the contents of the present disclosure. In addition, the source code of the ProtT5-XL-UniRef50 model and its more detailed application can be found at https://github.com/agemagician/ProtTrans, where the model was first released. ProtT5-XL-UniRef50 is based on the t5-3b model and is pre-trained on a large number of protein sequences in a self-supervised manner. The model can be used for protein feature extraction, and it is best to use features extracted from the encoder.

As shown in Figure 3, the protein feature extraction method 300 according to an embodiment of the present disclosure can start at step S310. That is, as described above, according to a preferred embodiment of the present disclosure, in step S310, the protein sequence of the sample to be predicted is used as input, and the ProtT5-XL-UniRef50 pre-trained model is used to extract the embedding layer vector output by the encoder, wherein each amino acid corresponds to a 1024-dimensional vector, and each sequence corresponds to an L×1024-dimensional feature matrix, wherein L is the amino acid sequence length of the protein.

Although the ProtT5-XL-UniRef50 pre-training model is used to extract the feature matrix of the protein sequence in the preferred embodiment of the present disclosure, those skilled in the art are fully motivated to use another or newly developed protein pre-training model to replace the ProtT5-XL-UniRef50 pre-training model used in the present disclosure when the input and output are essentially unchanged and the functions are similar and the effects are similar or better. Those skilled in the art should understand that even after such a replacement of the pre-training model, the replaced scheme still falls within the scope of protection claimed by the present disclosure.

Next, the obtained feature matrix can be compressed. Specifically, the feature matrix can be averaged by column to obtain a 1024-dimensional feature vector. More specifically, as shown in FIG3 , in step S320, the feature matrix obtained in step 310 is compressed, that is, the Sentence-T5 Encoder-only mean processing method is used to average the values of each dimension of the embedding layer feature values of all amino acids output by the encoder by column to obtain a 1024-dimensional feature vector. Here, the embedding layer feature values of the padding code are not calculated, so that the embedding layer feature values of all amino acids constituting a single protein are converted into 1024-dimensional feature values of the protein. Sentence-T5 Encoder-only mean is one of the processing methods for the features output by the encoder in Sentence-T5. For more information about Sentence-T5 Encoder-only mean, please refer to Jianmo Ni, Gustavo Hernandez Abrego, Noah Constant, Ji Ma, Keith Hall, Daniel Cer, and Yinfei Yang, “Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models”, in Findings of the Association for Computational Linguistics: ACL 2022, pages 1864–1874, Dublin, Ireland; Association for Computational Linguistics; doi: 10.18653/v1/2022.findings-acl.146. The entire contents of the above-mentioned documents are hereby incorporated into the present disclosure by reference, making them a part of the contents of the present disclosure.

Those skilled in the art should note that, although Sentence-T5 Encoder-only mean is a disclosed technology, its application in the method of the present disclosure is novel, because the application field involved in the present disclosure is completely different from the application field involved in the original disclosure of Sentence-T5 Encoder-only mean. The present disclosure uses the Sentence-T5 Encoder-only mean method to compress the extracted feature matrix, so that the extracted feature vectors can more deeply integrate the characteristics of the protein, which is helpful for the subsequent construction of a prediction model with better comprehensive performance based on this feature, and the final protein solubility prediction result is more accurate.

According to step S120 of FIG. 1 , next, the primary structural features of the protein are extracted based on the protein sequence.

In a preferred embodiment of the present disclosure, more specifically, the protein sequence is used as input and the iFeature tool is used to extract the primary structural features of the protein. As shown in Figure 3, in step S330, the protein sequence is used as input and the iFeature tool is used to extract the primary structural features of the protein.

For the iFeature tool, see Zhen Chen, Pei Zhao, Fuyi Li, André Leier, Tatiana T. Marquez-Lago, Yanan Wang, Geoffrey I. Webb, A. Ian Smith, Roger J. Daly, Kuo-Chen Chou, Jiangning Song, “iFeature: a Python package and web server for features extraction and selection from protein and peptide sequences”, Bioinformatics, Volume 34, Issue 14, July 2018, Pages 2499–2502, https://doi.org/10.1093/bioinformatics/bty140. The entire contents of the above-mentioned document are hereby incorporated into the present disclosure by reference, making it a part of the contents of the present disclosure.

Although the iFeature tool is used to extract the primary structural features of proteins in the preferred embodiment of the present disclosure, the input and output data are essentially unchanged, and the method functions are similar and the effects are similar or better, those skilled in the art are fully motivated to use another or newly developed extraction tool to replace the iFeature tool used in the present disclosure. Those skilled in the art should understand that even after such an extraction tool is replaced, the replaced solution still falls within the scope of protection claimed by the present disclosure.

In some embodiments of the present disclosure, the primary structural features of proteins extracted using the iFeature tool may include AAC (Amino Acid Composition), DPC (Di-Peptide Composition), DDE (Dipeptide Deviation from Expected Mean), TPC (Tri-Peptide Composition), CKSAAP (Composition of k-spaced Amino Acid Pairs), EAAC (Enhanced Amino Acid Composition), GAAC (Grouped Amino Acid Composition), and ), CKSAAGP(Composition of k-SpacedAmino Acid Group Pairs，Composition of k-spaced amino acid group pairs，GDPC(Grouped Di-Peptide Composition，Grouped Di-Peptide Composition，GTPC(Grouped Tri-Peptide Composition，Grouped Tri-Peptide Composition，Moran(Moran correlation，Moran correlation，Geary(Geary correlation，Geary correlation，NMBroto(Normalized Moreau-BrotoAutocorrelation，Normalized Moreau-BrotoAutocorrelation，CTDC(Composition,Transition and Distribution:Composition，Composition，Transition and Distribution:Composition，CTDT(Composition,Transition and Distribution:Transition，Composition,Transition and Distribution:Transition，CTDD(Composition,Transition and Distribution:Distribution，Composition,Transition and Distribution:Distribution，CTriad(Conjoint Triad，Conjugated Triad，KSCTriad，K-Spaced Conjoint Triad，SOCNumber(Sequence-Order-Coupling Number, sequence order coupling number), QSOrder (Quasi-sequence-order), PAAC (Pseudo-Amino Acid Composition), APAAC (Amphiphilic Pseudo-Amino Acid Composition), KNNprotein (K-Nearest Neighbor for proteins), PSSM (PSSM profile), AAINDEX (AAindex) and BLOSUM62 (BLOSUM62) are one or more of them.

In a preferred embodiment of the present disclosure, the primary structural features of proteins extracted using the iFeature tool include 11 categories, namely AAC, GAAC, Moran, Geary, NMBroto, CTDC, CTDT, CTDD, CTriad, PAAC and APAAC.

In addition, these features can be normalized, and the parameters of the standard normalization will be saved and used to process similar features of the input sequence during the prediction process, that is, the standard normalization parameters of the training set are saved for processing the same features of the independent test set.

According to step S130 of FIG. 1 , the features extracted by the protein pre-training model are then concatenated with the primary structure features of the protein to obtain a feature vector of the protein.

As shown in FIG. 3 , in step S340 , the feature vector obtained in step S320 is concatenated with the primary structural feature of the protein obtained in step S330 to obtain a feature vector of the protein.

Those skilled in the art should note that in the method disclosed herein, it is creatively proposed to combine the features extracted by the protein pre-training model with the primary structural features of the protein extracted by, for example, the iFeature tool to form protein features for prediction model input. This approach is not only proposed for the first time, but also, as described below, has achieved good results in protein solubility prediction.

Furthermore, in an alternative embodiment, in addition to the primary structural features of the protein, other features of the protein can be extracted based on the protein sequence, and the other features include molecular features, secondary structure and/or higher-order structural features of the protein sequence, thereby further optimizing the model.

In some specific embodiments, the iFeature tool can be used to extract other features of the protein. For example, the molecular features of the sequence can be selected from one or more of the following: sequence length, ZSCALE (Z-scale) and 48PseKRAAC (48pseudo K-tuple reduced amino acids composition); the secondary structure features can be selected from one or more of the following: SSEC (Secondary Structure Elements Content), SSEB (Secondary Structure Elements Binary); the advanced structure features can be selected from one or more of the following: Disorder, DisorderC, DisorderB (Disorder Binary), ASA (Accessible Solvent accessibility) and TA (Torsion angle).

In other specific embodiments, other features can also be extracted using the Bio.SeqUtils.ProtParam module of Biopython. For example, other features of the protein sequence are selected from one or more of the following: molecular weight, aromaticity, instability index, flexibility, isoelectric point, molar absorption coefficient and grand average of hydrophilicity; the secondary structure features include the proportion of helix/turn/sheet in the secondary structure of the protein.

In some preferred embodiments, other features of the protein include molecular weight, aromaticity, instability coefficient, elasticity, isoelectric point, molar absorptivity, overall average hydrophilicity and the proportion of helix/turn/fold in the secondary structure of the protein extracted using the Bio.SeqUtils.ProtParam module of Biopython.

For Biopython, see Peter J. A. Cock, Tiago Antao, Jeffrey T. Chang, Brad A. Chapman, Cymon J. Cox, Andrew Dalke, Iddo Friedberg, Thomas Hamelryck, Frank Kauff, Bartek Wilczynski, Michiel J. L. de Hoon, Biopython: freely available Python tools for computational molecular biology and bioinformatics, Bioinformatics, Volume 25, Issue 11, June 2009, Pages 1422–1423, https://doi.org/10.1093/bioinformatics/btp163. The entire contents of the above-mentioned documents are hereby incorporated into the present disclosure by reference, making them a part of the contents of the present disclosure.

Although the Bio.SeqUtils.ProtParam module of Biopython is used to extract other features of proteins in the preferred embodiment of the present disclosure, those skilled in the art are fully motivated to use other or newly developed extraction methods when the input and output data do not change in essence and the functions of the methods are similar and the effects are similar or better. The Biopython tool used in the present disclosure is replaced by a tool. Those skilled in the art should understand that even after such an extraction tool is replaced, the replaced scheme still falls within the scope of protection claimed in the present disclosure.

In a specific embodiment of the present disclosure, the iFeature tool is used to extract the primary structural features of the protein, including AAC, GAAC, Moran, Geary, NMBroto, CTDC, CTDT, CTDD, CTriad, PAAC and APAAC; at the same time, the Bio.SeqUtils.ProtParam module of Biopython is used to extract other features of the protein, including molecular weight, aromaticity, instability coefficient, elasticity, isoelectric point, molar absorptivity, total average hydrophilicity and the proportion of helix/turn/fold in the secondary structure of the protein, a total of 1502 features.

In this way, the features extracted by the protein pre-training model are spliced with the primary structural features of the protein and other features of the protein to obtain the feature vector of the spliced protein. Specifically, the 1024-dimensional feature vector extracted by the protein pre-training model is spliced with the primary structural features and other features of the extracted 1502 proteins to obtain a 2526-dimensional feature vector for each protein.

Through the above series of steps S310-S340, according to the method disclosed in the present invention, based on the protein sequence of the sample to be predicted, (1) protein feature vectors can be extracted through the protein pre-training model, (2) primary structural features of proteins can be extracted through the iFeature tool, and (3) other features of proteins can be extracted through the Bio.SeqUtils.ProtParam module of Biopython, and then the three types of features can be spliced. Thus, the spliced features can be used as input to predict the protein solubility of the sample to be predicted through the protein solubility prediction model, and the predicted protein solubility results can be output.

It is understandable that the present embodiment does not limit the order of splicing the three types of features, namely, features extracted through the protein pre-training model, primary structural features of the protein, and other features of the protein. It is only necessary to ensure that the order of splicing the three types of features is consistent during model building and actual prediction. Exemplarily, during model building, the features extracted through the protein pre-training model, primary structural features of the protein, and other features of the protein are spliced in order to obtain the feature vector of the protein; then, during the actual prediction process, the features extracted through the protein pre-training model, primary structural features of the protein, and other features of the protein are spliced in the same order to obtain the feature vector of the protein of the sample to be predicted.

In some embodiments, the method may further include performing dimensionality reduction processing on the feature vector of the spliced protein. The dimensionality reduction processing may further perform feature extraction on the mutation vector of the protein, extract features with relevance and effectiveness as much as possible, simplify the prediction model, reduce the computational complexity and shorten the computational time, so as to make the subsequent steps more efficient. For example, the principal component analysis (PCA) model may be used to perform dimensionality reduction on the feature vector of the spliced protein.

On the other hand, as described in FIG2 , the above series of steps can also be used to extract protein features from training samples, so that the extracted protein features are used as input data and input into a prediction model constructed by an automatic machine learning framework (such as the AutoGluon framework), and the protein solubility data in the training samples are used as output data to train a protein solubility prediction model based on the automatic machine learning framework, so as to ultimately determine the model parameters, thereby constructing a protein solubility prediction model with relatively satisfactory accuracy.

FIG. 4 is a more detailed flowchart of a method 400 for constructing a protein solubility prediction model used in the present disclosure.

Those skilled in the art should know that, as shown in Figure 4, in step S410, a data set including multiple training samples should be first obtained. Specifically, each training sample in the multiple training samples in the data set includes a sample protein sequence and its solubility data.

In a preferred embodiment of the present disclosure, the PSI:Biology dataset is selected as a training set to construct a binary classification model, and the source of the dataset is shown in Table 2 above. The sequence and solubility data of the protein are screened from the dataset, where the solubility data is divided into two categories: soluble and insoluble, as the training dataset; the probability value output by the classification model (such as the probability of being judged as a soluble protein) is used for the regression problem.

Then, as mentioned above, it is necessary to perform protein feature extraction based on the sample protein sequence in the data set. As shown in step S420 in FIG. 4 , feature extraction is performed on all training sample proteins according to the method of FIG. 3 . Specifically, first, based on the sample protein sequence, features are extracted using a protein pre-training model. Then, based on the sample protein sequence, the primary structural features of the protein are extracted. Finally, the features extracted by the protein pre-training model are spliced with the primary structural features of the protein to obtain a feature vector of the sample protein.

Furthermore, in an alternative embodiment, in addition to the primary structural features of the sample protein, other features of the sample protein can also be extracted based on the sample protein sequence. In this way, the features extracted by the protein pre-training model are spliced with the primary structural features of the extracted protein and other features of the extracted protein to obtain a feature vector of the spliced sample protein.

Next, the feature vector of the spliced sample protein is used as input data, and the solubility data of the sample protein is used as output data. The machine learning model is trained to obtain the final protein solubility prediction model.

Specifically, the protein solubility prediction model can be constructed and trained based on an automatic machine learning (eg, AutoML) framework, using one or more machine learning (eg, deep learning models) as candidate models.

Those skilled in the art will appreciate that the complexity and amount of learning involved in implementing a machine learning model is significant, and that it also requires extensive domain expertise to generate and compare multiple models before selecting the best model. Get the best model to achieve our ultimate goal. Automatic machine learning can simplify the process and automate the process of building the entire machine learning pipeline with minimal human intervention. In addition, using automatic machine learning can obviously train small and medium-sized datasets faster than larger datasets.

AutoGluon is a popular automatic machine learning framework. This AutoML open source toolkit developed by AWS helps achieve strong predictive performance in various machine learning and deep learning models.

AutoGluon (see doi:10.48550/arXiv.2003.06505, the entire contents of which are incorporated herein by reference as a part of the present disclosure) has achieved excellent performance in the evaluation of different automatic machine (deep) learning frameworks (see doi:10.48550/arXiv.2111.02705) due to its powerful feature engineering processing capabilities, automatic model selection and automatic combination and layer stacking technology, and automatic hyperparameter search technology, and can effectively avoid overfitting problems. It has been selected as the model automatic learning framework in the embodiments of the present disclosure.

As shown in FIG. 4 , in step S430 , AutoGluon is used as an automatic machine learning framework, and the protein solubility prediction model is constructed and trained based on the deep learning models NN_Torch and FASTAI as candidate models.

More specifically, when using AutoGluon, deep learning models NN_Torch and FASTAI are selected as candidate models, the Bayesian automatic hyperparameter selection strategy is enabled, and the search is performed ten times. The maximum number of model layers stacked is three, that is, the actual model usage parameters can be set as follows:

problem_type="binary"

eval_metric = 'mcc'

presets = 'best_quality'

num_bag_folds=5

num_bag_sets=5

num_stack_levels = 3

excluded_model_types=["GBM","CAT","XGB","RF","XT","KNN","LR"]

hyperparameter_tune_kwargs='bayes'

hyperparameters={"FASTAI":{},"NN_TORCH":{}}.

In the process of building the model, a five-fold cross validation was used (i.e., 80% of the data in the PSI:Biology dataset was used as the training set and 20% of the data was used as the validation set), whether the training set protein was soluble was used as the classification value, and MCC was used as the optimization indicator for evaluating the performance of the prediction model.

MCC (Matthews Correlation Coefficient) is an indicator commonly used to evaluate the prediction performance of classification models. It is recommended by the literature as an optimization indicator for binary classification models, especially when the number of sample categories is unbalanced. For example, see Chicco, D., Jurman, G. "The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification" evaluation”, BMC Genomics 21, 6 (2020); https://doi.org/10.1186/s12864-019-6413-7. Therefore, in a preferred embodiment of the present disclosure, MCC is selected as the optimization indicator (as a performance indicator for evaluating the classification model) to improve the generalization ability of the model.

As mentioned above, in the process of model construction, the feature vector of the spliced sample protein is used as input data, and the solubility data (i.e., soluble or insoluble) of the sample protein is used as the output data of the prediction model to train the model. As the model parameters are continuously optimized, the output data of the prediction model will be consistent with the solubility data of the sample protein.

Those skilled in the art should understand that the prediction model constructed by the method of, for example, FIG4 can be used for the prediction method shown in FIG1. As mentioned above, although in the process of model construction and training, the protein solubility data of the training samples include whether the protein is soluble, in the actual prediction process using the prediction model, the prediction result of the protein solubility can be the probability that the protein is soluble.

There are three independent test sets disclosed in the present invention, among which the NESG dataset is used as an independent test set for classification problems, and the CamSol dataset and the eSol dataset are used as independent test sets for regression problems.

Those skilled in the art should understand that in the embodiments of the present disclosure, the same steps as shown in FIG. 3 are used to extract the features of the proteins in each independent test set, and the features are used as the input of the model to output the prediction results of the protein solubility.

The results of the training set and the independent test set are shown in Tables 3 to 6. Among them, the comparative data in Tables 3 to 5 are all from: Vineet Thumuluri, Hannah-Marie Martiny, Jose J Almagro Armenteros, Jesper Salomon, Henrik Nielsen, Alexander Rosenberg Johansen, "NetSolP: predicting protein solubility in Escherichia coli using language models", Bioinformatics, Volume 38, Issue 4, February 2022, Pages 941–946, https://doi.org/10.1093/bioinformatics/btab801.

Table 3: Training set PSI: Bio VCXlogy dataset test results

Table 4: Test results of the independent test set NESG dataset

Table 5: Test results of the independent test set Camsol dataset

Regarding Table 5, it should be noted that the independent test set Camsol dataset contains 56 mutant sequences, of which the literature Trevino contains 22 protein variants, Miklos contains 3 protein variants, Tan contains 1 protein variant, and Dudgeon contains 30 protein variants. Different prediction models and the disclosed model predict the accuracy of changes in protein solubility after point mutations.

Table 6: Test results of the independent test set eSol dataset

Those skilled in the art should understand that the prediction of a regression problem is generally expressed by the Pearson correlation coefficient, and the larger the value, the higher the positive correlation between the predicted value and the true value.

The indicators in Tables 3 to 5 are explained as follows:

Area Under Curve (AUC) refers to the area under the receiver operating characteristic curve (ROC).

Accuracy (ACC) refers to the ratio of correctly classified samples to the total number of samples. The accuracy calculation formula is: ACC = (TP + TN) / (P + N). Generally, the higher the accuracy, the better the classifier.

Precision refers to the proportion of correctly predicted results among all the results predicted as positive samples. The precision calculation formula is: Precision = TP/(TP+FP). The closer the value is to 1, the better the performance.

MCC (Matthews correlation coefficient) is an indicator often used to evaluate the prediction performance of classification models. It describes the correlation coefficient between the actual classification and the predicted classification. The calculation formula of MCC is: MCC = (TP*TN-FP*FN)/(sqrt((TP+FP)*(TP+FN)*(TN+FP)*(TN+FN))). The value range of MCC is between -1 and +1, where +1 indicates perfect prediction, 0 indicates random prediction, and -1 indicates that the prediction is completely inconsistent with the actual observation.

Among them, TP (True Positive) and TN (True Negative) represent the number of correctly classified positive samples and negative samples, FP (False Positive) and FN (False Negative) represent the number of incorrectly classified positive samples and negative samples, and P and N represent the number of positive samples and negative samples.

From the comprehensive test results of the training set and three independent test sets, it can be seen that the prediction model constructed by the present invention not only has better classification performance, but also takes into account the application in regression problems (such as predicting the effect of amino acid mutations on protein solubility). Due to the different sources of different data sets, the experimental conditions (including culture medium, cell line, protein expression and purification conditions) are also different, resulting in the performance of the trained model being reduced when it is used in a generalized manner. However, the method flow of the present invention is simple and easy to transplant. By retraining the experimental data generated for specific experimental conditions as a training set, the performance of the model predicting solubility can be further improved, and even the process can be used to solve problems such as protein expression prediction, protein folding type and folding rate constant prediction.

Those skilled in the art will appreciate that, based on the protein solubility prediction method disclosed herein, a system for predicting protein solubility can be developed. FIG5 shows a schematic block diagram of a system for predicting protein solubility according to the present disclosure. Specifically, the system 500 for predicting protein solubility includes an acquisition module 510 and a processing module 520.

The acquisition module 510 is used to acquire a sample to be predicted, where the sample to be predicted includes a protein sequence of the sample to be predicted.

The processing module 520 is used to obtain the prediction result of the protein solubility of the sample to be predicted by inputting the protein sequence of the sample to be predicted. As shown in FIG5 , the processing module 520 may further include: a first feature extraction submodule 521, which is used to extract features based on the protein sequence of the sample to be predicted using the protein pre-training model; a second feature extraction submodule 522, which is used to extract features based on the protein sequence of the sample to be predicted using the protein pre-training model; The extraction submodule 522 is used to extract the primary structural features of the protein based on the protein sequence of the sample to be predicted; the feature splicing submodule 523 is used to splice the features extracted by the protein pre-training model with the primary structural features of the protein to obtain the feature vector of the protein of the sample to be predicted; the model prediction submodule 524 is used to input the feature vector of the protein of the sample to be predicted into the protein solubility prediction model based on the automatic machine learning framework to obtain the prediction result of the protein solubility.

Those skilled in the art should understand that the system for predicting protein solubility described above can be implemented by a computer. For example, the following operations can be implemented by executing a computer program: obtaining a sample to be predicted, the sample to be predicted includes a protein sequence of the sample to be predicted; by inputting the protein sequence of the sample to be predicted, a prediction result of the protein solubility of the sample to be predicted is obtained, and the operation further includes the following sub-operations: extracting features based on the protein sequence of the sample to be predicted using a protein pre-training model; extracting the primary structural features of the protein based on the protein sequence of the sample to be predicted; splicing the features extracted by the protein pre-training model with the primary structural features of the protein to obtain a feature vector of the protein of the sample to be predicted; inputting the feature vector of the protein of the sample to be predicted into the protein solubility prediction model to obtain a prediction result of the protein solubility of the sample to be predicted. That is, although in the system for predicting protein solubility according to the present disclosure, each operation function corresponding to the prediction method is described as a module or sub-module, those skilled in the art should understand that such a module or sub-module may not be composed of circuit components or other physical components, but a functional module constructed by a computer program.

In addition, it should be recognized by those skilled in the art that the method of the present disclosure can be implemented as a computer program. As described above in conjunction with the accompanying drawings, the method of the above embodiment is executed by one or more programs, and the instructions in the program cause the computer or processor to execute the algorithm described in conjunction with the accompanying drawings. These programs can be stored and provided to the computer or processor using various types of non-transient computer-readable media. Non-transient computer-readable media include various types of tangible storage media. Examples of non-transient computer-readable media include magnetic recording media (such as floppy disks, tapes, and hard disk drives), magneto-optical recording media (such as magneto-optical disks), CD-ROMs (compact disk read-only memories), CD-Rs, CD-R/Ws, and semiconductor memories (such as ROMs, PROMs (programmable ROMs), EPROMs (erasable PROMs), flash ROMs, and RAMs (random access memories)). Further, these programs can be provided to the computer using various types of transient computer-readable media. Examples of transient computer-readable media include electrical signals, optical signals, and electromagnetic waves. Transient computer-readable media can be used to provide programs to the computer through wired communication paths or wireless communication paths such as wires and optical fibers.

For example, according to one embodiment of the present disclosure, a non-transitory computer-readable storage medium may be provided for storing a computer program, wherein the computer program includes instructions, which, when executed by a processor of an electronic device, enable the electronic device to implement the protein solubility prediction method as described above.

In addition, according to the content disclosed in the present disclosure, a computer system can also be provided, the computer system comprising: a processor; a memory; and a computer program. The computer program is stored in the memory and is configured to be executed by the processor. The computer program includes instructions for implementing the protein solubility prediction method described above.

On the other hand, according to one embodiment of the present disclosure, a non-transitory computer-readable storage medium may be provided for storing a computer program, wherein the computer program includes instructions that, when executed by a processor of an electronic device, enable the electronic device to implement the method for constructing a protein solubility prediction model as described above.

In addition, according to the content disclosed in the present disclosure, a computer system can also be provided, the computer system comprising: a processor; a memory; and a computer program. The computer program is stored in the memory and is configured to be executed by the processor. The computer program includes instructions for implementing the method for constructing the protein solubility prediction model described above.

The implementation methods of the present disclosure are not limited to the above-mentioned embodiments. Without departing from the spirit and scope of the present disclosure, ordinary technicians in this field can make various changes and improvements to the present disclosure in form and details, and these are considered to fall within the protection scope of the present disclosure.

Claims (26)

A method for predicting protein solubility, characterized in that the method comprises: Based on the protein sequence, features are extracted using a protein pre-trained model; Extract the primary structural features of proteins based on protein sequences; The features extracted by the protein pre-training model are concatenated with the primary structure features of the protein to obtain a feature vector of the protein; The characteristic vector of the protein is input into a protein solubility prediction model to obtain a prediction result of the protein solubility. The method according to claim 1 is characterized in that the protein pre-training model is implemented by a combination of one or more of the following models: ESM-1b, UniRep, ProteinBert, TAPE, ProtGPT2, ProtTXL, ProtBert, ProtXLNet, ProtAlbert, ProtElectra, ProtT5-XL-BFD, ProtT5-XL-UniRef50 and ProtT5-XXL; preferably, the protein pre-training model is ProtT5-XL-BFD or ProtT5-XL-UniRef50; more preferably, the protein pre-training model is ProtT5-XL-UniRef50. The method according to claim 1, characterized in that the extracting features based on the protein sequence using a protein pre-training model comprises: Taking the protein sequence as input, the ProtT5-XL-UniRef50 pre-trained model is used to extract the embedding layer vector output by the encoder, where each amino acid corresponds to a 1024-dimensional vector and each sequence corresponds to an L×1024-dimensional feature matrix, where L is the length of the amino acid sequence of the protein; Take the average value of the above feature matrix by column to obtain a 1024-dimensional feature vector. The method according to claim 3 is characterized in that the step of averaging the feature matrix by columns to obtain a 1024-dimensional feature vector further comprises: The Sentence-T5Encoder-onlymean method is used to average the values of each dimension of the embedding layer feature values of all amino acids output by the encoder by column to obtain a 1024-dimensional feature vector. The method according to claim 1, characterized in that extracting the primary structural features of the protein based on the protein sequence comprises: Taking protein sequence as input, the iFeature tool is used to extract the primary structural features of the protein. The method according to claim 1, characterized in that the primary structural features of the protein are selected from one or more of AAC, DPC, DDE, TPC, CKSAAP, EAAC, GAAC, CKSAAGP, GDPC, GTPC, Moran, Geary, NMBroto, CTDC, CTDT, CTDD, CTriad, KSCTriad, SOCNumber, QSOrder, PAAC, APAAC, KNNprotein, PSSM, AAINDEX and BLOSUM62; preferably, the primary structural features of the protein include AAC, GAAC, Moran, Geary, NMBroto, CTDC, CTDT, CTDD, CTriad, PAAC and APAAC. The method according to claim 1, characterized in that the protein solubility prediction model is constructed by the following steps: Acquire a data set, wherein the data set includes a plurality of training samples, each of the training samples includes a sample protein sequence and solubility data thereof; Based on the sample protein sequence, features are extracted using a protein pre-trained model; Extract the primary structural features of the protein based on the sample protein sequence; The features extracted by the protein pre-training model are combined with the primary structure features of the protein to obtain a feature vector of the sample protein; The characteristic vector of the sample protein is used as input data, the solubility data of the sample protein is used as output data, and the machine learning model is trained to obtain a final protein solubility prediction model. The method according to claim 7, characterized in that the training of the machine learning model comprises: Based on the automatic machine learning framework, one or more machine learning models are selected as candidate models, and the protein solubility prediction model is constructed and trained. The method according to claim 8 is characterized in that the automatic machine learning framework is AutoGluon; and the candidate models are deep learning models NN_Torch and FASTAI. The method according to claim 1 or 7, characterized in that the method further comprises: Extract other features of proteins based on protein sequences; The features extracted by the protein pre-training model are spliced with the primary structure features of the protein and other features of the protein to obtain a spliced feature vector of the protein. The method according to claim 10, characterized in that the other characteristics of the protein are selected from one or more of molecular weight, aromaticity, instability coefficient, elasticity, isoelectric point, molar absorbance, hydrophilicity total average, the proportion of helix/turn/fold in the secondary structure of the protein, secondary structure element content (SSEC), secondary structure element binary (SSEB), disorder, disorder content (DisorderC), disorder binary (DisorderB), barrier-free solvent accessibility (ASA), torsion angle (TA), Z-scale (ZSCALE) and 48 pseudo K-tuple reduced amino acid composition (48PseKRAAC); preferably, the other characteristics include molecular weight, aromaticity, instability coefficient, elasticity, isoelectric point, molar absorbance, hydrophilicity total average and the proportion of helix/turn/fold in the secondary structure of the protein. A method for constructing a protein solubility prediction model, characterized in that the construction method comprises: Acquire a data set, wherein the data set includes a plurality of training samples, each of the training samples includes a sample protein sequence and solubility data thereof; Based on the sample protein sequence, features are extracted using a protein pre-trained model; Extract the primary structural features of the protein based on the sample protein sequence; The features extracted by the protein pre-training model are spliced with the primary structure features of the protein to obtain a feature vector of the sample protein; The characteristic vector of the sample protein is used as input data, the solubility data of the sample protein is used as output data, and the machine learning model is trained to obtain a final protein solubility prediction model. The method according to claim 12, characterized in that the protein pre-training model is implemented by a combination of one or more of the following models: ESM-1b, UniRep, ProteinBert, TAPE, ProtGPT2, ProtTXL, ProtBert, ProtXLNet, ProtAlbert, ProtElectra, ProtT5-XL-BFD, ProtT5-XL-UniRef50 and ProtT5-XXL; preferably, the protein pre-training model is ProtT5-XL-BFD or ProtT5-XL-UniRef50; more preferably, the protein pre-training model is ProtT5-XL-UniRef50. The method according to claim 12, characterized in that the extracting features based on the sample protein sequence using a protein pre-training model comprises: The sample protein sequence is used as input, and the ProtT5-XL-UniRef50 pre-trained model is used to extract the embedding layer vector output by the encoder, where each amino acid corresponds to a 1024-dimensional vector and each sequence corresponds to an L×1024-dimensional feature matrix, where L is the length of the amino acid sequence of the sample protein; Take the average value of the above feature matrix by column to obtain a 1024-dimensional feature vector. The method according to claim 14, characterized in that the step of averaging the feature matrix by columns to obtain a 1024-dimensional feature vector further comprises: The Sentence-T5Encoder-onlymean method is used to average the values of each dimension of the embedding layer feature values of all amino acids output by the encoder by column to obtain a 1024-dimensional feature vector. The method according to claim 12, characterized in that extracting the primary structural features of the protein based on the sample protein sequence comprises: Using the sample protein sequence as input, the iFeature tool is used to extract typical primary structural features of the protein. The method according to claim 12, characterized in that the primary structural features of the protein are selected from one or more of AAC, DPC, DDE, TPC, CKSAAP, EAAC, GAAC, CKSAAGP, GDPC, GTPC, Moran, Geary, NMBroto, CTDC, CTDT, CTDD, CTriad, KSCTriad, SOCNumber, QSOrder, PAAC, APAAC, KNNprotein, PSSM, AAINDEX and BLOSUM62; preferably, the primary structural features of the protein include AAC, GAAC, Moran, Geary, NMBroto, CTDC, CTDT, CTDD, CTriad, PAAC and APAAC. The method according to claim 12 is characterized in that the machine learning model is a deep learning model. The method according to claim 12, characterized in that the training of the machine learning model comprises: Based on the automatic machine learning framework, one or more machine learning models are selected as candidate models, and the protein solubility prediction model is constructed and trained. The method according to claim 19 is characterized in that the automatic machine learning framework is AutoGluon; and the candidate models are deep learning models NN_Torch and FASTAI. The method according to claim 12, characterized in that the method further comprises: Extract other features of proteins based on sample protein sequences; The features extracted by the protein pre-training model are spliced with the primary structure features of the protein and other features of the protein to obtain a spliced feature vector of the sample protein. The method according to claim 21, characterized in that the other characteristics of the protein are selected from one or more of molecular weight, aromaticity, instability coefficient, elasticity, isoelectric point, molar absorbance, total average hydrophilicity, the proportion of helix/turn/fold in the secondary structure of the protein, SSEC, SSEB, Disorder, DisorderC, DisorderB, ASA, TA, ZSCALE and 48PseKRAAC; preferably, the other characteristics include molecular weight, aromaticity, instability coefficient, elasticity, isoelectric point, molar absorbance, total average hydrophilicity and the proportion of helix/turn/fold in the secondary structure of the protein. A protein solubility prediction model constructed according to the method described in any one of claims 12 to 22. A computer-implemented system for predicting protein solubility, wherein the computer-implemented system implements the following operations by executing a computer program: Acquire a sample to be predicted, wherein the sample to be predicted includes a protein sequence of the sample to be predicted; By inputting the protein sequence of the sample to be predicted, a prediction result of the protein solubility of the sample to be predicted is obtained, and the operation further includes the following sub-operations: Based on the protein sequence of the sample to be predicted, a protein pre-training model is used to extract features; Based on the protein sequence of the sample to be predicted, extract the primary structural features of the protein; The features extracted by the protein pre-training model are spliced with the primary structure features of the protein to obtain a feature vector of the protein of the sample to be predicted; The characteristic vector of the protein of the sample to be predicted is input into the protein solubility prediction model to obtain the prediction result of the protein solubility of the sample to be predicted. A non-transitory computer-readable storage medium for storing a computer program, wherein the computer program includes instructions, which, when executed by a processor of an electronic device, cause the electronic device to implement the protein solubility prediction method as described in any one of claims 1 to 11 or the method for constructing a protein solubility prediction model as described in any one of claims 12 to 22. A computer system, comprising: processor; Memory; and A computer program, wherein the computer program is stored in the memory and is configured to be executed by the processor, and the computer program includes instructions for implementing the protein solubility prediction method according to any one of claims 1 to 11 or the method for constructing a protein solubility prediction model according to any one of claims 12 to 22.