WO2017214765A1 - Multi-thread fast storage lossless compression method and system for fastq data - Google Patents

Abstract

Provided is a multi-thread fast storage lossless compression method for FASTQ data, which is applied to compression of a DNA sequence. The method comprises: a data classification step of inputting original FASTQ data, and dividing a short reading of the original FASTQ data into three data flows, namely metadata, a mass fraction, and a base sequence (S11); a data compression step of: with respect to the metadata, using incremental encoding to detect and eliminate redundant information of the metadata; with respect to the mass fraction, using a bit level PPM prediction model and arithmetic coding for compression; and with respect to the base sequence, using improved arithmetic coding of a fixed order for compression (S12); and a data output step of archiving and merging compression results of different data flows, and outputting final data after the compression (S13). The solution can improve the compression efficiency and compression speed.

Description

 Multi-thread fast storage lossless compression method and system thereof for FASTQ data Technical field

The present invention relates to the field of data compression, and in particular, to a multi-thread fast storage lossless compression method and system for FASTQ data.

Background technique

With the development of DNA sequencing technology, the cost of genome sequencing is getting lower and lower. In 2014, a milestone in measuring the cost control of a human genome at $1,000 has arrived. Due to the increased sequencing efficiency, the amount of DNA sequence data has exploded. As the growth rate of DNA sequencing data far exceeds the growth rate of computer microprocessors and storage devices, the DNA tsunami generated by storing and analyzing DNA sequencing technology and large-scale genome projects has become a constraint to the further development of the DNA sequencing industry. An important bottleneck. Moreover, because DNA sequencing technology is being sequenced from high-throughput (High-Throughput Sequencing), also known as Next Generation Sequencing Sequencing) has developed into single-molecule sequencing technology (also known as third-generation sequencing technology). The short reading of FASTQ data has grown from a fixed length ranging from 50 to 200 bp to an indefinite length ranging from 1 kbp to 300 kbp. The data change is further restricted. With the development of the DNA sequencing industry, there is an urgent need for related data compression technologies to be put into use.

However, some of the current mainstream and efficient general-purpose compression software such as gzip (http://www.gzip.org/), bzip2 (http://gzip.org/) and LZMA (http://www.7-zip.org) /sdk.html). The gzip software first uses a variant compression method based on the LZ77 algorithm for the file to be compressed, and then uses the static Huffman coding or the dynamic Huffman coding method to compress the obtained result. The bzip2 software divides the data to be compressed into blocks (100~900KB each), and uses BWT (Burrows-Wheeler) for repeated character sequences. Transform) algorithm performs conversion processing, and then uses MTF (Move-To-Front transform) algorithm and Huffman coding (Huffman) Coding) to compress. LZMA software uses a dictionary encoding mechanism similar to LZ77 algorithm to compress the data stream, repeat sequence size and re-sequence position separately, and supports several hash chain variants, binary trees and radix trees as the basis of its dictionary search algorithm. .

However, these compression methods do not take into account the biological properties of DNA data, such as long repeats and complementary palindromes, resulting in less accurate compression of DNA sequence data and slower compression.

technical problem

In view of this, an object of the present invention is to provide a multi-thread fast storage lossless compression method and system thereof for FASTQ data, which aims to solve the problem of poor compression effect and slow compression speed for DNA sequence data in the prior art. .

Technical solution

The invention provides a multi-thread fast storage lossless compression method for FASTQ data, which is applied to compression of DNA sequences, characterized in that the method comprises:

Data classification step: inputting raw FASTQ data, and dividing the short reading of the original FASTQ data into three data streams of metadata, mass fraction and base sequence;

Data compression step: for metadata, using incremental coding to detect and eliminate redundant information of metadata; for quality scores, using bit-level PPM prediction model and arithmetic coding for compression; for base sequences, using fixed order Improved arithmetic coding for compression;

Data output step: Archive and merge the compression results of different data streams, and output the compressed final data.

Preferably, thread-level parallel programming of Pthreads is employed in the data compression step to simultaneously process compression of the three data streams.

Preferably, the data compression step specifically includes:

For the quality score, the data stream of the quality score is first compressed by the run length long read coding method to implement preprocessing;

The preprocessed compressed data is recompressed using a bit-level PPM prediction model and arithmetic coding.

Preferably, the data compression step specifically includes:

Determining, according to the base sequence, whether the compression mode of the DNA sequence is based on a compression mode of a non-reference gene or a compression mode based on a reference gene;

If the compression mode is based on a non-reference gene, the data stream of the base sequence is compressed using a modified arithmetic coding of a fixed order;

If it is based on the compression pattern of the reference gene, the DNA short-reading sequence is aligned and the redundancy is eliminated by the DNA data matching tool, the corresponding matching information is recorded and saved in the SAM format file, and then the modified arithmetic coding using the fixed order is used. Extract the saved SAM format file extract.

In another aspect, the present invention also provides a multi-threaded fast storage lossless compression system for FASTQ data, the system comprising:

a data classification module, configured to input original FASTQ data, and divide the short read of the original FASTQ data into three data streams of a metadata, a quality score, and a base sequence;

A data compression module for detecting and eliminating redundant information of metadata for metadata, using a bit-level PPM prediction model and arithmetic coding for quality scores; and fixing for base sequences Improved arithmetic coding of the order bits for compression;

The data output module is configured to archive and combine the compression results of different data streams, and output the compressed final data.

Preferably, the data compression module uses thread-level parallel programming of Pthreads to simultaneously process compression of the three data streams.

Preferably, the data compression module is specifically configured to:

For the quality score, the data stream of the quality score is first compressed by the run length long read coding method to implement preprocessing;

The preprocessed compressed data is recompressed using a bit-level PPM prediction model and arithmetic coding.

Preferably, the data compression module is specifically configured to:

Determining, according to the base sequence, whether the compression mode of the DNA sequence is based on a compression mode of a non-reference gene or a compression mode based on a reference gene;

If the compression mode is based on a non-reference gene, the data stream of the base sequence is compressed using a modified arithmetic coding of a fixed order;

If it is based on the compression pattern of the reference gene, the DNA short-reading sequence is aligned and the redundancy is eliminated by the DNA data matching tool, the corresponding matching information is recorded and saved in the SAM format file, and then the modified arithmetic coding using the fixed order is used. Extract the saved SAM format file extract.

Beneficial effect

The technical solution provided by the invention fully utilizes the biological characteristics of the DNA data, can obtain a very high compression ratio in two kinds of FASTQ data (ie, long reading and short reading), and splits the FASTQ data into high-efficiency coding methods. Three types of data streams are separately compressed separately, which greatly increases the compression ratio, and provides an interface to the upstream DNA data matching tool, which can be applied to the reference gene-based compression mode to play between the homologous species genomes. Highly similar, further improving the compression ratio of resequencing data. Also, by using Pthreads (POSIX Thread-level parallel programming of threads) can simultaneously compress and process the intermediate files generated by the three data streams, thereby greatly improving the compression speed, increasing the applicability, and thus improving the compression efficiency.

DRAWINGS

1 is a flowchart of a multi-thread fast storage lossless compression method for FASTQ data according to an embodiment of the present invention;

2 is a schematic diagram showing the internal structure of a multi-threaded fast storage lossless compression system 10 for FASTQ data according to an embodiment of the present invention.

Embodiments of the invention

The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

A specific embodiment of the present invention provides a multi-threaded fast storage lossless compression method for FASTQ data, which is applied to compression of a DNA sequence, wherein the method mainly includes the following steps:

S11. Data classification step: input original FASTQ data, and divide the short read of the original FASTQ data into three data streams of metadata, mass fraction and base sequence;

S12. Data compression step: for metadata, using incremental coding to detect and eliminate redundant information of metadata; for quality score, using bit level PPM prediction model and arithmetic coding for compression; for base sequence, using fixed Improved arithmetic coding of the order bits for compression;

S13. Data output step: Archive and merge the compression results of different data streams, and output the compressed final data.

The multi-threaded fast storage lossless compression method for FASTQ data provided by the invention fully utilizes the biological characteristics of DNA data, and can obtain extremely high compression ratio in two FASTQ data (ie, long read and short read). Designing an efficient coding method splits the FASTQ data into three types of data streams and separately compresses them separately, greatly increasing the compression ratio. Also, by using Pthreads (POSIX Thread-level parallel programming of threads) can simultaneously compress and process the intermediate files generated by the three data streams, thereby greatly improving the compression speed, increasing the applicability, and thus improving the compression efficiency.

A multi-threaded fast storage lossless compression method for FASTQ data provided by the present invention will be described in detail below.

Please refer to FIG. 1 , which is a flowchart of a multi-thread fast storage lossless compression method for FASTQ data according to an embodiment of the present invention.

In step S11, the data sorting step: inputting the original FASTQ data, and dividing the short read of the original FASTQ data into three data streams of metadata, quality score, and base sequence.

In this embodiment, DNA sequencing technology produces thousands of short reads, which are stored in a file in FASTQ format, containing all the information generated by sequencing. In the widely used FASTQ format, each short read contains four lines, each line being divided by a newline character. Each short read begins with the character '@' followed by the metadata as the first line to uniquely identify the short read. The second line is the base data, consisting of a sequence consisting of only five characters {'A', 'T', 'C', 'G', 'N'}, where the character 'N' indicates an ambiguous base. , can be expressed as any character in {'A', 'T', 'C', 'G'}. The third line begins with the character '+' followed by the same short read identifier as the first line. The last behavioral quality score line, one-to-one correspondence with the base, indicates the credibility of sequencing of each base character corresponding to the position.

In step S12, the data compression step: for the metadata, using the incremental coding method to detect and eliminate redundant information of the metadata; for the quality score, using the bit level PPM (Prediction) By partial matching) The prediction model and the arithmetic coding are compressed; for the base sequence, compression is performed using a modified arithmetic coding of fixed order bits.

In this embodiment, the data compression step uses Pthreads thread-level parallel programming to simultaneously process the compression of the three data streams, and can simultaneously compress and process the intermediate files generated by the three data streams, thereby greatly improving the compression speed. To enhance its applicability.

In the present embodiment, the metadata is detected by the incremental coding method and the redundant information of the metadata is eliminated, and then the compression is performed by the improved arithmetic coding of the fixed order, and the multi-threaded parallel processing is employed.

In this embodiment, the data compression step specifically includes:

For the quality score, the data stream of the quality score is first compressed by the run length long read coding method to implement preprocessing;

The preprocessed compressed data is recompressed using a bit-level PPM prediction model and arithmetic coding.

In the present embodiment, for the quality score, since it considers that it holds a large number of consecutively repeated identical characters, the run length long read code compression is used as a pre-processing, and the data stream of the quality score is first compressed, and then the bit level is used. The PPM prediction model and the arithmetic coding recompress the preprocessed compressed data and adopt multithreading parallel processing. In the present embodiment, such a bit-level PPM prediction model and arithmetic coding have a higher compression ratio and compression speed for lossless compression of a quality score of about 40 characters and subject to a pseudo-random distribution.

In this embodiment, the data compression step specifically includes:

Determining, according to the base sequence, whether the compression mode of the DNA sequence is based on a compression mode of a non-reference gene or a compression mode based on a reference gene;

If the compression mode is based on a non-reference gene, the data stream of the base sequence is compressed using a modified arithmetic coding of a fixed order;

If it is based on the compression pattern of the reference gene, the DNA short-reading sequence is aligned and the redundancy is eliminated by the DNA data matching tool, the corresponding matching information is recorded and saved in the SAM format file, and then the modified arithmetic coding using the fixed order is used. Extract the saved SAM format file extract.

In the present embodiment, because the performance of the DNA data matching tool directly affects the effect of data compression, the SAM format file is used as an intermediate file to provide an interface to the upstream DNA data matching tool, which provides a possibility for subsequent compression performance improvement.

In the present embodiment, for the base sequence, the present invention is applicable to two DNA data compression modes, that is, a compression mode based on a non-reference gene and a reference gene compression mode, by which the compression mode of the DNA sequence belongs. The mode is then processed separately. DNA data matching tool in the present embodiment, such as BWA (Burrows-Wheeler) Aligner) tool, Bowtie tool and CompMap tool, these DNA data matching tools can compare DNA short read sequences and eliminate redundancy, realize parallel fast matching of DNA short read sequences, and apply to DNA sequence compression storage methods. .

For non-reference gene based compression mode: This compression mode is composed of five characters consisting of {'A', 'T', 'C', 'G', 'N'}, due to the small number of characters. Adopting a fixed-order strategy can greatly reduce the waste of unnecessary spare bits, and reduce the size of space required for storing each character from the bit level, so a fixed-order arithmetic coding compression is adopted. That is, if the compression mode is based on a non-reference gene, the data stream of the base sequence is compressed using a modified arithmetic coding of a fixed order.

For reference gene-based compression mode: This compression mode can perform corresponding data compression based on the above DNA data matching tool, providing a corresponding interface, which can be applied to the preprocessing of base sequences, and the DNA short read sequences are compared and The redundancy is deleted, and the corresponding matching information is recorded and saved in a SAM format file, and the matching information includes: a matching position, a palindrome string mark, a matching character length, a matching type, and a non-matching character, and then the compressed mode puts the five kinds of information. Split into separate files and compress them with fixed arithmetic coding with fixed order. That is to say, if it is based on the compression mode of the reference gene, the DNA short reading sequence is compared by the DNA data matching tool and the redundancy is eliminated, the corresponding matching information is recorded and saved in the SAM format file, and then the fixed order is used. The improved arithmetic coding extracts and compresses the saved SAM format file.

In step S13, the data output step: archives the compression results of the different data streams, and outputs the compressed final data.

In the present embodiment, the technical solution of the present invention improves the compression performance of the FASTQ data, that is, the compression ratio and the compression speed, from two aspects. Among them, in the comparison of the high compression mode, the present invention is in two modes (ie, the compression mode based on the reference gene and based on the FASTQ data mainly based on the long reading of indefinite length or the short reading of fixed length). The compression mode of the non-reference gene) has a higher compression ratio and a faster compression speed than the mainstream general compression software. This advantage has been demonstrated by testing some data from the National Center for Biotechnology Information (NCBI, Http://www.ncbi.nlm.nih.gov/) Two FASTQ data downloaded: ERR385912 (641MB, short read length 51, short read), ERR654984 (1164MB, short read length 64~502, belongs to Long reading), through the comparison of test data, the compression ratio of the present invention is 10.7% and 15.2% higher than that of bzip2 software and gzip software, respectively, and the compression speed is 40.2% and 45.5% faster than bzip2 software and gzip software, respectively.

The multi-threaded fast storage lossless compression method for FASTQ data provided by the invention fully utilizes the biological characteristics of DNA data, and can obtain extremely high compression ratio in two FASTQ data (ie, long read and short read). Designing efficient coding methods splits FASTQ data into three types of data streams and compresses them separately, greatly increasing the compression ratio, and providing an interface to upstream DNA data matching tools that can be applied to reference-based genes. Compressed mode, which exerts a high degree of similarity between homologous species genomes, further improves the compression ratio of resequencing data. In addition, by using thread-level parallel programming of Pthreads, it is possible to simultaneously compress and process the intermediate files generated by the three data streams, thereby greatly increasing the compression speed, increasing the applicability, and thereby improving the compression efficiency.

The embodiment of the present invention further provides a multi-threaded fast storage lossless compression system 10 for FASTQ data, which mainly includes:

a data classification module 11 for inputting original FASTQ data, and dividing the short read of the original FASTQ data into three data streams of metadata, quality score and base sequence;

The data compression module 12 is configured to detect and eliminate redundant information of the metadata by using an incremental coding method for the metadata, compressing by using a bit-level PPM prediction model and arithmetic coding for the quality score, and utilizing the base sequence for the base sequence. Fixed arithmetic coding of fixed order bits for compression;

The data output module 13 is configured to archive and combine the compression results of different data streams, and output the compressed final data.

The multi-threaded fast storage lossless compression system 10 for FASTQ data provided by the present invention fully utilizes the biological characteristics of DNA data, and can obtain extremely high compression ratio in two FASTQ data (ie, long read and short read). By designing efficient coding methods, FASTQ data is split into three types of data streams and separately compressed separately, which greatly improves the compression ratio. In addition, by using thread-level parallel programming of Pthreads, it is possible to simultaneously compress and process the intermediate files generated by the three data streams, thereby greatly increasing the compression speed, increasing the applicability, and thereby improving the compression efficiency.

Referring to FIG. 2, a schematic structural diagram of a multi-threaded fast storage lossless compression system 10 for FASTQ data according to an embodiment of the present invention is shown.

In the present embodiment, the multi-threaded fast storage lossless compression system 10 for FASTQ data is applied to compression of DNA sequences, and mainly includes a data classification module 11, a data compression module 12, and a data output module 13.

The data classification module 11 is configured to input the original FASTQ data and divide the short read of the original FASTQ data into three data streams of a metadata, a quality score, and a base sequence.

In this embodiment, DNA sequencing technology produces thousands of short reads, which are stored in a file in FASTQ format, containing all the information generated by sequencing. In the widely used FASTQ format, each short read contains four lines, each line being divided by a newline character. Each short read begins with the character '@' followed by the metadata as the first line to uniquely identify the short read. The second line is the base data, consisting of a sequence consisting of only five characters {'A', 'T', 'C', 'G', 'N'}, where the character 'N' indicates an ambiguous base. , can be expressed as any character in {'A', 'T', 'C', 'G'}. The third line begins with the character '+' followed by the same short read identifier as the first line. The last behavioral quality score line, one-to-one correspondence with the base, indicates the credibility of sequencing of each base character corresponding to the position.

The data compression module 12 is configured to detect and eliminate redundant information of the metadata by using an incremental coding method for the metadata, compressing by using a bit-level PPM prediction model and arithmetic coding for the quality score, and utilizing the base sequence for the base sequence. Fixed arithmetic coding of fixed order bits for compression.

In this embodiment, the data compression module 12 uses the thread-level parallel programming mode of Pthreads to simultaneously process the compression of the three data streams, and can simultaneously compress and process the intermediate files generated by the three data streams, thereby greatly improving compression. Speed, enhance its applicability.

For the metadata, the data compression module 12 detects and eliminates the redundant information of the metadata by using the incremental coding method, and then performs compression using the improved arithmetic coding of the fixed order, and adopts multi-threaded parallel processing.

The data compression module 12 is specifically configured to:

For the quality score, the data stream of the quality score is first compressed by the run length long read coding method to implement preprocessing;

The preprocessed compressed data is recompressed using a bit-level PPM prediction model and arithmetic coding.

In the present embodiment, for the quality score, since it considers that it holds a large number of consecutively repeated identical characters, the data compression module 12 first uses the run length read coding compression as a pre-processing, and compresses the data stream of the quality score for the first time, and then The pre-processed compressed data is again compressed using a bit-level PPM prediction model and arithmetic coding, and multi-threaded parallel processing is employed. In the present embodiment, such a bit-level PPM prediction model and arithmetic coding have a higher compression ratio and compression speed for lossless compression of a quality score of about 40 characters and subject to a pseudo-random distribution.

The data compression module 12 is specifically configured to:

Determining, according to the base sequence, whether the compression mode of the DNA sequence is based on a compression mode of a non-reference gene or a compression mode based on a reference gene;

If the compression mode is based on a non-reference gene, the data stream of the base sequence is compressed using a modified arithmetic coding of a fixed order;

If it is based on the compression pattern of the reference gene, the DNA short-reading sequence is aligned and the redundancy is eliminated by the DNA data matching tool, the corresponding matching information is recorded and saved in the SAM format file, and then the modified arithmetic coding using the fixed order is used. Extract the saved SAM format file extract.

In the present embodiment, because the performance of the DNA data matching tool directly affects the effect of data compression, the SAM format file is used as an intermediate file to provide an interface to the upstream DNA data matching tool, which provides a possibility for subsequent compression performance improvement.

DNA data matching tool in the present embodiment, such as BWA (Burrows-Wheeler) Aligner) tool, Bowtie tool and CompMap tool, these DNA data matching tools can compare DNA short read sequences and eliminate redundancy, realize parallel fast matching of DNA short read sequences, and apply to DNA sequence compression storage methods. .

In the present embodiment, for the base sequence, the present invention is applicable to two DNA data compression modes, that is, a compression mode based on a non-reference gene and a reference gene compression mode, by which the compression mode of the DNA sequence belongs. The mode is then processed separately. The specific processing flow of the two DNA data compression modes is as shown in the foregoing step S12, and the repeated description is not repeated here.

The data output module 13 is configured to archive and combine the compression results of different data streams, and output the compressed final data.

The multi-threaded fast storage lossless compression system 10 for FASTQ data provided by the present invention fully utilizes the biological characteristics of DNA data, and can obtain extremely high compression ratio in two FASTQ data (ie, long read and short read). By designing efficient coding methods, FASTQ data is split into three types of data streams and compressed separately, which greatly improves the compression ratio, and provides an interface to the upstream DNA data matching tool, which can be applied to reference-based The compression pattern of genes plays a high degree of similarity between genomes of homologous species, further increasing the compression ratio of resequencing data. In addition, by using thread-level parallel programming of Pthreads, it is possible to simultaneously compress and process the intermediate files generated by the three data streams, thereby greatly increasing the compression speed, increasing the applicability, and thereby improving the compression efficiency.

It should be noted that, in the foregoing embodiment, each unit included is only divided according to functional logic, but is not limited to the above division, as long as the corresponding function can be implemented; in addition, the specific name of each functional unit is also They are only used to facilitate mutual differentiation and are not intended to limit the scope of the present invention.

In addition, those skilled in the art can understand that all or part of the steps of implementing the above embodiments may be completed by a program to instruct related hardware, and the corresponding program may be stored in a computer readable storage medium. Storage medium, such as ROM/RAM, disk or CD.

The above is only the preferred embodiment of the present invention, and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection of the present invention. Within the scope.

Claims (8)

A multi-threaded fast storage lossless compression method for FASTQ data, applied to compression of DNA sequences, characterized in that the method comprises: Data classification step: inputting raw FASTQ data, and dividing the short reading of the original FASTQ data into three data streams of metadata, mass fraction and base sequence; Data compression step: for metadata, using incremental coding to detect and eliminate redundant information of metadata; for quality scores, using bit-level PPM prediction model and arithmetic coding for compression; for base sequences, using fixed order Improved arithmetic coding for compression; Data output step: Archive and merge the compression results of different data streams, and output the compressed final data. The multi-threaded fast storage lossless compression method for FASTQ data according to claim 1, wherein a thread-level parallel programming manner of Pthreads is used in the data compression step to simultaneously process compression of the three data streams. . The multi-threaded fast storage lossless compression method for FASTQ data according to claim 2, wherein the data compression step specifically comprises: For the quality score, the data stream of the quality score is first compressed by the run length long read coding method to implement preprocessing; The preprocessed compressed data is recompressed using a bit-level PPM prediction model and arithmetic coding. The multi-threaded fast storage lossless compression method for FASTQ data according to claim 2, wherein the data compression step specifically comprises: Determining, according to the base sequence, whether the compression mode of the DNA sequence is based on a compression mode of a non-reference gene or a compression mode based on a reference gene; If the compression mode is based on a non-reference gene, the data stream of the base sequence is compressed using a modified arithmetic coding of a fixed order; If it is based on the compression pattern of the reference gene, the DNA short-reading sequence is aligned and the redundancy is eliminated by the DNA data matching tool, the corresponding matching information is recorded and saved in the SAM format file, and then the modified arithmetic coding using the fixed order is used. Extract the saved SAM format file extract. A multi-threaded fast storage lossless compression system for FASTQ data, characterized in that the system comprises: a data classification module, configured to input original FASTQ data, and divide the short read of the original FASTQ data into three data streams of a metadata, a quality score, and a base sequence; A data compression module for detecting and eliminating redundant information of metadata for metadata, using a bit-level PPM prediction model and arithmetic coding for quality scores; and fixing for base sequences Improved arithmetic coding of the order bits for compression; The data output module is configured to archive and combine the compression results of different data streams, and output the compressed final data. The multi-threaded fast storage lossless compression system for FASTQ data according to claim 5, wherein the data compression module uses a thread-level parallel programming manner of Pthreads to simultaneously process compression of the three data streams. The multi-threaded fast storage lossless compression system for FASTQ data according to claim 6, wherein the data compression module is specifically configured to: For the quality score, the data stream of the quality score is first compressed by the run length long read coding method to implement preprocessing; The preprocessed compressed data is recompressed using a bit-level PPM prediction model and arithmetic coding. The multi-threaded fast storage lossless compression system for FASTQ data according to claim 6, wherein the data compression module is specifically configured to: Determining, according to the base sequence, whether the compression mode of the DNA sequence is based on a compression mode of a non-reference gene or a compression mode based on a reference gene; If the compression mode is based on a non-reference gene, the data stream of the base sequence is compressed using a modified arithmetic coding of a fixed order; If it is based on the compression pattern of the reference gene, the DNA short-reading sequence is aligned and the redundancy is eliminated by the DNA data matching tool, the corresponding matching information is recorded and saved in the SAM format file, and then the modified arithmetic coding using the fixed order is used. Extract the saved SAM format file extract.