NL2037367B1

NL2037367B1 - Meteorological data compression method and system, and medium

Info

Publication number: NL2037367B1
Application number: NL2037367A
Authority: NL
Inventors: Zhang Ji; Yue Caijun; Chen Zhiqiang
Original assignee: Shanghai Marine Meteorological Center
Priority date: 2023-06-14
Filing date: 2024-03-29
Publication date: 2025-07-25
Also published as: CN116683915B; CN116683915A; NL2037367A

Abstract

Disclosed in the present invention are a meteorological data compression method and system, and a medium, which relate to the technical field of meteorological data processing. The method includes a data decoding step and a big data compression step. According to the method, a meteorological data compression algorithm based on scale features is configured to perform scale separation on original meteorological element data, acquire weather system information at different scales, extract key information, generate a plurality of irregular grid meteorological element data sets according to the extracted key information, and then, perform data fusion on the plurality of irregular grid meteorological element data sets to obtain a compressed irregular grid point data set. According to the present invention, the total data amount required for storage and transmission of meteorological data prediction is obviously reduced, the data storage space is reduced, and data transmission time is shortened.

Description

METEOROLOGICAL DATA COMPRESSION METHOD AND SYSTEM, AND

MEDIUM

TECHNICAL FIELD

[01] The present invention relates to the technical field of meteorological data processing, and in particular to a meteorological data compression method and system, and a medium.

BACKGROUND ART

[02] Due to frequent disasters such as strong winds, typhoons, sea fog and lightning, which cause huge losses to the world, marine meteorology has always been the focus of attention of the United Nations and the World

Meteorological Organization. With rapid development of field observation, satellite remote sensing, marine meteorological models, computers and communication technologies, great progress has been made in marine meteorology, and observation, forecasting, warning and service capabilities of offshore meteorology have been significantly enhanced. However, compared with the offshore metecrology, there is still a great gap in oceangoing meteorology, mainly due to the following reasons: 1) lack of three-dimensional observation apparatus, low coverage and lack of quality control; and 2) limitation of oceangoing communication capabilities, and the fact that traditional oceangoing meteorological services are basically performed by issuing meteorological information to the high seas in the form of voice and

Email, and digital facsimile maps based on short waves by the International Maritime Satellite Organization. In recent years, with continuous improvement of China's satellite communication capabilities, there has been some progress in the transmission of oceangoing data by means of short message or data packet transmission via Beidou and Tiantong satellites. However, no matter what way, the features of low image resolution, long reception time, limitations of number of bytes, and low frequency of publication, etc. generally exist. Moreover, in order to satisfy the transmission bandwidth limitation, a size of the facsimile map that can display prediction results of regional or global numerical prediction is strictly controlled, which inevitably omits a lot of important meteorological information, especially the information of small and medium-scale meteorological systems that are highly sensitive to disastrous weather. These information often plays an important role in navigation safety.

[03] With the rapid development of the global meteorological numerical prediction technology, the spatial and temporal resolution of grid forecast data which can accurately describe an atmospheric state is getting larger and larger, which brings great pressure to data storage and transmission. For example, the spatial resolution of the CMA-GFS global high-resolution numerical prediction model which is independently developed by China has reached 0.125 degree (about 1.5 G/single time), and the regional numerical prediction data is up to a hundred meter level. These data can not only accurately depict planetary scale and synoptic scale systems, but also effectively depict mesoscale and convective scale (from kilometer level to 100 km level) weather systems that are difficult to capture and predict, but meanwhile, it also causes exponential growth in numerical forecast data storage. A traditional meteorological data storage, transmission and processing manner can not satisfy the business service demand of meteorological big data. In practice, it is found that there are a lot of redundant data in these stored meteorological data, which are not important for describing weather systems of various scales. Existence of redundant data not only seriously interferes with representation and effective extraction of real system information, but also brings inconvenience to communication, data storage, transmission and application,

especially transmission of oceangoing data with extremely limited bandwidth. Therefore, how to effectively compress and store the massive data information is an important research direction.

[04] At present, data compression algorithms for the meteorological data can be divided into two types: lossless compression and lossy compression. Lossless compression can retain all information in source data, but a compression ratio is not high, generally 2:1 to 5:1. The main lossless compression coding includes predictive coding, transform coding, vector coding, arithmetic coding, etc. Lossy compression means that after reconstruction by using compressed data, the obtained data is different from the source data, but it will not cause misunderstanding of the information expressed by the source data. Lossy compression can obtain a higher compression ratio. Due to the limited compression capability, lossless compression is generally only applicable to meteorological data with a small amount of data or a small percentage of redundant data. In order to adapt to meteorological prediction data with a large amount of data, various improvement schemes based on traditional lossless compression are also proposed in the prior art, such as the meteorological prediction data compression method based on a video compression technology which is proposed in the Chinese patent application with

No. 202111420160.8. The method includes the steps: S1, acquiring original meteorological prediction data and extracting time slices and meteorological element data;

S2, preprocessing the meteorological prediction data, namely mapping the same weather element data of each station at the same time to an interval [0-1023*n], where n is the number of mapped channels; S3, reconstructing meteorological data three-dimensional lattice points, namely storing the meteorological prediction data according to three-dimensional lattice point data and forming a meteorological data map corresponding to three color channel data formats of a picture; S4, using a video compression tool to perform lossless compression coding with a 10-bit depth and a chroma brightness ratio of 4:4:4 on the meteorological prediction data; and S5, using the video compression tool to decode meteorological data on a coded file generated in S54, and performing inverse transformation on the meteorological prediction data. In the above compression solution, the meteorological prediction data of each station at the same time period are mapped to a 10-bit image storage range, meteorological prediction data frames are formed into meteorological prediction data sequences according to time dimensions, and then the meteorological prediction data frame sequences are encoded and compressed by using the video compression technology, which reduces a storage space of the meteorological prediction data, and the compression ratio can reach 10:1. However, the compression capability of the above solution is still difficult to satisfy the compression demand of high-resolution meteorological numerical prediction model data, which mainly consists of massive prediction data.

[05] Moreover, for massive cloud data with high redundancy, a lossy compression algorithm with controllable errors is usually employed at present, which is more effective. However, existing lossy compression algorithms are mostly used for image processing, voice data processing, electronic signal transmission, etc., and compression algorithms for the high-resolution meteorological numerical prediction model data, which mainly consists of mass prediction data, are rarely studied.

[06] To sum up, how to extract weather system data of various scales submerged in a large amount of redundant data without affecting weather system information, remove massive redundant data, and thus realize meteorological big data compression is a technical problem that needs to be solved urgently.

SUMMARY

[07] An objective of the present invention is to provide a meteorological data compression method and system, and a medium so as to overcome the defects in the prior art. 5 According to the present invention, by establishing a big data compression algorithm for extracting weather system information at different scales from massive high- resolution weather numerical prediction mode data, only key data which plays a key role in weather prediction is reserved, and a large amount of redundant data is removed from the meteorological data under the condition that weather system expression is not influenced, such that the total data amount required to be processed for prediction can be obviously reduced, the data storage space is reduced, and data transmission time is shortened.

Furthermore, oceangoing transmission of global meteorological numerical prediction data is made possible.

[08] To achieve the above objective, the present invention provides the technical solutions as follows:

[09] A meteorological data compression method includes the following steps:

[10] a data decoding step: reading high-resolution meteorological data in an original data format, and decoding meteorological element data of the high- resolution meteorological data on the basis of preset meteorological elements; and

[11] a big data compression step: processing, according to the original meteorological element data obtained by means of decoding, the original meteorological element data by using a meteorological data compression algorithm based on scale features to remove redundant data and obtain compressed data.

[12] The meteorological data compression algorithm based on scale features is configured to: perform scale separation on the original meteorological element data, acquire weather system information at different scales, extract key information, generate a plurality of irregular grid meteorological element data sets according to the extracted key information, where a weather system at one scale corresponds to an irregular grid meteorological element data set, and perform data fusion on the plurality of irregular grid meteorological element data sets to obtain a compressed irregular grid point data set.

[13] Furthermore, the method further includes a compressed data outputting step after the big data compression step as follows:

[14] performing data format conversion on the obtained irregular grid point data set to generate a data format file common to meteorology, and transmitting the common data format file to shore users.

[15] Furthermore, the common data format file is a NetCDF format file.

[16] Furthermore, the method further includes an encryption coding step after the big data compression step as follows:

[17] constructing a symmetric cipher table, performing lossless compression and encoding on the obtained irregular grid point data set, realizing reconstruction of the data set, and transmitting the reconstructed data to a ship-end communication satellite.

[18] Furthermore, the high-resolution meteorological data in an original data format is CMA-GFS global high- resolution numerical prediction model data in grib2 standard data format.

[19] In this case, data decoding processing includes the following steps:

[20] 5110, acquiring spatiotemporal resolution information of CMA-GFS data, which is defined as a resolution variable fbl in;

[21] 5120, acquiring horizontal grid point number information of the CMA-GFS data, including the total number of grid points, num lon in a horizontal X direction and the total number of grid points, num lat in a horizontal Y direction;

:

[22] S130, calculating horizontal grid boundary information of the CMA-GFS data according to steps S110 and S120, and calculating longitude and latitude information of all grid points;

[23] 5140, constructing an initial grid gridl according to the grid point latitude and longitude information; and

[24] S150, reading meteorological element values of all grid points in the initial grid gridl according to preset meteorological elements, and storing the meteorological element values in a grid array im data.

[25] Furthermore, when scale separation is performed on the original meteorological element data on the basis of two scales, the meteorological data compression algorithm based on scale features is configured to execute the steps as follows:

[26] 5210, acquiring weather system information at a first scale SCALEl and above of the horizontal direction, and generating a first data set, which includes the following steps:

[27] calculating coordinate information of grid points at the first scale SCALEl and above, taking an appropriate value interall near [SCALEl/{(fbl in*M)] according to a preset M value, and making num lon/interall and num lat/interall be positive integers, where M takes an integer of a comparable length to a longitude and latitude, and [SCALE1/{(fbl in*M)] represents rounding the fraction SCALE1/ (fbl in*M); letting resolution fbl inl=fbl in*interall, and repeating the steps 5120 and 5130 to obtain grid point latitude and longitude information when the resolution fbl inl=fbl in*interall, where the resolution fbl inl is smaller than the resolution of the initial grid gridl;

[28] denoting the grid point latitude and longitude information as a first-layer grid grid2, and forming a second-layer nested grid with the initial grid gridl;

[29] separately comparing numerical values of meteorological element values of grid points in the initial grid gridl embedded in each second-layer grid grid2, and acquiring extreme values of the meteorological element values of grid points in each first-layer grid grid? and initial grid positioning information corresponding thereto, thereby obtaining an irregular grid meteorological element data set denoted as a first data set, where the extreme values include a maximum value max and a minimum value min;

[30] $220, acquiring weather system information at a second scale SCALE2 in the horizontal direction, where the second scale SCALE2 is smaller than the first scale

SCALEl, and generating a second data set, which includes the steps:

[31] calculating coordinate information of grid points around the second scale SCALE2, taking an appropriate value interal2 near [SCALE2/ (fbl in*M)], making num lon/interal2 and num lat/interal2 be positive integers, letting resolution fbl in2=fbl in*interal2, and repeating the steps S120 and S130 to obtain grid point latitude and longitude information when resolution fbl inz=fbl in*interalz, where resolution fbl in2 is smaller than the resolution of the initial grid gridl and larger than the resolution fbl inl of the first-layer grid grid2;

[32] denoting the grid point latitude and longitude information as a second-layer grid grid3, and forming a third-layer nested grid with the initial grid gridl and the first-layer grid grid2;

[33] separately comparing numerical values of meteorological element values of grid points in the initial grid gridl embedded in each second-layer grid grid3, and acquiring extreme values of the meteorological element values of grid points in each second-layer grid grid3, where the extreme values include a maximum value max and a minimum value min; for each second-layer grid3, comparing the maximum value max and the minimum value min of the grid, and when the maximum value max is equal to the minimum value min, only retaining grid point coordinates and meteorological element values of the initial grid at an inner center point position of the grid to obtain an intermediate data set;

[34] for the intermediate data set, taking any grid point as a node, constructing eight child nodes according to eight directions of upper left, upper, upper right, left, right, lower left, lower and lower right so as to construct an improved two-dimensional planar octree structure with a two-dimensional horizontal structure, and performing Morton coding on the basis of the improved two- dimensional planar octree structure so as to obtain octree coding of each node; removing the point information whose octree code is 0 to obtain an irregular grid meteorological element data set denoted as a second data set; and

[35] $230, performing data fusion on the first data set and the second data set to obtain a compressed irregular grid point data set, i.e. compressed data.

[36] Furthermore, the manner by which the data fusion is performed is as follows: storing the grid points in the first data set and the second data set in order of longitude and latitude, and performing deduplication on overlapped grid point data.

[37] Furthermore, the meteorological elements include one or more of sea level pressure PRMSL, a temperature TMP, and humidity RH.

[38] The present invention further provides a meteorological data compression system. The system includes:

[39] a data decoding apparatus configured to read high- resolution meteorclogical data in an original data format and decode meteorological element data of the high- resolution meteorological data on the basis of preset meteorological elements; and

[40] a big data compression apparatus configured to process, according to the original meteorological element data obtained by means of decoding, the original meteorological element data by using a meteorological data compression algorithm based on scale features to remove redundant data and obtain compressed data.

[41] The meteorological data compression algorithm based on scale {features is configured to: perform scale separation on the original meteorological element data, acquire weather system information at different scales, extract key information, generate a plurality of irregular grid meteorological element data sets according to the extracted key information, where a weather system at one scale corresponds to an irregular grid meteorological element data set, and perform data fusion on the plurality of irregular grid meteorological element data sets to obtain a compressed irregular grid point data set.

[42] The present invention further provides a computer- readable storage medium, which is configured to store a computer program executable by a processing unit. The computer program, when executed by the processing unit, implements the meteorological data compression method mentioned above.

[43] Compared with the prior art, due to use of the above technical solutions, the present invention has the following advantages and positive effects as an example: by establishing the big data compression algorithm for extracting the weather system information at different scales from the massive high-resolution weather numerical prediction mode data, only key data which plays a key role in weather prediction is reserved, and a large amount of redundant data is removed from the meteorological data under the condition that weather system expression is not influenced, such that the total data amount required to be processed for prediction can be obviously reduced, the data storage space is reduced, and data transmission time is shortened. Furthermore, oceangoing transmission of global meteorological numerical prediction data is made possible.

[44] By utilizing the meteorological data compression algorithm based on scale features, which is put forward by the present invention, point cloud data extraction is performed, such that the data storage and transmission cost can be greatly reduced, and effective transmission of oceangoing numerical prediction data can be realized under the conditions of limited oceangoing satellite communication bandwidth, limited short message byte number, a high price, etc.

BRIEF DESCRIPTION OF THE DRAWINGS

[45] FIG. 1 is a flow of a scattered point cloud data compression algorithm based on octree coding provided in the prior art.

[46] FIG. 2 is a flowchart of a meteorological data compression method based on scale features provided in an example of the present invention.

[47] FIG. 3 is a flow chart of big data compression using improved octree coding provided in an example of the present invention.

[48] FIG. 4 is a schematic diagram of an improved third- layer nested grid provided in an example of the present invention.

[49] FIG. 5 is a schematic diagram of improved two- dimensional planar octree layout provided in an example of the present invention.

[50] FIG. 6 is a schematic diagram of morton coding of a two-dimensional planar octree structure provided in an example of the present invention.

[51] FIG. 7 shows a binary coding example of the morton coding in FIG. 6.

[52] Description of reference numerals:

[53] initial grid 100, first-layer grid 200, and second- layer grid 300.

DETAILED DESCRIPTION OF THE EMBODIMENTS

[54] A meteorological data compression method and system, and a medium disclosed in the present invention are further described in detail below with reference to the accompanying drawings and particular examples. It should be noted that the techniques (including methods and apparatuses) known to those of ordinary skill in the related field may not be discussed in detail but, where appropriate, should be considered a part of the description. Moreover, other instances of the exemplary examples may have different values. The structure, scale, size, etc. shown in the accompanying drawings of this description are only used for matching the content disclosed in the description and for those skilled in the art to understand and read, instead of being used to limit the limitations for implementing the present invention.

[55] In the description of the examples of the present application, “/Vmeans or, and “and/or” is used for describing an association relation of an associated object, which means that there may be three relations. For example, A and/or B may represent three situations: A exists alone, B exists alone, and A and B exist at the same time. In the description of the examples of the present application, “plurality of” means two and more.

[56] Examples

[57] The data compression solution provided by the present invention is mainly based on the combination of scale separation and scale nesting ideas to remove redundancy and duplication of data, and can effectively extract weather systems of various scales submerged in a large amount of redundant data on the premise of not affecting weather system information, so as to achieve the effect of “looking for a needle in the sea”. Furthermore, since the total amount of data to be processed is greatly reduced, oceangoing data transmission of oceangoing meteorological numerical prediction data becomes possible, and various problems of limited oceangoing transmission of marine meteorological data at present are effectively solved.

[58] Moreover, the meteorological data compression algorithm provided by the present invention also improves octree coding. Octree coding is a coding method based on three-dimensional spatial data, which is often used for solving the inconvenience caused by storage, transmission and application of massive point cloud data. The original scattered point cloud data is coded and sorted multiple times and then is written into a memory in the form of a binary file for system use after the redundant information is removed. Referring to FIG. 1, a data processing flow of the scattered point cloud data compression solution based on octree coding is illustrated, which is generally divided into four steps: preprocessing and octree coding, differential coding, improved run-length coding and arithmetic coding. Octree coding uses Morton codes (i.e.

Morton codes in FIG. 1) to represent leaf nodes with data in a three-dimensional space and a path from a root node to a leaf node, which regards each three-dimensional point as a voxel. Preprocessing is the mapping (i.e. amplification processing) of coordinates to positive integers when Morton coding is only suitable for positive integers. Differential coding is a kind of predictive coding, which stores the relative position data by storing first node data and a relative position of an adjacent point and the previous data, so as to remove information redundancy and reduce the storage space. Improved run- length coding is a lossless reversible coding method for

Morton codes of the same length by removing consecutive 0 and adding newline “\n”. Arithmetic coding maps a whole string of numbers to a representative fraction in the subinterval [0,1] and outputs same in binary coding.

According to the present invention, octree coding is improved, a three-dimensional octree structure is improved into an improved two-dimensional octree structure with a two-dimensional horizontal structure, and new improved

Morton coding based on adjacent data difference values and relative position identifiers is established on the basis of the improved two-dimensional octree structure to further remove redundant data.

[59] Specifically, referring to FIG. 2, a meteorological data compression method provided in this example at least includes a data decoding step and a big data compression step.

[60] $100, the data decoding step: read high-resolution meteorological data in an original data format, and decode meteorological element data of the high-resolution meteorological data on the basis of preset meteorological elements.

[61] By taking CMA-GFS global high-resolution numerical prediction model data independently developed by the China

Meteorological Administration as an example, the data decoding step mainly decodes meteorological element data of CMA-GFS global high-resolution numerical prediction model data in an original grib2 standard data format, and variables satisfying two conditions of continuous variable and scalar quantity can be employed.

[62] The meteorological elements may include one or more of sea level pressure PRMSL, a temperature TMP {such 2- meter temperature), and humidity RH.

[63] By taking the element of PRMSL of a CMA-GFS global high-resolution numerical prediction model as an example, the specific data decoding processing steps are as follows:

[64] S110, acquire spatiotemporal resolution information of CMA-GFS data, which is defined as a resolution variable fbl in.

[65] 5120, acquire horizontal grid point number information of the CMA-GFS data, comprising the total number of grid points, num lon in a horizontal X direction and the total number of grid points, num lat in a horizontal Y direction.

[66] $130, calculate horizontal grid boundary information of the CMA-GFS data according to steps $110 and S120, and calculate longitude and latitude information of all grid points.

[67] 5140, construct an initial grid gridl according to the grid point latitude and longitude information.

[68] S150, read PRMSL values of all grid points in the initial grid gridl according to preset meteorological elements, and store the values in a grid array im data.

[69] 5200, the big data compression step: process, according to the original meteorological element data obtained by means of decoding, the original meteorological element data by using a meteorological data compression algorithm based on scale features to remove redundant data and obtain compressed data.

[70] In this example, the big data compression step is mainly to perform information extraction on the original high-resolution meteorological data by using the meteorological data compression algorithm based on scale features to remove redundant data. Specifically, the meteorological data compression algorithm based on scale features is configured to: perform scale separation on the original meteorological element data, acquire weather system information at different scales, extract key information, generate a plurality of irregular grid meteorological element data sets according to the extracted key information, where a weather system at one scale corresponds to an irregular grid meteorological element data set, and perform data fusion on the plurality of irregular grid meteorological element data sets to obtain a compressed irregular grid point data set.

[71] Preferably, referring to FIG. 3, when scale separation is performed on the original meteorological element data on the basis of two scales, the meteorological data compression algorithm based on scale features is configured to execute the steps as follows:

[72] $210, acquire weather system information at a first scale SCALE1l and above of a horizontal direction, and generate a first data set.

[73] S220, acquire weather system information at a second scale SCALE2 in the horizontal direction, where the second scale SCALE2 is smaller than the first scale SCALE1, and generate a second data set.

[74] 5230, perform data fusion on the first data set and the second data set to obtain a compressed irregular grid point data set, i.e. compressed data.

[75] The first scale SCALEl and the second scale SCALE2 may be selected and set by a user according to actual needs, and accordingly, a scale feature collection window may be set for the user to set the first scale SCALEl and the second scale SCALEZ.

[76] In this example, step S210 may specifically includes the following steps:

[77] 5211, «calculate coordinate information of grid points at the first scale SCALEl and above, take an appropriate value interall near [SCALE1/ (fbl in*M)] according to a preset M value, and make num lon/interall and num lat/interall be positive integers, where M takes an integer of a comparable length to a longitude and latitude, and [SCALE1/{(fbl in*M)] represents rounding the fraction SCALE1/{(fbl in*M); let resolution fbl inl=fbl in* interall, repeat the steps S120 and S130 to obtain grid point latitude and longitude information when the resolution fbl inl=fbl in* interall, where the resolution fbl inl is smaller than the resolution of the initial grid gridl.

[78] S212, denote the grid point latitude and longitude information as a first-layer grid grid2, and forming a second-layer nested grid with the initial grid gridl.

[79] 5213, separately compare numerical values of meteorological element values of grid points in the initial grid gridl embedded in each first-layer grid grid2, and acquire extreme values of the meteorological element values of grid points in each first-layer grid grid? and initial grid positioning information corresponding thereto, thereby obtaining an irregular grid meteorological element data set denoted as a first data set, where the extreme values include a maximum value max and a minimum value min.

[80] In this example, step S220 may specifically includes the following steps:

[81] S221, calculate coordinate information of grid points around the second scale SCALEZ, take an appropriate value interal2 near [SCALE2/ (fbl in*M)], make num lon/interal2 and num lat/interal2 be positive integers, let resolution fbl in2=fbl in* interal2, and repeat the steps S120 and S130 to obtain grid point latitude and longitude information when resolution fbl inzZ=fbl in* interal2, where resolution fbl inZ is smaller than the resolution of the initial grid gridl and larger than the resolution fbl inl of the first-layer grid grid2.

[82] $222, denote the grid point latitude and longitude information as a second-layer grid grid3, and form a third-layer nested grid with the initial grid gridl and the first-layer grid grid2.

[83] 5223, separately compare numerical values of meteorological element values of grid points in the initial grid gridl embedded in each second-layer grid grid3, and acquire extreme values of the meteorological element values of grid points in each second-layer grid grid3, where the extreme values include a maximum value max and a minimum value min; for each second-layer grid3, compare the maximum value max and the minimum value min of the grid, and when the maximum value max is equal to the minimum value min, only retain grid point coordinates and meteorological element values of the initial grid at an inner center point position of the grid to obtain an intermediate data set.

[84] $224, for the intermediate data set, take any grid point as a node, construct eight child nodes according to eight directions of upper left, upper, upper right, left, right, lower left, lower and lower right so as to construct an improved two-dimensional planar octree structure with a two-dimensional horizontal structure, and perform Morton coding on the basis of the improved two- dimensional planar octree structure so as to obtain octree coding of each node; and remove the point information whose octree code is 0 to obtain an irregular grid meteorological element data set denoted as a second data set.

[85] In this example, in step $230, the manner by which the data fusion is performed is preferably as follows: store the grid points in the first data set and the second data set in order of longitude and latitude, and perform deduplication on overlapped grid point data.

[86] The above data compression algorithm is described in detail below in conjunction with particular examples. Let the preset M=100, SCALE1=500 km, SCALEZ=100 km, and the meteorological element be the sea level pressure PRMSL.

[87] Firstly, acquire weather system information of

SCALE=500 km and above of the horizontal direction.

[88] 1) Calculate coordinate information of grid points at the horizontal scale SCALE=500 km, take an appropriate value interal near [SCALE/ (fbl in*100)1)1, make num lon/interal and num lat/interal be positive integers, acquire resolution fbl in=fbl in*interal, repeat the steps

S120 and S130 in the data decoding stage to obtain grid point latitude and longitude information when resolution fbl in=fbl in*interal.

[89] 2) Denote the grid point latitude and longitude information as a first-layer grid2 of one layer, and form a second-layer nested grid with gridl, where as shown in

FIG. 4, by way of example but not limitation, 81 initial grids 100 are nested in a first-layer grid 200.

[90] 3) Separately compare numerical values of PRMSL values of various grid points of the initial grid gridl embedded in each first-layer grid grid2, and acquire extreme values of the PRMSL of grid points in each first- layer grid grid2 and initial grid positioning information corresponding thereto, thereby obtaining an irregular grid

PRMSL data set denoted as a first data set (namely the first-layer data set in FIG. 3), where the extreme values include a maximum value max and a minimum value min.

[91] Then, acquire weather system information of

SCALE=100 km of the horizontal direction.

[92] 4) Calculate coordinate information of grid points around the horizontal scale SCALE=100 km, where the process is similar to that of step 1) to obtain a second- layer grid grid3. 9 initial grids 100 are nested in a second-layer grid 300, and 9 second-layer grids 300 are nested in a first-layer grid 200. Thus, gridl, grid2 and grid3 jointly form a third-layer nested grid, as shown in

FIG. 4. In this example, the resolution fbl in2 of the second-layer grid grid3 is smaller than the resolution of the initial grid gridl (the resolution of the initial grid illustrated in FIG. 4 is 9 times that of the second-layer grid), while the resolution fbl inZ of the second-layer grid grid3 is larger than the resolution fbl inl of the first-layer grid grid2 (the resolution of the second-layer grid illustrated in FIG. 4 is 9 times that of the first- layer grid).

[93] 5) Separately compare numerical values of PRMSL values of grid points in the initial grid gridl embedded in each second-layer grid grid3, and acquire extreme values of the PRMSL values of grid points in each second- layer grid grid3, where the extreme values include a maximum value max and a minimum value min. For each second-layer grid grid3, compare the maximum value max and the minimum value min of the grid, and when the maximum value max is equal to the minimum value min, only retain grid point coordinates and PRMSL values of the initial grid at an inner center point position of the grid to obtain an intermediate data set.

[94] 6) For the intermediate data set, take any grid point as a node, construct eight child nodes according to eight directions of upper left, upper, upper right, left, right, lower left, lower and lower right so as to construct an improved two-dimensional planar octree structure with a two-dimensional horizontal structure as the octree structure as shown in FIG. 5, and perform

Morton coding so as to obtain octree coding of each node; and remove the point information whose octree code is 0 to obtain an irregular grid PRMSL data set denoted as a second data set.

[95] As shown in FIG. 6, when Morton coding is performed by using the improved two-dimensional planar octree structure, for any node, the eight orientations of upper left, upper, upper right, left, right, lower left, lower and lower right are child node 1, child node 2, child node 3, child node 4, child node 5, child node 6, child node 7 and child node 8 in sequence, and the codes corresponding to various child nodes are cd[1], cd[2], cd[3], cdl4}, cd[5], ed[6], cd [7], and cd [8] in sequence.

[96] Let the node code be CODE, there are:

[97] value [N]=node-child node N, Ne[1,8]

[98] if value[N]=0

[99] cd[N]=0

[100] Else

[101] cd[N]=1

[102] CODE=cd[l].cd[2].cd[3].cd[4].cd[5].cd[6].cd[7].cd[8]

[103] That is, the code of one node is equal to the sequence of codes of the child nodes in the eight directions of upper left, upper, upper right, left, right, lower left, lower and lower right of the node.

[104] Referring to FIG. 7, an example of a coding solution of certain node 1023 is given. In this case, the codes of cd[1l], ed[2], cd[3], cd[4], eced[5], ed[6], cd [7], and cd

[8] corresponding to child node 1, child node 2, child node 3, child node 4, child node 5, child node 6, child node 7 and child node 8 are 0, 1, 0, 1, 0, 0, 1, and 0 in sequence, thereby obtaining the code of the node 1023 as follows: CODE=01010010.

[105] After the point information with the node code of 00000000, another group of irregular grid PRMSL data set with redundant data removed, i.e. a second data set, can be obtained.

[106] In another embodiment of this example, after the big data compression step of 5200, a compression data outputting step of S300 is further included. The specific steps are as follows: perform data conversion on the obtained irregular grid point data set to generate a data format file common to meteorology, and transmit the common data format file to shore users.

[107] The common data format file may be a NetCDF format file as an instance.

[108] In another embodiment of this example, after the big data compression step of S200, an encryption coding step of S400 is further included. Specific steps are as follows: construct a symmetric cipher table, perform lossless compression and encoding on the obtained irregular grid point data set, realize reconstruction of the data set, and transmit the reconstructed data to a ship-end communication satellite.

[109] Thus, digital compression of the global high- resolution numerical prediction model data subjected to big data compression is further performed by means of encryption coding, such that the storage space occupied by data information 1s reduced, and it is possible to transmit the digital meteorological model data by means of ocean satellite communication.

[110] As another example of the present invention, the present invention further provides a meteorological data compression system.

[111] The system includes a data decoding apparatus and a big data compression apparatus.

[112] The data decoding apparatus is configured to read high-resolution meteorological data in an original data format and decode meteorological element data of the high- resolution meteorological data on the basis of preset meteorological elements.

[113] The big data compression apparatus is configured to process, according to the original meteorological element data obtained by means of decoding, the original meteorological element data by using a meteorological data compression algorithm based on scale features to remove redundant data and obtain compressed data. The meteorological data compression algorithm based on scale features is configured to: perform scale separation on the original meteorological element data, acquire weather system information at different scales, extract key information, generate a plurality of irregular grid meteorological element data sets according to the extracted key information, where a weather system at one scale corresponds to an irregular grid meteorological element data set, and perform data fusion on the plurality of irregular grid meteorological element data sets to obtain a compressed irregular grid point data set.

[114] In this example, the system may further includes a compressed data transmission device.

[115] The compressed data transmission device is configured to perform data {format conversion on the obtained irregular grid point data set to generate a data format file common to meteorology, and transmit the common data format file to shore users. The common data format file is a NetCDF format file.

[116] The system may also includes an encryption coding device. The encryption coding device is configured to construct a symmetric cipher table, perform lossless compression and encoding on the obtained irregular grid point data set, realize reconstruction of the data set, and transmit the reconstructed data to a ship-end communication satellite.

[117] Other technical features are described in the preceding examples and will not be repeated here.

[118] As another example of the present invention, a computer-readable storage medium is further provided, and is configured to store a computer program executable by a processing unit. The computer program, when executed by the processing unit, implements the meteorological data compression method mentioned above.

[119] The storage medium may include various media which may store program codes, such as a universal serial bus (USB) flash drive, a mobile hard disk drive, a read-only memory (ROM), a random access memory (RAM), a diskette and an optical disk.

[120] Other technical {features are described in the preceding examples and will not be repeated here.

[121] In the above description, the content disclosed in the present invention is not intended to limit itself to these aspects. But, components may be selectively and operatively combined in any number within the intended protection scope of the content of the present disclosure.

In addition, terms such as “comprising”, “including”, and “having” should by default be interpreted as inclusive or open-ended, and not exclusive or closed, unless expressly limited to the contrary. All technical, scientific or other terms are intended to have the meaning understood by those skilled in the art unless limited to the contrary.

Common terms found in dictionaries should not be too idealized or too impractical to interpret in the context of related art documents unless the content of the present disclosure explicitly limits them to that. Any variation or modification made by those of ordinary skill in the art of the present invention according to the content disclosed above falls within the protection scope of the claims.

Claims

CONCLUSIONS

1. A meteorological data compression method, comprising: a data decoding step: reading high-resolution meteorological data in an original data format, and decoding meteorological element data of the high-resolution meteorological data based on preset meteorological elements; and a bulk data compression step: processing, according to the original meteorological element data obtained by decoding, the original meteorological element data by using a meteorological data compression algorithm based on scale features, to remove redundant data and obtain compressed data, the meteorological data compression algorithm based on scale features being configured to: perform scale separation on the original meteorological element data, acquire weather system information at different scales, extract key information, generate a plurality of irregular grid meteorological element data sets according to the extracted key information, wherein a weather system at one scale corresponds to an irregular grid meteorological element data set, and perform data fusion on the plurality of irregular grid meteorological element data sets to obtain a compressed irregular grid point data set available.

The method of claim 1, further comprising a compressed data output step after the bulk data compression step as follows: performing data format conversion on the obtained irregular grid point data set to generate a data format file common to meteorology, and transmitting the common data format file to coastal users.

The method of claim 2, wherein the conventional data format file is a NetCDF format file.

The method of claim 1, further comprising an encryption encoding step after the bulk data compression step as follows: constructing a symmetric encryption table, performing lossless compression and encoding on the obtained irregular grid point data set, realizing reconstruction of the data set, and transmitting the reconstructed data to a ship-end communications satellite.

A method according to claim 1, wherein the high-resolution meteorological data in an original data format is CMA-GF5 global high-resolution numerical forecasting model data in grib2 standard data format, in this case, data decoding processing comprises the following steps: S110, acquiring spatio-temporal resolution information of CMA-GFS data, which is defined as a resolution variable fbl in; S120, acquiring horizontal grid point number information of the CMA-GFS data, including the total number of grid points, num lon in a horizontal X direction and the total number of grid points, num lat in a horizontal Y direction; S130, calculating horizontal grid boundary information of the CMA-GFS data according to steps S110 and S120, and calculating longitude and latitude information of all grid points; $5140, constructing an initial raster gridl according to grid point latitude and longitude information; and S150, reading meteorological element values from all grid points in the initial raster gridl according to preset meteorological elements, and storing the meteorological element values in a raster array im data.

A method according to claim 5, wherein when scale separation is performed on the original meteorological element data based on two scales, the meteorological data compression algorithm based on scale features is configured to perform the steps as follows: S210, acquiring weather system information at a first scale SCALE1l and above the horizontal direction, and generating a first data set, comprising the steps of: calculating coordinate information of grid points at the first scale SCALE1l and above, taking an appropriate value interall near [SCALEL1/(fbl in*M)] according to a preset M value, and making positive integers of num lon/interall and num lat/interall, where M takes an integer of a similar length to a longitude and latitude, and where [SCALE1/(fbl in*M)] represents rounding the fraction SCALE1/(fbl in*M); letting resolution fbl inl=fbl in*interall, and repeating steps S120 and S130 to obtain grid point longitude and latitude information when the resolution fbl inl=fbl in*interall, where the resolution fbl inl is less than the resolution of the initial grid gridl; denoting the grid point longitude and latitude information as a first-layer grid grid2, and forming a second-layer nested grid with the initial grid gridl; separately comparing numerical values of meteorological element values of grid points in the initial grid grid1 embedded in each second-layer grid grid2, and acquiring extreme values of the meteorological element values of grid points in each first-layer grid grid2 and initial grid positioning information corresponding thereto, thereby obtaining an irregular grid meteorological element data set designated as a first data set, wherein the extreme values include a maximum value max and a minimum value min; S220, acquiring weather system information at a second scale SCALE2 in the horizontal direction, the second scale SCALE2 being smaller than the first scale SCALE], and generating a second data set, comprising the steps of: calculating coordinate information of grid points around the second scale SCALE2, taking an appropriate value interal2 near [SCALE2/{({fbl in*M)], making num lon/interal2 and num lat/interalz positive integers, letting resolution fbl in2=fbl in*interal2, and repeating steps S120 and S130 to acquire grid point longitude and latitude information when resolution fbl in2=fbl in*interal2, where resolution fbl in2 is smaller than the resolution of the initial grid gridl and larger than the resolution fbl inl of the first-layer grid grid2; declaring the grid point longitude and latitude information as a second-layer grid grid3, and forming a third-layer nested grid with the initial grid grid1 and the first-layer grid grid2; separately comparing numerical values of meteorological element values of grid points in the initial grid grid1 embedded in each second-layer grid grid3, and acquiring extreme values of the meteorological element values of grid points in each second-layer grid grid3, wherein the extreme values include a maximum value max and a minimum value min; for each second-layer grid grid3, comparing the maximum value max and the minimum value min of the grid, and when the maximum value max equals the minimum value min, only retaining grid point coordinates and meteorological element values of the initial grid at an inner center point position of the grid to obtain an intermediate data set; for the intermediate data set, taking a grid point as a node, constructing eight child nodes along eight directions of top-left, top, top-right, left, right, bottom-left, bottom, and bottom-right to construct an improved two-dimensional planar octree structure with a two-dimensional horizontal structure, and performing Morton coding based on the improved two-dimensional planar octree structure to obtain octree coding of each node; removing the point information whose octree code is 0 to obtain an irregular grid meteorological element data set designated as a second data set; and S230, performing data fusion on the first data set and the second data set to obtain a compressed irregular grid point data set, i.e., compressed data.

The method of claim 6, wherein the manner in which the data fusion is performed is as follows: storing the grid points in the first data set and the second data set in order of longitude and latitude, and performing deduplication on overlapping grid point data.

The method of claim 1, wherein the meteorological elements comprise one or more of seal level pressure PRMSL, a temperature TMP and humidity RH.

9. A meteorological data compression system, comprising: a data decoding device configured to read high-resolution meteorological data in an original data format and decoding meteorological element data of the high-resolution meteorological data based on preset meteorological elements; and a bulk data compression device configured to process, according to the original meteorological element data obtained by decoding, the original meteorological element data by using a meteorological-

data compression algorithm based on scale features to remove redundant data and obtain compressed data, wherein a meteorological data compression algorithm based on scale features is configured to: perform scale separation on the original meteorological element data, acquire weather system information at different scales, extract key information, generate a plurality of irregular grid meteorological element data sets according to the extracted key information, wherein a weather system at one scale corresponds to an irregular grid meteorological element data set, and perform data fusion on the plurality of irregular grid meteorological element data sets to obtain a compressed irregular grid point data set.

10. A computer readable storage medium configured to store a computer program executable by a processing unit, the computer program, when executed by the processing unit, implementing the method of claims 1-8. -0-0-0-