CN119106035A

CN119106035A - A block storage method and related equipment

Info

Publication number: CN119106035A
Application number: CN202310673388.0A
Authority: CN
Inventors: 陈永清
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2023-06-07
Filing date: 2023-06-07
Publication date: 2024-12-10

Abstract

The embodiment of the invention discloses a block storage method, a device, electronic equipment and a computer readable storage medium, wherein after a current storage configuration file is acquired and compared with a corresponding historical storage configuration file, when a comparison result indicates that a new storage space exists in a current node, a metadata set is updated based on a space identifier of the new storage space, a block mapping relation is identified in the updated metadata set so as to obtain a block data index, when block data of a block to be stored is acquired, a target storage space corresponding to a target file and the target file is determined based on the block data index and the current block identifier of the block to be stored, and the block data is written into the target file in the target storage space.

Description

Block storage method and related equipment

Technical Field

The invention relates to the field of block chains, in particular to a block storage method and related equipment.

Background

In recent years, with the development of blockchain technology, blockchain systems are increasingly applied in the transaction field. In a blockchain system, each node may store blockdata on the chain. The existing block data storage method often writes the block data directly into a file system (folder) corresponding to the current node.

In the research and practice of the prior art, the inventor of the present application found that when storing block data to a file system, the upper storage limit of a blockchain is limited by the size of the file system (folder) or the storage space of a disk corresponding to the file system, and when the storage space is full, the chain stops working, so that the current node cannot store a new block, thus resulting in lower efficiency of block storage.

Disclosure of Invention

The embodiment of the invention provides a block storage method, a block storage device, electronic equipment and a computer readable storage medium, which can improve the block storage efficiency.

A block storage method, comprising:

Acquiring a current storage configuration file, and comparing the current storage configuration file with a corresponding historical storage configuration file;

When the comparison result indicates that a new storage space exists in the current node, updating a metadata set based on the space identification of the new storage space, wherein the metadata set indicates a file mapping relation among the space identification, the file identification and block identification information of the file;

Identifying a block mapping relation in the updated metadata set to obtain a block data index, wherein the block mapping relation comprises mapping relations between block identifiers in the block identifier information and the file identifiers and the space identifiers respectively;

When block data of a block to be stored is acquired, determining a target file and a target storage space corresponding to the target file based on the block data index and a current block identifier of the block to be stored;

and writing the block data into the target file in the target storage space.

Accordingly, an embodiment of the present invention provides a block storage device, including:

the acquisition unit is used for acquiring the current storage configuration file and comparing the current storage configuration file with the corresponding historical storage configuration file;

the updating unit is used for updating the metadata set based on the space identification of the newly added storage space when the comparison result indicates that the newly added storage space exists in the current node, and the metadata set indicates the file mapping relation among the space identification, the file identification and the block identification information of the file;

The identification unit is used for identifying a block mapping relation in the updated metadata set to obtain a block data index, wherein the block mapping relation comprises mapping relations between block identifiers in the block identifier information and the file identifiers and the space identifiers respectively;

The determining unit is used for determining a target file and a target storage space corresponding to the target file based on the block data index and the current block identifier of the block to be stored when the block data of the block to be stored is acquired;

And the writing unit is used for writing the block data into the target file in the target storage space.

In some embodiments, the identifying unit may be specifically configured to identify a block identifier section of each file corresponding to the file identifier in the block identifier information, convert the file mapping relationship into the block mapping relationship corresponding to the block identifier in the block identifier section, and construct the block data index corresponding to the current node according to the block mapping relationship.

In some embodiments, the identifying unit may be specifically configured to identify an initial file mapping relationship between the block identifier and the file identifier in the file block mapping relationship, identify an initial spatial mapping relationship between the block identifier and the spatial identifier in the spatial block mapping relationship, and fuse the initial file mapping relationship, the initial spatial mapping relationship, and the file spatial mapping relationship to obtain the block mapping relationship.

In some embodiments, the identification unit may be specifically configured to use the block identifier as a key value, and use the file identifier, the space identifier, and the block identifier interval corresponding to the block identifier as leaf nodes, and construct a search tree based on the block mapping relationship, the key value, and the leaf nodes, and use the search tree as the block data index.

In some embodiments, the block storage device may further include a creation unit, where the creation unit may be specifically configured to obtain a remaining storage capacity of the target file and compare the remaining storage capacity with a storage capacity occupied by the block data, create a candidate file and determine file metadata corresponding to the candidate file when the storage capacity exceeds the remaining storage capacity, update the block data index based on the file metadata, and use the updated block data index as the block data index.

In some embodiments, the creating unit may specifically be configured to query the block data index for a history file corresponding to a preset block identifier, adjust the current block identifier of the history file when the history file exists, and create the candidate file based on the current block identifier when the history file does not exist.

In some embodiments, the creating unit may be specifically configured to obtain storage space information of the current node, determine a candidate space identifier corresponding to the candidate file based on the storage space information, set a start block identifier and an end block identifier corresponding to the candidate file according to the current block identifier, and use the candidate file identifier, the start block identifier, the end block identifier, and the candidate space identifier as the file metadata.

In some embodiments, the creating unit may be specifically configured to extract, from the storage space information, dictionary ordering information of a storage space set corresponding to the block data index, and screen, based on the dictionary ordering information, a candidate storage space corresponding to the candidate file from the storage space set, so as to obtain a candidate space identifier of the candidate storage space.

In some embodiments, the creating unit may be specifically configured to extract a current remaining storage capacity of each storage space in a storage space set corresponding to the block data index from the storage space information, sort the storage spaces based on the current remaining storage capacity, and screen candidate storage spaces corresponding to the candidate files from the storage space set according to a sorting result, so as to obtain candidate space identifiers of the candidate storage spaces.

In some embodiments, the obtaining unit may be specifically configured to query a current space identifier set of the current storage configuration file for a corresponding history space identifier in the history storage configuration file, determine that the current storage configuration file is configured correctly when the current space identifier set includes the history space identifier, compare a space identifier in the current space identifier set with the history space identifier, determine that the current node has a new storage space when a space identifier other than the history space identifier exists in the current space identifier set, and determine that the current node does not have a new storage space when a space identifier other than the history space identifier does not exist in the current space identifier set.

In some embodiments, the obtaining unit may be specifically configured to store a block to be stored based on a current block data index when the current node has the current block data index, construct the block data index based on the metadata set when the current node does not have the block data index, and store the block to be stored based on the block data index.

In addition, the embodiment of the invention also provides electronic equipment, which comprises a processor and a memory, wherein the memory stores application programs, and the processor is used for running the application programs in the memory to realize the block storage method provided by the embodiment of the invention.

In addition, the embodiment of the invention also provides a computer readable storage medium, which stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor to execute the steps in any block storage method provided by the embodiment of the invention.

In addition, the embodiment of the application also provides a computer program product, which comprises a computer program or instructions, and the computer program or instructions realize the steps in the data method provided by the embodiment of the application when being executed by a processor.

According to the embodiment of the invention, after the current storage configuration file is obtained and compared with the corresponding historical storage configuration file, when the comparison result indicates that the current node has a new storage space, the metadata set is updated based on the space identification of the new storage space, the block mapping relation is identified in the updated metadata set to obtain the block data index, when the block data of the block to be stored is obtained, the target storage space corresponding to the target file and the target file is determined based on the block data index and the current block identification of the block to be stored, and the block data is written into the target file in the target storage space, and because the scheme can manage the relation among the storage spaces of the block, the file and the file through the current storage configuration file, the metadata set and the block data index, the new storage space is supported in the storage configuration file, and then the file writing of different storage spaces is distributed through the metadata set and the block data index, so that the block storage efficiency can be improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic diagram of a scenario of a block storage method according to an embodiment of the present invention;

FIG. 2 is a flowchart illustrating a block storage method according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of different versions of a storage configuration file provided by an embodiment of the present invention;

FIG. 4 is a schematic diagram of a data structure of a metadata collection provided by an embodiment of the present invention;

FIG. 5 is a schematic diagram of a block stored in a file according to an embodiment of the present invention;

FIG. 6 is a schematic diagram of a file stored in a storage space according to an embodiment of the present invention;

FIG. 7 is a diagram illustrating a data structure of a block data index according to an embodiment of the present invention;

FIG. 8 is a diagram illustrating a dictionary ordering of memory spaces according to an embodiment of the present invention;

FIG. 9 is a schematic diagram of screening candidate storage spaces after dictionary ordering according to an embodiment of the present invention;

FIG. 10 is a schematic diagram of a capacity screening candidate storage space based on a storage space according to an embodiment of the present invention;

FIG. 11 is a schematic flow chart of a new candidate file according to an embodiment of the present invention;

FIG. 12 is a schematic diagram of a metadata system according to an embodiment of the present invention;

FIG. 13 is a schematic diagram of a metadata system startup or restart procedure according to an embodiment of the present invention;

FIG. 14 is a flowchart of a block storage method according to an embodiment of the present invention;

FIG. 15 is a schematic diagram of a block memory device according to an embodiment of the present invention;

FIG. 16 is a schematic diagram of another block apparatus according to an embodiment of the present invention;

Fig. 17 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to fall within the scope of the invention.

The embodiment of the invention provides a block storage method, a block storage device, electronic equipment and a computer readable storage medium. The block storage device may be integrated in an electronic device, which may be a server or a terminal.

The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), basic cloud computing services such as big data and an artificial intelligent platform. The terminal may be, but is not limited to, a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, etc. The terminal and the server may be directly or indirectly connected through wired or wireless communication, and the present application is not limited herein.

For example, referring to fig. 1, taking the block storage device integrated in an electronic device as an example, after obtaining a current storage configuration file and comparing the current storage configuration file with a corresponding historical storage configuration file, when a comparison result indicates that a new storage space exists in a current node, updating a metadata set based on a space identifier of the new storage space, identifying a block mapping relationship in the updated metadata set to obtain a block data index, when obtaining block data of a block to be stored, determining a target file and a target storage space corresponding to the target file based on the block data index and the current block identifier of the block to be stored, and writing the block data into the target file in the target storage space, thereby improving the block storage efficiency.

The block storage method is mainly applied to a block chain system and can be applied to one or more nodes in the block chain system. The so-called blockchain (Blockchain) is a new mode of application for computer technologies such as distributed data storage, point-to-point transmission, consensus mechanisms, encryption algorithms, etc. The blockchain (Blockchain), essentially a de-centralized database, is a string of data blocks that are generated in association using cryptographic methods, each of which contains information from a batch of network transactions for verifying the validity (anti-counterfeit) of its information and generating the next block. The blockchain may include a blockchain underlying platform, a platform product services layer, and an application services layer.

The blockchain underlying platform may include processing modules for user management, basic services, smart contracts, and operational management. The system comprises a user management module, a basic service module, a public and private key generation (account management), a key management and a corresponding relation maintenance (authority management) of a user real identity and a blockchain address, and the like, wherein the user management module is responsible for identity information management of all blockchain participants, comprises maintenance of public and private key generation (account management), key management, maintenance of a corresponding relation between the user real identity and the blockchain address (authority management) and the like, and provides rule configuration (wind control audit) of risk control under the authorized condition, the basic service module is deployed on all blockchain node devices and is used for verifying the validity of service requests, recording the valid requests after the valid requests are completed, for a new service request, the basic service firstly carries out interface adaptation analysis and authentication processing (interface adaptation), then carries out encryption (consensus management) on the service information, and carries out recording and storage after the encryption, the complete and consistent transmission to a shared account book (network communication), the intelligent contract module is responsible for registration issuing of contracts and contract triggering and contract execution, and a developer can define contract logic through programming language, issue contract logic on the blockchain, call key or other event triggering execution according to contract logic, and contract rule triggering execution, and completion logic, and meanwhile, provision of contract upgrading and main function issuing and updating management functions are also provided, and management module, and can be deployed in real-time, and run to a product, and the network management module is suitable for setting and a product, and a running and a real-time service is arranged in a running and a running state and a network.

The platform product service layer provides basic capabilities and implementation frameworks of typical applications, and developers can complete the blockchain implementation of business logic based on the basic capabilities and the characteristics of the superposition business. The application service layer provides the application service based on the block chain scheme to the business participants for use.

It is to be understood that, in the embodiments of the present application, related data such as blocks or block data are related, and when the following embodiments of the present application are applied to specific products or technologies, permission or agreement is required, and collection, use and processing of related data are required to comply with related laws and regulations and standards of related countries and regions.

The following will describe in detail. The following description of the embodiments is not intended to limit the preferred embodiments.

The embodiment will be described from the perspective of a block storage device, which may be specifically integrated in an electronic device, where the electronic device may be a server, or may be a terminal, where the terminal may include a tablet computer, a notebook computer, a personal computer (PC, personal Computer), a wearable device, a virtual reality device, or other devices that may store a block.

A block storage method, comprising:

And when the comparison result indicates that a new storage space exists in the current node, updating the metadata set based on the space identification of the new storage space, wherein the metadata set indicates the space identification, the file identification and the file mapping relation between the block identification information of the file, the block mapping relation is identified in the updated metadata set so as to obtain a block data index, the block mapping relation comprises the mapping relation between the block identification in the block identification information and the file identification and the space identification respectively, when the block data of the block to be stored is obtained, the target storage space corresponding to the target file and the target file is determined based on the block data index and the current block identification of the block to be stored, and the block data is written into the target file in the target storage space.

As shown in fig. 2, the specific flow of the block storage method is as follows:

101. and acquiring the current storage configuration file, and comparing the current storage configuration file with the corresponding historical storage configuration file.

The current storage configuration file may be the latest storage configuration file in the current node. By a storage profile is understood a profile that configures the storage space in the current node. By storage space is understood a space or area in a file system, folder or disk for storing block data or other data, or a space in a file system, folder or disk for storing data. A so-called file system may be a mechanism for providing users with an underlying data access mechanism that divides the space in the device into blocks (or clusters) of a certain size, typically 512 bytes per block. Data is stored in these blocks, the size being modified to occupy an integer number of blocks. The file system software is responsible for organizing the blocks into files and directories and keeping track of which blocks are assigned to which files and which blocks are not used. The storage configuration file may include a set of space identifiers and a file version number of the configured storage space, where the set of space identifiers may include space identifiers of all configured storage spaces, for example, the set of space identifiers may include the following contents:

the method for obtaining the current storage configuration file may be various, and specifically may be as follows:

For example, when receiving a storage configuration file uploaded by a terminal, the uploaded storage configuration file may be directly used as a current storage configuration query, or a storage configuration file with the modification time closest to the current time may be queried in a preset storage area, the queried storage configuration file may be used as the current storage configuration file, or when the storage capacity of the storage space of the current node is smaller than a preset storage capacity threshold, space early warning information may be sent to the terminal corresponding to the current node, a storage configuration file returned by the receiving terminal based on the space early warning information may be used as the current storage configuration file, and the returned storage configuration file may be used as the current storage configuration file.

After the current storage configuration file is obtained, the current storage configuration file may be compared with the corresponding historical storage configuration file. The comparison mode can be various, for example, the history space identifier in the history storage configuration file can be queried in the current space identifier set of the current storage configuration file, when the current space identifier set contains the history space identifier, the current storage configuration file is determined to be correctly configured, the space identifier in the current identifier set is compared with the history space identifier, when the current space identifier set contains the space identifier except the history space identifier, the current node is determined to have the newly-added storage space, and when the current space identifier set does not contain the space identifier except the history space identifier, the current node is determined to not have the newly-added storage space.

The method for querying the historical space identifier in the historical storage configuration file in the current space identifier set of the current storage configuration file may be various, for example, the current storage space identifier set and the current version number may be extracted from the current storage configuration file, the historical storage configuration file set of the current node may be obtained, the historical storage configuration file corresponding to the current storage configuration file is screened out from the historical storage configuration file set based on the current version number, the space identifier in the space identifier set in the historical storage configuration file is obtained, the historical space identifier is obtained, and the historical space identifier is queried in the current space identifier set.

When the current space identifier set comprises the history space identifier, the current storage configuration file is determined to be correctly configured. The correct configuration can indicate that the current storage configuration file does not delete or modify the storage space in the current node in the configuration, so that the data security of the data of the history block stored in the current node can be ensured. Otherwise, when the current space identifier set does not include the history space identifier, the operation of deleting or modifying the storage space of the current node in the configuration in the current storage configuration file can be indicated, and in this case, the flow is stopped, so that the configuration of the storage space of the current node according to the current storage configuration file is prevented.

When the configuration of the current storage configuration file is determined to be correct, the space identifier in the current identifier set can be compared with the historical space identifier. When the space identifiers except the history space identifiers exist in the current space identifier set, determining that the current node has a new storage space. For example, taking the example that the current space identifier set includes disk1 (disk 1), disk2 (disk 2), disk3 (disk 3) and disk4 (disk 4), and the history space identifier includes disk1, disk2 and disk3, at this time, disk4 that does not exist in the history space identifier exists in the current space identifier set, so it can be determined that the current node exists in the newly added storage space disk4. When the space identifiers other than the history space identifier do not exist in the current space identifier set, the space identifier in the current space identifier set is the same as the history space identifier, and at this time, it is determined that the current node does not have a new storage space, for example, the current space identifier set includes a disk1 (disk 1), a disk2 (disk 2) and a disk3 (disk 3), the history space identifier includes a disk1, a disk2 and a disk3, and at this time, it can be found that the space identifier in the current space identifier set is the same as the history space identifier, and it can be determined that the current node does not have a new storage space.

It should be noted that, after the current node increases the storage space, the storage space information of the current node may be updated. In addition, in order to detect whether the storage space is newly increased in the storage configuration file configured each time, the history version information needs to be kept, so that the acquired storage configuration file of each version can be stored, that is, each time the storage configuration file (for example, the number of newly increased storage spaces or file systems) is modified, the new storage configuration file (the current storage configuration file) needs to be stored or saved in a lasting manner. For example, taking the storage space of the new disk4 of the current storage configuration file relative to the history storage configuration file as an example, the two versions of storage configuration files need to be stored, which can be specifically described in fig. 3. It should be noted that, in the current storage configuration file, compared with the historical storage configuration file, the storage space is not newly increased, and the current storage configuration file may be stored, or the current storage configuration file may not be stored.

In some embodiments, when it is determined that the current node has a new storage space, the metadata set may be updated based on the spatial identifier of the new storage space, which is described below.

102. And when the comparison result indicates that the newly added storage space exists currently, updating the metadata set based on the space identification of the newly added storage space.

The metadata set may include various types of metadata such as a block, a file, or a storage space. The metadata set indicates a file mapping relationship between the space identifier, the file identifier, and block identifier information of the file. The metadata may include a block identification of a block, a file identification of a file, block identification information, or a spatial identification of a storage space, etc. The block identification may include a block height (block height) of the block. The block identification information of the file may include a start block height (STARTHEIGHT) and an end block height (ENDHEIGHT) of the blocks stored within the file. The start block height may be the block height of the first block contained or stored in this file and the end block height may be the block height of the last block contained or stored in this file. When the file is being written, if the last block is written to the file, the ending block height of the file may be the preset maxUint64 or other preset block identifier, so if the ending block height of a file is maxUint64 or other preset block identifier, it may be indicated that the file is not completely written.

When the comparison result indicates that the current node has a new storage space, the metadata set may be updated in various ways based on the space identifier of the new storage space, which may be specifically as follows:

for example, when the comparison result indicates that the current node has a new storage space, a space identifier of the new storage space may be obtained, and the space identifier is added as metadata to the metadata set, so as to obtain an updated metadata set.

Wherein, metadata in the metadata set needs to be persistent, after the server or the process is restarted, metadata information can be read from the persistent data, and a block data index (memory index) can be regenerated in the memory, so leveldb can be adopted for persistent storage of the metadata set to save data. The metadata collection may be regarded as Db (dougabyte) data, which may be structural data of a key/value, and the metadata information (FileMeta) of the file is serialized by using a file identifier (file name) as a key. FileMeta may include a file name (filename), a start block height in the file (STARTHEIGHT), an end block height in the file (ENDHEIGHT), and a spatial identification of a storage space corresponding to the file (identification of a file system/folder/disk/hard disk), and may be specifically shown in fig. 4. When there is a new storage space, a space identifier of the new storage space may be added to the metadata set, where the new storage space is not allocated to a corresponding file, so in the updated metadata set, a key corresponding to the space identifier is empty, a file identifier, a start block height, and an end block height do not exist in a value, and for data of such key/value, it is indicated that the storage space corresponding to the space identifier is not allocated to a file and block data is written. When the block storage data needs to be stored in the newly added storage space, the creation of a file in the newly added storage space can be triggered, and after the creation of the file, the block data of the block to be stored can be selectively written in the created file, and the specific creation file and the writing of the block data can be described in detail below.

103. And identifying a block mapping relation in the updated metadata set to obtain a block data index.

The block mapping relationship includes a mapping relationship between the block identifiers in the block identifier information and the file identifiers and the space identifiers, respectively. For example, the mapping relationship between the block a and the file a storing the block a may be included, and the mapping relationship between the block a and the storage space a corresponding to the file a may be also included, where the file a is stored in the storage space a. Thus, it can be found that for storing a block, the block may be serialized into a byte stream, and then the byte stream is written into a file, so that storing the block is completed, and thus a plurality of blocks are stored into one file. This file is stored in (belonging to) a storage space (file system/folder/disk/hard disk). For example, 4 blocks (block 1, block2, block3, and block 4) may be stored to (wal. Bfdb.001) as shown in fig. 5. For another example, a plurality of files such as (wal.bdb.001), (wal.bdb.002), (wal.bdb.003), and (wal.bdb.004) may be stored in one storage space a, and specifically, as shown in fig. 6. Wal (WRITE AHEAD Log) is a pre-written Log, and is a common means in database systems for ensuring the atomicity and durability of data operations.

The block data index may be an index including a block mapping relationship. Metadata information such as file identifications, space identifications and the like corresponding to each block identification can be retrieved through the block data index. For high performance lookup file metadata information.

The block mapping relationship is identified in the updated metadata set, so that a plurality of block data indexes can be obtained, which can be specifically as follows:

For example, the block identification interval of each file corresponding to the file identification can be identified in the block identification information, the file mapping relationship is converted into the block mapping relationship corresponding to the block identification, and the block data index corresponding to the current node is constructed according to the block mapping relationship.

The block identifier section may be a section between the block identifier of the start block and the block identifier of the end block in each file. The block identification interval may be identified in various ways, for example, the block identification information of the file corresponding to each file identification is obtained, the start block height and the end block height are extracted from the block identification information, when the end block height is not the preset block identification (maxUint or other preset end block height identifications), the block height of the last written historical block is obtained, that is, the new block data cannot be rewritten in the file, the block identification interval at this time may be an interval between the start block height and the end block height, for example, the start block height is 3, the end block height is 5, the block identification interval at this time is exemplified by block 3-block 5, when the end block height is the preset block identification (maxUint or other preset end block height identifications), the file is not yet full of the block data, the block height of the last written historical block is obtained, for example, the block height of the start block height and the last written historical block is taken as the block identification interval, for example, the start block height is maxUint, the block height of the last written block is 6, and the block height of the last written block is 3-6.

After the block identification interval is identified, the file mapping relationship can be converted into a block mapping relationship corresponding to the block identification in the block identification interval. The file map may include a file spatial map, a file block map, and a spatial block map. The spatial mapping relationship of the file may include a mapping relationship between the file identifier and the spatial identifier, the block mapping relationship of the file may include a mapping relationship between the file identifier and block identifier information of the file, and the spatial block mapping relationship may include a mapping relationship between the spatial identifier and block identifier information of the file. The mode of converting the file mapping relationship into the block mapping relationship corresponding to the block identifier in the block identifier interval may be various, for example, an initial file mapping relationship between the block identifier and the file identifier may be identified in the file block mapping relationship, an initial spatial mapping relationship between the block identifier and the spatial identifier may be identified in the spatial block mapping relationship, and the initial file mapping relationship, the initial spatial mapping relationship and the file spatial mapping relationship are fused to obtain the block mapping relationship.

The method for identifying the initial file mapping relationship between the block identifier and the file identifier in the file block mapping relationship may be various, for example, based on the file block mapping relationship, determining a candidate file identifier corresponding to the block identifier interval, using the candidate file identifier as a file identifier corresponding to the block identifier in the block identifier interval, thereby obtaining a target file identifier corresponding to each block identifier, and determining the initial file mapping relationship based on the block identifier and the target file identifier corresponding to the block identifier. For example, taking the block identifier interval as the block 3-block 5 and the corresponding candidate file identifier as the file a as an example, the file corresponding to the block 3 is the file a, the file corresponding to the block 4 is the file a, and the file corresponding to the block 5 is the file a, so as to obtain the initial file mapping relationship between the block identifier and the file identifier of each block.

The method for identifying the initial spatial mapping relationship between the block identifier and the spatial identifier in the spatial block mapping relationship is similar to the method for identifying the initial file mapping relationship, which is described in detail above, and will not be described in detail here.

After the initial file mapping relation and the initial space mapping relation are identified, the initial file mapping relation, the initial space mapping relation and the file space mapping relation can be fused to the block mapping relation. The method of fusing the initial file mapping relationship, the initial spatial mapping relationship and the file spatial mapping relationship may be various, for example, the initial file mapping relationship and the initial spatial mapping relationship may be associated to obtain a spatial identifier corresponding to each block identifier and a file identifier corresponding to the block identifier, the spatial identifier and the file identifier are associated based on the file spatial mapping relationship, so as to obtain file metadata (file identifier and corresponding spatial identifier) corresponding to each block identifier, and the block mapping relationship is constructed based on the block identifier and the corresponding file metadata.

After fusing the initial file mapping relation, the initial space mapping relation and the file space mapping relation, the obtained block mapping relation can be fused, and a block data index corresponding to the current node is constructed. There are various ways of constructing the block data index, for example, the block identifier may be used as a key value, the file identifier, the space identifier and the block identifier interval corresponding to the block identifier respectively are used as leaf nodes, the search tree is constructed based on the block mapping relationship, the key value and the leaf nodes, and the search tree is used as the block data index.

The search tree is understood to be tree data which contains keys and leaf nodes and satisfies the block mapping relation. The search tree may be of various types, and may include, for example, a B-tree (self-balancing tree), a binary ordering tree, a balanced tree, a red-black tree, or other types of tree data, and so forth. For example, taking a search tree as a B-tree as an example, assuming that one file stores 1000MB of data, 1PB of total data needs 1000000 (one million files), and searching file metadata information from one million file objects according to block heights, a data structure of a corresponding block data index may be as shown in fig. 7, supporting 1PB of data, where the time complexity is read at O (m), and m is generally less than 10.

104. When the block data of the block to be stored is acquired, determining a target file and a target storage space corresponding to the target file based on the block data index and the current block identification of the block to be stored.

For example, when obtaining the block data of the block to be stored, the current block identifier of the block to be stored may be obtained, the current file identifier and the current space identifier corresponding to the current block identifier are queried in the block data index, and the target file corresponding to the block to be stored and the target storage space corresponding to the target file are determined based on the query result and the block data index.

The method for obtaining the block data of the block to be stored may be various, for example, the block to be stored distributed by the blockchain system may be directly received, the block to be stored may be serialized to obtain a byte stream of the block to be stored, the byte stream may be used as the block data of the block to be stored, or the block data of the area to be stored uploaded or sent by the terminal may be directly received, and so on.

Serialization, among other things, may refer to the process of converting a data structure or object state into a usable format (e.g., stored as a file, buffered, or transmitted over a network) to allow for subsequent restoration of the original state in the same or another computer environment.

The method includes that a target file corresponding to a block to be stored and a target storage space corresponding to the target file are determined based on a query result and a block data index, for example, when a current file identifier and a current space identifier exist, a file corresponding to the current file identifier is used as the target file, a storage space corresponding to the current space identifier is used as a target storage space corresponding to the target file, or when the current file identifier and the current space identifier do not exist, a last written target historical block is determined based on the current block identifier, a candidate file written by the target historical block is searched in the block data index, an ending block identifier of the candidate file is determined, when the ending block identifier is a preset block identifier, the candidate file is used as the target file, the current space identifier corresponding to the file identifier of the target file is screened out from the block data index, and the storage space corresponding to the current space identifier is used as the target storage space corresponding to the target file.

When the current file identifier and the current space identifier do not exist, the block data index can be updated based on the current block identifier, the file identifier of the target file and the space identifier of the target storage space after the target file and the target storage space are determined, so that an updated block data index is obtained, and the updated block data index is used as the block data index.

Optionally, in some embodiments, after determining the target file and the target storage space corresponding to the target file based on the block data index and the current block identifier of the block to be stored, the remaining storage space of the target file may be detected, and the creation of a new file may be triggered. There are various ways of creating the new file, for example, the remaining storage capacity of the target file may be obtained, the remaining storage capacity is compared with the storage capacity occupied by the block data, when the storage capacity exceeds the remaining storage capacity, a candidate file is created, file metadata corresponding to the candidate file is determined, the block data index is updated based on the file metadata, and the updated block data index is used as the block data index.

When the storage capacity exceeds the remaining storage capacity, it means that the target node is full after writing the block data, and cannot write new block data, at this time, a candidate file (new file) needs to be newly added or created in order that the block data of the next block can be stored. There are various ways of creating the candidate file, for example, a history file corresponding to the preset block identifier may be queried in the block data index, when the history file exists, the preset block identifier in the history file is adjusted to be the current block identifier, the candidate file is created based on the current block identifier, and when the history file does not exist, the candidate file is created based on the current block identifier.

The history file may be understood as a file in which the last block before the candidate file is written with block data. The preset block identifier may be a preset end block height that is used to indicate that the file has not been completely written, for example, maxUint64 or other preset block identifiers. Since a candidate file needs to be newly created, which indicates that the history file is full, for the file that is full, the ending block height should be the block height of the last written block, rather than the preset block identifier. Therefore, when the history file exists, the preset block identifier in the history file can be adjusted to be the current block identifier, and then the candidate file is created based on the current block identifier. When the history file exists, it is indicated that there is an error or abnormality in the ending block height in the history file. The cause of such errors or anomalies may be various, and may include, for example, a server or process crash or the presence of a bug (bug), etc.

In some embodiments, when there is no history file, meaning that there is no error or abnormality in the ending block height in the previous file of the candidate file, the candidate file may be created based on the current block identification.

The method for creating the candidate file based on the current block identifier may be various, for example, a new file may be created, and the candidate file identifier corresponding to the new file may be determined based on the current block identifier, so as to obtain the candidate file, or the candidate file identifier of the candidate file may be determined based on the current block identifier, and a new file may be created, so as to obtain the candidate file.

The method for determining the candidate file identifier based on the current block identifier may be various, for example, a next block identifier of the current block identifier may be used as the candidate block identifier, and the candidate file identifier may be generated based on the candidate block identifier and the file identifier template. For example, taking the current block identification as block 3 as an example, the next block identification may be block 4.

After creating the candidate file, file metadata corresponding to the candidate file may be determined. The file metadata may include a file identification, a start block identification (start block high) within the candidate file, an end block identification (end block high), a spatial identification of the storage space to which the candidate file corresponds, and so on. The method for determining the file metadata of the candidate file may be various, for example, the method may include obtaining the storage space information of the current node, determining the candidate space identifier corresponding to the candidate file based on the storage space information, setting the start block identifier and the end block identifier corresponding to the candidate file according to the current block identifier, and using the candidate file identifier, the start block identifier, the end block identifier and the candidate space identifier as the file metadata may be specifically as follows:

(1) And acquiring storage space information of the current node, and determining a candidate space identifier corresponding to the candidate file based on the storage space information.

The storage space information may be information related to the storage space of the current node, for example, may include a space identifier set of all storage spaces of the current node, a storage capacity of each storage space, a remaining capacity, or ordering information of each storage space, and so on.

The method for obtaining the storage space information of the current node may be various, and specifically may be as follows:

for example, node attribute information of the current node may be acquired, storage space information may be extracted from the node attribute information, or attribute information corresponding to the storage space of the current node may also be acquired, storage space information may be extracted from the attribute information, and so on.

After the storage space information of the current node is acquired, the candidate space identifier corresponding to the candidate file can be determined based on the storage space information. The method for determining the candidate space identifier corresponding to the candidate file may be various based on the storage space information, for example, dictionary ordering information of a storage space set corresponding to the block data index may be extracted from the storage space information, candidate storage spaces corresponding to the candidate file may be screened out from the storage space set based on the dictionary ordering information to obtain the candidate space identifier of the candidate storage space, or the current remaining storage capacity of each storage space in the storage space set corresponding to the block data index may be extracted from the storage space information, the storage spaces may be ordered based on the current remaining storage capacity, and candidate storage spaces corresponding to the candidate file may be screened out from the storage spaces according to the ordering result to obtain the candidate storage space identifier of the candidate storage space.

The dictionary ordering information may be understood as information for ordering the spatial identifiers of the storage spaces in the storage space set based on a dictionary order. For example, taking the space identifiers of the storage space sets configured in the storage configuration file as examples, respectively, the storage space sets can be ordered according to a word order to obtain an ordering result of the disk1/disk2/disk3, and then the storage space identifiers in the storage configuration file are modified to OnlineFs:/disk1,/disk2,/disk3 to obtain dictionary ordering information, which can be shown in fig. 8 specifically.

After the dictionary ordering information is extracted, candidate storage spaces corresponding to the candidate files can be screened out from the storage space set based on the dictionary ordering information. The candidate storage space can be screened in various modes, for example, the residual storage capacity of the historical storage space stored by the last file of the candidate file is extracted from the storage space information, when the residual storage capacity of the historical storage space is smaller than a preset capacity threshold value, the next storage space of the historical storage space can be used as the candidate storage space based on dictionary ordering information, so that the candidate space identification of the candidate storage space can be obtained, and when the residual storage capacity of the historical storage space is larger than the preset capacity threshold value, the historical storage space can be used as the candidate storage space, so that the candidate space identification of the candidate storage space can be obtained. For example, taking the storage space of the current node including/disk 1,/disk 2 and/disk 3 as an example, when the residual storage capacity in/disk 1 is smaller than the preset capacity threshold, the method can switch to the next storage space (i.e.,/disk 2) of/disk 1 based on the dictionary ordering information, take/disk 2 as the candidate storage space corresponding to the candidate file, after/disk 2 is fully written, the method can switch to the next storage space (i.e.,/disk 3) of/disk 2, and so on, as shown in fig. 9, and when the residual storage capacity in/disk 1 is larger than the preset capacity threshold, the method also means that the file can be continuously stored in/disk 1 at this time, and the method can continuously identify the candidate space of the candidate storage space corresponding to the candidate file.

It should be noted that, because the scheme supports the newly added storage space, the storage space of the current node has a function of transversely expanding, so that the upper limit of the whole capacity of the storage space is improved, the storage flexibility of the block data of the current node can be greatly increased, and the block chain can not stop working because the storage space is full, so that the storage efficiency of the block storage can be greatly improved.

The current remaining storage capacity may be a remaining capacity of a storage space corresponding when the candidate file is created. And sequencing the storage spaces based on the current residual storage capacity, thereby obtaining sequencing results of the storage spaces. And then, screening candidate storage spaces corresponding to the candidate files from the storage space set based on the sorting result to obtain candidate space identifiers of the candidate storage spaces. The method for screening the candidate storage space may be various, for example, based on the sorting result, the storage space with the largest current remaining storage capacity may be screened out from the storage space set to obtain the candidate storage space, and the candidate space identifier of the candidate storage space may be determined, or the storage space with the smallest current remaining storage capacity may be screened out from the storage space set to obtain the candidate storage space, and the candidate space identifier of the candidate storage space may be determined, and so on.

Taking the storage space set including/disk 1,/disk 2 and/disk 3,/disk2 with the largest current residual capacity and/disk 1 with the smallest current residual capacity as an example, when the candidate storage space is the storage space with the largest current residual capacity, the storage space with the largest current residual capacity (next_filesystem=max (freeDisk 1, freeDisk2, freeDisk 3)) can be screened out from the storage space set, so that/disk 2 is used as the candidate space identifier of the candidate storage space, specifically as shown in fig. 10, and when the candidate storage space is the storage space with the smallest current residual capacity, the storage space with the lowest current residual capacity can be screened out from the storage space set, thereby taking/disk 1 as the candidate space identifier of the candidate storage space.

(2) And setting a start block identifier and an end block identifier corresponding to the candidate file according to the current block identifier.

The start block identifier may be the block identifier of the first block set to be stored, or may be understood as the block identifier of the first block set to be stored in the candidate file. The block identification here may be a block height (block height). The ending block identifier may be a block identifier of a last block for which ending storage is set, and when the candidate file is not full of block data, the ending block may be a preset block identifier, for example, may be maxUint64 or other preset block identifiers, and so on.

The ways of setting the start block identifier and the end block identifier corresponding to the candidate file according to the current block identifier may be various, and specifically may be as follows:

for example, the next block identifier of the current block identifier may be used as the start block identifier of the candidate file, and the preset block identifier may be used as the end block identifier of the candidate file.

Taking the current block identifier as the block 3 and the preset block identifier as maxUint and 64 as examples, the starting block identifier corresponding to the candidate file may be the block 4, and the ending block identifier corresponding to the candidate file may be maxUint and 64.

(3) And taking the candidate file identification, the starting block identification, the ending block identification and the candidate space identification as file metadata.

For example, the start block identifier and the end block identifier may be used as block identifier information of the candidate file, and the block identifier information, the file identifier, and the candidate space identifier may be used as file metadata corresponding to the candidate file.

After determining the file metadata corresponding to the candidate file, the block data index may be updated based on the file metadata, for example, the file identifier of the candidate file may be used as a key, the candidate space identifier and the block identifier information may be used as a value, so that the file identifier, the candidate space identifier and the block identifier information are permanently written into the metadata set to obtain a target metadata set, the current block mapping relationship is identified in the target metadata set, the block data index is updated based on the current block mapping relationship, and the updated block data index is used as the block data index.

After updating the block data index and taking the updated block data index as the block data index, after obtaining the block data of the block to be stored, the step of determining the target file and the target storage space corresponding to the target file based on the block data index and the current block identifier of the block to be stored can be performed back, so that the block data can be written into the file of the storage space based on the block data index for all the block data of the block to be stored, and the block storage efficiency is improved.

The newly created candidate file is mainly used for writing the block data of the next block of the block to be stored when the target file corresponding to the block data of the block to be stored is full. For the time of creating the candidate file, not only the candidate file may be created after the target file corresponding to the block to be stored is determined, but also the candidate file may be created after the block data of the block to be stored is written into the target file. The overall flow of creating a candidate file may be as shown in fig. 11, where after the creating, it may be detected whether there is a previous file (i.e. a target file) whose end block identifier is a preset block identifier (maxUint) and when there is a previous file, the target file may be found, and the end block height (ENDHEIGHT) of the target file may be modified to the block height of the current latest block (i.e. the current block identifier), then a new file may be created and a file name (file identifier) may be specified, and when there is no previous file, a new file may be directly created and a file name may be specified. After creating a new file, a spatial identifier of a storage space corresponding to the new file is acquired, then a start block height (STARTHEIGHT) and an end block height (ENDHEIGHT) of the new file are set, then persistent writing is performed to the metadata set, and finally the block data index (memory index) is updated.

105. And writing the block data into the target file in the target storage space.

For example, the target file may be stored in a target storage space, and then, the block data may be written to the target file in the target storage space, thereby completing the block storage of the block to be stored, or the block data may be written to the target file, the target file written to the block data may be stored in the target storage space, thereby completing the block storage of the block to be stored, and so on.

Optionally, in some embodiments, after the current node does not have the new storage space, the current node may query the current block data index, store the block to be stored based on the current block data index when the current node has the current block data index, construct the block data index based on the metadata set when the current node does not have the current block data index, and store the block to be stored based on the current block data index.

The current block data index may be stored in the cache, and may not be available in the current node when the process is started or restarted. When the current block data index does not exist, the current block data index needs to be constructed based on the metadata set, and the manner of constructing the current block data index is similar to that of constructing the block data index, which is described in detail above, and will not be described in detail here.

After the current block data index exists or the current block data index is constructed based on the metadata set, after the block data of the block to be stored is acquired, the target file and the target storage space corresponding to the target file are determined based on the current block data index and the current block of the block to be stored, and the block data is written into the target file in the target storage space, which is described in detail above, and will not be described in detail here.

For block storage, the present solution may implement block storage and lateral expansion of storage space in the block storage process by using a metadata system, where the metadata system may include a storage configuration file, a block data index, and a metadata set (Db), and may be specifically shown in fig. 12. The storage configuration file may include a space identification and version number of the configured storage space. The block data index may be an index of a B-tree structure, with a block height (height) as a key, fileMeta as a value, and according to the block height, metadata information such as a file identifier corresponding to the block height and a space identifier of a storage space can be queried, where the metadata set is also structural data of a key/value, the file identifier is a key, and fileMeta serialized data is a value. The metadata collection may be used for persistent metadata, and when the metadata system/process is started or restarted, metadata information may be read directly from the metadata collection, and a chunk data index may be constructed based on the read metadata information. When the block data of the block to be stored is acquired, the block data can be stored based on the block data index.

When the metadata system is started or restarted, the metadata system may also read the current storage configuration file, check for errors and update, then read metadata information in the metadata set, and construct a block data index based on the read metadata information, as shown in fig. 13. In addition, the metadata system may be deployed in one or more nodes in a blockchain system to enable blockwise storage of the one or more nodes.

In this scheme, the metadata set may be updated by adding a storage configuration file of the storage space, and the block data index may be constructed or updated. In addition, a file can be newly built in the storage space, the newly built file is switched to the corresponding storage space, block identification information of the newly built file is set, the information is persisted to a metadata set, and a block data index is built or updated based on the metadata set, so that the storage space has the capability of transversely expanding, a new storage space can be added under the condition that a single storage space (file system/folder/disk/hard disk) is not enough, intelligent switching is performed on the storage space, and the upper limit of the whole capacity of the storage space is improved. Particularly, in the case of cloud deployment, the method is more convenient and simpler to use, and the performance is the same as that of using a single storage space, so that the efficiency of block storage can be improved.

As can be seen from the above, in the embodiment of the present application, after the current storage configuration file is obtained and compared with the corresponding historical storage configuration file, when the comparison result indicates that the current node has a new storage space, the metadata set is updated based on the space identifier of the new storage space, the block mapping relationship is identified in the updated metadata set to obtain the block data index, when the block data of the block to be stored is obtained, the target storage space corresponding to the target file and the target file is determined based on the block data index and the current block identifier of the block to be stored, and the block data is written into the target storage space.

According to the method described in the above embodiments, examples are described in further detail below.

In this embodiment, the block storage device is specifically integrated in an electronic device, the electronic device is a server, the block is identified as a block height (block height), the preset block is identified as maxUint64, and the storage space is illustrated as a file system.

As shown in fig. 14, a block storage method specifically includes the following steps:

201. the server obtains the current storage configuration file.

For example, when receiving a storage configuration file uploaded by a terminal, the server may directly use the uploaded storage configuration file as a current storage configuration query, or may query a preset storage area for a storage configuration file with the modification time closest to the current time, use the queried storage configuration file as the current storage configuration file, or may send storage early warning information to a terminal corresponding to the current node when the storage capacity of a file system of the current node is smaller than a preset storage capacity threshold, receive a storage configuration file returned by the terminal based on the space early warning information, use the returned storage configuration file as the current storage configuration file, and so on.

202. The server compares the current storage configuration file with the corresponding historical storage configuration file.

For example, the server extracts a current file system identifier set and a current version number from the current storage configuration file, obtains a history storage configuration file set of the current node, screens a history storage configuration file corresponding to the current storage configuration file from the history storage configuration file set based on the current version number, obtains a space identifier in a space identifier set in the history storage configuration file, obtains a history space identifier, and queries the history space identifier in the current space identifier set.

When the current space identification set contains the history space identification, the server determines that the configuration of the current storage configuration file is correct, compares the space identification in the current identification set with the history space identification, determines that the current node has a newly added file system when the space identification outside the history space identification exists in the current space identification set, and determines that the current node does not have the newly added file system when the space identification outside the history space identification does not exist in the current space identification set.

203. And when the comparison result indicates that the current node has the newly added file system, the server updates the metadata set based on the space identification of the newly added file system.

For example, when the comparison result indicates that the current node has a new file system, the server may acquire a spatial identifier of the new file system, and add the spatial identifier as metadata to the metadata set to obtain an updated metadata set.

204. The server identifies a block mapping relationship in the updated metadata set to obtain a block data index.

For example, the server acquires the block height information of the file corresponding to each file identifier, extracts the start block height and the end block height from the block height information, and when the end block height is not maxUint64, the block height section at this time may be a section between the start block height and the end block height, and when the end block height is maxUint, acquires the block height of the last written history block, and uses the block heights of the start block height and the history block as the block height section.

The server identifies an initial file mapping relation between the block height and the file identifier in the file block mapping relation, identifies an initial space mapping relation between the block height and the space identifier in the space block mapping relation, and fuses the initial file mapping relation, the initial space mapping relation and the file space mapping relation to obtain the block mapping relation.

The server correlates the initial file mapping relation with the initial space mapping relation to obtain a space identifier corresponding to each block height and a file identifier corresponding to the block height, correlates the space identifier with the file identifier based on the file space mapping relation, so as to obtain file metadata (file identifier and corresponding space identifier) corresponding to each block height, and constructs a block mapping relation based on the block height and the corresponding file metadata.

The server may use the block height as a key value, use a file identifier, a space identifier and a block height section corresponding to the block height as leaf nodes, construct a search tree (B-tree) based on the block mapping relationship, the key value and the leaf nodes, and use the search tree as a block data index.

205. And when the comparison result indicates that the current node does not have the newly added file system, the server constructs a block data index based on the metadata set.

For example, when the comparison result indicates that the newly added file system does not exist currently, the server queries the current block data index at the current node, takes the current block data index as the block data index when the current block data index exists at the current node, and constructs the block data index based on the metadata set when the current block data index does not exist at the current node.

206. When the server acquires the block data of the block to be stored, determining a target file and a target file system corresponding to the target file based on the block data index and the current block height of the block to be stored.

For example, the server may receive the block to be stored distributed by the blockchain system, sequence the block to be stored to obtain a byte stream of the block to be stored, use the byte stream as the block data of the block to be stored, or directly receive the block data of the area to be stored uploaded or sent by the terminal, and so on.

The server may query the block data index for the current file identifier and the current space identifier corresponding to the current block height, when the current file identifier and the current space identifier exist, take the file corresponding to the current file identifier as a target file, take the file system corresponding to the current space identifier as a target file system corresponding to the target file, or when the current file identifier and the current space identifier do not exist, determine the last written target historical block based on the current block height, retrieve the candidate file written by the target historical block in the block data index, determine the ending block height of the candidate file, take the candidate file as the target file when the ending block height is maxUint, screen the file system corresponding to the current space identifier as the target file system corresponding to the target file in the block data index.

When the current file identifier and the current space identifier do not exist, the server can update the block data index based on the current block height, the file identifier of the target file and the space identifier of the target file system after determining the target file and the target file system, so as to obtain an updated block data index, and the updated block data index is used as the block data index.

In some embodiments, after determining the target file and the target file system corresponding to the target file based on the block data index and the current block height of the block to be stored, the server may further obtain a remaining storage capacity of the target file, compare the remaining storage capacity with a storage capacity occupied by the block data, query maxUint a corresponding historical file in the block data index when the storage capacity exceeds the remaining storage capacity, adjust maxUint a in the historical file to be the current block height when the historical file exists, create a new file based on the current block height, and determine a candidate file identifier corresponding to the new file based on the current block height, thereby obtaining the candidate file, or determine a candidate file identifier of the candidate file based on the current block height, and create a new file to obtain the candidate file.

In some embodiments, when the history file does not exist, the server may directly create the candidate file, and the process of creating the candidate file may be described above, which is not described in detail herein.

After creating the candidate file, the server can use the next block height of the current block height as the candidate block height, and generate a candidate file identifier based on the candidate block height and the file identifier template.

The server may acquire node attribute information of the current node, extract file system information from the node attribute information, or may also acquire attribute information corresponding to a file system of the current node, extract file system information from the attribute information, and so on.

The server extracts dictionary ordering information of a file system set corresponding to the block data index from file system information, extracts the residual storage capacity of a historical file system stored by a previous file of a candidate file from the file system information, when the residual storage capacity of the historical file system is smaller than a preset capacity threshold, the next file system of the historical file system can be used as the candidate file system based on the dictionary ordering information, so as to obtain a candidate space identifier of the candidate file system, when the residual storage capacity of the historical file system is larger than the preset capacity threshold, the historical file system can be used as the candidate file system, so as to obtain the candidate space identifier of the candidate file system, or the current residual storage capacity of each file system in the file system set corresponding to the block data index can be extracted from the file system information, the file system is ordered based on the current residual storage capacity, the candidate file system is selected from the file system set based on the ordering result, the candidate space identifier of the candidate file system with the largest current residual storage capacity is obtained, or the file system with the smallest residual storage capacity can be selected from the file system set, so as to obtain the candidate space identifier of the candidate file system, and the candidate file system is determined.

The server may take the next block height of the current block height as the starting block height of the candidate file and maxUint64 as the ending block height of the candidate file. Then, the start block height and the end block height are used as block height information of candidate files, and the block height information, the file identification and the candidate space identification are used as file metadata corresponding to the candidate files.

The server can take the file identification of the candidate file as a key and take the candidate space identification and the block height information as values, so that the file identification, the candidate space identification and the block height information are permanently written into a metadata set to obtain a target metadata set, a current block mapping relation is identified in the target metadata set, the block data index is updated based on the current block mapping relation, and the updated block data index is taken as the block data index.

After the server updates the block data index and takes the updated block data index as the block data index, after the block data of the block to be stored is obtained, the server can return to execute the step of determining the target file and the target file system corresponding to the target file based on the block data index and the current block height of the block to be stored.

207. The server writes the block data to the target file in the target file system.

For example, the server may store the target file into a target file system, then write the block data into the target file in the target file system, thereby completing the block storage of the block to be stored, or write the block data into the target file, thereby completing the block storage of the block to be stored, etc. as can be seen from the above, the server of the embodiment obtains the current storage configuration file, compares the current storage configuration file with the corresponding historical storage configuration file, when the comparison result indicates that the current node has a newly added file system, updates the metadata set based on the spatial identifier of the newly added file system, identifies the block mapping relationship in the updated metadata set, so as to obtain the block data index, when the block data of the block to be stored is obtained, determines the target file and the target file system corresponding to the target file based on the block data index and the current block height of the block to be stored, and writes the block data into the target file in the target file system, and as can manage the block data set and the block data index and the file system according to the current storage configuration file, and the file system and the file indexing relationship between the block data and the file system can be stored, thereby realizing the file expansion.

In order to better implement the above method, the embodiment of the present invention further provides a block storage device, where the block storage device may be integrated in an electronic device, such as a server or a terminal, where the terminal may include a tablet computer, a notebook computer, and/or a personal computer.

For example, as shown in fig. 15, the block storage device may include an acquisition unit 301, an update unit 302, an identification unit 303, a determination unit 304, and a writing unit 305, as follows:

(1) An acquisition unit 301;

The obtaining unit 301 is configured to obtain a current storage configuration file, and compare the current storage configuration file with a corresponding historical storage configuration file.

For example, the obtaining unit 301 may specifically be configured to obtain a current storage configuration file, query a current space identifier set of the current storage configuration file for a history space identifier in the history storage configuration file, determine that the current storage configuration file is configured correctly when the current space identifier set includes the history space identifier, compare the space identifier in the current identifier set with the history space identifier, determine that a new storage space exists in the current node when a space identifier other than the history space identifier exists in the current space identifier set, and determine that the current node does not have the new storage space when a space identifier other than the history space identifier does not exist in the current space identifier set.

(2) An updating unit 302;

and an updating unit 302, configured to update, when the comparison result indicates that the current node has a new storage space, a metadata set based on a space identifier of the new storage space, where the metadata set indicates a file mapping relationship between the space identifier, a file identifier, and block identifier information of the file.

For example, the updating unit 302 may be specifically configured to obtain, when the comparison result indicates that the current node has a new storage space, a space identifier of the new storage space, and add the space identifier as metadata to the metadata set, to obtain an updated metadata set.

(3) An identification unit 303;

the identifying unit 303 is configured to identify a block mapping relationship in the updated metadata set to obtain a block data index, where the block mapping relationship includes a mapping relationship between a block identifier in the block identifier information and a file identifier and a space identifier, respectively.

For example, the identifying unit 303 may specifically be configured to identify, in the block identifier information, a block identifier section of each file corresponding to the file identifier, convert the file mapping relationship into a block mapping relationship corresponding to the block identifier, and construct a block data index corresponding to the current node according to the block mapping relationship.

(4) A determination unit 304;

The determining unit 304 is configured to determine, when block data of the block to be stored is acquired, the target file and a target storage space corresponding to the target file based on the block data index and a current block identifier of the block to be stored.

For example, the determining unit 304 may be specifically configured to, when obtaining the block data of the block to be stored, obtain a current block identifier of the block to be stored, query a current file identifier and a current space identifier corresponding to the current block identifier in the block data index, and determine, based on the query result and the block data index, a target file corresponding to the block to be stored and a target storage space corresponding to the target file.

(5) A writing unit 305;

a writing unit 305, configured to write the block data into the target file in the target storage space.

For example, the writing unit 305 may specifically be configured to store the target file into the target storage space, and then write the block data into the target file in the target storage space, thereby completing the block storage of the block to be stored, or may also write the block data into the target file, store the target file into the target storage space, thereby completing the block storage of the block to be stored, and so on.

Optionally, in some embodiments, the block storage device may further include a creating unit 306, as shown in fig. 16, specifically may be as follows:

the creating unit 306 is configured to create a candidate file, and update the block data index based on file data corresponding to the candidate file.

For example, the creating unit 306 may specifically be configured to obtain a remaining storage capacity of the target file, compare the remaining storage capacity with a storage capacity occupied by the block data, create a candidate file when the storage capacity exceeds the remaining storage capacity, determine file metadata corresponding to the candidate file, update the block data index based on the file metadata, and use the updated block data index as the block data index.

In the implementation, each unit may be implemented as an independent entity, or may be implemented as the same entity or several entities in any combination, and the implementation of each unit may be referred to the foregoing method embodiment, which is not described herein again.

As can be seen from the foregoing, in this embodiment, after the obtaining unit 301 obtains the current storage configuration file and compares the current storage configuration file with the corresponding historical storage configuration file, the updating unit 302 updates the metadata set based on the space identifier of the newly added storage space when the comparison result indicates that the newly added storage space exists in the current node, the identifying unit 303 identifies the block mapping relationship in the updated metadata set to obtain the block data index, the determining unit 304 determines the target storage space corresponding to the target file and the target file based on the block data index and the current block identifier of the block to be stored when the block data to be stored is obtained, and the writing unit 305 writes the block data into the target file in the target storage space.

The embodiment of the invention also provides an electronic device, as shown in fig. 17, which shows a schematic structural diagram of the electronic device according to the embodiment of the invention, specifically:

the electronic device may include one or more processing cores 'processors 401, one or more computer-readable storage media's memory 402, power supply 403, and input unit 404, among other components. It will be appreciated by those skilled in the art that the electronic device structure shown in fig. 17 is not limiting of the electronic device and may include more or fewer components than shown, or may combine certain components, or a different arrangement of components. Wherein:

The processor 401 is a control center of the electronic device, connects various parts of the entire electronic device using various interfaces and lines, and performs various functions of the electronic device and processes data by running or executing software programs and/or modules stored in the memory 402, and calling data stored in the memory 402. Optionally, the processor 401 may include one or more processing cores, and preferably the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, an application program, etc., and the modem processor mainly processes wireless communication. It will be appreciated that the modem processor described above may not be integrated into the processor 401.

The memory 402 may be used to store software programs and modules, and the processor 401 executes various functional applications and data processing by executing the software programs and modules stored in the memory 402. The memory 402 may mainly include a storage program area that may store an operating system, application programs required for at least one function (such as a sound playing function, an image playing function, etc.), etc., and a storage data area that may store data created according to the use of the electronic device, etc. In addition, memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

The electronic device further comprises a power supply 403 for supplying power to the various components, preferably the power supply 403 may be logically connected to the processor 401 by a power management system, so that functions of managing charging, discharging, and power consumption are performed by the power management system. The power supply 403 may also include one or more of any of a direct current or alternating current power supply, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and the like.

The electronic device may further comprise an input unit 404, which input unit 404 may be used for receiving input digital or character information and generating keyboard, mouse, joystick, optical or trackball signal inputs in connection with user settings and function control.

Although not shown, the electronic device may further include a display unit or the like, which is not described herein. In particular, in this embodiment, the processor 401 in the electronic device loads executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 executes the application programs stored in the memory 402, so as to implement various functions as follows:

For example, the electronic device may take a current storage configuration file, query a current space identifier set of the current storage configuration file for a history space identifier in the history storage configuration file, determine that the current storage configuration file is configured correctly when the current space identifier set includes the history space identifier, compare the space identifier in the current identifier set with the history space identifier, determine that a new storage space exists in the current node when a space identifier other than the history space identifier exists in the current space identifier set, and determine that the current node does not have the new storage space when the current space identifier set does not have the space identifier other than the history space identifier. When the comparison result indicates that the current node has a new storage space, a space identifier of the new storage space can be acquired, and the space identifier is used as metadata to be added to the metadata set to obtain an updated metadata set. And identifying a block identification interval of each file corresponding to the file identification in the block identification information, converting the file mapping relation into a block mapping relation corresponding to the block identification, and constructing a block data index corresponding to the current node according to the block mapping relation. When the block data of the block to be stored is obtained, a current block identifier of the block to be stored is obtained, a current file identifier and a current space identifier corresponding to the current block identifier are queried in a block data index, and a target file corresponding to the block to be stored and a target storage space corresponding to the target file are determined based on a query result and the block data index. And writing the block data into the target file, and storing the target file written with the block data into the target storage space, thereby completing the block storage of the block to be stored. After determining the target file and the target storage space corresponding to the target file, the residual storage capacity of the target file can be obtained, the residual storage capacity is compared with the storage capacity occupied by the block data, when the storage capacity exceeds the residual storage capacity, a candidate file is created, file metadata corresponding to the candidate file is determined, the block data index is updated based on the file metadata, and the updated block data index is used as the block data index.

The specific implementation of each operation may be referred to the previous embodiments, and will not be described herein.

As can be seen from the above, in the embodiment of the present invention, after the current storage configuration file is obtained and compared with the corresponding historical storage configuration file, when the comparison result indicates that the current node has a new storage space, the metadata set is updated based on the space identifier of the new storage space, the block mapping relationship is identified in the updated metadata set to obtain the block data index, when the block data of the block to be stored is obtained, the target storage space corresponding to the target file and the target file is determined based on the block data index and the current block identifier of the block to be stored, and the block data is written into the target storage space.

Those of ordinary skill in the art will appreciate that all or a portion of the steps of the various methods of the above embodiments may be performed by instructions, or by instructions controlling associated hardware, which may be stored in a computer-readable storage medium and loaded and executed by a processor.

To this end, an embodiment of the present invention provides a computer readable storage medium having stored therein a plurality of instructions capable of being loaded by a processor to perform the steps of any of the block storage methods provided by the embodiments of the present invention. For example, the instructions may perform the steps of:

The specific implementation of each operation above may be referred to the previous embodiments, and will not be described herein.

The computer readable storage medium may include, among others, read Only Memory (ROM), random access Memory (RAM, random Access Memory), magnetic or optical disks, and the like.

The steps in any block storage method provided by the embodiment of the present invention can be executed due to the instructions stored in the computer readable storage medium, so that the beneficial effects of any block storage method provided by the embodiment of the present invention can be achieved, which are detailed in the previous embodiments and are not described herein.

Wherein according to an aspect of the application, a computer program product or a computer program is provided, the computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The computer instructions are read from the computer-readable storage medium by a processor of a computer device, and executed by the processor, cause the computer device to perform the methods provided in various alternative implementations of the block storage aspect or the block data storage aspect described above.

The above description of the block storage method, the apparatus, the electronic device and the computer readable storage medium provided by the embodiments of the present invention has been provided in detail, and specific examples are provided herein to illustrate the principles and embodiments of the present invention, and the above description of the embodiments is only for aiding in understanding of the method and core concept of the present invention, and meanwhile, for those skilled in the art, according to the concept of the present invention, there are variations in the specific embodiments and application scope, so the disclosure should not be interpreted as limiting the invention.

Claims

1. A block storage method, comprising:

and writing the block data into the target file in the target storage space.

2. The method according to claim 1, wherein identifying a block mapping relationship in the updated metadata set to obtain a block data index comprises:

Identifying a block identification interval of each file corresponding to the file identification in the block identification information;

Converting the file mapping relation into the block mapping relation corresponding to the block identifier in the block identifier interval;

And constructing the block data index corresponding to the current node according to the block mapping relation.

3. The block storage method according to claim 2, wherein the file mapping relationship includes a file spatial mapping relationship, a file block mapping relationship, and a spatial block mapping relationship, and the converting the file mapping relationship into the block mapping relationship corresponding to the block identifier in the block identifier interval includes:

identifying an initial file mapping relation between the block identifier and the file identifier in the file block mapping relation;

Identifying an initial spatial mapping relation between the block identifier and the spatial identifier in the spatial block mapping relation;

and fusing the initial file mapping relation, the initial space mapping relation and the file space mapping relation to obtain the block mapping relation.

4. The block storage method according to claim 2, wherein the constructing the block data index corresponding to the current node according to the block mapping relationship includes:

taking the block identifier as a key value, and taking the file identifier, the space identifier and the block identifier interval respectively corresponding to the block identifier as leaf nodes;

and constructing a search tree based on the block mapping relation, the key value and the leaf node, and taking the search tree as the block data index.

5. The block storage method according to claim 1, wherein after determining a target file and a target storage space corresponding to the target file based on the block data index and the current block identifier of the block to be stored, the method further comprises:

obtaining the residual storage capacity of the target file, and comparing the residual storage capacity with the storage capacity occupied by the block data;

When the storage capacity exceeds the residual storage capacity, creating a candidate file, and determining file metadata corresponding to the candidate file;

And updating the block data index based on the file metadata, and taking the updated block data index as the block data index.

6. The block storage method of claim 5, wherein creating the candidate file comprises:

inquiring a history file corresponding to a preset block identifier in the block data index;

When the history file exists, adjusting the current block identifier by the preset block identifier in the history file, and creating the candidate file based on the current block identifier;

and when the history file does not exist, creating the candidate file based on the current block identification.

7. The block storage method of claim 5, wherein the determining file metadata corresponding to the candidate file comprises:

Acquiring storage space information of the current node, and determining a candidate space identifier corresponding to the candidate file based on the storage space information;

setting a start block identifier and an end block identifier corresponding to the candidate file according to the current block identifier;

And taking the candidate file identification, the starting block identification, the ending block identification and the candidate space identification as the file metadata.

8. The block storage method according to claim 7, wherein the determining the candidate space identifier corresponding to the candidate file based on the storage space information includes:

extracting dictionary ordering information of a storage space set corresponding to the block data index from the storage space information;

And screening out candidate storage spaces corresponding to the candidate files from the storage space set based on the dictionary ordering information so as to obtain candidate space identifiers of the candidate storage spaces.

9. The block storage method according to claim 7, wherein determining candidate space identifiers corresponding to candidate files based on the storage space information comprises:

Extracting the current residual storage capacity of each storage space in the storage space set corresponding to the block data index from the storage space information;

Sorting the storage spaces based on the current remaining storage capacity;

And screening the candidate storage space corresponding to the candidate file from the storage space set according to the sorting result to obtain a candidate space identification of the candidate storage space.

10. The block storage method of claim 1, wherein comparing the current storage profile with a corresponding historical storage profile comprises:

inquiring a history space identifier in the corresponding history storage configuration file in a current space identifier set of the current storage configuration file;

when the current space identifier set comprises the history space identifier, determining that the current storage configuration file is correctly configured, and comparing the space identifier in the current space identifier set with the history space identifier;

when the space identifiers except the history space identifiers exist in the current space identifier set, determining that a new storage space exists in the current node;

And when the space identifiers except the historical space identifiers do not exist in the current space identifier set, determining that the current node does not have a new storage space.

11. The block storage method of claim 10, wherein after determining that the current node does not have a new storage space, further comprising:

When the current node has a current block data index, storing a block to be stored based on the current block data index;

And when the block data index does not exist in the current node, constructing the block data index based on the metadata set, and storing the block to be stored based on the block data index.

12. A block storage device, comprising:

13. An electronic device comprising a processor and a memory, the memory storing an application, the processor being configured to run the application in the memory to perform the steps in the tile storage method of any one of claims 1 to 11.

14. A computer program product comprising computer programs/instructions which when executed by a processor implement the steps of the block storage method of any of claims 1 to 11.

15. A computer readable storage medium storing a plurality of instructions adapted to be loaded by a processor to perform the steps in the block storage method of any one of claims 1 to 11.