CN110046133A - A kind of metadata management method, the apparatus and system of storage file system - Google Patents

Abstract

The invention discloses a kind of metadata management methods of storage file system, are applied to directory path server, the target access object including receiving and determining according to the request of data that client is sent client；If target access object is directory metadata, reads directory metadata and directory metadata is returned into client；If target access object is attribute metadata, corresponding file system ID is sent to meta data server, so that meta data server reads corresponding attribute metadata and attribute metadata is returned to client.The metadata management method can ensure the high-performance and scalability of storage file system by meta data server management attribute metadata using directory path server admin directory metadata, while can effectively improve the access efficiency of metadata.The invention also discloses meta data management device, system and the computer readable storage mediums of a kind of storage file system, all have above-mentioned technical effect.

Description

A metadata management method, device and system for a storage file system

technical field

The present invention relates to the field of computer technology, in particular to a metadata management method for a storage file system; and also relates to a metadata management device, a system and a computer-readable storage medium for a storage file system.

Background technique

The data includes three parts: directory path metadata, file metadata and data objects. The directory path metadata includes the file name of the file and the file access control attribute, which constitute the namespace of the file system. The metadata of the file itself includes the properties of the file, such as file size, owner, last access time, location of data objects, etc. The current way of metadata management is to manage all kinds of metadata in a unified way through a single server. However, due to the different metadata content, nature and access frequency of each metadata, the storage management of metadata is inconsistent, and the access efficiency is low.

Therefore, how to provide a metadata management solution to improve metadata access efficiency is a technical problem to be solved urgently by those skilled in the art.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a metadata management method for a storage file system, which can improve the efficiency of metadata access and ensure the performance and scalability of the storage file system; another purpose of the present invention is to provide a metadata storage file system. The data management device, system and computer-readable storage medium all have the above technical effects.

In order to solve the above technical problems, the present invention provides a metadata management method for a storage file system, which is applied to a directory path server, including:

Receive data requests sent by clients;

Determine the target access object of the client according to the data request;

If the target access object is directory metadata, read the directory metadata, and return the directory metadata to the client;

If the target access object is attribute metadata, send the corresponding file system ID to the metadata server, so that the metadata server reads the corresponding attribute metadata and returns the attribute metadata to the metadata server. described client.

Optionally, the determining the target access object of the client according to the data request includes:

parsing the data request;

If the data request includes a full-path execution directory, determine that the target access object of the client is the directory metadata;

If the data request does not include the full-path execution directory, it is determined that the target access object of the client is the attribute metadata.

Optionally, the reading the catalog metadata includes:

Searching for the directory metadata in the cache of the directory path server with the full path name of the file as an index item;

If the directory metadata exists in the cache of the directory path server, read the directory metadata from the cache of the directory path server;

If the directory metadata does not exist in the cache of the directory path server, the directory metadata is read from the disk of the directory path server.

Optionally, the metadata server reads the corresponding attribute metadata, including:

Searching for the attribute metadata in the cache of the metadata server with the file system ID as an index item;

If the attribute metadata exists in the cache of the metadata server, read the attribute metadata from the cache of the metadata server;

If the attribute metadata does not exist in the cache of the metadata server, the attribute metadata is read from the disk of the metadata server.

In order to solve the above technical problems, the present invention also provides a metadata management device for a storage file system, including:

A determination module, configured to receive a data request sent by a client and determine a target access object of the client according to the data request;

a reading module, configured to read the directory metadata and return the directory metadata to the client if the target access object is directory metadata;

A sending module, configured to send the corresponding file system ID to the metadata server if the target access object is attribute metadata, so that the metadata server reads the corresponding attribute metadata and sends the attribute Metadata is returned to the client.

Optionally, the determining module includes:

a parsing unit for parsing the data request;

a determining unit, configured to determine that the target access object of the client is the directory metadata if the data request contains a full-path execution directory; if the data request does not contain the full-path execution directory, determine The target access object of the client is the attribute metadata.

Optionally, the reading module includes:

a search unit, configured to search for the directory metadata in the cache of the directory path server with the full path name of the file as an index item;

a first reading unit, configured to read the directory metadata from the cache of the directory path server if the directory metadata exists in the cache of the directory path server;

The second reading unit is configured to read the directory metadata from the disk of the directory path server if the directory metadata does not exist in the cache of the directory path server.

In order to solve the above technical problems, the present invention also provides a metadata management system for a storage file system, including:

The directory path server is used to receive the data request sent by the client and determine the target access object of the client according to the data request; if the target access object is directory metadata, read the directory metadata and store the The directory metadata is returned to the client; if the target access object is attribute metadata, then the corresponding file system ID is sent to the metadata server;

The metadata server is configured to read the corresponding attribute metadata and return the attribute metadata to the client.

In order to solve the above technical problems, the present invention also provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the storage as described in any of the above is implemented. Steps of a metadata management method for a file system.

The metadata management method for a storage file system provided by the present invention is applied to directory path metadata, including: receiving a data request sent by a client; determining a target access object of the client according to the data request; If the object is directory metadata, read the directory metadata and return the directory metadata to the client; if the target access object is attribute metadata, send the corresponding file system ID to the metadata server, so that the metadata server reads the corresponding attribute metadata and returns the attribute metadata to the client.

It can be seen that the metadata management method of the storage file system provided by the present invention performs hierarchical management on various metadata, adopts the directory path server to manage the directory metadata, and manages the attribute metadata through the metadata server, thereby ensuring the storage file system. high performance and scalability, and can accurately locate metadata, effectively improving the efficiency of metadata access.

The metadata management device, system and computer-readable medium for storing a file system provided by the present invention all have the above technical effects.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present invention more clearly, the following briefly introduces the prior art and the accompanying drawings required in the embodiments. Obviously, the drawings in the following description are only some of the present invention. In the embodiments, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without any creative effort.

1 is a schematic flowchart of a method for managing metadata of a storage file system according to an embodiment of the present invention;

2 is a schematic diagram of a metadata management device for a storage file system according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a metadata management system for a storage file system according to an embodiment of the present invention.

Detailed ways

The core of the present invention is to provide a metadata management method for a storage file system, which can improve the metadata access efficiency and ensure the high performance and easy expansion of the storage file system; another core of the present invention is to provide a storage file system. The metadata management device, system, and computer-readable storage medium all have the above technical effects.

In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments These are some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Please refer to FIG. 1, which is a schematic flowchart of a method for managing metadata of a storage file system according to an embodiment of the present invention; with reference to FIG. 1, the method for managing metadata includes:

S100: Receive a data request sent by the client;

S200: Determine the target access object of the client according to the data request;

Specifically, for the directory metadata and attribute metadata, this embodiment adopts a hierarchical management method, that is, the directory metadata service is completed by managing the directory metadata through the directory path server. Manage attribute metadata through a metadata server, and provide attribute metadata services. The above attribute metadata is a general term for each independent metadata that is different from the directory metadata.

For different data requests, the processing process will be performed in different servers. If the data request is to operate on the file directory, the data request will be processed in the directory path server; if the data request is to operate on attribute metadata, then Handle this data request in the metadata server. The directory path server acts as an interface for accessing metadata in the entire file storage system. Regardless of whether the client's target access object is directory metadata or attribute metadata, the client first sends a data request to the directory path server, so that the directory path server receives the data sent by the client. After receiving the data request, determine the specific type of the client's target access object according to the data request, that is, determine whether the client's target access object is directory metadata or attribute metadata, and then perform subsequent corresponding operations according to the analysis results.

In a specific embodiment, determining the target access object of the client according to the data request includes: parsing the data request; if the data request contains a full-path execution directory, determining that the target access object of the client is directory metadata; If the request does not contain the full-path execution directory, it is determined that the target access object of the client is attribute metadata.

Specifically, in this embodiment, after the directory path server receives the data request sent by the client, it parses the data request, and if the data request contains a full-path execution directory, it indicates that the target access object of the client is directory metadata; If the data request does not contain the full-path execution directory, it indicates that the target access object of the client is attribute metadata.

S300: If the target access object is directory metadata, read the directory metadata and return the directory metadata to the client;

Specifically, on the basis of determining that the target access object of the client is the directory metadata, this step aims to read the directory metadata, and return the read directory metadata to the client. Specifically, the directory path server allocates a globally unified file system ID to all files in the storage file system including the directory file, and manages the file allocation in a unified manner to ensure that the files are globally unique. The search index item is set in the directory path server, including the globally unique file system ID (fsID), the full path name of the file (dir), the parent directory ID (pid), and the file system name (fsname), that is, the last item of the directory path, Access Control Properties (ACP). When the client accesses the directory metadata, the full path of the file is used as the index item, and when locating the attribute metadata, the globally unique file system ID is used as the index item. Therefore, after determining that the target access object of the client is the directory metadata, the directory path server reads the directory metadata with the full path name of the file as an index item, and returns the read directory metadata to the client.

In addition, each time a file is created, the obtained access control attributes can be recorded in the current index item, so that when a file is created in the current directory, it is not necessary to traverse the previous directory, but the access control attributes of the current directory and the newly created file can be recursively used. Access Control Attributes Build new access control attributes to greatly improve access performance.

In a specific embodiment, the above-mentioned reading of the directory metadata includes: searching for the directory metadata in the cache of the directory path server with the full path name of the file as an index item; if the directory metadata exists in the cache of the directory path server, then The directory metadata is read from the cache of the directory path server; if the directory metadata does not exist in the cache of the directory path server, the directory metadata is read from the disk of the directory path server.

Specifically, in order to reduce the network communication overhead, in this embodiment, the directory path server has a caching mechanism, which can cache the directory metadata loaded into the memory. Therefore, when the directory path server reads the directory metadata, it first searches for the corresponding directory metadata in its cache with the full path name of the file as the index item. If the directory metadata does not exist in the cache of the directory path server, the directory path server accesses the local storage and reads the directory metadata from the disk.

S400: If the target access object is attribute metadata, send the corresponding file system ID to the metadata server, so that the metadata server reads the corresponding attribute metadata and returns the attribute metadata to the client.

Specifically, when the target access object of the client is attribute metadata, the directory path server further sends the corresponding file system ID to the metadata server. Therefore, the metadata server reads the corresponding attribute metadata after receiving the file system ID, and returns the read attribute metadata to the client according to the opposite path. Therefore, after receiving the attribute metadata of the file, the client can access the back-end storage according to the location information of the file, and exchange data with it.

The attribute metadata includes the following information: file system ID, file system name, that is, the last item of the full path name, file type, file location information, and other information, such as file size, last access time, and the like. Also, the filesystem ID is obtained when the file is created. The location information of the file is determined by the location number where the data object that saves the file is located and the file system ID that assigns the file.

In a specific implementation, the metadata server reading the corresponding attribute metadata includes: using the file system ID as an index item to search for the attribute metadata in the cache of the metadata server; if the attribute metadata exists in the cache of the metadata server If there is no attribute metadata in the cache of the metadata server, the attribute metadata is read from the disk of the metadata server.

Specifically, in order to reduce the network communication overhead, in this embodiment, the metadata server also has a caching mechanism, which can cache the attribute metadata transferred into the memory. Therefore, after the metadata server receives the file system ID sent by the directory path server, it first uses the file system ID as an index item to search for the corresponding attribute metadata in the cache of the metadata server. If the attribute exists in the cache of the metadata server Metadata, the attribute metadata is read from the cache; if the attribute metadata does not exist in the cache of the metadata server, the metadata server accesses the local storage and reads the attribute metadata from the disk.

To sum up, the metadata management method for a storage file system provided by the present invention performs hierarchical management on metadata, uses directory metadata to manage directory path metadata, and manages attribute metadata through a metadata server, thereby ensuring storage The high performance and scalability of the file system can accurately locate the metadata and effectively improve the access efficiency of the metadata.

The present invention also provides an apparatus for managing metadata of a storage file system, and the apparatus described below may refer to the method described above in correspondence with each other. Please refer to FIG. 2, which is a schematic diagram of a metadata management device for a storage file system according to an embodiment of the present invention; with reference to FIG. 2, the metadata management device includes:

A determination module 11, configured to receive the data request sent by the client and determine the target access object of the client according to the data request;

The reading module 12 is configured to read the directory metadata and return the directory metadata to the client if the target access object is the directory metadata;

The sending module 13 is configured to send the corresponding file system ID to the metadata server if the target access object is attribute metadata, so that the metadata server reads the corresponding attribute metadata and returns the attribute metadata to the client.

On the basis of the foregoing embodiment, optionally, the determining module 11 includes:

Parsing unit for parsing data requests;

The determining unit is used to determine that the target access object of the client is the directory metadata if the data request contains the full-path execution directory; if the data request does not contain the full-path execution directory, determine that the target access object of the client is attribute metadata.

On the basis of the above embodiment, optionally, the reading module 12 includes:

The search unit is used to search the directory metadata in the cache of the directory path server with the full path name of the file as the index item;

a first reading unit, configured to read the directory metadata from the cache of the directory path server if the directory metadata exists in the cache of the directory path server;

The second reading unit is configured to read the directory metadata from the disk of the directory path server if the directory metadata does not exist in the cache of the directory path server.

The present invention also provides a metadata management system for a storage file system. Please refer to FIG. 3. FIG. 3 is a schematic diagram of a metadata management system for a storage file system according to an embodiment of the present invention. Referring to FIG. 3, the metadata The data management system includes:

The directory path server 10 is used to receive the data request sent by the client and determine the target access object of the client according to the data request; if the target access object is directory metadata, read the directory metadata and return the directory metadata to the client; If the target access object is attribute metadata, send the corresponding file system ID to the metadata server;

The metadata server 20 is configured to read the corresponding attribute metadata and return the attribute metadata to the client.

For the introduction of the system provided by the present invention, please refer to the embodiments of the above method, which are not repeated in the present invention.

The present invention also provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the following steps are implemented:

Receive the data request sent by the client; determine the target access object of the client according to the data request; if the target access object is directory metadata, read the directory metadata and return the directory metadata to the client; if the target access object is an attribute element data, the corresponding file system ID is sent to the metadata server, so that the metadata server reads the corresponding attribute metadata and returns the attribute metadata to the client.

The computer-readable storage medium may include: a USB flash drive, a removable hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, etc., which can store program codes. medium.

For the introduction of the computer-readable storage medium provided by the present invention, please refer to the above-mentioned method embodiments, which are not repeated in the present invention.

The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the various embodiments can be referred to each other. For the apparatuses, devices, and computer-readable storage media disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and reference may be made to the descriptions of the methods for related parts.

Professionals may further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two, in order to clearly illustrate the possibilities of hardware and software. Interchangeability, the above description has generally described the components and steps of each example in terms of functionality. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of the present invention.

The steps of a method or algorithm described in conjunction with the embodiments disclosed herein may be directly implemented in hardware, a software module executed by a processor, or a combination of the two. The software module can be placed in random access memory (RAM), internal memory, read only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other in the technical field. in any other known form of storage medium.

The metadata management method, device, system and computer-readable storage medium of the storage file system provided by the present invention have been described in detail above. The principles and implementations of the present invention are described herein by using specific examples, and the descriptions of the above embodiments are only used to help understand the method and the core idea of the present invention. It should be pointed out that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and modifications can also be made to the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims (9)

1. a kind of metadata management method of storage file system, which is characterized in that be applied to directory path server, comprising:

Receive the request of data that client is sent；

The target access object of the client is determined according to the request of data；

If the target access object is directory metadata, the directory metadata is read, and the directory metadata is returned Back to the client；

If the target access object is attribute metadata, corresponding file system ID is sent to meta data server, with So that the meta data server is read the corresponding attribute metadata and the attribute metadata is returned into the client.

2. metadata management method according to claim 1, which is characterized in that described to determine institute according to the request of data State the target access object of client, comprising:

Parse the request of data；

If executing catalogue comprising complete trails in the request of data, it is determined that the target access object of the client is the mesh Record metadata；

If not including the complete trails in the request of data executes catalogue, it is determined that the target access object of the client is The attribute metadata.

3. metadata management method according to claim 2, which is characterized in that described to read the directory metadata, packet It includes:

The directory metadata is searched in the caching of the directory path server using file full path name as index entry；

If there are the directory metadatas in the caching of the directory path server, from the slow of the directory path server Deposit the middle reading directory metadata；

If the directory metadata is not present in the caching of the directory path server, from the directory path server The directory metadata is read in disk.

4. metadata management method according to claim 3, which is characterized in that the meta data server reads corresponding The attribute metadata, comprising:

The attribute metadata is searched in the caching of meta data server using the file system ID as index entry；

If there are the attribute metadatas in the caching of the meta data server, from the caching of the meta data server Read the attribute metadata；

If the attribute metadata is not present in the caching of the meta data server, from the disk of the meta data server It is middle to read the attribute metadata.

5. a kind of meta data management device of storage file system characterized by comprising

Determining module, for receiving the request of data of client transmission and determining according to the request of data mesh of the client Mark access object；

Read module reads the directory metadata and will be described if being directory metadata for the target access object Directory metadata returns to the client；

Corresponding file system ID is sent to member if being attribute metadata for the target access object by sending module Data server, so that the meta data server reads the corresponding attribute metadata and returns to the attribute metadata To the client.

6. meta data management device according to claim 5, which is characterized in that the determining module includes:

Resolution unit, for parsing the request of data；

Determination unit, if for executing catalogue comprising complete trails in the request of data, it is determined that the target of the client is visited Ask that object is the directory metadata；If not including the complete trails in the request of data executes catalogue, it is determined that the visitor The target access object at family end is the attribute metadata.

7. metadata management method according to claim 6, which is characterized in that the read module includes:

Searching unit, for searching the mesh in the caching of the directory path server using file full path name as index entry Record metadata；

First reading unit, if in the caching of the directory path server there are the directory metadata, from described The directory metadata is read in the caching of directory path server；

Second reading unit, if for the directory metadata to be not present in the caching of the directory path server, from institute It states and reads the directory metadata in the disk of directory path server.

8. a kind of metadata management system of storage file system characterized by comprising

Directory path server, for receiving the request of data of client transmission and determining the client according to the request of data The target access object at end；If the target access object is directory metadata, the directory metadata is read and will be described Directory metadata returns to the client；If the target access object is attribute metadata, by corresponding file system ID is sent to meta data server；

The meta data server, for reading the corresponding attribute metadata and returning to the attribute metadata described Client.

9. a kind of computer readable storage medium, which is characterized in that the computer-readable recording medium storage has computer journey Sequence realizes the member such as the described in any item storage file systems of Claims 1-4 when the computer program is executed by processor The step of data managing method.