CN1997972A - Emulated storage system supporting instant volume restore - Google Patents

Abstract

In a back-up storage system, an apparatus and methods for mounting a data volume corresponding to a back-up data set to a host computer. In one example, a method includes mounting a data volume on a host computer, the data volume comprising at least one data file, the data file corresponding to a most recently backed-up version of the at least one data file stored on a backup storage system, and storing, on the backup storage system, data corresponding to a second version of the at least one data file that is more recent than the most recently backed-up version of the at least one data file stored on the backup storage system while preserving the most recently backed-up version of the at least one data file.

Description

Emulated storage system supporting instant volume recovery

technical field

Aspects of the present invention relate to data storage, and more specifically to emulation of tape storage systems to provide the equivalent of using existing full backups and future incremental backups, and enabling end users to restore from said backups Data means and methods.

Background technique

Many computer systems include one or more mainframes and one or more data storage systems that store data used by the mainframes. These hosts and storage systems are typically connected together by a network such as Fiber Channel, Ethernet or other type of communication network. Fiber Channel is a standard that combines the speed of channel-based transport mechanisms with the flexibility of network-based transport mechanisms and allows multiple initiators to communicate over the network with multiple targets, which can be Any device coupled to the network. Fiber Channel is typically implemented using fast transport media, such as fiber optic cables, and is a popular choice in storage networks that transport large amounts of data.

An embodiment of a typical networked computing environment includes several hosts and backup storage systems as shown in FIG. 1 . One or more application servers 102 are coupled to a plurality of user computers 104 through a local area network 103 (LAN). Both application server 102 and user computer 104 may be considered "hosts." Application server 102 is coupled to one or more primary storage devices 106 through a storage area network 108 (SAN). Primary storage device 106 may be, for example, a disk array available from, for example, EMC Corporation, IBM Corporation, and other similar companies. Alternatively, a data transfer bus (not shown) or other network connection may provide the interconnection between the application server and primary storage system 106 . The connection of the data transfer bus and/or Fiber Channel network may operate using a protocol, such as the Small Computer System Interface (SCSI) protocol, which controls the transmission of certain formatted information package.

It will be appreciated that the networked computing environment illustrated in Figure 1 is typical of large systems, such as those used by large financial institutions or large corporations. It will be appreciated that many networked computing environments do not necessarily include all of the components enumerated in FIG. 1 . For example, a smaller networked computing environment can be simplified to include hosts connected directly to the storage system, or connected to the storage system through a LAN. Additionally, although FIG. 1 illustrates separate user computers 104, application server 102, and media server 114, these functions may be combined into more than one computer.

In addition to primary storage device 106 , many networked computing environments include at least one secondary or backup storage system 110 . The backup storage system 110 is typically a tape library, although other high-capacity, reliable secondary storage systems may be used. These secondary storage systems are typically slower than primary storage systems, but include certain types of removable media (for example, tape, magnetic disks, or optical discs) that can be moved and offsite storage.

In an illustrative embodiment, application server 102 may be able to communicate directly with backup storage system 110 via, for example, Ethernet or other communication link 112 . However, such connections may be relatively slow and resource-intensive, such as processor time or network bandwidth. Thus, the illustrated system may include one or more media servers 114, which may provide a communication connection between the SAN 108 and the backup storage system 110 via, for example, Fiber Channel.

Media server 114 may run software including a backup/restore application that controls the communication between a host (e.g., user computer 104, media server 114, and/or application server 102), primary storage device 106, and backup storage system 110. data transmission. Backup/restore applications are available from companies such as Veritas, Legato, and others. For data protection, data from various hosts and/or primary storage devices in a networked computing system can be periodically backed up to the backup storage system 110 by a backup/restore application, which is known in the art openly known.

Of course, as discussed above, it will be appreciated that many networked computing environments may be smaller and include fewer components than the exemplary networked computing environment shown in Figure 1 . Therefore, it will still be appreciated that the media server 114 is actually connected to the application server 112 in a separate host, and that the backup/restore application can be coupled to the backup storage system 110 at any location (directly or indirectly, such as through a network). executed on the host.

An example of a typical backup storage system is a tape library that includes a number of tape cartridges and at least one tape drive, and a robot that controls loading of the tape cartridges into the tape drive and unloading from the tape drive. The backup/restore application provides instructions to the robot to determine the location of a particular cartridge, such as tape number 0001, and the tape drive to load the cartridge so that data can be written to the tape. The backup/restore application also controls the format of the data written to tape. Typically, a backup/restore application can use SCSI commands, or other standardized commands, to instruct robotics and control tape drives to write data to tape and to restore written data from tape.

Traditional tape library backup systems suffer from several issues, including speed, reliability, and fixed capacity. Many large companies need to back up terabytes of data every week. However, even expensive, high-end tapes are typically only capable of reading/writing data at 30-40 megabytes per second (MB/s), which translates to about 50 gigabytes per hour (GB/hr). Therefore, backing up one or two terabytes of data to a tape backup system may require at least 10-20 hours of continuous data conversion time.

Also, most tape manufacturers do not guarantee the possibility of storing (or restoring) data to/from the tape if the tape is lost (during a move or load operation, possibly due to human action or robotic mechanism relative frequently occurs in a typical tape library) or if the tape is exposed to non-ideal conditions, for example, extremes of temperature or humidity. Therefore, a great deal of effort is required to keep the storage tapes in a controlled environment. Furthermore, the complex mechanism of the tape library, including the robotics, is expensive to maintain, and the individual tape cartridges are relatively expensive and have a limited useful life.

Contents of the invention

The backup storage system provided by the embodiments of the present invention overcomes or alleviates some or all of the problems of the traditional tape library system, and can provide more flexibility than the traditional tape library system.

In summary, various aspects of embodiments of the present invention provide a RAM-based storage system that emulates traditional tape backup storage systems such that the devices and media identified by the backup/restore application are identical to the actual Same as tape library. The storage system of the present invention uses software and hardware to emulate real tape media and replaces it with one or more random storage disk arrays, adapting tape format, linear, sequential data transformations to data stored on the disks. Additionally, an application running on hardware and/or software is provided for restoring data stored in the backup storage system.

In accordance with certain aspects of embodiments of the present invention, means are provided for converting continuous tape format data into an appropriate format for random access I/O. In one embodiment, an apparatus is provided that includes provision for mounting a converted representation of tape format data on a host computer as an NFS (Network File System) or CIFS (Common Internet File System) mounted volume.

According to other aspects and embodiments of the present invention, means are provided for converting writes to a mounted file system for "safe storage", whereby the original data remains unchanged. In one embodiment, means are provided for tracking real-time transformations of raw data to enable random access I/O. In another embodiment, means are provided for converting newly written data backups into tape format data suitable for I/O characterized by sequential tape.

In one embodiment, the method includes the step of: mounting a data volume on the host computer, the data volume including at least one data file, the data file being the latest backup version of the at least one data file stored in the backup storage system Correspondingly, the data corresponding to the second version of the at least one data file is stored in the backup storage system, and when the latest backup version of the at least one data file is saved, the data is higher than the At least one data file is updated with the latest backup version. The method may also include concatenating the latest backup version of the at least one data file with the second version of the at least one data file. In one embodiment, the method may also include creating a data structure identifying the latest backup version of the at least one data file and the second version of the at least one data file. In another embodiment, the second version of the at least one data file may be a modified version of the latest backup version of the at least one data file.

According to another embodiment, the backup storage system includes a backup storage medium for storing backup data sets, and a controller, the controller includes at least one processor configured to execute the method for executing the method described above. A series of instructions.

In another embodiment, the provided computer-readable medium stores a data structure, the data structure includes a first marking toolkit, and the first marking toolkit uniquely marks at least one data file corresponding to the backup data set system files, and at least one second marking toolkit, where the second marking toolkit marks respective storage locations in the storage medium, and the storage medium stores the latest version of at least each data file in the backup data set.

Description of drawings

The drawings are not drawn to scale. In the drawings, each identical or substantially identical part in each of the illustrated drawings is denoted by the same numeral. For clarity, not every part is labeled in every drawing. In the attached picture:

Figure 1 is a block diagram of one embodiment of a large networked computing environment including a backup storage system;

Figure 2 is a block diagram of one embodiment of a networked computing environment including a storage system in accordance with aspects of the present invention;

Figure 3 is a block diagram of one embodiment of a storage system in accordance with various aspects of the present invention;

Figure 4 is a block diagram illustrating a virtual design of one embodiment of a storage system in accordance with various aspects of the present invention;

Figure 5 is a schematic diagram of an embodiment of a system file according to various aspects of the present invention;

Accompanying drawing 6 is an embodiment of tape directory according to various aspects of the present invention;

Figure 7 is a graphical depiction of one embodiment of a method for creating a synthetic full backup in accordance with various aspects of the present invention;

Figure 8 is a schematic diagram of one embodiment of a sequence of backup data sets including synthetic full backups in accordance with various aspects of the present invention;

Figure 9 is a diagram of one embodiment of the structure of a metadata cache;

Figure 10 is a diagram of one embodiment of a virtual encoded tape storing a synthetic full backup data set; and

Figure 11 is a diagram of another embodiment of a virtual encoded tape storing a synthetic full backup data set;

Figure 12 is a flowchart of one embodiment of a method for restoring data in a backup storage system in accordance with various aspects of the present invention;

Figure 13 is a block diagram of another embodiment of a networked computing environment including a backup storage system in accordance with aspects of the present invention;

Figure 14 is a diagram of one embodiment of a file descriptor structure in accordance with various aspects of the invention;

Accompanying drawing 15 illustrates an embodiment how file data is stored in magnetic tape format with diagram;

Figure 16 diagrammatically illustrates the file descriptors of the file description of Figure 15;

Accompanying drawing 17 is a flowchart of an embodiment of the method for writing data into a mounted data volume according to the present invention;

Accompanying drawing 18 is a schematic diagram of newly written files;

Accompanying drawing 19 is according to various aspects of the present invention, the schematic diagram of an embodiment of the relationship among original data, newly written file and final modification file; And

Figure 20 is a diagram representing one embodiment of a file descriptor for the modified file of Figure 19;

Detailed ways

Various embodiments and aspects are described in more detail below with reference to the accompanying figures. It will be appreciated that the invention is not limited in its application to the construction and arrangement of parts set forth hereinafter, or to the details illustrated in the drawings. The invention is capable of other embodiments and of being practiced and carried out in various ways. Also, the phraseology and terminology used herein are for the purpose of description and should not be regarded as limiting. The use of "including", "comprising", "having", "comprising", "involving" and their combinations, the use of these words and terms means including the items listed thereafter and equivalents and attachments .

As used herein, the term "host" refers to any computer having at least a processor, such as personal computers, workstations, mainframes, network clients and servers, etc., capable of communicating with other devices, such as, storage system or other host. Hosts may include media servers and application servers (as previously described in connection with Figure 1) and user computers (which may be user workstations, personal computers, mainframes, etc.). Also, in the published document, the term "networked computing environment" includes any computing environment in which multiple hosts are connected to one or more shared storage systems in such a way that the storage systems can communicate with each host communication. Fiber Channel is one example of a communication network that may be used in embodiments of the present invention. However, it will be appreciated that the networks described herein are not limited to Fiber Channel, and that various network components may communicate with each other over any network connection, for example, Token Ring or Ethernet instead, or in addition to Fiber Channel, Or through a combination of different network connections. Furthermore, aspects of the present invention may also be used in data transfer bus topologies such as SCSI or parallel SCSI.

According to various embodiments and aspects of the present invention, the provided virtual removable media library backup storage system can use one or more disk arrays to simulate removable media based on the storage system. Using embodiments of the present invention, data can be backed up to a disk array by using a backup/restore application similar to that used to back up data to removable media (e.g., tape, disk, CD, etc.) , eliminating the need for users to make any modifications or adjustments to existing backup programs or purchase new backup/restore applications. In one embodiment, described in detail herein, the removable media is emulated as tape, and the backup storage system of the present invention emulates a tape library system that includes the tape and the robotics used to handle the tape in a conventional tape library system.

A storage system according to various aspects of the present invention includes hardware and software interfaced with a host (running a backup/restore application) and a backup storage medium. The storage system can be designed to emulate magnetic tape, or other types of removable storage media, so that backup/restore applications view the device and media as a real tape library and treat linear, sequential, tape format data Converted to data suitable for storage on RAM disk. In the manner described, the storage system of the present invention can provide increased functionality (for example, allowing users to query individually backed up user files, as discussed below) without requiring new backup/restore application software or strategies.

Referring to FIG. 2, one embodiment of a networked computer environment including a backup storage system 170 in accordance with aspects of the present invention is illustrated in block diagram form. As an example, host 120 is coupled to storage system 170 via network connection 121 . Network connection 121 may be, for example, a Fiber Channel connection to allow high-speed transfer of data between host 120 and storage system 170 . It will be appreciated that host 120 may be, or may include, one or more application servers 102 (see FIG. 1 ) and/or media server 114 (see FIG. 1 ), and may be selected from existing Any computer or backup data from primary storage device 110 (see Appendix 1). Additionally, one or more user computers 136 may be coupled to storage system 170 via another network connection 138, such as an Ethernet connection. As discussed in detail below, the storage system enables the user of the user computer 136 to read and selectively restore backed-up user files from the storage system.

The storage system includes backup storage media 126, which may be, for example, one or more disk arrays, as explained in more detail below. The backup storage medium 126 provides actual storage space for the backup data from the host 120 . However, storage system 170 may also include software and additional hardware that emulates a removable media storage system, such as a tape library, so that when running a backup/restore application on host 120, data appears to be backed up to a traditional Removable storage media. Thus, as explained in FIG. 2, storage system 170 may include "emulated media" 134, which represents, for example, virtual or emulated removable storage media (eg, tape). The "emulated medium" 134 exists in the host through storage system software and/or hardware, and appears in the host 120 as a physical storage medium. A further interface between the emulated medium 134 and the actual backup storage medium 126 may be a storage system controller (not shown) and a switch network 132 that receives data from the host 120 and stores the data to the backup storage medium 126, as discussed in more detail below. In this way, the storage system "emulates" a traditional tape-based storage system into the host 120.

According to one embodiment, the storage system may include a “logical metadata cache” 242 that stores metadata related to user data backed up from the host 120 to the storage system 170 . As used herein, the term "metadata" refers to information representing user data, and data describing attributes of actual user data. Logical metadata cache 242 represents a queryable data set that enables a user and/or software application to randomly determine the location of a backup user file, compare a user file to another file, or access and process a backup user file. Two software application embodiments may use data stored in logical metadata cache 242, including synthetic full backup application 240 and end-user restore application 300, discussed more fully below.

In summary, the synthetic full backup application 240 has the capability to create a new synthetic full backup data set from an existing full backup data set and one or more incremental backup data sets. Synthetic full backups can eliminate the need to complete periodic (eg, weekly) full backups, thus saving considerable time and network resources. The details of the composite full backup application 240 are discussed further below. The end user restore application 300, also discussed further below, enables an end user (for example, the operator of the user computer 136) to browse, query, read and/or restore previously backed up files from the storage system 170. user files.

As discussed above, storage system 170 includes the hardware and software to interface with host 120 and backup storage medium 126 . Hardware and software in conjunction with embodiments of the present invention can emulate a traditional tape library backup system so that from the perspective of the host computer 120, data is apparently backed up to tape, but is actually backed up to another storage medium, e.g., For example, numerous disk arrays.

Referring to FIG. 3, one embodiment of a storage system 170 in accordance with aspects of the present invention is illustrated in block diagram form. In one embodiment, the hardware of the storage system 170 includes a storage system controller 122 and a switch network 132 connecting the storage system controller 122 and the backup storage medium 126 . The storage system controller 122 includes a processor 127 (which may be a single processor or multiple processors) and a memory 129 (for example, RAM, ROM, PROM, EEPROM, flash memory, etc., or a combination thereof), which can Run all or part of the storage system software. Memory 129 may also be used to store metadata associated with the data stored in backup storage medium 126 . Software (comprising program codes to implement the embodiments of the present invention) is usually stored in a readable/writable non-volatile recording medium, for example, RAM, ROM, optical disc, magnetic disk or magnetic tape, etc., and then copied to the memory 129, wherein the software is executed by the processor 127. The program code may be written in any of a number of programming languages, for example, Java, Visual Basic, C, C#, or C++, Fortran, Pascal, Eiffel, Basic, COBAL, and combinations thereof , since the invention is not limited to a particular programming language. Typically, in operation, processor 127 causes data (for example, code to carry out embodiments of the present invention) to be read from a non-volatile recording medium into another form of memory, such as RAM, allowing Access processor information at a faster rate than non-volatile recording media.

As shown in FIG. 3 , the controller 122 also includes a series of interface adapters 124 a , 124 b and 124 c that connect the controller 122 and the host 120 to the switch network 132 . As illustrated, host 120 is coupled to the storage system through interface adapter 124a, which may be, for example, a Fiber Channel interface adapter. Through the controller 122 of the storage system, the host 120 backs up data to the backup storage medium 126 and can restore the data backed up from the backup storage medium 126 .

In an exemplary embodiment, switch network 132 may include one or more Fiber Channel switches 128a, 128b. Storage system controller 122 includes a number of Fiber Channel interface adapters 124b and 124c to couple the storage system controller to Fiber Channel switches 128a, 128b. The controller 122 of the storage system allows data to be backed up to the backup storage medium 126 via the Fiber Channel switches 128a, 128b. As shown in FIG. 3 , the switch network 132 may further include one or more Ethernet switches 130a, 130b, and the Ethernet switches 130a, 130b are coupled to the controller 122 of the storage system through Ethernet interface adapters 125a, 125b. In one embodiment, storage system controller 122 further includes another Ethernet interface adapter 125c, which may be coupled to, for example, LAN 103 to enable storage system 170 to communicate with a host (eg, a user computer), as discussed below.

In the embodiment illustrated in FIG. 3 , the storage system controller 122 is coupled to the backup storage medium 126 through a switch network, and the switch network includes two types of fiber channel switches and two types of Ethernet switches. Providing at least two of each type of switch in storage system 170 emulates any single point of failure in the system. In other words, even if one switch (eg, Fiber Channel switch 128a) fails, storage system controller 122 will still be able to communicate with backup data medium 126 through the other switch. The described arrangement has advantages in terms of stability and speed. For example, as discussed above, stability is improved by providing spare components and eliminating single points of failure. Furthermore, in some embodiments, the storage system controller can increase the overall backup speed by using some or all of the Fiber Channel switches in parallel to backup data to the backup storage medium 126 . However, it will be appreciated that it is not required that the system include two or more switches of each type, nor that the switch network include both Fiber Channel and Ethernet switches. Still further, in embodiments where the backup storage medium 126 includes a single disk array, no switches are needed at all.

As discussed above, in one embodiment, backup storage medium 126 may include one or more disk arrays. In a preferred embodiment, backup storage medium 126 includes a plurality of ATA or SATA disks. The disks are "for sale" commodities that are less expensive than traditional commodity storage arrays from manufacturers such as EMC, IBM, etc. Also, the disks are comparable in cost to traditional tape-based backup storage systems when the cost of removable media (eg, magnetic tape) is a factor and the media has a limited useful life. Additionally, the disk reads/writes data substantially faster than tape. For example, over a single fiber optic network connection, data can be backed up to disk at a rate of at least about 150MB/s, which translates to about 540GB/hr, significantly faster than backups to tape (eg, via disk sequential) . Additionally, several Fiber Channel connections can be performed in parallel, further increasing speed. According to embodiments of the present invention, backup storage system media may be assembled to implement any RAID (Redundant Array of Inexpensive Disks) sequence. For example, in one embodiment, the backup storage medium may perform RAID-5 tasks.

As discussed above, embodiments of the present invention emulate a traditional tape library backup system by using disk arrays instead of tape cartridges as the actual backup storage medium, thus providing a "virtual tape library." The actual tape cartridges present in traditional tape libraries are replaced by the term "virtual encoded tape" as used in this paper. It will be appreciated that for purposes of this disclosure, the term "virtual tape library" refers to an emulated tape library that may be implemented in software and/or physical hardware, for example, as one or more disk arrays. It will further be appreciated that although the discussion primarily refers to emulated tape, the storage system can also emulate other storage media, for example, CD-ROM or DVD-ROM, and that the term "virtual encoded tape" generally refers to the emulated storage medium, For example, emulate a tape or emulate a CD. In one embodiment, the virtual encoded tape actually corresponds to one or more hard disks.

Thus, in one embodiment, a software interface is provided to emulate a tape library to a backup/restore application, as if data were backed up to tape. However, the actual tape library is replaced by one or more disk arrays, so that the data is actually backed up to these disk arrays. It will be appreciated that other types of removable media storage systems may be emulated, and that the invention is not limited to emulation of tape library storage systems. The following discussion will explain the operation of various features and software included in storage system 170 .

It will be appreciated that while software may be depicted as being "included" in storage system 170 and possibly executed by processor 127 of storage system controller 122 (see FIG. 3 ), not all software is required to be present in storage system control run in the device 122. Software programs, for example, synthetic full backup applications and end-user restore applications may run on the host computer and/or user computer, in which part may be distributed through all or some of the storage system controllers, host computer and user computer. Accordingly, it will be appreciated that a storage system controller is not required to comprise a physical entity, such as a computer. Storage system 170 communicates with software stored in hosts such as, for example, media server 114 and application server 102 . Additionally, a storage system may contain several application software that can run and reside on the same or different hosts. Also, it will be appreciated that storage system 170 is not limited to discrete pieces of a device, although in some embodiments storage system 170 may be embodied as a discrete piece of a device. In one embodiment, storage system 170 may be provided as a self-contained unit that acts as a "stop and start" in place of a traditional tape library backup system (eg, no modifications to existing backup processors and policies ). The storage system unit may be used in networked computing environments, including conventional backup systems, to provide redundancy or additional storage capacity.

As discussed above, according to one embodiment, host 120 (which may be, for example, application server 102 or media server 114 in FIG. In the storage medium 126 , the network connection 121 couples the host 120 to the storage system 170 . It will be appreciated that while the following discussion will primarily relate to backups of data on emulated media, the principles apply to restoring backup data from emulated media as well. Data flow between host 120 and emulated medium 134 can be controlled by a backup/restore application, as discussed above. From the point of view of the backup/restore application, it is obvious that the data is actually backed up to a true version of the emulated media.

Referring to FIG. 4 , storage system software 150 may include one or more logical abstraction layers representing emulated media and providing an interface between backup/restore application 140 and backup storage media 126 resident in host 120 . Software 150 accepts tape format data from backup/restore application 140 and translates the data into data suitable for storage on random access disks (eg, hard disks, optical disks, and the like). In one embodiment, software 150 runs on processor 127 of storage system controller 122 and may be stored in memory 129 (see FIG. 3 ).

According to one embodiment, software 150 may include layers, as referred to herein, of virtual tape library (VTL) layer 142, which may provide emulation of SCSI tapes, tape drives, and formats for converting tapes to tape drives and from tape drives to Automated tape format. Backup/restore application 140 may communicate with VTL 142 (eg, backup or read data into emulated media) using, for example, SCSI commands (indicated by arrow 144). Thus, the VTL can form the software interface between other storage system software and hardware and the backup/restore application, emulated storage system media 134 (in FIG. 2 ) appears in the backup/restore application, and allows emulated media to be appears in the backup/restore application for removable backup storage media.

A second software layer referred to herein, such as the file system layer 146 may provide an interface between the emulated storage medium (represented in the VTL) and the real backup storage medium 126 . In one embodiment, file system 146 functions as a trimming system to communicate with backup storage medium 126 , for example, using SCSI commands (indicated by arrow 148 ) to read and write data from backup storage medium 126 .

In one embodiment, the VTL provides generic tape library support and can support any SCSI media converter. Emulated tape devices may include, but are not limited to, IBM LTO-1 and LTO-2 tape devices, Quantum SuperDLT320 tape devices, Quantum P3000 tape library systems, or Storage TekL180 tape library systems. In VTL, each virtual encoded tape is a file that grows dynamically as data is stored. This is in contrast to conventional tape cassettes which have fixed dimensions. One or more virtual encoded tapes may be stored in the system file, as further described below with reference to FIG. 5 .

In FIG. 5, illustrating an example of a data structure in file system software 146, a system file 200 is illustrated in accordance with an embodiment of the present invention. In the depicted embodiment, system file 200 includes header 202 and data 204 . The header 202 may include information identifying each virtual coded tape stored in the system file. Regardless of whether the virtual coded tape is write-protected or not, the header may include information such as new/modified data of the virtual coded tape and the like. In one embodiment, header 202 includes information that uniquely identifies each encoded tape and distinguishes each encoded tape from other virtual encoded tapes stored in the storage system. For example, the information may include the name and identification code of the virtually encoded tape (eg, corresponding to a bar code normally represented on a real tape so that the tape can be marked by an automated device). Header 202 may also include additional information, for example, the size of each virtual encoded tape, last modified data, and the like.

According to one embodiment of the present invention, the size of header 202 can be fully utilized to reflect the type of stored data (for example, a virtual encoded tape representing data backed up from one or more host systems), and a series of obvious The set of data that the system can track (eg, a virtual encoded tape). For example, data typically backed up to tape storage systems is typically characterized by larger datasets representing digital systems and user files. Since the data set is very large, a series of discrete data files that are tracked may be relatively small. Accordingly, in one embodiment, the size of the header 202 is chosen based on the fact that there is too much data to store to effectively keep track of (e.g., the header is too large) and there is no space for storing a sufficient number of coded tapes ( For example, the header is too small) on a compromise basis. In an exemplary embodiment, header 202 utilizes the first 32MB of system file 200 . However, it will be appreciated that the header 202 can be of different sizes based on the needs and characteristics of the system, and that one can choose a different size for the header 202 depending on the needs and capacity of the system.

It will be appreciated that from the point of view of the backup/restore application, a virtual encoded tape with all the same attributes and characteristics appears as a real tape cartridge. In other words, to backup/restore applications, the virtual encoded tape essentially appears as written tape. However, in a preferred embodiment, the data stored on the virtual encoded tape is not stored in a sequential format to the backup storage medium 126, but rather, the data that appears to be written to the virtual encoded tape is actually written as a random-access Free, disk-formatted data is stored in files on the storage system. Metadata is used to link stored data to virtually encoded tapes so that backup/restore applications can read and write encoded tape formats.

Thus, in general terms of a preferred embodiment, user and/or system data (referred to herein as "file data") is received by storage system 170 from host computer 120 and stored in a disk array of supplemental backup storage media 126 middle. Software 150 (see FIG. 4 ) and/or storage system hardware writes this file data to backup storage medium 126 in the format of a system file, as described in more detail below. Metadata extracted from the backup file data by the storage system controller is used to track attributes of user and/or system files being backed up. For example, the metadata for each file may include the file's name, date of creation or last modification of the file, any encrypted information about the file, and other information. In addition, metadata is established for each file by the storage system, which links the file to the virtual encoded tape. Using the metadata described, the software presents an emulation of the tape cartridge to the host; however, the actual file data is not stored in tape format, but in system files, as described below. Storing data in system files, rather than in a sequentially encoded tape format, can advantageously allow fast, efficient, and random access to individual files without requiring scanning of sequential data to find particular files.

As discussed above, according to one embodiment, file data (e.g., user and/or system data) is stored on the backup storage medium as system files, each system file includes a header and data, the data is real user and/or System Files. The header 202 of each copy of system files 200 includes a tape directory 206 that includes metadata linking user and/or system files to virtual encoded tapes. The term "metadata" herein refers neither to user nor system file data, but to data describing attributes of real user and/or system data. According to one embodiment, a tape directory may define, down to the byte level, the layout of data in virtually encoded tape. In one embodiment, tape directory 206 has a table structure, as shown in FIG. 6 . The table includes a column 220 for the type of information stored (e.g., Data, File Mark FM, etc.), a column 222 indicating the size of disk bytes used in bytes, and a column for calculating the number of disk bytes used to store the file data column 224. Thus, the tape directory allows the controller to randomly access (in reverse order) any data file stored in the backup storage medium 126 . For example, referring to FIG. 6 , the data file 226 may be quickly located on the virtual encoded tape because the tape directory indicates that the data of the file 226 begins with the block diagram of the initial stage of the system file 200 . The one block diagram described has no size due to the response to file markers (FM). Filemarks are not stored in the system, for example, filemarks correspond to zero data. Since filemarks are used by traditional tapes, tape directories include filemarks, so backup/restore applications write filemarks along data files and expect to see filemarks when browsing a virtual encoded tape. Therefore, file marks are tracked in the tape directory. However, filemarks do not represent any data, and thus are not stored in the data section of system files. Therefore, the data for file 226 begins at the beginning of the data section of the system file (indicated by arrow 205), and its length is 1024 bytes (eg, a disk byte is 1024 bytes). It will be appreciated that other file data may be stored in bytes other than 1024 bytes, depending on the total amount of data, eg, the size of the data file. For example, larger data files may use larger disk bytes to store for efficiency.

In one embodiment, the tape directory may be included in a "file descriptor" associated with each data file backed up to the storage system. The file descriptors contain metadata associated with data files 204 stored in the storage system. In one embodiment, file descriptors may be implemented according to a standard format, for example, the tape archive file format (an extension for compressed files) used by most UNIX-based systems (multi-user computer operating systems). Each file descriptor may include information such as the name of the corresponding user file, the data of the newly created/modified user file, the size of the user file, any access restrictions on the user file, and the like. The additional information stored in the file descriptor may further include information describing a directory structure from which data may be copied. Accordingly, a file descriptor may include queryable metadata about the corresponding data file, as discussed in more detail below.

From the backup/restore application's point of view, any virtual encoded tape may contain multiple file data and corresponding file descriptors. From the point of view of the storage system software, data files are stored in system files, which can be linked, for example, for special backup jobs. For example, a backup performed by a host at a specific time may generate a system file corresponding to one or more virtual encoded tapes. Thus, the virtual encoded tape may be of any size, and the virtual encoded tape may grow dynamically as more user files are stored on the virtual encoded tape.

Referring again to FIG. 3, storage system 170 may include composite full backup software application 240, as described above. In one embodiment, host 120 backs up data to emulated media 134, forming one or more virtual encoded tapes. In some computer environments, "full backups", eg, backup copies of all data stored in the primary storage system of the network (see Figure 1), may be performed periodically (eg, weekly). This process is usually very long due to the large amount of data being copied. Therefore, in many computing environments, additional backups, so-called incremental backups, may be performed between successive full backups (eg, daily). Incremental backup is a process in which only the data has been changed since the latest backup (whether incremental or full) is saved. Typically, changed data is a backup in the file repository, even though most of the data in the file is not changed frequently. Therefore, incremental backups are usually smaller and can complete faster than full backups. It will be appreciated that although in many environments where full backups are typically performed once a week and incremental backups are performed daily during the week, the use of schedules is not required. For example, an environment may require incremental backups several times a day. The principles of the present invention apply to any environment where full backups (and random incremental backups) are used, regardless of the frequency of execution.

During full backup processing, the host can create one or more virtual encoded tapes containing backup data containing numerous data files. For clarity, the following discussion will assume that a full backup produces only one virtual encoded tape. However, it will be appreciated that a full backup may result in more than one virtual encoded tape and that the principles of the present invention may be applied to any number of virtual encoded tapes.

According to one embodiment, a method for creating a synthetic full backup data set from an existing full backup data set and one or more incremental backup data sets is provided. The method can avoid the requirement to periodically (eg, weekly) perform full backups, thereby saving the user considerable time and network resources. Furthermore, as is known to those of ordinary skill in the art, restoring data based on a full backup and one or more incremental backups is a time-consuming process, for example, if the latest version of the file exists In an incremental backup, the backup/restore application will typically store the files based on the latest full backup and then apply any changes made in the incremental backup. Therefore, providing a synthetic full backup may have the added advantage of allowing the backup storage application to save data files more quickly based on the synthetic full backup, without requiring multiple saves to be done from the full backup and one or more incremental backups. It will be appreciated that the term "latest version" as used herein generally refers to the latest copy of a data file (eg, when the data file was last saved), regardless of whether the file has a new version number. The term "version" as used herein generally refers to a copy of the same file that may be modified in some way or may be saved multiple times.

Referring to FIG. 7, a diagrammatic depiction of an exemplary synthetic full backup procedure. Host 120 may perform full backup 230 at the first time, for example, on a weekend. Host 120 may perform continuous incremental backups 232a, 232b, 232c, 232d, and 232e, for example, for each day of the week. Storage system 170 may create a new synthetic full backup data set 234, as described below.

According to one embodiment, storage system 170 may include a software application, referred to herein as composite full backup application 240 (see FIG. 3 ). Synthetic full backup application 240 may run in storage system controller 122 (see FIG. 2 ) or in host 120 . The synthetic full backup application includes the software commands and interfaces necessary to create a synthetic full backup data set 234 . In one embodiment, the synthetic full backup application may perform a logical merge of the metadata representing each full backup data set 230 and incremental backup data set 232 to produce a new virtual encoded tape containing the synthetic full backup data set 234 .

For example, referring to FIG. 8, the existing complete backup data set may include user files F1, F2, F3 and F4. The first incremental backup data set 232a may include a modified version of the user file F2' of F2, and a modified version of F3' of the user file F3'. The second incremental backup data set 232b may include the user file F1' of the modified version of F1, and the further modified version F2" of F2, and the new user file F5. Therefore, from the full backup data set 230 and the two incremental data The synthetic complete backup data set 234 formed in the logical merging of sets 232a and 232b includes the latest version of each user file F1, F2, F3, F4 and F5.As shown in accompanying drawing 8, the synthetic complete backup data set here Including user files F1', F2", F3', F4 and F5.

Referring again to FIGS. 3 and 4 , the file system software 146 may create a logical metadata cache 242 that stores metadata associated with each user file stored in the emulation medium 134 . It will be appreciated that the logical metadata cache need not be an actual data cache, but may replace queryable collection data stored in the storage medium 126 . In another example, logical metadata cache 242 may be implemented as a database. Metadata is stored in the database, and traditional database commands (for example, SQL commands) can be used to complete the logical combination of the full backup data set and one or more incremental backup data sets to create a new synthetic full backup data set.

As discussed above, each data file stored on emulation medium 134 may include a file descriptor containing metadata associated with the data file, including the location of the file in backup storage medium 126 . In one embodiment, a backup/restore application running on host computer 120 saves data in tape stream format on emulated medium 134 . The embodiment in data structure 250 represents the tape format illustrated in FIG. 9 . As discussed above, system file data structures include headers that may contain information about the data file, for example, the file descriptor for the data file, new and/or modified file data, security information, the host system from which the file originated directory structure, and other information linking the files to the virtual encoded tape. The header is associated with data 254, which is the actual user and system files that have been backed up from the host, primary storage system, etc. The system file data structure may also randomly include pads 256 which may properly align the next header to the region boundary.

As shown in FIG. 9, in one embodiment header data is placed in a logical metadata cache 242 to allow fast lookup and random access to a continuous tape data format. The use of a logical metadata cache, accomplished by the file system software 148 stored in the system controller 122, allows translation of the linear, sequential tape data format stored in the emulation medium 134 into the data stored in the supplemental backup storage medium 126. Random access data format on physical disk. Logical metadata cache 242 stores headers 252 that include file descriptors for data files, security information that is used to control access to data files, as discussed in more detail below, indicators 256 corresponds to the real location of the data file in the virtual encoded tape and backup storage medium 126 . In one embodiment, the logical metadata cache stores data related to all data files for each share of data in the full backup data set 230 and the incremental data set 232 .

According to one embodiment, the synthetic full backup application software 240 uses information stored in the logical metadata cache to create a new synthetic full backup data set. The synthetic full backup data set is linked to a virtual encoded tape created by the synthetic full backup application 240 . For backup/restore applications, the synthetic full backup data set is ostensibly stored on synthetic virtual encoded tape. As discussed above, synthetic full backup datasets can be created by performing a logical merge of existing full and incremental backup datasets. A logical merge may include comparing each data file contained in each existing full backup data set and incremental backup data set, and creating a composite of the latest modified version of each user file, as in See discussion with reference to Figure 8.

According to one embodiment, the synthesized virtual coded tape 260 includes indicators indicating the location of data files within other virtual coded tapes, notably, as shown in FIG. There are full backup data sets and incremental backup data sets. Considering the embodiment given in the previous accompanying drawing 8, the synthetic virtual encoded tape 260 includes an indicator 266 which points out (marked with arrow 268) the user file F4 (since the existing complete backup data set includes the latest version) in the existing full backup data set on the virtual encoded tape 262, for example, the location of the user file F3′ in the incremental data set 232a on the virtual encoded tape 264.

The composite virtual coded tape may also include a list 270 containing the identification codes (optionally named) of all virtual coded tapes that include the data indicated by the indicator 266 . The attached coded tape list 270 is important to keep track of the real data and to prevent the attached dummy encoded tape from being degaussed. In the described embodiment, the synthetic full backup data set does not include any actual user files, but an additional set of indicators indicates where the user files are located in the backed up storage medium 126 . Therefore, the description (location of the user files in the backup storage medium 126) is required to prevent the real user files (stored in the otherwise virtual encoded tape) from being deleted. This may be accomplished in part by keeping a record of the virtual encoded tapes containing the data (subsidiary encoded tape directory 270), and protecting each of said virtual encoded tapes from overwriting or erasure. The synthesized virtual encoded tape may also include encoded tape data 272, eg, the size of the synthesized virtual encoded tape, its location in the backup storage medium 126, and the like. Additionally, the synthesized virtual coded tape has an identification code and/or name 274 .

According to another embodiment, the synthetic virtual encoded tape may also include a combination of pointers and actually stored user files. Referring to FIG. 11 , in one embodiment, the composite virtual encoded tape includes an indicator 266 indicating that the data file (latest version, as discussed with reference to FIG. The location in the backup data set 230 . The synthesized virtual encoded tape may also include data 278 containing actual data files copied from incremental data set 232 as indicated by arrow 280 . In the manner described, incremental backup data sets may be deleted after synthetic full backup data set 276 is created, thereby saving storage space. The resulting virtual encoded tape is relatively small due to the inclusion of all or part of the pointer rather than a copy of all user files.

It will be appreciated that a synthetic full backup may include any combination of indicators and stored file data and is not limited to the examples given above. For example, a synthetic full backup may include pointers to data files for some files stored in some incremental and/or full backup, and include data files from other existing full and/or incremental backups Stored data files copied in . And alternatively, a synthetic full backup can be created from an existing full backup and any associated incremental backups, which do not include any indicators, but include the preferred full and/or incremental backups from the The latest version of the real data file copied in .

In one embodiment, synthesizing a full backup application may include a differential operation that compares user and system file metadata for each existing full backup data set and incremental data set to determine the The location of the latest version of the data file. For example, differential operations can be used to compare new and/or modified data, version numbers (if available), etc., with different versions of the same data file in different backup sets selecting the most recent version of the data file. However, users can open user files and save files (thus changing modified data) without actually changing any data in the file. Therefore, the system can perform more complex differential operations, and can analyze data in system or user files to determine whether the data has indeed been modified. Transformations of the differential operation and other comparable types of algorithms are well known in the art. Additionally, as discussed above, when metadata is stored in a database format, database commands, eg, SQL commands, can be used to perform logical merges. The present invention may employ any of the differential operations described to ensure that the most recent or up-to-date version of each user file is selected from all data sets compared to existing backups to suitably generate a composite full backup data set.

Those of ordinary skill in the art will appreciate that a synthetic full backup application can create a complete backup data set, and can obtain an actual full backup without requiring the host to perform it. Not only is the added host and processor cost of transferring data to the backup storage system avoided, but in embodiments the composite full backup application can be executed on the storage system, which can significantly reduce network bandwidth utilization. As shown in FIG. 7 , further synthetic full backup data sets may be created using the first synthetic full backup data set 234 and subsequent incremental backup data sets 236 . Among the significant time advantages provided, files or objects are not frequently modified, frequently copied. In fact, the synthetic full backup data set may retain pointers in the files that were just copied.

As discussed with reference to FIG. 3 , the storage system may include a software application involving an end-user recovery application 300 . Thus, according to another embodiment, a method for identifying and restoring backup data is provided to end users without requiring IT workgroup inventions, and without requiring any changes to existing backup/restore processors and/or policies. Change. In a typical backup storage system, the backup/restore application running in the host 120 is controlled by the IT workgroup, and there is no possibility for end users to access the backed up data without the invention of the IT workgroup or very difficult. According to various aspects of embodiments of the present invention, storage system software is provided to allow end users to locate and restore files by interfacing, for example, to network-based or other backup storage media 126 .

It will be appreciated that due to the use of the synthetic full backup application 240, the end user restores the application 300 which may be running on the storage system's controller 122 (see FIG. 2) or on the host 120. The terminal restore application includes the software commands and interfaces necessary to allow authorized users to query the logical metadata cache, restore at random, files backed up from the backup storage medium 126 .

According to one embodiment, the provided software includes a user interface installed on the user computer 136 and/or executed on the user computer 136 . The user interface may be any type of interface that allows a user to determine the location of a file in the backup storage medium. For example, the user interface may be a graphical user interface, may be web-based, or may be a textual interface. User computers are coupled to storage system 170 through network connection 138, which may be, for example, an Ethernet connection. Through network connection 138 , an operator of user computer 136 may access data stored in storage system 170 .

In one embodiment, the end user restores the application 300 including the user's credentials and/or authorization features. For example, a user may register through a user interface on the user's computer using a user name and password. The user computer can communicate the user name and password to the storage system (eg, to the end user recovery application) and can use preferred user authorization means to determine whether the user has accessed the storage system. Some examples of user authorization means may include, but are not limited to, Microsoft Active Directory servers, Unix "Yellow Pages" servers, or standard directory access protocols. The registration/user authorization means may communicate with the end user recovery application in exchange for the user's privileges. For example, some users may be allowed to only query files created by themselves, or files with certain privileges or marked as owners. Other users, for example, system operators or authorized persons can access all backup files and so on.

According to one embodiment, the end-user restore application uses the logical metadata cache to obtain information about all data files backed up to the backup storage medium. The terminal recovery application presents the user with a user interface, hierarchical directory structure where user files are stored, e.g. backup time/data, username, initial user computer directory structure (obtained when files were backed up), or other file characteristics. In one embodiment, the directory structure that appears to the user can be changed according to the user's privileges. The end-user restore application can receive browsing requests (for example, through a user interface, the user can browse the directory structure to the location of the desired file) or the user can query files by name, date, etc.

According to one embodiment, the user can restore backup files from the storage system. For example, once the user locates the desired file, as discussed above, the user may download the file from the storage system via network connection 138 . In one embodiment, the download program may be downloaded in any manner other than network-based downloads, as known to those of ordinary skill in the art.

By allowing end-user access to other files that allow browsing/downloading, and by facilitating access through a user interface (e.g., web-based termination), an end-user recovery application can enable users to query and restore their own files without altering any backups policy or procedure.

According to another embodiment, apparatus and methods are provided whereby a user may install a network attachment view of a backup data set stored in the backup storage medium 126 . This allows users to browse and access data in installed datasets, as users will be entering data on any local or network drives coupled to their computers. Thus, for example, a user can restore the availability of application server (eg, when system main storage 106 fails, see FIG. 1 ) data without performing a recovery procedure through the media server 114 (see FIG. 1 ). Restoring application server data using the setup described in this article can be orders of magnitude faster than typical media servers for volume restores. It will be appreciated that the term "mount" as used herein refers to making a data volume or network component, such as a network drive, available to the host operating system. A data volume may include, for example, a single data file or system file, a plurality of files, or a directory structure including a plurality of files. Common mount protocols include common parts of NFS (Network File System) and CIFS (Common Internet File System). These protocols allow a host to access resources on another host over a network connection through an interface that makes the remote resource appear to reside locally on the host.

Referring to FIG. 12 , there is illustrated a flow diagram of one embodiment of a method for performing volume mounts in accordance with various aspects of the present invention. In the first step 290, the user selects a data volume to be mounted, and sends a volume mount request to the controller 122 of the backup storage system (refer to FIG. 3 ). Typically, users can restore data from a full backup dataset (not just an incremental backup dataset) to obtain a complete and correct representation of the backup information. If the current full backup data set does not exist, (for example, the network administrator may perform full backups weekly, so if the user wishes to restore data during the week, the current full backup may not be available), a new synthetic full backup may be created. Backup (as above) and use to restore selected data.

According to one embodiment, backup storage system 170 may include a software application, with respect to volume restore application 310 herein (see FIG. 13 ), that may control and implement methods for performing data volume mount and restore procedures. Volume restore application 310 is similar to a composite full backup and end-user restore application and can execute on host and/or user computers, where a portion of the program is distributed across all or part of the storage system controller, host, and user computers.

Referring again to FIG. 12, after requesting a volume mount, the volume recovery application may be challenged regardless of whether a current full backup is available (step 292). If not, the volume recovery application may communicate with the synthetic full backup application 240 (see FIG. 2) to execute the synthetic full backup handler and create a new current backup data set (step 294). The volume recovery application may output to a regular full backup data set or a synthetic full backup data set to perform volume mount requests, which may be common to NFS or CIFS. In particular, the volume recovery application queries the logical metadata cache 242 to determine the preferred metadata representing the selected full backup volumes marked in step 290 .

According to one embodiment, the mount request (in step 290) may cause the volume recovery application to create one or more file descriptor structures to facilitate the export of mounted volumes, as common to NFS or CIFS (step 296). With respect to Figure 14, one embodiment of a file descriptor structure 320 that may be created by a volume recovery application is illustrated, the file descriptor 320 corresponding to the tape format (e.g. system file 332, see Figure 15) system files. As described above, a file descriptor includes queryable metadata corresponding to data files and system files stored in the storage system. The file descriptor 320 may include a number of sections containing information such as a filename 322 and a permission file (access control list) 324 for the data files contained in the volume to be mounted. In addition, the file descriptor includes one or more indicators 326 to determine the location of the source data of the data file (for example, whether the data file is stored in the storage medium 126), the length 328 of the data file, and the link list file A pointer 330 to the next entry (eg, next data file) in the descriptor structure. If the "next" file is empty, such as indicated by reference numeral 331, then it indicates that the data file is the last of the system files represented by file descriptor 320 (e.g., last link list entry) data file. As explained in Figure 14, each system file included in the data volume to be mounted will be represented by a file descriptor structure. Once a file descriptor 320 is established for each system file in the requested volume, the file descriptor can respond to NFS or CIFS requests to locate and export the associated data files.

As discussed above, in one embodiment, file descriptors may be implemented according to a standard format, for example, the tape archive file format (tar: a compressed file format) used by most UNIX-based systems (multi-user computer operating systems). file extension). As illustrated in Figure 15, an example of a typical system file 332 would be written in tape format (eg, tar format) as part of a tape (eg, tar) data source. FIG. 16 illustrates file descriptors 340 corresponding to system files 332 . As illustrated in FIG. 15 , files written in tape format include header 336 and actual data 338 stored in system files 332 . The data 338 may correspond to one or more data files. In the illustrated embodiment, the system file 332 is 1032 bytes in length, however, it will be understood that the file may be of any length, depending on the size and writing format of the file.

File descriptor 340 for file 332 is included in header 336 . According to the explanation of the accompanying drawing 16 and the general embodiment given with reference to the accompanying drawing 14, the file descriptor 340 includes the file name 341 and the security information 334 of the storage data of each data file known to the system file , an indicator 342, the length 346 of the corresponding data file, and the "next" entry indicating the next data file known to the system file, in the embodiment used for explanation, the "next" entry is empty 348 .

Referring again to accompanying drawing 12, once all the file descriptors of the files in the data volume to be mounted have been established, the volume recovery application program uses the file system as an NFS or CIFS share on the basis of the established file descriptors (step 298) Output to a specific user mount point. At the mount point, the mount is complete (step 299), and the mounted data volume is available for users to read and/or write data, as will be described in detail below.

According to one embodiment, NFS or CIFS read operations (eg, a user wishes to browse data in a mounted data volume) are provided by querying the file descriptor 320 for a matching file specification. It will be appreciated that, according to one embodiment, the user need not query the file descriptors themselves. Rather, the volume recovery application includes a user interface that presents the data to the user, eg, in a typical dictionary-structured format. The volume recovery application may include software that translates a user request for a particular file into a query command that accesses the logical metadata cache and looks up a file descriptor 320 for a matching system file. Once the location of the file is determined, the conversion to the data in the user's computer can be done by tracking the connection list (for example, tracking the pointer stored in the file descriptor to determine the location of the actual data) and building a buffer for the file data, said File data will be sent to the requesting user.

According to another embodiment, the device provided to the user may also write new data to the installation volume. As discussed above, mounted volume data may appear to users as normal network drives or other network storage data. However, in fact, the original mounted volume data is the data that is actually backed up, and this data usually needs to be preserved; at least until another backup data set is created. Therefore, allowing users to actually modify the original backup data may not be ideal. To avoid modifying the original backup data, and still allow the user to modify the data corresponding to the mounted volume, means are provided (obvious to the user) to offload writes to the storage system, as discussed below.

Referring to FIG. 17, there is illustrated a flowchart of one embodiment of a method of processing write requests in accordance with various aspects of the present invention. In a first step 350, the user requests an NFS or CIFS write operation (typically by selecting the "Save" option during editing or browsing a data file). The volume recovery application then executes the write request by determining the available storage space, writing the data to that space, and updating the appropriate file descriptors to reference the latest written data.

According to one embodiment, the volume recovery application queries whether space has been allocated for the written data (step 352), and if not, the volume recovery application allocates storage space (step 354). Storage space may be allocated in backup storage medium 126 (see FIG. 13). The allocated storage space can be specially designed to only hold write data (random associated metadata).

Referring to FIG. 18, an embodiment of NFS or CIFS write data stored in the backup storage medium 126 is explained. Write data 360 includes, for example, two write portions, W1 362 and W2 364, corresponding to stored data resulting as a result of a write command provided by the volume recovery application. For example, W1 and W2 correspond to modified data files included in the mounted data volume. It will be appreciated that although the illustrated example corresponds to two write requests, the principles of the invention may be applied to any number of write requests, and that files may be adapted to be altered to reflect any suitable number of write requests. The written data 360 may also include a header 366 that includes metadata that forms a self-describing relationship between the original data (eg, file 332 ) and the most recently written data 360 . In particular, the header may include offset information indicating the written data portions W1 and W2 that exist logically relative to the original data, as further described in connection with FIG. 19 .

Referring to Figure 19, an embodiment of the system file layout when two write requests are made is illustrated. The raw data file 332 is stored in the backup storage system medium 126 (refer to FIG. 13 ), and appears to the user by installing the program described above. As shown in FIG. 19, system files 332 are written in tape format, and data portion 338 may include numerous data files (eg, user files). The data starts at offset zero bytes (point 370 ) and ends at point 372 after the end 1032 bytes. Write to file 360 corresponds to a user request to write data into file 332 . For example, the user can modify two data files included in the system file 332 to obtain the written file 360 including W1 and W2. As described above, the write file 360 may be stored separately from the file 332 in the storage medium, so that the original backup data is not changed. Logical modification system files 380 are used for illustration and represent files 332 that include changes made by users through write requests (eg, written to file 360). In other words, in the modified system file 380, W1 and W2 (user-modified data files) may be used to replace the original data files included in the data portion of the original data file 332 without deleting the backup data.

As shown in FIG. 19 , the modified system file corresponds to the logical combination of the original system file 332 and the written file 360 . As shown, the original system file data 338 begins with the offset zero of the original file. At offset 64 (indicated by reference numeral 384 ), the first portion W1 of the modified data begins and ends at a position 9 bytes along offset 73 indicated by reference numeral 386 . Thus, W1, the user-modified data file from the user write request may be used to replace the original data file that determines the location of the compensation 64 in the original system file 332 . As shown, W1 is 9 bytes in length because W1 begins with offset zero written to file 360 (390) and ends with offset nine written to file 360 (392). The starting position of W1 in the modified file (offset 64 in the illustrated example), ie the relative relationship between the written file 360 and the original file 332, is determined by the information stored in the header 366. The W2 section is also included in the modified file 380, starting at offset 1032 (the original end of the file, indicated by reference numeral 372), and extending logically to the 100 byte file. Also, the length of W2 is determined by the location information in header 366 . The new endpoint of the file is indicated by reference numeral 388 .

It will be appreciated that the latest written data represented by file 360 is not actually stored as part of original file 332, although modified file 380 is logically new and represents a user-modified version of the original file. Instead, as discussed above, the most recently written data is stored in a specific location on the storage medium that identifies written data. In this way, the integrity of the original backup data files is maintained, while allowing the user to explicitly write to the mounted volume, as they may be the original location or a network drive.

Modified file 380 includes header 382 that includes a file descriptor representing the modified file. Referring to FIG. 20, an embodiment of a file descriptor 400 is illustrated. File descriptor 400 includes a name portion 402 identifying the file name of modified file 380 and a security portion 404 identifying permission attributes of modified file 380 . The file descriptor 400 also includes a number of data sections including a pointer to the original file 332 and a pointer to the write file 360 to access the data stored in each of the original file and the write file. The expression for modifying the file 380 is derived by continuously following the linked list of pointers given in the file descriptor 400 .

Referring to FIG. 19 and FIG. 20, a particular embodiment of a file descriptor for modifying a file is explained. In the first data portion 406, an indicator is located which determines the location of the first data file which, in the modified file 380, is located at offset zero bytes, as indicated by reference numeral 408 in FIG. 19 . Subsequent portion 410 indicates the length of the data file, the location of which is indicated by indicator 406 . In the illustrative embodiment, the length is 64 bytes, as can be derived from Figure 19 (data provided between the zero offset point, reference numeral 408, and the 64 byte offset, indicated by reference numeral 384). The next section 412 indicates that the next data file in the modified file 380 is W1, as shown in FIG. 19 . Accordingly, an indicator 414 indicating the location of the data corresponding to W1 is stored in the zero offset point most recently written to file 360 (reference numeral 390 in FIG. 19). Length section 416 indicates that W1 is 9 bytes in length, as can also be seen in FIG. Next section 418 indicates that the next data file in modified file 380 is a data file from original system file 332 . The indicator in section 420 indicates the offset 73 location of the next data file in modified file 380, as indicated by reference numeral 386 in FIG. 19 . Part 422 indicates that the length of the data file is 959 bytes, and reference may also be made to FIG. 19 . The next section 424 indicates that the subsequent data file is W2. Again, the indicator in the part 426 indicates the location of W2, that is, the location of the latest written file 360 at offset 9, which can be referred to FIG. 19 . Section 428 indicates that W2 is 100 bytes in length, and the next section 430 includes a slot indicating that W2 is the latest data file in modified file 380, as shown in FIG. 19 . Accordingly, the file descriptor 400 includes a "road map" that identifies the structure and data locations of the modified file 380 included in the modified file 380 .

Volume recovery applications and methods described above represent continuous tape format data in a suitable format random access I/O system, such as NFS or CIFS. Link list file descriptors, such as file descriptor 400, can be used to convert continuous tape format data to random access data by recording the location of each data file for a particular tar source on the storage medium, e.g., and each One data file in the tar source is done with other data files relative to the location of the tar source. Additionally, according to one embodiment, the volume recovery application may include provisions for converting (e.g., writing) the data to a representation in a tape format (e.g., tar) so that the backup/restore application can access the data in the general manner described above . According to one embodiment, the instant recovery application includes a facility to generate a virtual encoded tape properly formatted with tape headers, pads, data and filemarks in the manner described above in relation to file system software. In another embodiment, the volume recovery application interfaces with the file system software to create a new virtual encoded tape as discussed above that includes the most recently written and modified files.

It will be appreciated that although aspects of the present invention, such as the synthetic full backup application, end-user restore application, and volume restore application, are described herein primarily in terms of software, said and other aspects may optionally be described in software, hardware, or firmware, or any combination thereof. Thus, for example, embodiments of the present invention may include any computer-readable medium (e.g., computer memory, floppy disks, compact disks, tapes, etc.) encoded with instructions (e.g., a multitude of instructions) when stored in When executed on a processor of a storage system, at least in part implements the functions of a composite full backup application and/or an end-user restore application, as described in detail above.

In general, embodiments and aspects of the present invention include storage systems and methods that emulate traditional tape backup systems, but may provide enhanced functionality, for example, the ability to create synthetic backups and allow end users to browse and restore backup files . However, it will be appreciated that aspects of the present invention may be used for more than just backup of computer data. Because the storage system of the present invention can be used to economically store large amounts of data, and can randomly access stored data in reverse order within hard disk access times, embodiments of the present invention may find application outside of traditional backup storage systems. For example, embodiments of the present invention may be used to store a greater selection of video or audio data representing movies and music, and enable video and/or audio as desired.

Having thus described several aspects of at least one embodiment of this invention, it is to be appreciated various alterations, modifications, and improvements will readily occur to those skilled in the art. The alterations, modifications and improvements are intended to be part of this disclosure, and are intended to be within the scope of the present invention. Accordingly, the foregoing description and drawings are by way of example only.

Claims (11)

1. method, it comprises the steps:

Book is installed on the main frame, and this book comprises at least one one data file, and described data file is corresponding with the up-to-date backup version of at least one one data file in being stored in backup storage system;

The corresponding data of second version of storage and described at least one one data file in backup storage system, when preserving the up-to-date backup version of a at least data file, described data are upgraded than the up-to-date backup version that is stored at least one one data file in the backup storage system.

2. according to the method for claim 1, further may further comprise the steps:

Connect the up-to-date backup version of described at least one one data file and second version of described at least one one data file.

3. according to the method for claim 1, further may further comprise the steps:

Newdata structure, this data structure had both indicated the up-to-date backup version of described at least one one data file, also indicated second version of described at least one one data file.

4. according to the method for claim 3, second version of wherein said at least one one data file is the revision of the up-to-date backup version of described at least one one data file.

5. according to the process of claim 1 wherein that the step of installation data volume comprises that execution NFS installs or CIFS one of installs.

6. according to the method for claim 1, wherein the step of installation data volume comprises that foundation comprises the filec descriptor of the metadata relevant with the up-to-date backup version of described at least one one data file, and described metadata comprises the designator of the memory location of up-to-date backup version in backup storage medium that indicates described at least one one data file.

7. backup storage system, this system comprises:

The backup storage medium that is used for the store backup data collection; And

Controller, this controller comprise at least one realizes the method in the claim 1 with execution through the processor of configuration a series of instructions.

8. as claim 7 backup storage system required for protection, wherein said backup data set is the full back-up data sets of synthesizing.

9. the computer-readable medium that is composed of numerous order numbers, when carrying out at least one processor, described numerous order numbers are realized the method for claim 1.

10. as claim 9 computer-readable medium required for protection, wherein said processor is included in the backup storage system.

11. a computer readable media store that is composed of numerous order numbers has data structure, this data structure comprises:

First indicates kit, and this first sign kit indicates the system file corresponding with the backup data set that comprises at least one one data file uniquely; And at least one second sign kit, this second memory location separately that indicates in kit sign storage medium, described storage medium stores the latest edition of concentrated each one data file at least of Backup Data.

Images