CN102200935A - Method for storing data in stream media system - Google Patents
Method for storing data in stream media system Download PDFInfo
- Publication number
- CN102200935A CN102200935A CN2011101056797A CN201110105679A CN102200935A CN 102200935 A CN102200935 A CN 102200935A CN 2011101056797 A CN2011101056797 A CN 2011101056797A CN 201110105679 A CN201110105679 A CN 201110105679A CN 102200935 A CN102200935 A CN 102200935A
- Authority
- CN
- China
- Prior art keywords
- disk
- block
- outskirt
- inner region
- blocks
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title abstract description 12
- 238000013500 data storage Methods 0.000 claims description 12
- 230000007257 malfunction Effects 0.000 claims description 3
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 241001269238 Data Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000002950 deficient Effects 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a method for storing data in a stream media system. The method comprises the following steps of: 1, cutting a stream media file into slices according to the same playing time span; 2, dividing each disk of a server into two regions including an internal region and an external region with the same capacity, respectively dividing the internal region and the external region into disk blocks with the same size and establishing index information; 3, dividing the slices into multiple slice blocks according to the sizes of the disk blocks and uniformly allocating the slice blocks onto the disk blocks of the external regions of all disks; 4, uniformly allocating the slice blocks onto the disk blocks of the internal regions of all disks and ensuring that the same slice block is allocated to the internal regions and the external regions of different disks; 5, writing the slice blocks into the internal regions and the located external regions of the slice blocks, and generating allocation index information for disk numbers and disk block numbers allocated for the internal regions and the located external regions of the slice blocks; and 6, re-allocating the disk blocks if a write error is generated due to an error of a disk block during the writing, and reading the correct slice from another region for duplication. By using the method, the IO (Input/Output) capacity of the system is improved, the data restoration is simple, and the system is simple.
Description
Technical field
The present invention relates to the stream media system field, specifically is a kind of streaming media system data storage means.
Background technology
What stream media system was stored is jumbo media data, finish a large amount of concurrent real-time flow transmission to these media datas, except needs network I/O bandwidth correspondingly, the disk I bandwidth that matches will be arranged also.How with data storage on streaming media server, on server, how to organize each disk, directly determined the concurrent ability of server and the security of media data.
In some present stream medium distributed storage systems, data are divided into a lot of sections, section is by the disk block size, strip-type ground is stored on each disk one by one, the number of disks of system configuration is probably less than the piece number of each section, that is to say that section all has content at each disk, all disks read concomitantly, to improve the IO bandwidth of system.But disk is an equipment that is easier to damage, and along with the prolongation of service time, disk performance descends gradually, causes reading some disk block overlong time, obviously reduces the IO of system bandwidth.
If certain disk breaks down, system is out of service, and the service that provides is provided again, if do not do data backup, the data of overwhelming majority section on this disk on the streaming media server all will be lost, and be referred to as junk data, even may cause heavy losses.During on-demand access, what see will be the picture that jumps, and when more serious, may cause the collapse of streaming media server.In order to recover this part data, simple effective method is exactly the content that empties on the whole server, upload again or come from other server copy, but this data volume is very huge, will consume the long time and the very big network bandwidth.But, if only copy the blocks of data that those are lost, and regenerate the index record in when visit, again the complexity of system software is had relatively high expectations, and be unfavorable for large-scale use.
Summary of the invention
At the defective that exists in the prior art, the object of the present invention is to provide a kind of streaming media system data storage means, the IO ability of raising system, section is preserved two parts, lays respectively at different disk, when one of them disk breaks down, read the Backup Data of another disk, it is simple that data are recovered, and needn't consume for a long time and than the macroreticular bandwidth, requires lower to the complexity of system software.
For reaching above purpose, the technical scheme that the present invention takes is: a kind of streaming media system data storage means comprises the steps:
(1) files in stream media is cut into section by identical reproduction time length;
(2) each disk of server is divided into the identical inner region of capacity and outskirt two parts, inner region and outskirt are divided into equal-sized disk block, and set up index information;
(3) the disk block size is pressed in described section, be divided into a plurality of slicing blocks, and be evenly distributed on the disk block of outskirt of all disks;
(4) described slicing block is evenly distributed on the disk block of inner region of all disks, and guarantees that same slicing block is assigned to the inner region and the outskirt of different disk;
(5) described slicing block is write the outskirt and the inner region at its place, and be disk number and disk block number generation allocation index information that slicing block place outskirt and inner region distribute;
(6) fashionable if slicing block is write, a certain district disk block is wrong, causes write error, then redistributes disk block in this district, and reads correct slicing block from another district and duplicate.
On the basis of technique scheme, the storage size of described section has nothing in common with each other.
On the basis of technique scheme, when described section distributes disk,, distribute the disk of residual memory space maximum earlier according to the size of disk residual memory space.
On the basis of technique scheme, read according to following steps the storage back:
1. the time point of playing according to request is determined the section of system's visit;
2. according to the allocation index information of described section, obtain being stored in the disk number, disk block number of outskirt and inner region;
3. if outskirt place disk operate as normal, and this disk block number effectively, then preferentially reads outskirt; Otherwise read the disk block number of inner region place disk;
4. if find that repeatedly reading a certain disk block mistake occurs, this disk block of mark is unavailable, and redistributes disk block, initiates to duplicate.
On the basis of technique scheme, the section of described system visit is tried to achieve divided by section time size by the play time of request.
On the basis of technique scheme, if during described disk cisco unity malfunction, from the inner region visit of other disk, online replacing disk copies data to disk after the replacing from other disk.
Beneficial effect of the present invention is:
I. will cut into slices and press the disk block size, and be divided into a lot of slicing blocks, and be stored in one by one on the disk of streaming media server, disk is read and write media content concomitantly, improves the whole disk I ability of system.
If II. disk breaks down in the discovery system, can onlinely extract this disk, be positioned at all data on this disk, because at other Backup Data is arranged all above the disk, system can also normally provide service, guarantees that broadcasting pictures is steady.
III. after inserting a new disk, record according to the section index, with the section that should be stored on other disk on the new disk, copy new disk to, it is simple that data are recovered, guarantee that data block has two parts of copies in order to the damage of disk, needn't consume for a long time and, require lower the complexity of system software than the macroreticular bandwidth.
Embodiment
Below embodiments of the invention are described in further detail.
A kind of streaming media system data storage means, in the present embodiment, streaming media server has 4 disks (D0, D1, D2, D3), and the capacity of each disk is 1GB, and the disk storage block size is 2MB.
Step is as follows during the present embodiment storage:
(1) files in stream media is cut into section by identical reproduction time length, and the storage size of cutting into slices among the described different embodiment has nothing in common with each other, in the present embodiment, slice size is 14MB.
(2) respectively each disk of server (D0, D1, D2, D3) is divided into outskirt and inner region two parts, size all is 500MB, outer regional address be [0,500MB), interior regional address be [500MB, 1000MB).Inner region and outskirt are divided into equal-sized disk block, are unit with the disk block, for disk inner region and outskirt are set up index information, outskirt available disk piece number be [0,249), inner region available disk piece number be [250,499).Size and disk block according to section determine that slicing block quantity is 7, and what deposit outskirt in is designated as M0 respectively, M1, and M2...., M6, what deposit inner region in is designated as S0, S1, S2...., S6 respectively.
Distribute outskirt when (3) storing earlier, when selecting disk block, consider the load balancing of disk,, select the disk of outskirt remaining space maximum to distribute according to the size of disk residual memory space for slicing block.Earlier slicing block M0 is distributed in first disk block B0 of first disk D0, the slicing block with other according to said method distributes again.
(4) slicing block is assigned to inner region according to the method described above, the selection of disk number should can not be for same with outskirt, from remaining disk, select the disk of inner region vacant space maximum again, when a plurality of disk inner region vacant spaces equate, select one at random, can not can avoid two mutually redundant situations of disk like this by the disk select progressively.
(5) described slicing block writes the outskirt and the inner region at its place, and is disk number, disk block number generation allocation index information that slicing block place outskirt and inner region distribute.Qie Pian first slicing block M0 for example is distributed in first disk block B0 of first disk D0, be designated as M0, D0, B0}, the data block of subsequent allocations is designated as { M1, D1, B0}, { M2, D2, B0}, { M3, D3, B0}, { M4, D0, B1}, { M5, D1, B1}, { M6, D2, B1}.The index information of setting up at inner region can be { S0, D1, B0}, { S1, D2, B0}, { S2, D3, B0}, { S3, D0, B1}, { S4, D2, B1}, { S5, D3, B1}, { S6, D0, B1}.
(6) fashionable if slicing block is write, a certain district disk block is wrong, causes write error, then redistributes disk block in this district, and reads correct slicing block from another district and duplicate.
When streaming media server read described section and plays, section was stored in all disks, and these disks all can be received read request in proper order, and finish read operation.When streaming media server need support that a large amount of concurrent differences are cut into slices read operation, system disk IO operation can be evenly distributed to all disks, improves entire system IO handling capacity greatly.And it is as follows that it reads order:
1. the time point of playing according to request again divided by section time size, is determined the section of system's visit.
2. according to the allocation index information of described section, obtain being stored in the disk number, disk block number of outskirt and inner region.
3. since the IO of outskirt read faster than inner region, the preferential disk block of selecting to read outskirt of system.When the outskirt disk block is out of order or this disk when being pulled out, system reads the disk at intra block place, the steady free of discontinuities operation of assurance system.
4. if find that repeatedly reading a certain disk block mistake occurs, this disk block of mark is unavailable, and redistributes disk block, initiates to duplicate.If during described disk cisco unity malfunction, from the inner region visit of other disk, online replacing disk copies data to disk after the replacing from other disk.
The present invention is not limited to above-mentioned embodiment, for those skilled in the art, under the prerequisite that does not break away from the principle of the invention, can also make some improvements and modifications, and these improvements and modifications also are considered as within protection scope of the present invention.
Claims (6)
1. a streaming media system data storage means is characterized in that, comprises the steps:
(1) files in stream media is cut into section by identical reproduction time length;
(2) each disk of server is divided into the identical inner region of capacity and outskirt two parts, inner region and outskirt are divided into equal-sized disk block, and set up index information;
(3) the disk block size is pressed in described section, be divided into a plurality of slicing blocks, and be evenly distributed on the disk block of outskirt of all disks;
(4) described slicing block is evenly distributed on the disk block of inner region of all disks, and guarantees that same slicing block is assigned to the inner region and the outskirt of different disk;
(5) described slicing block is write the outskirt and the inner region at its place, and be disk number and disk block number generation allocation index information that slicing block place outskirt and inner region distribute;
(6) fashionable if slicing block is write, a certain district disk block is wrong, causes write error, then redistributes disk block in this district, and reads correct slicing block from another district and duplicate.
2. a kind of streaming media system data storage means as claimed in claim 1, it is characterized in that: the storage size of described section has nothing in common with each other.
3. a kind of streaming media system data storage means as claimed in claim 1 is characterized in that: when described section distributes disk, according to the size of disk residual memory space, distribute the disk of residual memory space maximum earlier.
4. a kind of streaming media system data storage means as claimed in claim 1 is characterized in that: read according to following steps the storage back:
1. the time point of playing according to request is determined the section of system's visit;
2. according to the allocation index information of described section, obtain being stored in the disk number, disk block number of outskirt and inner region;
3. if outskirt place disk operate as normal, and this disk block number effectively, then preferentially reads outskirt; Otherwise read the disk block number of inner region place disk;
4. if find that repeatedly reading a certain disk block mistake occurs, this disk block of mark is unavailable, and redistributes disk block, initiates to duplicate.
5. a kind of streaming media system data storage means as claimed in claim 4 is characterized in that: the section of described system visit is tried to achieve divided by section time size by the play time of request.
6. a kind of streaming media system data storage means as claimed in claim 4 is characterized in that: if during described disk cisco unity malfunction, from the inner region visit of other disk, online replacing disk copies data to disk after the replacing from other disk.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011101056797A CN102200935A (en) | 2011-04-27 | 2011-04-27 | Method for storing data in stream media system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011101056797A CN102200935A (en) | 2011-04-27 | 2011-04-27 | Method for storing data in stream media system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102200935A true CN102200935A (en) | 2011-09-28 |
Family
ID=44661626
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2011101056797A Pending CN102200935A (en) | 2011-04-27 | 2011-04-27 | Method for storing data in stream media system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102200935A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102857554A (en) * | 2012-07-26 | 2013-01-02 | 福建网龙计算机网络信息技术有限公司 | Data redundancy processing method based on distributed storage system |
| CN104980786A (en) * | 2015-07-01 | 2015-10-14 | 广东工业大学 | Multi-path audio and video stream storage method with function of accidental stopping prevention |
| CN105094705A (en) * | 2015-07-27 | 2015-11-25 | 武汉兴图新科电子股份有限公司 | Method for optimizing disk storage strategy |
| CN111405313A (en) * | 2019-06-28 | 2020-07-10 | 杭州海康威视系统技术有限公司 | Method and system for storing streaming media data |
| CN113485966A (en) * | 2021-07-05 | 2021-10-08 | 中国联合网络通信集团有限公司 | Multimedia storage method and device, electronic equipment and readable storage medium |
| CN113641298A (en) * | 2020-04-27 | 2021-11-12 | 伊姆西Ip控股有限责任公司 | Data storage method, device and computer program product |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070204104A1 (en) * | 2006-02-17 | 2007-08-30 | Shenzhen Qifeng Chuangzhi Technology Corp. | Transparent backup service for networked computers |
| CN101127898A (en) * | 2007-09-20 | 2008-02-20 | Ut斯达康通讯有限公司 | Slice storage and streaming service method for stream media system and multimedia files |
| CN101571820A (en) * | 2009-05-27 | 2009-11-04 | 成都市华为赛门铁克科技有限公司 | Redundancy protected method, device and system |
-
2011
- 2011-04-27 CN CN2011101056797A patent/CN102200935A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070204104A1 (en) * | 2006-02-17 | 2007-08-30 | Shenzhen Qifeng Chuangzhi Technology Corp. | Transparent backup service for networked computers |
| CN101127898A (en) * | 2007-09-20 | 2008-02-20 | Ut斯达康通讯有限公司 | Slice storage and streaming service method for stream media system and multimedia files |
| CN101571820A (en) * | 2009-05-27 | 2009-11-04 | 成都市华为赛门铁克科技有限公司 | Redundancy protected method, device and system |
Non-Patent Citations (1)
| Title |
|---|
| 万继光: "集群多媒体存储系统的数据组织研究", 《中国博士学位论文全文数据库》, 15 May 2009 (2009-05-15), pages 16 - 20 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102857554A (en) * | 2012-07-26 | 2013-01-02 | 福建网龙计算机网络信息技术有限公司 | Data redundancy processing method based on distributed storage system |
| CN102857554B (en) * | 2012-07-26 | 2016-07-06 | 福建网龙计算机网络信息技术有限公司 | Data redundancy processing method is carried out based on distributed memory system |
| CN104980786A (en) * | 2015-07-01 | 2015-10-14 | 广东工业大学 | Multi-path audio and video stream storage method with function of accidental stopping prevention |
| CN104980786B (en) * | 2015-07-01 | 2019-02-22 | 广东工业大学 | A multi-channel audio and video stream storage method for preventing accidental suspension |
| CN105094705A (en) * | 2015-07-27 | 2015-11-25 | 武汉兴图新科电子股份有限公司 | Method for optimizing disk storage strategy |
| CN111405313A (en) * | 2019-06-28 | 2020-07-10 | 杭州海康威视系统技术有限公司 | Method and system for storing streaming media data |
| CN111405313B (en) * | 2019-06-28 | 2023-11-14 | 杭州海康威视系统技术有限公司 | Method and system for storing streaming media data |
| CN113641298A (en) * | 2020-04-27 | 2021-11-12 | 伊姆西Ip控股有限责任公司 | Data storage method, device and computer program product |
| CN113485966A (en) * | 2021-07-05 | 2021-10-08 | 中国联合网络通信集团有限公司 | Multimedia storage method and device, electronic equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10977124B2 (en) | Distributed storage system, data storage method, and software program | |
| US9858013B1 (en) | Scalable distributed storage system infrastructure | |
| US6571351B1 (en) | Tightly coupled secondary storage system and file system | |
| CN102622185B (en) | The method of storage file and storage allocation method in multiple storage unit | |
| CN102411637B (en) | Metadata management method of distributed file system | |
| CN102024016B (en) | Rapid data restoration method for distributed file system (DFS) | |
| CN102033786B (en) | Method for repairing consistency of copies in object storage system | |
| CN102200935A (en) | Method for storing data in stream media system | |
| US11144396B1 (en) | Raid reliability with a provisional spare disk | |
| US20150363126A1 (en) | Logical zone mapping | |
| CN103514249A (en) | Method and system for automatic data reduction and storage device | |
| WO2007115492A1 (en) | A method of storing files in a data storage device and data storage device thereof | |
| CN107291889A (en) | A kind of date storage method and system | |
| CN102780836B (en) | High-standard definition compatibility and high-safety news non-linear editing network system | |
| CN105828017B (en) | A kind of cloud storage access system and method towards video conference | |
| CN103699457A (en) | Method and device for restoring disk arrays based on stripping | |
| US9558206B2 (en) | Asymmetric distributed data storage system | |
| CN102999399B (en) | The method and apparatus that a kind of JBOD array is automatically renewed | |
| CN102117297A (en) | Streaming media file processing method, device and system | |
| CN104580439A (en) | Method for achieving uniform data distribution in cloud storage system | |
| CN105100815A (en) | Distributed Metadata Management Method for Streaming Data Based on Time Series | |
| CN110825552A (en) | Data storage method, data recovery method, node and storage medium | |
| WO2014094568A1 (en) | Data storage planning method and device | |
| CN105867836A (en) | Storage management method and apparatus as well as stream media system | |
| CN104424052A (en) | Automatic redundant distributed storage system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C12 | Rejection of a patent application after its publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20110928 |