JP4065535B2 - Code data creation method and apparatus - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a technique for creating code data from fragmentary image data received via a network.
[0002]
[Prior art]
On the Internet, accessing a WWW server from a Web browser and browsing information such as document data and image data is actively performed. As described above, in a system environment including a WWW server for publishing information on the Internet and a client for browsing the information, each client uses a web browser to browse the information published by the WWW server. be able to.
[0003]
The WWW server normally stores a document describing information to be disclosed, which is called a home page, in HTML, which is accessed by a client-side web browser and displayed on a client-side computer. Also, the client-side web browser can obtain necessary information by following the links in the displayed page. Furthermore, as a method of downloading a file managed by the WWW server, there is a method called File Transfer Protocol (hereinafter abbreviated as “FTP”). FTP is a mechanism for transferring the contents of a file on a WWW server to a client computer at once through a network.
[0004]
Further, there is Flashpix / IIP as a protocol for fragmentally accessing an image file managed by a server and displaying it on a client. The Internet Imaging Protocol (IIP) is an optimal protocol for an image data file format called Flashpix, and can request a part of image data through a network. Access at this time is performed in units of tiles of Flashpix.
[0005]
On the other hand, consider the case where this Flashpix / IIP is applied to JPEG2000 as it is. Here, in JPEG2000, the code data of each scalability (Scalability) is composed of difference data from the scalability data one level lower than the scalability. Therefore, the code data received on the client side is cached, all the code data is transferred to the decoder and re-decoded from the beginning, and the code data received this time is stopped by stopping the decoder halfway. There is a method of passing and decoding from the previous continuation.
[0006]
Among such methods, a method has been conventionally proposed in which only a necessary portion of data is extracted from image code data having multi-resolution data and converted to another code data (see, for example, Patent Document 1). .
[0007]
The image data serving as a source in Patent Document 1 is code data that can manage multi-resolution data by using wavelet transform or wavelet-like conversion. A method is described in which code data necessary for processing data in a spatial area selected by the user is extracted from the source code data and converted into one independent code data. At this time, the partial code data extracted from the source code data corresponds to the JPEG2000 code block, and includes code data necessary for processing a request from the user this time.
[0008]
Here, when the code data sent from the server is re-decoded from the beginning on the client side, these pieces of code data do not have a mechanism like a cache, and directly, one code data file compliant with JPEG2000. It is possible to recreate it.
[0009]
[Patent Document 1]
US Pat. No. 6,041,143
[0010]
[Problems to be solved by the invention]
However, since fragmented code data sent from the server side is sent in the order requested from the client side, in order to convert the code data received on the client side into one code data compliant with JPEG2000, There is a problem that complicated processing on the client side is necessary.
[0011]
It is also possible to configure fragmented code data sent from the server as a cache file of a unique format, and the JPEG2000 decoder on the client side can directly read this cache file. However, in this case, the JPEG2000 decoder requires a process for reading an original cache file, which complicates the process and that a general-purpose JPEG2000 decoder cannot be used.
[0012]
Therefore, when JPEG2000 code data is received in units of packets, the packet data is cached, and when the code data is passed to the decoder, a zero length packet (hereinafter referred to as a zero length packet) This is abbreviated as “ZLP”.) By inserting data and creating a single JPEG2000-compliant code file together with the already received packet data, it is possible to perform general operations without rewriting the main header. It is conceivable to create a JPEG2000 file that can be processed by a typical JPEG2000 decoder.
[0013]
However, the code data created using all received packet data considers the display format desired by the user in the image size, image quality, and image area requested by the user using the client side application. Absent. Therefore, in a decoder that receives such code data, a display screen may be created by using a decoding result generated from a part of the data, and 1 using all the cached code data. Creating two JPEG2000 code data files has a problem that wasteful processing occurs in the processing of the decoder itself.
[0014]
Furthermore, the more code data that is cached, the more data copy operations that occur when creating code data to be passed to the decoder. This leads to an increase in the time required for display by the client, causing a problem of performance degradation. In particular, when the user issues a command for enlarging or scrolling the image, cache data that is not directly required for display increases, and the above problem frequently occurs.
[0015]
Furthermore, the above-mentioned Patent Document 1 has the following problems. That is, when the technique described in Patent Document 1 is applied as it is to IIP that is communication through a network, code data of all code blocks of all layers is sent for each request from the client. Even the received code data is transmitted. In addition, the header portion of the code data to be transmitted is rewritten to the information (for example, image size, hierarchy information, etc.) of the image data requested this time, and the original information of the code data managed by the server is acquired. I can't.
[0016]
The present invention has been made in view of such circumstances, and uses fragmented code data cached in the client and fragmented code data received from the server as much as necessary. Code data creation method and apparatus capable of suitably creating code data that can be used by a general-purpose JPEG2000 decoder in a client and capable of performing decoding processing of the code data and display processing of image data at high speed The purpose is to provide.
[0017]
[Means for Solving the Problems]
  In order to solve the above problems, the present inventionThe encoded data creation method has the following configuration. That is,
  Encoded image data divided into a plurality of tiles and encoded as JPEG2000 is stored in and managed as unit encoded data of hierarchical elements represented by tiles, layers, resolutions, components, and positions. In a client that downloads data and generates a new encoded data file for decodingAn encoded data creation method comprising:
  According to the designation of the region in the image to be decoded and the image size to be output, the required unit encoded data is specified, and the requesting step of requesting the specified unit encoded data from the server;
  A creation step of creating a management table for managing in units of tiles which unit encoded data belongs in the hierarchical structure;
  Update of updating the corresponding tile management table with the information indicating the location of the unit encoded data to be cached in the predetermined memory as well as caching the received unit encoded data in the predetermined memory as a result of the request by the request step Process,
  The unit encoded data cached in the memory in response to the update of the management table in the update step, and a code indicating that there is no corresponding unit encoded data for unrequested unit encoded data An encoded data file generation step for generating an encoded data file in JPEG2000 format in tile units,
  In the requesting step, when a new area in the image to be decoded or a new output image size is designated, the requesting process is not performed in the memory in the unit encoded data necessary for decoding based on the management table. Request unit encoded data in cache state from the server,
  Furthermore,
  In accordance with the management table, a determination step of determining whether there is a tile in which all unit encoded data is cached in the memory;
  A deletion step of deleting, from the management table, information on cached unit encoded data related to the tile when it is determined that there is a tile in which all the unit encoded data is cached in the determination step;Is provided.
[0018]
In the code data creation method according to the present invention, when it is determined by the determination step that all the code data constituting the independent code data is stored in the storage unit, the code unit is stored in the storage unit. The method further comprises a replacement step of replacing the code data thus obtained with the independent code data divided in the division step.
[0019]
Furthermore, the present invention is a code data creating device in a second computer comprising storage means for storing fragmented first code data among code data managed by the first computer,
Of the code data managed by the first computer, first storage means for storing fragmentary first code data;
Calculating means for calculating the second code data that is deficient from the code data required for creating JPEG2000 code data in the second computer and the first code data stored in the storage means;
Requesting means for requesting the calculated second code data to the first computer;
Obtaining means for obtaining the second code data from the first computer;
Second storage means for storing the acquired second code data;
Dividing means for analyzing the header information of the acquired second code data and dividing the code data into a plurality of independent code data;
Determining means for determining whether or not all the code data constituting the independent code data is stored in the first or second storage means for each unit divided by the dividing means;
A third storage means for storing dummy code data in a portion that is not stored, when not all code data constituting the independent code data is stored;
Creating means for creating JPEG2000 code data using code data stored in the first, second and third storage means;
It is characterized by providing.
[0020]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
[0021]
<First Embodiment>
FIG. 1 is a diagram illustrating a state in which a plurality of computers are connected on a network, and is a schematic diagram illustrating a configuration of a network system including a server and a client according to the first embodiment of the present invention.
[0022]
In FIG. 1, reference numeral 100 denotes a network represented by the Internet. Reference numerals 101 and 103 denote server computers connected to the network 100, which execute software necessary for the WWW server function, including an IIP server for JPEG 2000 for transmitting image data. A large amount of image data is also stored. Reference numerals 102a and 102b denote client computers in which necessary software such as a web browser and a JPEG2000 decoder is executed.
[0023]
FIG. 2 is a diagram illustrating an example of a hardware configuration of each computer system of the server computers 101 and 103 or the client computers 102a and 102b in FIG. In FIG. 2, 201 is a CPU, which controls the entire system. A keyboard 202 is used for inputting information and the like to the system together with the mouse 201a. Reference numeral 203 denotes a display unit, which includes a CRT, a liquid crystal display, or the like.
[0024]
On the other hand, 204 is a ROM, and 205 is a RAM, which constitutes a storage device in this system, and stores programs executed by the system, data used by the system, and the like. Reference numeral 206 denotes a hard disk device, and 207 denotes a flexible disk, which constitutes an external storage device used in the file system of this system. Reference numeral 208 denotes a printer. Furthermore, reference numeral 209 denotes a part for controlling the network, from which a resource on the network such as a server computer connected to the network 100 (for example, the Internet) is accessed.
[0025]
In the present embodiment, already generated JPEG2000 files are managed by the server computer. On the other hand, the client computer uses the JPEG2000 IIP to access only the necessary parts in the JPEG2000 file requested by the user using an image display application or the like, and receives the encoded data in pieces. Then, the received fragmented code data is cached on, for example, the hard disk 206 on the client computer.
[0026]
In the following, a part for converting fragmented JPEG2000 code data cached in the hard disk 206 or the like into JPEG2000-compliant code data to be passed to a JPEG2000 decoder will be described.
[0027]
In addition, the user opens a homepage using a Windows (registered trademark) machine or the like, and clicks a link to a JPEG2000 image written therein, whereby the JPEG2000 image has an image size suitable for the user's application. Acquire and cache the fragmentary data necessary for display at high resolution. A case is assumed in which one bit stream is created from the cache data, decoded, and displayed.
[0028]
Here, general JPEG2000 code data will be described. FIG. 3 is a conceptual diagram showing a configuration of a JPEG2000 file recorded along a layer-resolution level-component-position progression (hereinafter referred to as “SNR Progression”). When conforming to SNR Progression, recording is performed in the order of layer / resolution / component / position. Defining the arrangement of such code data is called “progression order”.
[0029]
FIG. 4 is a diagram for explaining the resolution scalability of JPEG2000. That is, it is a diagram for illustrating the relationship between the resolution (image size) and the Resolution number. In FIG. 4, the resolution number of the image with the smallest resolution is set to 0, and the width and height of the image double each time the resolution number increases by one. In each layer, data is stored in ascending order of resolution number. The layer number corresponds to the S / N ratio for the original image of the image to be restored, and the S / N ratio becomes worse as the layer number is smaller.
[0030]
Further, the maximum values of resolution number, layer number, and component number in one JPEG2000 file are preset by the encoder, encoded according to the parameters, and the information is stored in the code data. . Each packet is composed of a packet header part that manages information of code-blocks stored in the packet and code data of each code-block.
[0031]
By using such JPEG2000 code data, the user does not need to acquire all the image data in the server, and only receives the code data of the part necessary for display on the client side from the server. . As a unit of received data on the user side (client computer), a JPEG2000 packet or a code block unit which is a smaller encoding unit than the packet can be considered. In the present embodiment, a packet unit is assumed as a data unit received by the user from the server.
[0032]
FIG. 5 is a conceptual diagram for explaining a request and response of data in packet units between the server and the client in the first embodiment.
[0033]
In FIG. 5, the client 501 requests the server 502 for necessary data by designating the tile number, resolution level, layer, component, and position number of the image. Then, the server 502 analyzes the code stream of the image 503, extracts packet data corresponding to the tile number, resolution level, layer, component, and position number of the specified image and sends it back to the client 501.
[0034]
Next, an example of a JPEG2000 file managed by the server computer used in this embodiment will be described with reference to FIGS. 6A and 6B. FIG. 6A is a diagram illustrating an example in which original image data stored in the server computer is divided into tiles. FIG. 6A shows an example in which the image is divided into 4 tiles in each of the vertical and horizontal directions, that is, 16 tiles in the entire image.
[0035]
FIG. 6B is a diagram showing a data structure of JPEG2000 code data (that is, data obtained by encoding the image data with “SNR Progression”) related to the tile-divided image data shown in FIG. 6A. The code data shown in FIG. 6B includes three types of layers 0 to 2 (Lay0, Lay1, Lay2), resolution levels 4 types (Res0, Res1, Res2, Res3), and component and position are each It is a structural example of the code data at the time of encoding by JPEG2000 in case of 1 type (Com0, Pos0).
[0036]
For example, when the maximum resolution image size of JPEG2000 is 1024 × 1024 pixels, the image size of each tile is 256 × 256 pixels. In addition, when each tile has four resolution levels, the image size at each resolution is 256 × 256, 128 × 128, 64 × 64, and 32 × 32 pixels.
[0037]
Further, for the main header portion in the JPEG2000 code data, as shown in FIG. 6B, Xsiz and Ysiz that are parameters indicating the image size are both 1024, indicating the image size of the entire image. On the other hand, XTsiz and YTsiz indicating the tile size have a value of 256 which is the tile size described above. With these parameters, this JPEG2000 code data has code data divided into 16 tiles. Then, the code data of each tile is stored by being separated by “Tile part Header” as shown in FIG. 6B.
[0038]
FIG. 7 is a flowchart for explaining an outline of a processing procedure in an application on the client computer in the first embodiment. In the present embodiment, a JPEG2000 image display application is activated on the client side, and the flowchart shown in FIG. 7 shows a part for displaying an image after designating a JPEG2000 file to be displayed by another means such as a web. It is a processing flow.
[0039]
First, an internal variable (parameter) for controlling future processing is initialized (step S707). Specifically, the server inquires about information such as the image size and encoding parameters of the designated JPEG2000 file, and stores information for managing the cache of encoded data to be displayed as shown in FIG. Create it on a file and initialize its variables (parameters). That is, FIG. 13 is a diagram for explaining the logical structure of the code data cache controlled on the client side. In FIG. 13, reference numeral 1300 denotes the logical structure of the entire cache file, which can be divided into an image management information field 1302 for the entire cache, a Main header data field 1303, and a data field 1301 for each Tile, as shown in the figure.
[0040]
Here, in the data field 1301 related to Tile, there are fields for the number of Tiles when this image data is divided into Tile. Reference numeral 1304 denotes details of the data field 1301 regarding Tile. These are divided into a Tile data management data field 1304a, a Tile header field 1304b, and a packet data management data 1304c. The packet data management data field 1304c includes fields for the number of packets existing in the Tile.
[0041]
Details of the Tile data management data field 1304a are shown in 1305. As shown in 1305, the Tile data management data field 1304a includes parameters of a Tile data length 1305a, a Tile number 1305b, a packet number 1305c, a Resolution number 1305d, a Layer number 1305e, a Component number 1305f, and a Position number 1305g.
[0042]
Further, details of the packet data management data 1304 c are shown in 1306. As shown in 1306, the packet data management data 1304c includes a packet data length 1306a, a pointer value 1306b, a packet number 1306c, and basic packet information 1306d. The actual packet data received from the server is stored ahead of the pointer indicated by the pointer value 1306b.
[0043]
As can be seen from FIG. 13, when the packet data is fully filled, there are a lot of information for managing the table, and the number of bytes of the original code data managed by the server becomes larger. . That is, since the client side generates one code data from the cache information and the cache information, the number of bytes related to the code data managed on the client side becomes maximum when all the tiles are fully filled. The size is at least twice the size of the original code data managed by the server. The data 1304 related to each Tile is stored in a file, and the field indicated by 1301 in 1300 is associated with the corresponding data by the file name.
[0044]
After the internal variables are initialized in step S707, the user performs an operation for display (step S700). For example, in the first display of the designated file at the time of starting the application, an image size that fits in the opened window size is calculated and displayed with an image size that matches the window size. Then, the user's operation may be accepted thereafter in step S700.
[0045]
Next, all the packets that are newly required by the user operation in step S700 are calculated (step S701). Thereafter, a request is issued to the server for the newly required packet data calculated in step S701 (step S702). Next, the requested packet data is received from the server and cached on the client computer (step S703).
[0046]
Further, code data is generated by converting the code data managed in the cache format into a plurality of JPEG2000 code data formats so that high-speed reading is possible (step S704). Details of the code data creation processing will be described later. Thereafter, the JPEG2000 code data created in step S704 is decoded and displayed (step S705). Then, it is determined whether or not the user has requested termination (step S706). As a result, when the user requests termination (Yes), this application is terminated. On the other hand, when the user makes another display request without requesting termination (No), the process returns to step S701 to execute the above process.
[0047]
Hereinafter, specific processing contents of the processing procedure in the application on the client computer will be described.
[0048]
First, suppose that immediately after the client application is started, the initial image display size of this application is set to 128 × 128 pixels, and image data that falls within this size is displayed. In this case, considering that the necessary packet calculated in step S702 is to display data having the lowest resolution image size and the maximum SNR, in all 16 tiles from tile 0 to tile 15 in FIG. 6B. , “Lay0 / Res0 / Com0 / Pos0”, “Lay1 / Res0 / Com0 / Pos0”, and “Lay2 / Res0 / Com0 / Pos0” are required 3 × 16 packets.
[0049]
Therefore, the client program requests these packets from the server program. On the other hand, the server program performs a process of cutting out 3 × 16 packets of all tiles. After that, the client program receives these packets and then decodes and displays them.
[0050]
Next, detailed description of the code data creation in step S704 in FIG. 7 will be given with reference to FIGS. FIG. 8A is a flowchart for explaining an overview of the code data generation process in step S704 in FIG. First, the main header of JPEG2000 code data related to the necessary packet received from the server is analyzed (step S800). Next, the number of tiles in the code data is calculated from the result of analyzing the main header in step S800 (step S801). In the present embodiment, the calculation of the number of tiles is performed by the following equation (1).
[0051]
Xtiles = (Xsiz + XTsiz-1) / XTsiz
Ytiles = (Ysiz + YTsiz-1) / YTsiz (1)
Tiles = Xtiles × Ytiles
Then, the processes in and after step S802 are performed by the number of tiles obtained by Expression (1).
[0052]
Step S802 is a part that performs code conversion processing of each tile. Details of this processing will be described later. Then, it is determined whether or not the code conversion processing for all tiles has been completed (step S803). As a result, when there is an unprocessed tile (No), the process returns to step S802. On the other hand, if the code conversion process has been completed for all tiles (Yes), the process ends and the process proceeds to step S705.
[0053]
FIG. 8B is a flowchart for explaining in detail the code conversion processing in step S802. First, it is determined whether or not the tile to be processed this time is fully filled, that is, whether all packet data is cached (step S804). Specifically, it can be determined by looking at the packet data length 1306a or the pointer value 1306b to the packet data shown in FIG. These values are “0” when not received, and when packet data is received, values such as the number of bytes of the packet data, the address or file name actually stored, and the like are stored. Thereby, the value of the data length 1306a or the pointer value 1306b is determined for all the packets in the tile, and if each value is not “0”, it can be determined that the packet is full. On the other hand, if there is even a value of “0”, it can be determined that it is not yet full. As a result, if it is fully filled (Yes), the process proceeds to step S805. Otherwise (No), the process proceeds to step S807.
[0054]
In step S805, it is determined whether code data in a full filling state has been created. As a result, if it has not been created yet (No), the process proceeds to step S806, and if it has been created (Yes), the code conversion process ends. In step S806, a flag indicating that complete code data has been created is set. Details of the processing relating to the setting of the flag will be described later. Then, “code file creation” processing for creating code data from actual cache data is performed (step S807). Details of the code file creation processing will be described later.
[0055]
Here, the detailed processing content of step S806 of FIG. 8 is demonstrated. FIG. 14 is a flowchart for explaining in detail a set of flags indicating that complete code data has been created in step S806 of FIG. This process is relatively easy. First, the file of the Tile data management data 1304 of the fully filled tile is deleted (step S1400). As a result, the areas of the data 1304 related to Tile, the management data 1305 for Tile data, and the management data 1306 for Packet data shown in FIG. 13 are deleted. Next, the complete code data is set by replacing with the code data file for each tile shown in FIG. 11 (step S1401). That is, the setting of the flag indicating that complete code data has been created is to change the value of data related to Tile indicated by 1301 in 1300 shown in FIG.
[0056]
By performing the above processing, when the client application needs the data of this tile after the processing, it is only necessary to pass the file name stored in the field indicated by 1301 to the application. It does not occur at all and can be performed at high speed. Furthermore, since the amount of data that must be held as a cache directly has the code data as compared with the case where it is held in the form of a management table, the data capacity can be reduced.
[0057]
Whether or not the tile is fully filled may be determined by checking the Tile data length field 1305a held as a cache. FIG. 15 is a diagram showing a state when a tile is fully filled and replaced with complete code data. As a specific confirmation method, when the field shown in FIG. 15 is a 16-bit integer field, when the field is full-filled and replaced, it becomes the first 16 bits of the code data. In this case, the beginning of the code data is the SOC marker code (0xFF4F), and when this value is read as an integer, the leading bit (15th bit) is “1”, and thus becomes a negative value. The result is the same even when the integer is 32 bits long, and is (0xFF4FFF51) because it is the SOC marker code and the SIZ marker code. That is, when the head of the management data of Tile data is read as an integer of the required number of bits, if it is a positive integer, it is still being filled, and if it is a negative integer, it has been converted to code data with full filling. It can be easily determined that there is.
[0058]
As described above, after the code file is generated from the cache data that has been full-filled once, the code file creation process (step S807) is controlled again so that the cache is filled. The code file creation process can be performed at high speed.
[0059]
FIG. 9 is a flowchart for explaining in detail the code file creation processing in step S807 shown in FIG. First, the number of packets included in the tile is set to iPMax (step S900). Next, “Tile part Header” is provisionally output (step S901). Here, the temporary output is because the “Tile part Header” has a field for storing the code length of the tile, and the code length of the tile is a value that can be calculated when the following processing is completed.
[0060]
Next, “0” is set as an initial value in the variables iP and iL (steps S902 and S903). Here, the variable iP is a variable indicating the packet ID, and the variable iL is a variable indicating the code length of the tile. Next, it is determined whether or not the packet indicated by the variable iP is cached (step S904). As a result, if the variable iP is cached (Yes), the process proceeds to step S905. If the variable iP is not cached (No), the process proceeds to step S907.
[0061]
In step S907, a “Zero Length Packet” code (hereinafter abbreviated as “ZLP code”) indicating that there is no corresponding packet data is output. Then, “1” which is the number of bytes of this ZLP code is added to the variable iL (step S908). On the other hand, in step S905, the corresponding packet data is output. Then, the number of bytes is added to the variable iL to be updated (step S906).
[0062]
After the processes of steps S906 and S908, the variable iP is incremented (step S909). And it is judged whether it processed about all the packets (step S910). As a result, when all the packets have not been processed yet (No), the process returns to step S904 and the above processing is performed. On the other hand, when all the packets have been processed (Yes), the “SOT marker” that is the “Tile part Header” is updated from the variable iL and output (step S911), and the code file generation process ends. Then, as described above, the code file is decoded by a general-purpose JPEG2000 decoder, and the decoded image data is displayed (step S705).
[0063]
That is, as described above, the client according to the present embodiment includes a storage unit (for example, the hard disk 206) that stores fragmentary first code data among the code data managed by the server, and stores JPEG2000 code data. It functions as a code data creation device to be created. In the client according to the present embodiment, first, code data necessary for creating JPEG2000 code data in the client and first code data (for example, in the current display unit 203) stored in the storage unit such as the hard disk 206 in advance. Second code data that is insufficient from the displayed image data (for example, code data for image data that is necessary for enlarged display on the display unit 203, the first code data) Is not included). Then, the server requests the calculated second code data, acquires the second code data from the server, and stores the acquired second code data in a storage unit such as the hard disk 206. Next, the header information of the acquired second code data is analyzed, and the target code data is divided into a plurality of independent code data. Furthermore, for each divided unit, it is determined whether or not all the code data constituting the independent code data is stored in the storage means such as the hard disk 206. When not all of the code data constituting the independent code data is stored, dummy code data is stored in the unstored portion, and the stored code data is output as JPEG2000 code data. .
[0064]
In the client (code data creation device), code data is handled in units of packets. Further, the client (code data creation device) is characterized in that the dummy code data is zero length packet data defined by JPEG2000. Furthermore, the server and the client can communicate with each other via a network.
[0065]
Furthermore, the client (code data creation device) further includes display means (for example, a display unit 203) for displaying image data, and the first code data is code data of the image data, and the display unit 203 The code data necessary for the calculation of the second code data is set in accordance with the movement or enlargement display of the display area of the image data displayed on the display means. The client (code data creation device) decodes the output JPEG2000 code data and displays the decoded image data on a display unit such as the display unit 203.
[0066]
Here, the code file creation processing shown in the flowchart will be described using a specific example. FIG. 10 is a diagram showing the configuration of the code file of each tile after the code file processing on the initial screen in the first embodiment of the present invention. Here, the packet requested from the client computer is “Lay0 / Res0 / Com0 / Pos0”, “Lay1 / Res0 / Com0 / Pos0”, and “Lay2 / Res0 / Com0 / Pos0” for all tiles as described above. .
[0067]
Therefore, as shown in FIG. 10, for all 16 tiles, code data storing ZLP is created for packets other than the above three packet data, as indicated by reference numeral 1001. In addition, the main header portion of the code data created for each tile is changed to 256 in which each field of Xsiz and Ysiz in the SIZ marker code in the main header is the size of the tile. Thereby, an independent code file can be created for each tile.
[0068]
That is, in the client (code data creation device) according to the present embodiment, code data (image data) is divided into tile units of a predetermined size (for example, units calculated by the method described in Expression (1)). Features. Further, the client is characterized in that each header information of independent code data is changed to a size of a divided tile unit.
[0069]
Next, a case where enlargement display is instructed by the user in step S701 will be described. In this enlarged display, it is assumed that the image is first enlarged twice as much as the initial screen. In this case, if the display window size is 128 × 128 pixels, four tiles can be displayed. For example, assume that Resolution 1 (image size: 64 × 64 pixels) of four tiles Tile5, Tile6, Tile9, and Tile10 illustrated in FIG. 6A is displayed. In this case, in step S702, “Lay0 / Res1 / Com0 / Pos0”, “Lay1 / Res1 / Com0 / Pos0”, and “Lay2 / Res1 / Com0 / Pos0” of the four tiles Tile5, Tile6, Tile9, and Tile10 are clients. Requested from the computer to the server.
[0070]
Further, when the user issues a request to double the size, if the display window size is changed from 128 × 128 pixels to 256 × 256 pixels, “Lay0 / Res2 / Com0 / Pos0” is further added to the four tiles. "Lay1 / Res2 / Com0 / Pos0" and "Lay2 / Res2 / Com0 / Pos0" are required.
[0071]
Thereafter, it is assumed that there is a request from the user to further double the size. In this case, if the window size of the display area is not changed, only one tile can be displayed. So, if only Tile5 is displayed, Tile5's "Lay0 / Res3 / Com0 / Pos0", "Lay1 / Res3 / Com0 / Pos0", and "Lay2 / Res3 / Com0 / Pos0" will be requested from the server and displayed. . As a result, Tile5 is completely filled with cache. FIG. 11 is a diagram for explaining an example of the state of the code file for each tile of the code data in the first embodiment.
[0072]
As shown in FIG. 11, the code files of the tiles Tile0, Tile1, Tile2, Tile3, Tile4, Tile7, Tile8, Tile11, Tile12, Tile13, Tile14, and Tile15 are denoted by reference numeral 1001 as shown in FIG. It is in the state shown. Each tile of Tile6, Tile9, and Tile10 is in a state indicated by reference numeral 1101, and Tile5 is in a state indicated by reference numeral 1102. Until this state is reached, the process proceeds to step S806 depending on whether or not complete code data has been created in step S805. However, after the Tile5 code conversion process, the process proceeds to step S806 after the determination in step S805. First, the process ends. This eliminates the need for code conversion processing for Tile5 thereafter. As a result, the code file creation process can be accelerated as the cache is filled. That is, according to the present embodiment, when the area to be decoded / displayed is performed in units of tiles, it is possible to perform processing only on tiles necessary for decoding / display.
[0073]
Conventionally, when one JPEG2000 code data file is generated from a cache file, it is difficult to make a multithread for each tile to be displayed. This is because in order to generate one code data file, it is essential to serialize the processing here. However, by generating a code data file for each tile as in this embodiment, generation of a cache file from a packet request required for each tile to be displayed, creation of code data from the cache file, decoding and display of code data Can be multithreaded as threads for each tile. Therefore, the speed can be increased by multi-threading as compared with the conventional single thread.
[0074]
Furthermore, in a system that takes time to process file I / O, instead of creating one large code data file, generating a plurality of small code files minimizes time loss due to file I / O. It can also be limited.
[0075]
<Second Embodiment>
In the first embodiment described above, it has been described that the code conversion process is performed on all cached tiles. However, in the present embodiment, the code conversion process is performed only on the tiles necessary for display. A case in which code conversion processing is performed at a higher speed than in the first embodiment will be described.
[0076]
FIG. 12 is a flowchart for explaining the outline of the code data creation processing according to the second embodiment of the present invention. In addition, the same number is attached | subjected to the part of the same process as the flowchart of FIG. 8A in 1st Embodiment mentioned above. Here, only a different part from 1st Embodiment is demonstrated.
[0077]
After calculating the number of tiles in step S801, the command operated by the user in step S701 is analyzed (step S1200). Then, using this analysis result, it is determined whether or not the tile to be processed is a tile displayed in the current display (step S1201). As a result, when it is determined that the tile is not used for display (No), the process proceeds to step S803. On the other hand, if it is determined that the tile is used for display (Yes), the process proceeds to step S1202.
[0078]
In step S1202, when a new packet is requested in step S703, it is determined whether or not the packet of the tile to be processed this time has been acquired. As a result, if a new packet has been acquired (Yes), the process proceeds to step S802. If a new packet has not been acquired (No), the process proceeds to step S803. The subsequent processing procedure is the same as the procedure described in the first embodiment. By these processes, it is possible to perform the code conversion process only on tiles that are necessary for display and have not been completely filled. Therefore, the encoding process can be performed at a higher speed than in the first embodiment. it can.
[0079]
<Third Embodiment>
Hereinafter, an image data transmitting apparatus according to a third embodiment of the present invention will be described in detail with reference to the drawings. In the present embodiment, a method for creating one piece of code data from the cache will be described. The configuration of the network system including the server and the client according to the present embodiment is the same as that in FIG. 1, and the hardware configuration of each computer system is also the same as that in FIG. The processing procedure in the application on the client computer is also the same as the flowchart shown in FIG.
[0080]
FIG. 16 is a flowchart for explaining the processing procedure when one code data file is created from the cache. As shown in FIG. 16, first, the information held in the field 1303 in the cache is converted into a format according to the JPEG2000 code data format, and a main header (Main Header) is output to the output file (step S1600). Next, the number of tiles in the code data is calculated from the result of analyzing the main header (step S1601). The calculation formula at this time is the same as that in step S801 in FIG. In the following, the processing in step S1602 and subsequent steps is performed for the number of tiles obtained by the above-described equation (1).
[0081]
Then, for each tile, it is determined whether the tile is fully filled and there is complete code data of the tile (step S1602). This determination method is as described above. As a result, when it is determined that it is fully filled (Yes), the process proceeds to step S1604, and when it is not fully filled (No), the process proceeds to step S1603.
[0082]
In step 1603, a code file is created in the same manner as the code file creation processing procedure shown in FIG. After this processing, the process proceeds to step S1606, and the code data is output. On the other hand, in step S1604 and subsequent steps, conversion from complete code data of tiles is performed. Therefore, first, a process of skipping the main header portion is performed (step S1604). As a specific example of this processing, for example, it is only necessary to skip from the beginning of the code data shown in FIG. 15 to the SOT marker code. Next, the value of the tile index (Tile Index) in the tile part header (Tile part Header) is replaced with the index (Index) number of the tile (step S1605). Further, the tile part header and subsequent EOC marker codes (but not including the EOC marker code) are copied (step S1606). Then, it is determined whether or not all tiles have been processed (step S1607), and if not processed (No), the process returns to step S1602 and the above processing is repeated. On the other hand, if all the tiles have been processed (Yes), the process ends.
[0083]
Through the processing described above, one piece of code data can be easily created from the cache file in the format according to the present embodiment, and since there is no processing such as data conversion, it can be created at high speed.
[0084]
<Other embodiments>
In general, when one JPEG2000 code data file is generated from a cache file, it is difficult to make a multithread for each tile to be displayed. This is because it is essential to serialize the processing in order to generate one code data file. However, as described above, by generating a code data file for each tile, from a packet request necessary for each tile to be displayed, generation of a cache file, creation of code data from the cache file, decoding of code data, Multithreading can be performed with each tile as a thread until display. For this reason, it is possible to increase the speed by using multi-threading as compared with the case of single thread so far.
[0085]
Note that the present invention can be applied to a system composed of a plurality of devices (for example, a host computer, an interface device, a reader, a printer, etc.), but a device (for example, a copier, a facsimile machine, etc.) composed of a single device. You may apply to.
[0086]
Also, an object of the present invention is to supply a recording medium (or storage medium) in which a program code of software that realizes the functions of the above-described embodiments is recorded to a system or apparatus, and a computer (or CPU or CPU) of the system or apparatus. Needless to say, this can also be achieved when the MPU) reads and executes the program code stored in the recording medium. In this case, the program code itself read from the recording medium realizes the functions of the above-described embodiment, and the recording medium on which the program code is recorded constitutes the present invention. Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an operating system (OS) running on the computer based on the instruction of the program code. It goes without saying that a case where the function of the above-described embodiment is realized by performing part or all of the actual processing and the processing is included.
[0087]
Further, after the program code read from the recording medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer, the function expansion is performed based on the instruction of the program code. It goes without saying that the CPU or the like provided in the card or the function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.
[0088]
When the present invention is applied to the recording medium, program code corresponding to the flowchart described above is stored in the recording medium.
[0089]
As described above, when the file creation method described in the above embodiment according to the present invention is used, fragmented data of JPEG2000 encoded data sent using IIP is cached, and normal JPEG2000 is stored from this cache file. When JPEG2000 encoded data that can be processed by the decoder is created, an independent JPEG2000 code data file is generated for each tile necessary for the current decoding and display, so that processing up to decoding / display can be performed at high speed.
[0090]
It is also possible to minimize the data capacity or file capacity for configuring the cache or generating code data from the cache. Furthermore, when code data is generated from the cache, since processing such as data conversion can be omitted, it can be generated at high speed.
[0091]
Further, when the area to be decoded / displayed is performed in units of tiles, it is possible to perform processing only on tiles necessary for decoding / display. Furthermore, the speed can be increased by multithreading. Furthermore, in a system that requires file I / O processing time, instead of creating one large code data file, a plurality of small code files are generated, thereby minimizing time loss due to file I / O. It can also be limited. Alternatively, conversion can be easily performed even when one piece of code data is created.
[0092]
【The invention's effect】
As described above, according to the present invention, a general-purpose JPEG2000 decoder is used in a client by using fragmented code data cached in the client and fragmented code data received from the server as much as necessary. Code data that can be used in the above-described process and that can decode the code data and display the image data at high speed.
[Brief description of the drawings]
FIG. 1 is a schematic diagram showing a configuration of a network system including a server and a client according to a first embodiment of the present invention.
FIG. 2 is a diagram illustrating an example of a hardware configuration of each computer system of the server computers 101 and 103 or the client computers 102a and 102b in FIG.
FIG. 3 is a conceptual diagram illustrating a configuration of a JPEG2000 file recorded in accordance with Layer-resolution level-component-position progression (SNR Progression).
FIG. 4 is a diagram illustrating resolution scalability of JPEG2000.
FIG. 5 is a conceptual diagram for explaining a request and response for data in packet units between a server and a client in the first embodiment.
FIG. 6A is a diagram showing an example in which original image data stored in a server computer is divided into tiles.
6B is a diagram showing a data structure of JPEG2000 code data related to the tile-divided image data shown in FIG. 6A. FIG.
FIG. 7 is a flowchart for explaining an outline of a processing procedure in an application on a client computer in the first embodiment.
FIG. 8A is a flowchart for explaining the outline of the process of creating the code data in step S704 in FIG.
FIG. 8B is a flowchart for explaining in detail the code conversion processing in step S802.
FIG. 9 is a flowchart for explaining in detail the code file creation processing in step S807 shown in FIG. 8;
FIG. 10 is a diagram showing a configuration of a code file of each tile after code file processing on the initial screen in the first embodiment of the present invention.
FIG. 11 is a diagram for explaining an example of a state of a code file for each tile of code data in the first embodiment.
FIG. 12 is a flowchart for explaining an overview of code data creation processing according to the second embodiment of the present invention;
FIG. 13 is a diagram for explaining a logical structure of a cache of code data controlled on the client side.
FIG. 14 is a flowchart for explaining in detail a set of flags indicating that complete code data has been created in step S806 of FIG. 8;
FIG. 15 is a diagram illustrating a state when a tile is fully filled and replaced with complete code data.
FIG. 16 is a flowchart for explaining a processing procedure when one code data file is created from a cache;
[Explanation of symbols]
501 server
502 clients

Claims (6)

The unit encoding is performed from a server that stores and manages encoded image data divided into a plurality of tiles and encoded as JPEG 2000 as unit encoded data of hierarchical elements represented by tiles, layers, resolutions, components, and positions. A method of creating encoded data in a client that downloads data and generates a new encoded data file for decoding ,
According to the designation of the region in the image to be decoded and the image size to be output, the required unit encoded data is specified, and the requesting step of requesting the specified unit encoded data from the server;
A creation step of creating a management table for managing in units of tiles which unit encoded data belongs in the hierarchical structure;
Update of updating the corresponding tile management table with the information indicating the location of the unit encoded data to be cached in the predetermined memory as well as caching the received unit encoded data in the predetermined memory as a result of the request by the request step Process,
The unit encoded data cached in the memory in response to the update of the management table in the update step, and a code indicating that there is no corresponding unit encoded data for unrequested unit encoded data An encoded data file generation step for generating an encoded data file in JPEG2000 format in tile units,
In the requesting step, when a new area in the image to be decoded or a new output image size is designated, the requesting process is not performed in the memory in the unit encoded data necessary for decoding based on the management table. Request unit encoded data in cache state from the server,
Furthermore,
In accordance with the management table, a determination step of determining whether there is a tile in which all unit encoded data is cached in the memory;
And a deletion step of deleting, from the management table, information on cached unit encoded data related to the tile when it is determined that there is a tile in which all the unit encoded data is cached in the determination step. Encoded data creation method. The encoded data creation method according to claim 1 , wherein the code indicating that the unit encoded data does not exist is zero length packet data. The client comprises display means for displaying image data;
A decoding step of decoding the JPEG2000 coded data file for each tile generated by the encoded data file generation step,
Encoded data creation method according to claim 1 or 2, characterized by further comprising a table Shimesuru display step the decoded image data to said display means. The unit encoding is performed from a server that stores and manages encoded image data divided into a plurality of tiles and encoded as JPEG 2000 as unit encoded data of hierarchical elements represented by tiles, layers, resolutions, components, and positions. An encoded data creation device that functions as a client that downloads data and generates a new encoded data file for decoding ,
A request unit that specifies the unit encoded data required according to the designation of the area in the image to be decoded and the image size to be output, and requests the specified unit encoded data from the server;
Creating means for creating a management table for managing in units of tiles which unit encoded data belongs in the hierarchical structure;
Update of updating the corresponding tile management table with the information indicating the location of the unit encoded data to be cached in the predetermined memory as well as caching the received unit encoded data as a result of the request by the request means Means,
It is composed of unit encoded data cached in the memory in response to the update of the management table by the updating means and a code indicating that there is no corresponding unit encoded data for unrequested unit encoded data. An encoded data file generating means for generating an encoded data file in JPEG2000 format for each tile,
When a new area in an image to be decoded or a new image size to be output is designated, the requesting unit is not stored in the memory in the unit encoded data necessary for decoding based on the management table. Request unit encoded data in cache state from the server,
Furthermore,
In accordance with the management table, a determination unit that determines whether there is a tile in which all unit encoded data is cached in the memory;
And a deletion unit that deletes, from the management table, information on cached unit encoded data related to the tile when the determination unit determines that there is a tile in which all the unit encoded data is cached. Encoded data creation device. A computer program that causes a computer to function as the encoded data creation device according to claim 4 by being read and executed by a computer . A computer-readable storage medium storing the computer program according to claim 5 .

Images