JP3277971B2 - Data multiplexing apparatus, data multiplexing method, and data recording medium - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data encoding method suitable for recording a plurality of data such as moving pictures and sounds encoded at a variable rate on a single recording medium such as a disk. Related to the device.

[0002]

2. Description of the Related Art For example, when moving image and audio data is digitized and recorded on a recording medium such as a disk, the data amount is enormous if it is used, so that data compression encoding is generally performed.

When data recorded on a disc is decoded by a decoding device (decoder), for example, in the case of an application such as a movie, moving image data (video) and audio data (eg, speech) are synchronized. Need to play. Therefore, in the encoding device (encoder), the multiplexing process of the encoded data is performed together with the encoding of the data (however, the multiplexing of the data is not always performed simultaneously with the encoding of the data).

[0004] When multiplexing data, a plurality of encoded data such as moving images and sounds are read at appropriate time intervals and at an appropriate rate (rate), and time information (time) required for synchronous reproduction is read. After adding information such as a stamp), the data is output as single data (bit stream).

[0005] Generally, a bit stream has a hierarchical structure. FIG. 5 shows a data structure of a bit stream defined in MPEG (ISO 11172).
As can be seen from FIG. 5, the bit stream is composed of a plurality of packs. Each pack is composed of a plurality of packets, and a packet start code is placed at the beginning of the pack, and is used for synchronization of the entire system (encoder and decoder, etc.) and management of buffers (this point will be described later). SCR (System ClockReferenc)
e), system header, etc. are placed.

At the beginning of each packet, an identification number (ID) of a stream, a PTS (Presentation Time Stamp) indicating a time at which data decoded by a decoder is reproduced,
Indicates the time at which the decoder decodes the data,
Along with S, DT used for synchronous reproduction of data
A time stamp such as S (Decoding Time Stamp) and information such as a buffer size are placed, followed by compression-encoded data such as a moving image or audio.

A packet in which encoded data of a coded moving image is inserted is called a video packet, and a packet in which audio encoded data is inserted is called an audio packet. Different types of data (for example, coded video and audio data) are not mixed in the same packet.

[0008] In MPEG, packs and packets are of variable length. In MPEG, the syntax of a bit stream is specified so that a maximum of 32 (types) of audio and a maximum of 16 (types) of video data can be simultaneously multiplexed. For details of the MPEG multiplexing process, see I.
SO 11172 1-AnnexA "Descri
Ption of the System Codin
g Layer "(informative).

For example, in MPEG, there are degrees of freedom such as the length of a pack or a packet being variable, and therefore various methods of multiplexing data can be considered.

[0010] The first limitation in the data multiplexing process is that the data must be multiplexed so that the decoder buffer does not overflow or underflow. For this purpose, on the encoder side (including the data encoding device), a buffer having the same capacity as the decoder buffer is provided, and management and control are performed so that overflow and underflow do not occur in this buffer.

In the following description, unless otherwise specified.
The buffer indicates a buffer memory on the encoder side.

Further, as a specific problem in the data multiplexing process, in addition to the buffer management, the lengths of the pack and the packet as described above, the pack configuration method (the number of packets per pack, etc.) are different. An insertion ratio of a type of data packet (for example, a video packet and an audio packet) to a pack is included.

However, if the input data such as the moving image and the sound are encoded at a fixed rate, the above problem can be solved relatively easily. This will be described below with examples.

For example, it is assumed that single moving image data and single audio data are independently coded at a fixed rate and then multiplexed. It is assumed that the packet length is 2048 bytes (fixed) for both video and audio.
It is assumed that video data is encoded at 150,000 bytes / sec, and audio data is encoded at 24000 bytes / sec.

In this case, the coding rate of the moving picture is 6.25 times (= 150,000 / 24000) that of the sound, and the packet length is the same, so that the moving picture data and the sound data are reproduced synchronously. Suffices to make the insertion ratio of video packets per pack 6.25 times that of audio packets, so that between 6 and 7 video packets,
What is necessary is just to insert one audio packet. Assuming that each pack is composed of three packets, the bit stream generated under the above conditions has a configuration as shown in FIG.

As can be seen from this example, in the multiplexing processing of a plurality of data encoded at a fixed rate, the length of the pack and the packet and the configuration of the pack are determined in advance, so that the packet is inserted into the pack. The ratio can be calculated in advance, and multiplexing can be performed relatively easily.

Further, regarding the management of the buffer, it is possible to control the accumulation amount by constantly monitoring the accumulation amount of data in the buffer.

For example, when the video code buffer (for temporarily storing encoded moving image data) is likely to overflow, the value of the quantization step in the quantization circuit is increased as will be described later to increase the code. Buffer overflow is suppressed by temporarily reducing the amount of generated data. If the buffer is likely to underflow, buffer underflow is suppressed by inserting pseudo data unrelated to video or audio data and temporarily increasing the amount of generated information of encoded data.

[0019]

As described above, when data such as a moving image and audio is encoded with a fixed maximum coding rate, the multiplexing apparatus uses the moving image data at a predetermined rate. The bit stream can be generated relatively easily by alternately inserting the and the audio data into the packet.

However, even when the video and audio data are efficiently compression-encoded and encoded at a low rate, multiplexing is required to suppress buffer underflow and guarantee a fixed rate. In processing, pseudo data irrelevant to reproduction must be inserted. Therefore, it cannot be said that the recording capacity of the recording medium is effectively utilized.

For example, when encoding a moving image, the value of a quantization step in a quantization circuit in an encoding device may be changed in order to suppress overflow in a buffer and guarantee a fixed rate. Done. That is, the quantization step is increased for a part where the picture is relatively complicated and the image compression is difficult (encoding efficiency is low), and conversely, the picture is relatively flat and the compression is easy (encoding is difficult). The quantization step is reduced for the (highly efficient) part. For this reason, uniform image quality cannot be guaranteed for the entire moving image data.

In order to guarantee uniform image quality, a method of encoding a moving image at a variable rate is conceivable. In other words, for portions with low coding efficiency, the coding rate is temporarily increased, and coding is performed with many coded bits. Conversely, for portions with high coding efficiency, the coding rate is temporarily set. Make it smaller. By adaptively controlling the coding rate as described above, it is possible to obtain uniform image quality throughout the entire moving image. In this case, since the data encoding rate is variable, there is no need to insert similar data, and the recording capacity of the recording medium can be effectively used.

A description relating to a multiplexing device and a demultiplexing device for variable rate data is disclosed in British Patent No. 2259229A.
(Date of A Publication 0
3.0.1993), but there is no description of the details of the control of the code buffer on the multiplexer side.

By the way, in the above-mentioned variable rate coding, the coding rate is adaptively changed, so that the coding rate is unpredictable and different packets (for example, video packets and audio packets) in the multiplexing process are different. ) Cannot be determined in advance, and the multiplexing process becomes more difficult than in the case of a fixed rate.

In view of the above situation, the present invention provides a multiplexing apparatus for a plurality of data (for example, video data and audio data) encoded at a variable rate.
A method and a recording medium are proposed.

[0026]

SUMMARY OF THE INVENTION A data multiplexing apparatus according to the present invention comprises: an image encoding means for encoding at least one image signal at a variable rate; and an audio encoding means for encoding at least one audio signal. First storage means for temporarily storing at least one encoded image signal encoded by the image encoding means, and temporarily storing at least one encoded audio signal encoded by the audio encoding means. Second storage means for temporarily storing the coded image signal and coded audio signal stored in the first and second storage means, and the first storage means based on the accumulated amount of accumulated said encoded image signal to, and control means for controlling the transfer of the coded image signal to the multiplexing means from said first storage means, the control hand
The step-by-step control may include limiting the transfer or transferring
The transfer rate is controlled by controlling the transfer rate .

[0027] When the encoded image signal is not present in the first storage means, the control means suspends the transfer of the encoded image signal to the multiplexing means, and controls the first accumulation means. When there is the encoded image signal, the encoded image signal is transferred to the multiplexing means at a predetermined transfer rate. Furthermore, the predetermined transfer rate is equal to or greater than the sum of the maximum coding rate of maximum coding rate and the speech encoding means of said image encoding means.

Also, the input to the multiplexing means is
2 based on the storage amount of the encoded audio signal in the storage means.
Switching means for switching based on
I can .

Further, the data multiplexing apparatus of the present invention has recording means such as a modulation recording circuit for recording the multiplexed coded image signal and coded audio signal on a recording medium.

[0030]

In the multiplexing device configured as described above,
When a picture with a flat design and easy compression is continuously input to the video encoder, the generated code amount is reduced,
The video code buffer is likely to underflow, at which time the buffer controller limits data transfer to the multiplexing circuit.

The buffer controller controls the reading of each code buffer so that the coded codes are transferred at a transfer rate equal to or higher than the sum of the maximum coding rates of all the encoders. Therefore, even when an image having a complicated pattern and difficult to compress is continuously input to the video encoder, overflow of the code buffer is suppressed.

Further, since the counter switches at regular time intervals, the inputs to the multiplexing circuit are sequentially changed, and the encoded codes output from each code buffer are multiplexed at regular time intervals. .

[0033]

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing an embodiment of a data encoding apparatus according to the present invention. First, a case where input data is single moving image data (variable rate coding) and single audio data (fixed rate coding) will be described as a first embodiment with reference to FIG.

FIG. 1 is a block diagram showing the configuration of an embodiment of the data encoding device of the present invention. In the present embodiment, it is assumed that input data to the multiplexing device is single moving image data and single audio data. Also, moving image data has a variable rate (maximum coding rate Rvmax = 10 Mb).
ps), the audio data has a fixed rate (coding rate Ra
= 1.5 Mbps) and then multiplexed. Further, the pack length is fixed (for example, 2048 bytes), and the packet length is variable. The overall transfer rate is assumed to be 17 Mbps or less even when additional information such as a header (maximum generation rate Rsmax = 4.5 Mbps) is added.

Moving image data d captured from the camera 1
1 is input to the video encoder 3 and compression encoding is performed. Moving image data d1 captured from camera 1
May be temporarily stored in the storage device 15 and then sent to the video encoder 3. In the video encoder 3, processes such as DCT (discrete cosine transform), quantization, and VLC (variable length coding) are performed at a variable rate. The moving image data (video code c1) compressed and encoded through the above process is input to the video code buffer 5.

The video code c1 generated by the video encoder 3 is input to the video code buffer 5, and is temporarily stored. At that time, the video encoder 3
Performs encoding of moving image data at a variable rate, so that the video code buffer 5 stores video code o
1 also becomes temporally variable.

The storage amount o1 of the video code c1 in the video code buffer 5 is fed back to the video encoder 3 and used for allocating the code amount when the next picture is compression-coded. Specifically, a value such as a quantization step is changed in a quantizer in the video encoder 3 and rate control is performed so that the video code buffer 5 does not overflow. The storage amount o1 of the video code d1 in the video code buffer 5
Is also supplied to a buffer controller 14 described later.

On the other hand, the audio data d2 captured from the microphone 2 is input to the audio encoder 4 and encoded at a fixed rate, for example, linear quantization (linear P
CM). Audio data d taken from microphone 2
2 may be temporarily stored in the storage device 16 and output to the audio encoder 4.

The audio data (audio code c2) generated by the audio encoder 4 is input to the audio code buffer 6. The audio code buffer 6 receives the audio code c2 generated by the audio encoder 4 and temporarily stores the audio code c2.

At this time, since the audio encoder 4 encodes the audio data at a fixed rate, if the audio code c2 is transferred to the data multiplexing circuit 7 at the same rate, the audio code c2 in the audio code buffer 6 Is constant without any temporal variation. However, in practice, the audio code buffer 6 repeats a transfer state at a transfer rate of (Rvmax + Ra) or more and a standby state in which the audio code c2 is not transferred (that is, a transfer rate of 0 Mbps) at fixed time intervals. The accumulated amount o2 of c2 fluctuates at regular time intervals while repeating the same pattern.

The data multiplexing circuit 7 multiplexes data by alternately inputting video and audio codes from respective buffers. Switching of the input code to the data multiplexing circuit 7 is executed by the switch 8. The counter 9 monitors the data storage amount o2 of the audio code buffer 6, and monitors the data storage amount o2.
2, the input of the switch 8 is switched between the audio code buffer 6 side and the video code buffer side 5 at fixed time intervals.

As described above, by controlling the switching of the switch 8 based on the amount of data stored in the audio code buffer 6 for storing the fixed-rate encoded code, multiplexing based on the fixed-rate encoded code on a time basis. Can be realized.

The buffer controller 13 controls reading of the audio code buffer 6 in synchronization with the switching of the switch 8. From the audio code buffer 6, the switch 8 outputs a fixed amount of audio code
c2 is transferred.

During the time when the audio code c2 is not transferred, the switch 8 is switched to the video code buffer 5 side. The buffer controller 14 controls the video signal c1 so that the stored video code c1 is transferred at an appropriate transfer rate (for example, 11.5 Mbps) during a certain period during which the switch 8 is connected to the video code buffer 5 side. The reading of the code buffer 5 is controlled.

As described above, when the video code c1 does not exist (is not stored) in the video code buffer 5, the buffer controller 14 suspends the transfer of the video code c1 to the switch 8, and By waiting until c1 is accumulated, underflow in the video code buffer 5 is suppressed. When the video code c1 is accumulated in the video code buffer 5, the buffer controller 14 starts reading the video code buffer 5 again, and sends the video code c to the switch 8.
Transfer c1.

Thereafter, in the data multiplexing circuit 7, the additional information insertion unit 10 adds a counter 9 to the video code and audio code multiplexed by the switch 8.
Based on the count value and the like, information such as a time code is added to form a packet, and a bit stream bs is constructed. The bit stream bs is transferred to the modulation recording circuit 11.

The information transfer from the video code buffer 5 and the audio code buffer 6 to the data encoding device 7 is performed in order to suppress the overflow of the video code buffer 5 and the audio code buffer 6 so that the encoding rate of all information is maximized. Control is performed so that the transfer rate is equal to or greater than the sum of the values (Rvmax + Ra). The transfer amount of the video code c1 also depends on the size of the pack being created.

The switching between standby and transfer of the video code buffer 5 is controlled by the buffer controller 14 based on the storage amount o1 of the video code c1 stored in the video code buffer 5.

The bit stream bs input to the modulation recording circuit 11 is recorded on a disk 12 inserted in a drive after being modulated by a predetermined method.

FIG. 2 shows the results of the data storage amounts in the video and audio code buffers 5 and 6 in the first embodiment.
Shown in The horizontal axis indicates time, and the vertical axis indicates the number of bits accumulated (accumulated) in the buffer. FIG. 2A shows the audio code output amount output from the audio code buffer 6 to the data multiplexing circuit 7. B indicates an audio code input amount input from the audio encoder 4 to the audio code buffer 6. The interval between A and C indicates the capacity of the audio code buffer 6. The interval between A and B indicates the storage amount o2 of the audio code stored in the audio code buffer 6.

D indicates the amount of video code output from the video code buffer 5 to the data multiplexing circuit 7.
E indicates a video code input amount input from the video encoder 3 to the video code buffer 5. The interval between D and F indicates the capacity of the video code buffer 5. D and E
Represents the storage amount o1 of the video code stored in the video code buffer 5.

In the video code buffer 5, coded data is input at a variable rate, and the output depends on the amount of buffer storage, the size of the pack being created, or the transfer time interval of the audio code. The amount of accumulated data fluctuates randomly.

On the other hand, in the audio code buffer 6,
As shown in an enlarged manner in FIG. 3, an audio code is input at a fixed rate, a transfer state (a slope portion of A) at a transfer rate (11.5 Mbps) or higher (Rvmax + Ra) and a standby state where the audio code c2 is not transferred. (That is, the transfer rate of 0 Mbps, the horizontal portion of A) is repeated at fixed time intervals, so that the accumulated amount o2 of the audio code c2 fluctuates while repeating the same pattern at fixed time intervals. However, the average amount of accumulated data per unit time does not change.

Also, it can be seen that data is written and read in each code buffer so that overflow and underflow do not occur.

In the horizontal portion of the video code output amount D, the buffer controller 14 monitors the storage amount o1 of the video code c1 in the video code buffer 5 and determines that the video code c1 is not stored.
A standby state (that is, a transfer rate of 0 Mbps) is shown, and the other slopes are video codes at a transfer rate of (Rvmax + Ra) or more, for example, 11.5 Mbps.
The figure shows a state in which c1 is sent to the data multiplexing circuit 7.

However, the output from the video code buffer 5 to the data multiplexing circuit 7 is
This is performed during a period (horizontal portion of A) during which data is not output to the data multiplexing circuit 7 from. For this reason, actually, the period during which the D video code c1 is output (the slope portion of D) has a finer horizontal portion corresponding to the slope portion of A, but is shown by a straight line for simplicity. . still,
Similarly, the inclined portion of F actually has a finer horizontal portion corresponding to the inclined portion of A, but is shown by a straight line for simplicity.

In the above embodiment, the data encoded at the variable rate is the moving image data. However, the present invention is not limited to this. The present invention is also applicable to other data such as audio data and subtitle data.

In the above embodiment, the audio data is
Although the coding is performed at a fixed rate, the coding may be performed at a variable rate (the maximum rate is Ramax). In this case, the information transfer from the video code buffer 5 and the audio code buffer 6 to the data encoding device 7 is performed by summing the maximum value of the encoding rates of all information (Rvmax + R
(amax) or higher.

When audio data is encoded at a variable rate, the data storage amount of the audio code buffer 6 becomes
The counter 9 randomly varies, so that the counter 9 counts a reference clock (not shown) without using the audio code accumulation amount o2 as a time reference, and controls the switching of the switch 8 at fixed time intervals.

In the first embodiment, one variable-rate coded data (moving picture data) and one fixed-rate coded data (speech data) are input to the data coding apparatus. May be plural.
Therefore, next, as a second embodiment (embodiment 2), a case where input data to a data encoding device is generalized and a plurality of variable rate encoded data and a plurality of fixed rate encoded data are processed will be described. This will be described with reference to FIG.

FIG. 4 shows a second embodiment of the data encoding apparatus according to the present invention. This is a generalization of the first embodiment, and the basic configuration of the device is the same as that of the first embodiment.

N VBRs (VBR: Variable Bit Rate)
The source data 21 (21 _{1 to} 21 _N ) includes N VBRs.
Is input to the encoder 22 (22 ₁ to 22 _N), are respectively encoded at a variable rate. Encoded N variable rate encoded data (VBR: Variable Bit Rate stream)
Are N VBR code buffers 23 respectively corresponding to
It is temporarily stored in (23 ₁ to 23 _N).

Further, M CBRs (CBR: Constant Bit R)
ate) source data 29 (29 ₁ to 29 _M) are inputted to the M CBR encoder 30 (30 ₁ to 30 _M),
Each is encoded at a fixed rate. M fixed-rate encoded data (CBR: Constant Bit Rate st)
ream) are temporarily stored in the corresponding M CBR code buffers 31 (31 _{1 to} 31 _M ). The input data to the data multiplexing circuit 32 is N variable rate encoded data and M fixed rate encoded data.

In the data multiplexing circuit 32 shown in FIG.
Data is multiplexed by inputting respective code data from the M CBR code buffers 31 and the N VBR code buffers 23 in an appropriate order. Switching of input code data to the data multiplexing circuit 32 is performed by the switch 25.

The counter 27 monitors the data storage amount o (m) of one of the M CBR code buffers 31, and based on this data storage amount o (m), the switch 25
Is sequentially switched to each VBR code buffer 23 and each video code buffer 31 at a fixed time interval. The buffer controller 28 controls reading of each CBR code buffer 31 in synchronization with the switching of the switch 25. From each of the CBR code buffers 31, a certain amount of CB is output during a certain time during which the switch 25 is connected.
R codes are sequentially transferred. This transfer rate is CBR
And the VBR encoders 22 and 30 take a value equal to or greater than the sum of the maximum values of all the encoding rates.

At the time when no CBR code is transferred, the switch 25 sets the VBR code buffer 23
Can be switched to the side. The buffer controller 24 controls each of the VBR codes so that the VBR codes stored in the respective VBR code buffers 23 are transferred at an appropriate transfer rate during a certain time during which the switch 25 is switched to the VBR code buffer 23 side. The reading of the VBR code buffer 23 is controlled. This transfer rate is based on CBR and VB
It is assumed that the value is equal to or larger than the sum of all the encoding rates of the R encoders 22 and 30.

The priority of the VBR code to be read is determined in advance, and the switch 25 performs the scheduling. However, when the VBR code to be read is not stored in the VBR code buffer 23, the transfer of the VBR code is temporarily stopped, and the process waits until the VBR code is stored (or another VBR code). If sufficient data is stored in the code buffer, the code data is read.) As a result, underflow in the VBR code buffer is suppressed.

Thereafter, when the VBR code is accumulated in the VBR code buffer, the buffer controller 24 starts reading the VBR code buffer again, and transfers the VBR code to the switch 25.

In the data multiplexing circuit 32, the additional information insertion unit 26 adds information such as a time code based on the count value of the counter 27 and the like, forms a packet, and constructs a bit stream. The bit stream is transferred to the modulation recording circuit 11 as in the case of FIG. After the bit stream input to the modulation recording circuit 11 is modulated, the
2 recorded.

[0070]

Since the present invention is configured as described above, it has the following effects.

By controlling the transfer / standby of the coded code to the multiplexing circuit in accordance with the storage amount of the code buffer in which the coded code of the variable rate is stored, the generation of the code data in the variable rate coding circuit is performed. Even if the amount is variable, underflow in the code buffer is suppressed.

Further, by setting the transfer rate from the code buffer to the multiplexing circuit to be equal to or greater than the sum of the maximum values of the encoding rates of all the encoders, overflow in the code buffer is suppressed.

Further, by switching data input to the data multiplexing circuit at fixed time intervals, the multiplexed bit stream contains a predetermined coded code at fixed time intervals. For this reason, when decoding, the delay time is small, and synchronization between the multiplexed data can be easily achieved.

Furthermore, since a code encoded at a fixed rate is used as a time reference, multiplexing at a fixed time interval can be realized with a simple configuration.

As described above, since the underflow of the code buffer is suppressed without generating the pseudo data, it is possible to realize a recording apparatus using the storage capacity of the recording medium efficiently. In addition, the recording medium created by the recording device can include data for a longer time because the recording medium does not include the pseudo data.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration example of a data multiplexing device according to an embodiment of the present invention.

FIG. 2 is a diagram for explaining an accumulated amount of encoded codes in a code buffer in the embodiment of FIG. 1;

FIG. 3 is an enlarged view showing an encoded code storage amount of an audio code buffer of FIG. 2;

FIG. 4 is a block diagram showing a configuration example of another embodiment of the data multiplexing device of the present invention.

Fig. 5 MPEG (ISO 11172 1-AnnexA "Description
FIG. 3 is a diagram illustrating a data structure of a bitstream indicated in “of the System Coding Layer” (informative)).

FIG. 6 is a diagram illustrating an output bit stream from a data multiplexing apparatus to which fixed-rate encoded data is input.

[Explanation of symbols]

 Reference Signs List 1 camera 2 microphone 3 video encoder 4 audio encoder 5 video code buffer 6 audio code buffer 7 data multiplexing circuit 8 switch 9 counter 10 additional information insertion unit 11 modulation recording circuit 12 disk 13, 14 buffer controller

Continuation of the front page (56) References JP-A-5-2834 (JP, A) JP-A-4-267682 (JP, A) JP-A-5-161115 (JP, A) European Patent Application Publication 515101 (EP, A2) (58) Field surveyed (Int. Cl. ⁷ , DB name) H04N 5/91-5/956 H04N 7/24 G11B 20/10

Claims (14)

(57) [Claims] 1. A decoder for decoding an encoded signal.
Buffer overflow or underflow
A data multiplexing device for multiplexing the coded signal so as not to cause the multiplexed signal; an image coding means for coding at least one image signal at a variable rate; and a voice coding device for coding at least one voice signal. Means, first storage means for temporarily storing at least one encoded image signal encoded by the image encoding means, and at least one encoded audio signal encoded by the audio encoding means A multiplexing unit for multiplexing the coded image signal and the coded audio signal stored in the first and second storage units; and based on the accumulated amount of accumulated said encoded image signal in the storage means, and control means for controlling the transfer of the coded image signal to the multiplexing means from said first storage means, said system The control by means of the transfer restriction youth properly
Is performed by controlling the transfer rate of the transfer.
That, data multiplexing apparatus characterized by. 2. The method according to claim 1, wherein the control unit suspends the transfer of the coded image signal to the multiplexing unit when the coded image signal does not exist in the first storage unit. 2. The data multiplexing device according to claim 1, wherein when the coded image signal is present, the coded image signal is transferred to the multiplexing unit at a predetermined transfer rate. 3. The data according to claim 2, wherein the predetermined transfer rate is equal to or greater than a sum of a maximum encoding rate of the image encoding unit and a maximum encoding rate of the audio encoding unit. Multiplexer. The inputs to wherein said multiplexing means, according to claim 1, characterized by further comprising a switching means Ru switched based on the storage amount of the second the encoded audio signal in the storage means or 4. The data multiplexing device according to any one of 3 . 5. The recording apparatus according to claim 1, further comprising recording means for recording the coded image signal and the coded audio signal multiplexed by the multiplexing means on a recording medium.
5. The data multiplexing apparatus according to any one of 1, 2, 3, and 4 . 6. A decoder for decoding an encoded signal.
Buffer overflow or underflow
A data multiplexing method for multiplexing the coded signals , wherein at least one image signal is coded at a variable rate, at least one audio signal is coded, and at least one coded coding is performed. Temporarily storing the image signal in the first storage means, temporarily storing the encoded at least one encoded audio signal in the second storage means, storing the encoded image signal and the encoded the audio signal multiplexed with multiplexer, the first based on the accumulated amount of accumulated said encoded image signal in the storage unit, the coded image signal from the first storage unit to the multiplexer The transfer of the transfer,
Or a step of controlling the transfer rate of the transfer by controlling the transfer rate . 7. The step of controlling the transfer includes, if the encoded image signal is not present in the first storage means, temporarily suspending the transfer of the encoded image signal to the multiplexer. 7. The data multiplexing method according to claim 6, wherein when the coded image signal is stored in the storage unit, the coded image signal is transferred to the multiplexer at a predetermined transfer rate. 8. The data according to claim 7, wherein the predetermined transfer rate is equal to or greater than a sum of a maximum encoding rate of the encoded image signal and a maximum encoding rate of the encoded audio signal. Multiplexing method. 9. An input to said multiplexer based on an amount of said encoded speech signal stored in said second storage means.
9. Switching according to claim 6, 7 or 8.
The data multiplexing method described in any of the above. 10. The apparatus according to claim 6 , further comprising a step of recording the coded image signal and the coded audio signal multiplexed by the multiplexer on a recording medium. data multiplexing method according to any. 11. A decoder for decoding an encoded signal.
Coder buffer overflow or underflow
In multiplexed recorded data recording medium of the encoded signal so as not to cause, at least one image signal encoded at a variable rate, encoding at least one audio signal, at least one encoded Temporarily storing an encoded image signal in a first storage means, temporarily storing at least one encoded audio signal in a second storage means, and storing the encoded image signal and A multiplexed audio signal is multiplexed by a multiplexer, and the encoding from the first storage unit to the multiplexer is performed based on the amount of the encoded image signal stored in the first storage unit. Transfer of image signals is restricted by the transfer
Or by controlling a transfer rate of the transfer, and recording the encoded image signal and the encoded audio signal multiplexed by the multiplexer on a recording medium. Data recording medium. 12. A decoder for decoding an encoded signal.
A storage unit having the same capacity as the coder buffer;
In a data multiplexing device for multiplexing signals, An image for encoding at least one image signal at a variable rate
Image encoding means; Audio encoding means for encoding at least one audio signal
When, At least one encoded by the image encoding means
Coded image signal First storage means for temporally storing
When, At least one encoded by the audio encoding means
Storage means for temporarily storing the encoded audio signal of
When, The encoded image stored in the first and second storage means;
Multiplexing means for multiplexing the image signal and the encoded audio signal, Of the encoded image signal stored in the first storage means.
Multiplexing from the first storage means based on the storage amount
Control means for controlling the transfer of said encoded image signal to the means
With The control by the control means may limit the transfer.
Is performed by controlling the transfer rate of the transfer.
, A data multiplexing device characterized by the above-mentioned. 13. A decoder for decoding an encoded signal.
A storage unit having the same capacity as the coder buffer;
Data multiplexing method for data multiplexing equipment that multiplexes signals
In the law, Encoding at least one image signal at a variable rate; Encoding at least one audio signal; Temporarily transforming the encoded at least one encoded image signal
At the first storage means, Temporarily transforming the encoded at least one encoded audio signal
Is stored in the second storage means, The stored encoded image signal and encoded audio signal are
Multiplexing with multiplexing, Of the encoded image signal stored in the first storage means.
Multiplexing from the first storage means based on the storage amount
The transfer of the encoded image signal to the
Or by controlling the transfer rate of the transfer
A data multiplexing method. 14. A decoder for decoding an encoded signal.
Multiplexer having storage means of the same capacity as the coder buffer
Coded signal multiplexed by Data recording
In the medium, Encoding at least one image signal at a variable rate; Encoding at least one audio signal; Temporarily transforming the encoded at least one encoded image signal
At the first storage means, Temporarily transforming the encoded at least one encoded audio signal
Is stored in the second storage means, The stored encoded image signal and encoded audio signal are
Multiplex with a multiplexer, Of the encoded image signal stored in the first storage means.
Multiplexing from the first storage means based on the storage amount
The transfer of the encoded image signal to the
Or by controlling the transfer rate of the transfer
And The encoded image signal multiplexed by the multiplexer; and
Formed by recording the encoded audio signal on a recording medium.
A data recording medium characterized by being recorded.