WO2023197809A1 - High-frequency audio signal encoding and decoding method and related apparatuses - Google Patents

Abstract

Disclosed are a high-frequency audio signal encoding and decoding method and related apparatuses, which can be applied to various scenarios such as cloud technology, artificial intelligence, intelligent transportation, auxiliary driving, and a vehicle-mounted scenario. The encoding method comprises: acquiring a plurality of encoding modes and acquiring an original high-frequency audio signal frame decomposed from an original audio signal frame, wherein the encoding modes have corresponding priorities, and the numbers of encoding bits of the encoding modes are ascended in the order of descending the priorities of the encoding modes; according to the priorities of the encoding modes, determining, from among the plurality of encoding modes, an encoding mode having an encoding error within a preset error interval as a target encoding mode; and sending to a receiving end a high-frequency code stream obtained by encoding the original high-frequency audio signal frame by using the target encoding mode. According to the present application, an encoding mode having a small number of encoding bits is selected while ensuring encoding quality, thereby achieving satisfactory effects in both the number of encoding bits and the encoding quality, and having a lower number of encoding bits and a high-quality audio.

Description

A coding and decoding method for high-frequency audio signals and related devices

This application claims the priority of the Chinese patent application submitted to the China Patent Office on April 15, 2022, with the application number 202210395889.2 and the application title "A high-frequency audio signal encoding and decoding method and related devices", and its entire content is approved by This reference is incorporated into this application.

Technical field

The present application relates to the field of computer technology, and in particular to the encoding and decoding of high-frequency audio signals.

Background technique

Audio coding and decoding occupies an important position in modern communication systems. By compressing and encoding audio signals, the network bandwidth pressure of audio signals in network transmission can be reduced, and the storage and transmission costs of audio signals can be saved.

The high-frequency component of the audio signal (i.e., the high-frequency audio signal) has rich information and has a greater impact on the sound quality. The loss of the high-frequency audio signal will lead to problems such as muffled sound, reduced intelligibility, and reduced fidelity. Compared with the low-frequency component of the audio signal (i.e., low-frequency audio signal), it has the characteristics of low energy proportion, low harmonic components, and low human ear resolution, so it has a large coding compression space.

The current high-frequency audio signal encoding methods either sacrifice encoding quality in order to reduce the number of encoding bits, or increase the number of encoding bits in order to improve encoding quality. It is difficult to achieve satisfactory results in both the number of encoding bits and encoding quality.

Contents of the invention

In order to solve the above technical problems, the present application provides a coding and decoding method for high-frequency audio signals and related devices. When the coding quality allows, a coding method with a small number of coding bits can be selected to achieve a balance between the number of coding bits and the coding quality. The quality is relatively satisfactory, with a lower number of encoding bits and high-quality audio.

The embodiments of this application disclose the following technical solutions:

On the one hand, embodiments of the present application provide a method for encoding high-frequency audio signals. The method includes:

Obtain multiple encoding methods, and obtain the original high-frequency audio signal frame decomposed from the original audio signal frame;

Obtain the priorities corresponding to the multiple encoding methods, and in the order of the priorities from high to low, the number of encoding bits of the encoding method increases;

According to the priority of the coding method, the coding method whose coding error is within the error preset interval is determined from the multiple coding methods as the target coding method. The coding error of the coding method is the encoding method of the original high-frequency audio signal frame. Generated by coding;

The high-frequency code stream obtained by encoding the original high-frequency audio signal frame using the target encoding method is sent to the receiving end. The high-frequency code stream has an encoding identifier, and the encoding identifier is used to indicate that the encoding obtained by The encoding method used for high-frequency code streams.

On the one hand, embodiments of the present application provide another method for decoding high-frequency audio signals. The method includes:

Receive the high-frequency code stream sent by the transmitting end, the high-frequency code stream has a coding identifier, and the coding identifier is used to indicate the coding method used to encode the high-frequency code stream;

Analyze to obtain the encoding identifier corresponding to the high-frequency code stream, and determine the encoding method indicated by the encoding identifier;

The high-frequency code stream is decoded according to the decoding method corresponding to the encoding method to obtain a high-frequency audio signal frame.

On the one hand, embodiments of the present application provide a device for encoding high-frequency audio signals. The device includes an acquisition unit, Determine unit and send unit:

The acquisition unit is used to acquire multiple encoding methods and acquire original high-frequency audio signal frames decomposed from original audio signal frames;

The acquisition unit is also used to acquire the priorities corresponding to the multiple encoding methods. According to the order of the priorities from high to low, the number of encoding bits of the encoding methods increases;

The determination unit is configured to determine, from the plurality of coding methods, a coding method with a coding error within a preset error interval as a target coding method according to the priority of the coding method. The coding error of the coding method is determined by using the coding method. It is generated by encoding the original high-frequency audio signal frame;

The sending unit is configured to send a high-frequency code stream obtained by encoding the original high-frequency audio signal frame using the target encoding method to the receiving end. The high-frequency code stream has a coding identifier, and the coding identifier Used to indicate the encoding method used to obtain the high-frequency code stream.

On the one hand, embodiments of the present application provide another device for decoding high-frequency audio signals. The device includes a receiving unit, an analysis unit, and a decoding unit:

The receiving unit is used to receive the high-frequency code stream sent by the transmitting end. The high-frequency code stream has a coding identifier, and the coding identifier is used to indicate the coding method used to encode the high-frequency code stream;

The analysis unit is used to analyze and obtain the encoding identifier corresponding to the high-frequency code stream, and determine the encoding method indicated by the encoding identifier;

The decoding unit is configured to decode the high-frequency code stream according to the decoding method corresponding to the encoding method to obtain a high-frequency audio signal frame.

On the one hand, embodiments of the present application provide a computer device, which includes a processor and a memory:

The memory is used to store a computer program and transmit the computer program to the processor;

The processor is configured to execute the method described in any of the foregoing aspects according to instructions in the computer program.

On the one hand, embodiments of the present application provide a computer-readable storage medium used to store a computer program, and the computer program is used to execute the method described in any of the foregoing aspects.

On the one hand, embodiments of the present application provide a computer program product, including a computer program, which implements the method described in any of the foregoing aspects when executed by a processor.

It can be seen from the above technical solution that this application proposes a high-frequency audio signal encoding and decoding method based on the mixing of multiple encoding methods based on coding error judgment for the original high-frequency audio signal. Specifically, for the original audio signal For each original audio signal frame in, obtain multiple encoding methods and obtain the original high-frequency audio signal frame decomposed from the original audio signal frame. The encoding methods have corresponding priorities, in order from high to low priority, The encoding method has an increasing number of coded bits. Then according to the priority of the coding method, the coding method with coding error within the error preset interval is determined from multiple coding methods as the target coding method. The coding error of the coding method is generated by encoding the original high-frequency audio signal frame using the coding method. , so that the coding error can be used as the criterion, the target coding method can be determined with the optimal number of coding bits as the goal, and the high-frequency code stream obtained by encoding the original high-frequency audio signal frame using the target coding method is sent to the receiving end. Therefore, when the encoding quality permits, the encoding method with a small number of encoding bits is selected, which reduces the bandwidth of audio signal transmission. Since the high-frequency code stream has a coding identifier, the coding identifier is used to indicate the encoding method used to obtain the high-frequency code stream, so that the decoding end can determine which encoding to use based on the coding identifier. Method to decode the received high-frequency code stream. It can be seen that this application can choose to use a coding method with a small number of coded bits when the coding quality permits, to achieve relatively satisfactory results in both the number of coded bits and coding quality, with a lower number of coded bits and high-quality audio. .

Description of the drawings

Figure 1 is an application scenario architecture diagram of a high-frequency audio signal encoding and decoding method provided by an embodiment of the present application;

Figure 2 is a flow chart of a high-frequency audio signal encoding method provided by an embodiment of the present application;

Figure 3 is a coding flow chart of an SBR method provided by an embodiment of the present application;

Figure 4 is a decoding flow chart of an SBR method provided by an embodiment of the present application;

Figure 5 is a schematic diagram of a low-frequency audio signal copy and correction of a high-frequency copy signal provided by an embodiment of the present application;

Figure 6 is a coding flow chart of a CELP coding method provided by an embodiment of the present application;

Figure 7 is a decoding flow chart of a CELP encoding method provided by an embodiment of the present application;

Figure 8 is a flow chart of a coding error determination method provided by an embodiment of the present application;

Figure 9 is an acoustic equal loudness curve measured by the International Acoustic Standards Organization provided by the embodiment of the present application;

Figure 10 is a calculated auditory perception weighting coefficient diagram provided by an embodiment of the present application;

Figure 11 is a flow chart of a method for decoding high-frequency audio signals provided by an embodiment of the present application;

Figure 12 is an overall implementation architecture diagram of a high-frequency audio signal encoding and decoding method provided by an embodiment of the present application;

Figure 13 is a structural diagram of a high-frequency audio signal encoding device provided by an embodiment of the present application;

Figure 14 is a structural diagram of a high-frequency audio signal decoding device provided by an embodiment of the present application;

Figure 15 is a structural diagram of a terminal provided by an embodiment of the present application;

Figure 16 is a structural diagram of a server provided by an embodiment of the present application.

Detailed ways

The embodiments of the present application are described below with reference to the accompanying drawings.

Audio codec plays an important role in modern communication systems. For example, in a voice call application, the audio signal is collected through a microphone, and the analog audio signal is converted into a digital audio signal through an analog-to-digital conversion circuit. The digital audio signal is compressed by an encoder, and then packaged and sent to the receiver according to the communication network transmission format and protocol. After receiving the data packet, the receiving end unpacks and outputs the encoded code stream, and then regenerates the audio digital signal through the decoder. Finally, the audio digital signal is played through the speaker. Audio coding and decoding can effectively reduce the bandwidth of audio signal transmission, play a decisive role in saving audio signal storage and transmission costs, and ensuring the integrity of audio signals during communication network transmission.

The high-frequency audio signal of the audio signal has richer information, which has a greater impact on the sound quality. Compared with the low-frequency audio signal of the audio signal, the high-frequency audio signal has a lower proportion of energy, lower harmonic components, and lower resolution of the human ear. Low-level characteristics, so it has a large coding compression space.

The high-frequency audio signal coding methods provided by related technologies either sacrifice coding quality in order to reduce the number of coding bits (such as the blind expansion method), or increase the number of coding bits in order to improve the coding quality (such as code-excited linear prediction (Code-Excitation Linear Prediction)). Excited Linear Prediction (CELP) coding method), it is difficult to achieve satisfactory results in terms of the number of coding bits and coding quality.

In order to solve the above technical problems, embodiments of the present application provide a encoding and decoding method for high-frequency audio signals. It is a high-frequency audio signal coding and decoding method based on the mixing of multiple coding methods based on coding error judgment. When the coding quality allows, the coding method with a small number of coding bits can be selected to achieve the best performance in terms of the number of coding bits and coding quality. All achieve relatively satisfactory results, with lower encoding bit count and high-quality audio. Among them, the audio signal can be speech, music, etc.

As shown in Figure 1, Figure 1 shows an application scenario architecture diagram of a high-frequency audio signal encoding and decoding method. This application scenario may include a sending end 101 and a receiving end 102. Among them, the sending end 101 and the receiving end 102 can both be terminals, or the sending end 101 can be a terminal, the receiving end 102 can be a server, and so on. The terminal may be, for example, a mobile phone, a computer, an intelligent voice interaction device, a smart home appliance, a vehicle-mounted terminal, an aircraft, etc., but is not limited thereto. The server can be, for example, an independent server, a server in a cluster, or a cloud server. When the sending end 101 is a terminal and the receiving end 102 is a server, the terminal and the server can be connected through wired or wireless means. The embodiment of this application will be introduced using a voice call scenario as an example. In this case, both the sending end 101 and the receiving end 102 can be terminals, and the terminal is a mobile phone. The above-mentioned terminals and servers are computer equipment.

In the voice call scenario, the sending end 101 can collect the original audio signal through the corresponding microphone (in this case, the audio signal can be the voice of the corresponding user of the sending end 101). Before sending the original audio signal to the receiving end 102, the sending end 101 can To encode the original audio signal, the embodiment of this application mainly introduces the encoding of the original high-frequency audio signal in the original audio signal.

For each original audio signal frame in the original audio signal, the sending end 101 can obtain multiple encoding methods, and obtain the original high-frequency audio signal frame decomposed from the original audio signal frame. The encoding methods have corresponding priorities. The priority of the method is used to indicate the priority of encoding using this encoding method. Normally, in order to reduce the bandwidth of audio signal transmission as much as possible, the number of encoding bits of the encoding method increases in order from high to low priority.

Then the sending end 101 determines the coding method with coding error within the preset error interval from multiple coding methods as the target coding method according to the priority of the coding method. The coding error of the coding method is calculated by using the coding method to encode the original high-frequency audio signal frame. Generated by encoding, the target encoding method can be determined with the encoding error as the criterion and the optimal number of encoding bits as the goal. The transmitting end 101 sends the high-frequency code stream obtained by encoding the original high-frequency audio signal frame using the target encoding method to the receiving end 102, so that when the encoding quality allows, the encoding method with a small number of encoding bits is selected to reduce the cost. The bandwidth of audio signal transmission. Since the high-frequency code stream has an encoding identifier, the encoding identifier is used to indicate the encoding method used to obtain the high-frequency code stream, so that the receiving end 102 can determine which encoding method to use to decode the received high-frequency code stream based on the encoding identifier. .

The receiving end 102 determines the decoding method corresponding to the identified encoding method according to the encoding identification, thereby using the corresponding decoding method to decode the high-frequency code stream to obtain the high-frequency audio signal frame, and play it through the corresponding speaker.

It should be noted that the embodiments of this application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, vehicle-mounted scenarios, etc., and are specifically applied to voice calls, video conferencing, in-person communication in these scenarios. Computer interaction scenes and so on.

Next, the encoding method of high-frequency audio signals provided by the embodiments of the present application will be introduced in detail from the perspective of the transmitter with reference to the accompanying drawings.

Referring to Figure 2, Figure 2 shows a flow chart of a method for encoding high-frequency audio signals. This embodiment can be executed by a computer device. The method includes:

S201. Obtain multiple encoding methods, and obtain the original high-frequency audio signal frame decomposed from the original audio signal frame.

When the sending end obtains the original audio signal, the bandwidth of the audio signal transmission is effectively reduced, and the original audio signal can be encoded first, thereby transmitting the encoded code stream to the receiving end. The original audio signal may include high-frequency audio signals. Based on the characteristics of high-frequency audio signals, high-frequency audio signals have a large coding space. When encoding high-frequency audio signals, embodiments of the present application provide a high-frequency audio signal encoding method that mixes multiple encoding methods based on coding error determination. Specifically, for each original audio signal frame in the original audio signal, the transmitting end can obtain multiple encoding methods, and obtain the original high-frequency audio signal frame decomposed from the original audio signal frame.

In the embodiment of the present application, the multiple encoding methods available for selection can be any number of existing encoding methods, such as audio super resolution (Speech Super Resolution, SSR) method, frequency band replication (Spectral Band Replication) , SBR) method, CELP coding method and other methods in combination. Among them, the SSR method is a blind expansion method, and the SBR method and CELP coding method are a non-blind expansion method. Of course, various coding methods can also include other The encoding method is not limited in the embodiments of this application. The embodiments of this application mainly introduce multiple coding methods including SSR method, SBR method and CELP coding method as examples.

The encoding and decoding principles of each encoding method are introduced in turn below:

The SSR method is a blind frequency band expansion method, which can also be called a blind expansion method. The SSR method does not send coding parameters to the receiving end when encoding, so this method does not occupy the number of coding bits. When decoding at the receiving end, there is a certain correlation between the low-frequency audio signal and the high-frequency audio signal. Through some prediction methods, such as the neural network model in deep learning, the high-frequency audio signal is mapped from the low-frequency audio signal. This method aims to reconstruct a high-resolution audio signal with a lower-resolution audio signal as input. With the rapid development of deep neural networks, the SSR method can predict the characteristic information of high-frequency audio signals based on the characteristic information of the input low-frequency audio signals based on the neural network model, thereby fictionalizing the high-frequency audio signals.

The SBR method is a non-blind expansion method that requires reconstructing high-frequency audio signals based on a small number of coding parameters transmitted by the transmitter. Since high-frequency reconstruction is supported by auxiliary information, this method has better reconstruction quality. As shown in Figure 3 and Figure 4 below, Figure 3 is the encoding flow chart of the SBR mode, and Figure 4 is the decoding flow chart of the SBR mode. Figure 3 illustrates the encoding method of Advanced Audio Coding (AAC)+SBR, and Figure 4 illustrates the corresponding AAC+SBR decoding method. In Figure 3, the original audio signal is first decomposed into a high-frequency audio signal and a low-frequency audio signal. For example, the high-frequency audio signal is obtained through a Quadrature Mirror Filter (QMF) filter bank, and the high-frequency audio signal is obtained through a 2:1 downsampler. The low-frequency audio signal uses an AAC encoder to generate the encoding parameters of the low-frequency audio signal, while the high-frequency audio signal is encoded based on the SBR encoder. The low-frequency audio signal is copied to the high-frequency band to obtain the high-frequency copy signal, and then the high-frequency replica signal is obtained according to the envelope. The envelope features are extracted, used to correct the high-frequency replica signal (the process is shown in Figure 5), and the encoding parameters are extracted and sent to the receiving end. As can be seen from Figure 5, (a) in Figure 5 is a schematic diagram of the high-frequency energy curve after directly copying the low-frequency audio signal to the high-frequency band to obtain the high-frequency copy signal 501, and the high-frequency energy is different from the actual high-frequency energy. There is a slight difference. The envelope feature can more accurately reflect the high-frequency energy. Therefore, the high-frequency copy signal is corrected based on the envelope feature obtained by envelope extraction to obtain the high-frequency reconstructed signal 502. The high-frequency energy curve obtained at this time can be See (b) in Figure 5. The encoded high-frequency audio signal and low-frequency audio signal obtained through the above process can be combined by a bit stream multiplexer to obtain the corresponding encoded code stream.

Through the introduction of the encoding process of the SBR method above, the SBR method only needs to transmit limited parameters to the receiving end.

The decoding process at the receiving end shown in Figure 4 uses a code stream decomposer to decompose the coded code stream into coded low-frequency audio signals and coded high-frequency audio signals. First, the encoded low-frequency audio signal is decoded. The encoded low-frequency audio signal is generated by the AAC decoder, which is passed through the QMF analysis filter and then participates in high-frequency reconstruction. The high-frequency reconstruction process is to decode the SBR decoder to obtain the required encoding parameters, copy the low-frequency audio signal to the high-frequency band, and obtain the high-frequency replica signal. The envelope features obtained by envelope extraction correct the high-frequency copy signal to generate a high-frequency reconstructed signal. Finally, after a certain delay, the high- and low-frequency signals are aligned and combined into a full-band audio signal through a comprehensive filter.

The CELP coding method is an effective speech compression coding method with a medium and low number of coding bits. It uses the codebook as the excitation source. It has the advantages of low code rate, high synthetic speech quality, and strong anti-noise ability. It operates at a code rate of 4.8 to 16kbps. It has been widely used. Currently, there are many types of encoders using CELP encoding. Figures 6 and 7 are respectively the encoding flow chart of the CELP encoding method and the decoding flow chart of the CELP encoding method. In Figure 6, after the original audio signal is preprocessed such as high-pass filtering, a set of linear prediction filter coefficients is obtained through linear predictive coding (LPC), and the LPC parameters (such as linear prediction filter coefficients) are converted into LSP parameters And quantized to facilitate transmission to the receiving end. Preprocessed original audio signal s(n) and LPC prediction filtering results The difference is the residual signal. After the residual signal passes through the perceptual weighting filter, the filtered residual signal is obtained. Based on the filtered residual signal e(n), and based on the principle of minimum perceptual weighted error, search for the best Fixed codebook and adaptive codebook to calculate fixed codebook gain (Gc) and adaptive codebook gain (Ga). The coding parameters obtained during these coding processes are encapsulated and packaged and transmitted to the receiving end.

During the decoding process, as shown in Figure 7, the receiving end parses all coding parameters from the received data packet through the decoder, and generates a fixed codebook excitation signal based on the fixed codebook and fixed codebook gain, and based on the adaptive codebook The adaptive codebook gain and the adaptive codebook gain generate an adaptive codebook excitation signal. The sum of the two excitations is filtered and post-processed by a synthesis filter to obtain the final audio signal. Among them, the filter coefficients of the synthesis filter are obtained by interpolating the LSP parameters.

S202: Obtain the priorities corresponding to the multiple encoding methods, and in order of the priorities from high to low, the number of encoding bits of the encoding methods increases.

In the embodiment of the present application, the coding method has a corresponding priority. The priority of the coding method is used to indicate the priority of using this coding method for encoding. Normally, in order to reduce the bandwidth of audio signal transmission as much as possible, according to the priority, from In order from high to low, the number of encoding bits of the encoding method increases, that is, the bandwidth occupied by the transmission or the compressed storage space increases.

When multiple coding methods include SSR, SBR and CELP coding, since the number of coded bits is sequentially SSR, SBR and CELP from small to large, the priority from high to low is SSR, SBR and CELP. Encoding.

S203. According to the priority of the encoding method, determine the encoding method whose encoding error is within the preset error interval from the multiple encoding methods as the target encoding method. The encoding error of the encoding method is the encoding method of the original high-frequency audio. Signal frames are encoded.

In related technologies, when blind expansion is used, it has the advantage of not occupying the number of coding bits for high-frequency reconstruction. The high-frequency audio signal frame is completely predicted based on the characteristic information of the low-frequency audio signal frame. Although the high-frequency and low-frequency audio signal frames have certain correlation, but there is no absolute correspondence, so there is a large error between the high-frequency audio signal frame obtained by high-frequency reconstruction and the original high-frequency audio signal frame; the SBR method can only ensure envelope matching, cannot further reduce the error, and takes up Less number of coding bits; high-frequency reconstruction based on CELP coding ensures that the reconstructed high-frequency audio signal frame has a consistent envelope with the original high-frequency audio signal frame through LSP parameters, and at the same time, codebook excitation is used to further reduce the reconstructed high-frequency audio signal frame. The error between the high-frequency audio signal frame and the original high-frequency audio signal frame, but takes up more coding bits. Based on the above analysis, the goal of the embodiments of the present application is to achieve relatively satisfactory results in both the number of coding bits and the coding quality (that is, the error is small). To this end, the embodiments of the present application are based on the coding error judgment and the priority of the coding method (priority level reflects the number of encoding bits), select an appropriate encoding method to encode the current original high-frequency audio signal frame. Specifically, based on the priority of the encoding method, the sending end determines the encoding method with the encoding error within the error preset interval from multiple encoding methods as the target encoding method. The encoding error of the encoding method is the encoding method of the original high-frequency audio signal. Frames are encoded.

The coding error is generated by encoding the original high-frequency audio signal frame using a coding method, and can be the difference between the reconstructed high-frequency audio signal frame (ie, the high-frequency reconstructed signal frame) and the original high-frequency audio signal frame. Error can reflect the encoding quality. The smaller the encoding error, the higher the encoding quality. Considering the priority and coding quality comprehensively, if the coding quality allows, you can choose to use a coding method with a small number of coded bits to achieve a satisfactory effect in terms of the number of coded bits and coding quality, with a lower number of coded bits. and quality audio. The error preset interval may be greater than the error threshold (Thrd). In this case, the encoding error is less than or equal to Thrd. It can be considered that the encoding error is within the error preset interval. Otherwise, the encoding error exceeds the error preset interval.

It should be noted that the embodiments of this application provide multiple ways to implement S203. In a possible implementation, the coding error of each coding method in multiple coding methods can be determined separately. If there is a coding method in which the coding error is within the error preset interval, the coding error in the coding method is within the error preset interval. Among the methods, the encoding method with the highest priority is determined as the target encoding method. Taking multiple coding methods including SSR method, SBR method and CELP coding method as an example, the order of priority from high to low is SSR method, SBR method and CELP coding method. Determine the encoding of SSR method, SBR method and CELP coding method respectively. Error, if the coding error between the SSR mode and the SBR mode is within the error preset interval among the coding modes, since the priority of the SSR mode is higher than the priority of the SBR mode, the SSR mode will be used as the target encoding mode. Of course, if there are only coding errors in the SSR mode that are within the error preset interval, then the SSR mode is directly used as the target coding mode, or if there are only coding errors in the SBR mode that are within the error preset interval, Then directly use the SBR method as the target encoding method.

If there is no coding method with a coding error within the preset error interval, the coding method can be selected according to the actual situation. For example, for scenes with higher quality requirements, the encoding method with the smallest encoding error is used as the target encoding method to ensure encoding quality. For another example, for scenarios with higher bandwidth requirements, the encoding method with the highest priority is used as the target encoding method. Continuing to take multiple coding methods including SSR method, SBR method and CELP coding method as an example, the order of priority from high to low is SSR method, SBR method and CELP coding method. Determine the SSR method, SBR method and CELP coding method respectively. Coding error. The coding error of any coding method exceeds the preset error interval. For scenes with higher quality requirements, since the coding error of the CELP coding method is the smallest and its coding quality is the highest, the CELP coding method is used as the target coding method; for In scenarios with high bandwidth requirements, since the SSR method has the highest priority, the SSR method is used as the target encoding method.

In another possible implementation, you can perform ladder attempts from high to low according to priority, and sequentially determine whether the encoding error of the encoding method is within the error preset interval. The encoding error of the currently selected encoding method is within the error preset interval. within, stop trying and select the currently selected encoding method as the target encoding method for encoding. Specifically, the sending end selects the undetermined encoding method from multiple encoding methods in order from high to low priority, and determines the encoding error of the undetermined encoding method. If the encoding error of the undetermined encoding method is within the error preset interval, then Determine the pending encoding method as the target encoding method and stop selecting the pending encoding method. If the pending encoding method is the last encoding method among multiple encoding methods (i.e., the encoding method with the lowest priority), it means that the encoding errors of the previously tried encoding methods are beyond the error preset interval, then for the last encoding method It is possible to directly use the last encoding method as the target encoding method without performing the step of determining the encoding error of the encoding method.

Continuing to take multiple coding methods including SSR method, SBR method and CELP coding method as an example, in order of priority from high to low are SSR method, SBR method and CELP coding method, then the SSR method is first selected as the pending coding method, and the SSR The coding error of the method. If the coding error of the SSR method is within the error preset interval, the SSR method is determined as the target encoding method; if the coding error of the SSR method exceeds the error preset interval, the SBR method continues to be selected as the pending encoding method and determined Coding error of the SBR method. If the coding error of the SBR method is within the error preset interval, the SBR method is determined as the target encoding method; if the coding error of the SBR method exceeds the error preset interval, the CELP encoding method is directly used as the target encoding method. .

S204. Send the high-frequency code stream obtained by encoding the original high-frequency audio signal frame using the target encoding method to the receiving end. The high-frequency code stream has an encoding identifier, and the encoding identifier is used to indicate that the encoding has been obtained. The encoding method used by the high-frequency code stream.

After the target encoding method is determined, the high-frequency code stream obtained by encoding the original high-frequency audio signal frame using the target encoding method can be sent to the receiving end. The high-frequency code stream has a coding identifier, and the coding identifier is used to indicate that the high-frequency code stream obtained by encoding is The encoding method used for the video code stream, so that the receiving end can know which encoding method corresponds to the decoding method for decoding. Among them, the encoding identifier can be used to uniquely identify the encoding method, and the encoding identifier can be in various possible forms, such as numbers, symbols, letters, etc.

It can be understood that since when encoding each original high-frequency audio signal frame, it is necessary to select an appropriate encoding method for encoding based on the priority of the encoding method and the encoding error. Therefore, the difference between different original high-frequency audio signal frames The encoding methods may be different, thus achieving mixed encoding of multiple encoding methods.

When selecting a coding method for the original high-frequency audio signal, determining the coding error is a critical step. Next, the method of determining the coding error of any coding method is introduced. As shown in Figure 8, the method includes:

S801. Obtain the original low-frequency audio signal frame decomposed from the original audio signal frame.

S802. According to the original low-frequency audio signal frame, use any encoding method to perform high-frequency reconstruction to obtain a high-frequency reconstructed signal frame.

It should be noted that, according to the coding and decoding principles of different coding methods introduced above, the high-frequency reconstruction methods may be different for different coding methods. If any encoding method is the audio super-resolution method, the implementation method of S802 can be to obtain the neural network model corresponding to the audio super-resolution method, and predict the high-frequency reconstructed signal frame through the neural network model based on the original low-frequency audio signal frame. Among them, the neural network model is obtained by training training samples. The training samples are sample low-frequency audio signal frames with labels. In the training phase, the low-frequency features corresponding to the sample low-frequency audio signal frames can be used as the input of the neural network model. The high-frequency reconstructed signal frame is the output of the neural network model. After training with large-scale training samples, a neural network that can predict the high-frequency reconstructed signal frame based on the sample low-frequency audio signal frame is obtained. network model. When used, the original low-frequency audio signal frame is used as the input of the neural network model, the low-frequency features are extracted through the neural network model, and then the high-frequency reconstructed signal frame is predicted based on the low-frequency features. In some cases, the low-frequency features of the original low-frequency audio signal frame can also be extracted through other methods, and the low-frequency features can be used as the input of the neural network model to output a high-frequency reconstructed signal frame. The neural network model may be a convolutional neural network (Convolutional Neural Networks, CNN), a long short-term memory network (Long Short-Term Memory, LSTM), etc., which are not limited in the embodiments of the present application.

If any encoding method is a frequency band copy method, S802 can be implemented by copying the original low-frequency audio signal frame to the high-frequency band to obtain a high-frequency copy signal frame. Directly copy the low-frequency audio signal frame to the high-frequency band. The high-frequency energy of the high-frequency copied signal frame is slightly different from the actual high-frequency energy. The envelope feature can more accurately reflect the high-frequency energy, so the original high-frequency energy can be extracted. The envelope characteristics of the high-frequency audio signal frame are then used to correct the high-frequency copied signal frame to obtain the high-frequency reconstructed signal frame.

If any encoding method is a code-excited linear prediction method, the implementation of S802 can be to obtain the encoding parameters from the high-frequency code stream and obtain the pitch period (pitch) of the original low-frequency audio signal frame, and then proceed based on the encoding parameters and pitch period. High-frequency reconstruction is performed to obtain a high-frequency reconstructed signal frame. The coding parameters may include LSP parameters, codebook data (such as fixed codebook and adaptive codebook), and gain data (such as fixed codebook gain and adaptive codebook gain).

S803. Perform error analysis based on the high-frequency reconstructed signal frame and the original high-frequency audio signal frame to obtain the corresponding coding error.

After obtaining the high-frequency reconstructed signal frame, error analysis can be performed based on the high-frequency reconstructed signal frame and the original high-frequency audio signal frame to obtain the corresponding coding error. The coding error can reflect the difference between the high-frequency reconstructed signal frame and the original high-frequency audio signal frame. The coding error is used to measure the coding quality of the coding method.

Based on the role of coding error, it can be understood that in one possible implementation, S803 can be implemented by calculating the difference signal between the high-frequency reconstructed signal frame and the original high-frequency audio signal frame, and then using the difference signal to determine Coding errors. If the high-frequency reconstructed signal frame is represented by S’ and the original high-frequency audio signal frame is represented by S, then S’ and S are subtracted to obtain a difference signal, and the difference signal can be represented as Err.

Since the difference signal can already reflect the error between the high-frequency reconstructed signal frame and the original high-frequency audio signal frame, in a possible implementation, the difference signal can be used as the coding error to more accurately reflect the error. Encoding error in encoding method.

In some cases, the error reflected in the difference signal is the error of the signal itself, and the signal usually needs to be played to the user, and the error at the user's auditory perception level may be different from the error of the signal itself. Therefore, in another Among possible implementation methods, psychoacoustic perception analysis method can be used for the error signal, and the error size at the auditory perception level can be quantified through psychoacoustic perception. Based on this, when calculating the coding error, the difference energy can be obtained by calculating the auditory perception weighted energy of the difference signal, and the auditory perception weighted energy of the original high-frequency audio signal frame can be calculated to obtain the original energy. The difference energy and the original energy can be calculated The ratio of is taken as coding error. Among them, the difference energy and the original energy are the auditory perception weighted energy. If the difference energy is expressed as EP_err(i) and the original energy is expressed as EP_s(i), then the coding error calculation formula can be:

Among them, w(i) is the coding error, EP_err(i) is the difference energy, and EP_s(i) is the original energy. Compare w(i) with the error preset interval Thrd. When w(i)>Thrd, it means that the encoding error exceeds the error preset interval. On the contrary, it is within the error preset interval.

In this way, the coding error can be measured from the aspect of auditory perception, thereby ensuring the coding quality at the auditory perception level.

The main basis of auditory perception is "loudness". "Loudness" changes with the intensity of the audio signal, but is also affected by frequency. That is, audio signals of the same intensity and different frequencies have different auditory perceptions for the human ear. Figure 9 is an acoustic equal loudness curve measured by the International Acoustic Standards Organization provided by the embodiment of the present application. The acoustic equal loudness curve is a curve describing the relationship between sound pressure loudness and frequency under equal loudness conditions, and is one of the important auditory characteristics. That is, what sound pressure level intensity does the audio signal at different frequencies need to reach in order to obtain consistent hearing loudness for the user. In order to illustrate the meaning of this curve, let’s take an example below, such as any equal loudness curve in Figure 9. It can be seen that for medium and low frequencies (below 1kHz), the lower the frequency, the sound pressure intensity (i.e. energy) required for equal loudness ), simply put, more energy is needed to give users the same hearing experience. For mid-to-high frequencies (above 1kHz), audio in different frequency bands has different acoustic auditory perception characteristics. In this case, the calculation process of auditory perception weighted energy can be:

1) Framing and windowing:

For the input audio signal (such as the difference signal or the original high-frequency audio signal frame in the embodiment of the present application), an analysis window of 20 ms is usually used for one frame (consistent with the encoder frame definition). The window function can choose Hanning window or Hanning window. Bright windows.

2) Power spectrum calculation:

Perform Fourier transform on the audio signal obtained after windowing and framing, and find the energy p(i,j) of each frequency point in the i-th frame, j=0~K-1, where K is the total number of frequency points.

3) Calculate auditory perception weighted energy:

The energy of each frequency point k is multiplied by different auditory perception weighting coefficients and then accumulated to obtain the auditory perception weighted energy value of the audio signal of this frame. The calculation formula is as follows:

Among them, EP(i) is the auditory perception weighted energy of the i-th frame audio signal, i is the frame number, k is the frequency point number, and cof(k) is the auditory perception weighting coefficient of the k-th frequency point.

In this way, when the i-th frame audio signal is the current original high-frequency audio signal frame, the calculated EP(i) is expressed as the original energy EP_s(i); when the i-th frame audio signal is the corresponding difference signal, the calculated The obtained EP(i) is expressed as the difference energy EP_err(i).

For the auditory perception weighting coefficient, the embodiment of the present application uses psychoacoustic equal loudness curve data based on the BS3383 standard to calculate it. The calculation formula is as follows:
cof(freq)＝(10^loud/20)/1000 (3)

Among them, freq represents the frequency point, cof(freq) is equivalent to the auditory perception weighting coefficient of the kth frequency point, and loud represents the loudness value of the frequency point freq.

It should be noted that the loudness value loud of frequency point freq can be calculated by the following formula:
loud＝4.2+afy*(dB-cfy)/(1+bfy*(dB-cfy)) (4)
afy＝af(j-1)+(freq-ff(j-1))*(af(j)-af(j-1))/(ff(j)-ff(j-1)) (5)
bfy＝bf(j-1)+(freq-ff(j-1))*(bf(j)-bf(j-1))/(ff(j)-ff(j-1)) (6)
cfy＝cf(j-1)+(freq-ff(j-1))*(cf(j)-cf(j-1))/(ff(j)-ff(j-1)) (7)

Among them, ff, af, bf, cf correspond to the data in the equal loudness curve data table disclosed in the BS3383 standard, which can be obtained through the equal loudness curve data table query, j is the number in the equal loudness curve data table, and freq is required Calculate the frequency point of the loudness value loud. The loudness value loud is calculated by using the linear interpolation method to interpolate the data in the equal loudness curve data table.

It can be understood that the freq of the loudness value calculated through the above formula is usually the frequency point corresponding to the number between j-1 and j. The auditory perception weighting coefficient diagram calculated based on the above formula can be seen in Figure 10, which reflects the auditory perception weighting coefficient corresponding to different frequency points.

Embodiments of the present application also provide a method for decoding high-frequency audio signals. This method is introduced from the perspective of the receiving end. See Figure 11. The method includes:

S1101. Receive the high-frequency code stream sent by the transmitting end. The high-frequency code stream has a coding identifier, and the coding identifier is used to indicate the coding method used to encode the high-frequency code stream.

S1102. Analyze and obtain the encoding identifier corresponding to the high-frequency code stream, and determine the encoding method indicated by the encoding identifier.

S1103. Decode the high-frequency code stream according to the decoding method corresponding to the encoding method to obtain a high-frequency audio signal frame.

After the receiving end receives the high-frequency code stream, it can parse the high-frequency code stream and the corresponding encoding identifier, and then decode the high-frequency code stream according to the decoding method corresponding to the encoding method indicated by the encoding identifier to obtain the high-frequency audio signal frame. .

It can be seen from the above technical solution that this application proposes a high-frequency audio signal encoding and decoding method based on the mixing of multiple encoding methods based on coding error judgment for the original high-frequency audio signal. Specifically, for the original audio signal For each original audio signal frame in, obtain the original high-frequency audio signal frame decomposed from the original audio signal frame and multiple encoding methods. The encoding methods have corresponding priorities. The priority of the encoding method is used to indicate the use of this encoding. The priority of encoding method. Usually, in order to reduce the bandwidth of audio signal transmission as much as possible, the number of encoding bits of the encoding method increases in order from high to low priority. Then according to the priority of the coding method, the coding method with coding error within the error preset interval is determined from multiple coding methods as the target coding method. The coding error of the coding method is generated by encoding the original high-frequency audio signal frame using the coding method. , so that the coding error can be used as the criterion, the target coding method can be determined with the optimal number of coding bits as the goal, and the high-frequency code stream obtained by encoding the original high-frequency audio signal frame using the target coding method is sent to the receiving end. Therefore, when the encoding quality permits, the encoding method with a small number of encoding bits is selected, which reduces the bandwidth of audio signal transmission. Since the high-frequency code stream has an encoding identifier, the encoding identifier is used to indicate the encoding method used to encode the high-frequency code stream, so that the decoding end can determine which encoding method to use to decode the received high-frequency code stream based on the encoding identifier. It can be seen that this application can choose to use a coding method with a small number of coded bits when the coding quality permits, to achieve relatively satisfactory results in both the number of coded bits and coding quality, with a lower number of coded bits and high-quality audio. .

This application also provides a coding and decoding method for high-frequency audio signals, which is introduced from the perspective of the overall architecture of the transmitter and receiver. The embodiments of the present application use multiple coding methods including SSR method, SBR method and CELP coding method, which has the advantages of The priorities from high to low are SSR mode, SBR mode and CELP encoding mode as an example. The overall implementation architecture of encoding and decoding high-frequency audio signals can be seen in Figure 12.

Among them, the original high-frequency audio signal frame and the original low-frequency audio signal frame are input (see 1201 in Figure 12). The original high-frequency audio signal frame and the original low-frequency audio signal frame are the original audio signal frames after high and low frequency decomposition (for example, through QMF Filter bank decomposition), the original low-frequency audio signal frame can be used for subsequent high-frequency reconstruction.

In the high-frequency audio coding process, first try to encode through SSR to obtain the high-frequency code stream (see 1202 in Figure 12), and then perform high-frequency reconstruction (see 1203 in Figure 12), and reconstruct it based on the high frequency The high-frequency reconstructed signal frame and the original high-frequency audio signal frame are used to determine whether the coding error is within the error preset interval (see 1204 in Figure 12). If so, send the high-frequency code stream to the receiving end (see Figure 12 (shown as 1209 in Figure 12), if not, try to encode through SBR to obtain the high-frequency code stream (see 1205 in Figure 12), and then perform high-frequency reconstruction (see 1206 in Figure 12) to determine the encoding Whether the error is within the error preset interval (see 1207 in Figure 12), if so, perform the steps of sending the high-frequency code stream to the receiving end (see 1209 in Figure 12), if not, continue to try CELP encoding method (see 1208 in Figure 12), perform the step of sending the high-frequency code stream to the receiving end (see 1209 in Figure 12), where the high-frequency code stream has a corresponding encoding identifier. In the high-frequency audio decoding process, the high-frequency code stream and encoding identifier are obtained through analysis (see 1210 in Figure 12), and the high-frequency code stream is decoded using the decoding method corresponding to the encoding method indicated by the encoding identifier (see Figure 12) (shown as 1211 in Figure 12), after the above process, the high-frequency audio signal frame (see 1212 in Figure 12) is obtained by decoding.

It should be noted that, based on the implementation methods provided in the above aspects, this application can also be further combined to provide more implementation methods.

Based on the encoding method of high-frequency audio signals provided in the corresponding embodiment of Figure 2, embodiments of the present application also provide a high-frequency audio signal encoding device 1300. Referring to Figure 13, the high-frequency audio signal encoding device 1300 includes an acquisition unit 1301, a determination unit 1302 and a sending unit 1303:

The acquisition unit 1301 is used to acquire multiple encoding methods and acquire original high-frequency audio signal frames decomposed from original audio signal frames;

The obtaining unit 1301 is also used to obtain the priorities corresponding to the multiple coding methods. According to the order of the priorities from high to low, the number of coded bits of the coding method increases;

The determination unit 1302 is configured to determine, from the plurality of coding methods, a coding method with a coding error within a preset error interval as a target coding method according to the priority of the coding method. The coding error of the coding method is determined by using the coding method. The original high-frequency audio signal frame is generated by encoding;

The sending unit 1303 is configured to send a high-frequency code stream obtained by encoding the original high-frequency audio signal frame using the target encoding method to the receiving end. The high-frequency code stream has a coding identifier, and the coding The identifier is used to indicate the encoding method used to obtain the high-frequency code stream.

In a possible implementation, the determining unit 1302 is specifically used to:

Select the undetermined encoding method from the plurality of encoding methods in order from high to low priority;

Determine the coding error of the undetermined coding method;

If the coding error of the undetermined encoding method is within the error preset interval, the undetermined encoding method is determined as the target encoding method, and the continued selection of the undetermined encoding method is stopped.

In a possible implementation, the determining unit 1302 is specifically used to:

Determine the coding error of each coding method in the plurality of coding methods respectively;

From the coding methods with coding errors within the error preset interval, the coding method with the highest priority is determined as the target coding method.

In a possible implementation, for any of the multiple coding methods, the device further includes a reconstruction unit and an error analysis unit:

The acquisition unit 1301 is also used to acquire the original low-frequency audio signal frame decomposed from the original audio signal frame;

The reconstruction unit is configured to perform high-frequency reconstruction using any of the encoding methods according to the original low-frequency audio signal frame to obtain a high-frequency reconstructed signal frame;

The error analysis unit is configured to perform error analysis based on the high-frequency reconstructed signal frame and the original high-frequency audio signal frame to obtain the corresponding coding error.

In a possible implementation, the error analysis unit is specifically used to:

Calculate a difference signal between the high-frequency reconstructed signal frame and the original high-frequency audio signal frame;

The coding error is determined using the difference signal.

In a possible implementation, the error analysis unit is specifically used to:

Use the difference signal as the coding error;

or,

Perform auditory perception weighted energy calculation on the difference signal to obtain the difference energy, and perform auditory perception weighted energy calculation on the original high-frequency audio signal frame to obtain the original energy;

The ratio of the difference energy to the original energy is used as the coding error.

In a possible implementation, if any of the encoding methods is an audio super-resolution method, the reconstruction unit is specifically used to:

Obtain the neural network model corresponding to the audio super-resolution method;

Perform feature extraction on the original low-frequency audio signal frame to obtain low-frequency features;

According to the low-frequency characteristics, the high-frequency reconstructed signal frame is obtained by predicting through the neural network model.

In a possible implementation, if any of the coding methods is a frequency band replication method, the reconstruction unit is specifically used to:

Copy the original low-frequency audio signal frame to the high-frequency band to obtain a high-frequency copied signal frame;

Extract envelope features of the original high-frequency audio signal frame;

The high-frequency replica signal frame is corrected using the envelope feature to obtain the high-frequency reconstructed signal frame.

In a possible implementation, if any of the coding methods is a code-excited linear prediction coding method, the reconstruction unit is specifically used to:

Obtain coding parameters from the high-frequency code stream, and obtain the pitch period of the original low-frequency audio signal frame;

High-frequency reconstruction is performed according to the coding parameters and the pitch period to obtain the high-frequency reconstructed signal frame.

Based on the decoding method of high-frequency audio signals provided in the corresponding embodiment of Figure 11, embodiments of the present application also provide a device 1400 for decoding high-frequency audio signals. Referring to Figure 14, the high-frequency audio signal decoding device 1400 includes a receiving unit 1401, parsing unit 1402 and decoding unit 1403:

The receiving unit 1401 is used to receive the high-frequency code stream sent by the transmitting end. The high-frequency code stream has a coding identifier, and the coding identifier is used to indicate the coding method used to encode the high-frequency code stream;

The analysis unit 1402 is used to analyze and obtain the encoding identifier corresponding to the high-frequency code stream, and determine the encoding method indicated by the encoding identifier;

The decoding unit 1403 is configured to decode the high-frequency code stream according to the decoding method corresponding to the encoding method to obtain a high-frequency audio signal frame.

An embodiment of the present application also provides a computer device that can execute a coding and decoding method for high-frequency audio signals. The computer device may be, for example, a terminal. Taking the terminal as a smartphone as an example:

FIG. 15 shows a block diagram of a partial structure of a smart phone provided by an embodiment of the present application. Referring to Figure 15, the smartphone includes: radio frequency (English full name: Radio Frequency, English abbreviation: RF) circuit 1510, memory 1520, input unit 1530, display unit 1540, sensor 1550, audio circuit 1560, wireless fidelity (English abbreviation: WiFi ) module 1570, processor 1580, and power supply 1590 and other components. The input unit 1530 may include a touch panel 1531 and other input devices 1532, the display unit 1540 may include a display panel 1541, and the audio circuit 1560 may include a speaker 1561 and a microphone 1562. It can be understood that the structure of the smart phone shown in Figure 15 does not constitute a limitation to the smart phone, and may include more or less components than shown in the figure, or combine certain components, or arrange different components.

The memory 1520 can be used to store software programs and modules. The processor 1580 executes various functional applications and data processing of the smart phone by running the software programs and modules stored in the memory 1520 . The memory 1520 may mainly include a storage program area and a storage data area, where the storage program area may store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the storage data area may store data based on Data created by the use of smartphones (such as audio data, phone books, etc.), etc. In addition, memory 1520 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.

The processor 1580 is the control center of the smartphone, using various interfaces and lines to connect various parts of the entire smartphone, by running or executing software programs and/or modules stored in the memory 1520, and calling data stored in the memory 1520 , perform various functions of the smartphone and process data. Optionally, the processor 1580 may include one or more processing units; preferably, the processor 1580 may integrate an application processor and a modem processor, where the application processor mainly processes operating systems, user interfaces, application programs, etc. , the modem processor mainly handles wireless communications. It can be understood that the above modem processor may not be integrated into the processor 1580.

In this embodiment, the processor 1580 in the smartphone can perform the following steps:

Obtain multiple encoding methods, and obtain the original high-frequency audio signal frame decomposed from the original audio signal frame;

Obtain the priorities corresponding to the multiple encoding methods, and in the order of the priorities from high to low, the number of encoding bits of the encoding method increases;

According to the priority of the coding method, the coding method whose coding error is within the error preset interval is determined from the multiple coding methods as the target coding method. The coding error of the coding method is the encoding method of the original high-frequency audio signal frame. Generated by coding;

Send the high-frequency code stream obtained by encoding the original high-frequency audio signal frame using the target encoding method to At the receiving end, the high-frequency code stream has an encoding identifier, and the encoding identifier is used to indicate the encoding method used to obtain the high-frequency code stream.

Alternatively, processor 1580 may perform the following steps:

Receive the high-frequency code stream sent by the transmitting end, the high-frequency code stream has a coding identifier, and the coding identifier is used to indicate the coding method used to encode the high-frequency code stream;

Analyze to obtain the encoding identifier corresponding to the high-frequency code stream, and determine the encoding method indicated by the encoding identifier;

The high-frequency code stream is decoded according to the decoding method corresponding to the encoding method to obtain a high-frequency audio signal frame.

The embodiment of the present application also provides a server, as shown in Figure 16. Figure 16 is a structural diagram of the server 1600 provided by the embodiment of the present application. The server 1600 may vary greatly due to different configurations or performance, and may include a Or more than one central processing unit (Central Processing Units, CPU for short) 1622 (for example, one or more processors) and memory 1632, one or more storage media 1630 (for example, one or more storage media 1630 for storing application programs 1642 or data 1644 mass storage device). Among them, the memory 1632 and the storage medium 1630 may be short-term storage or persistent storage. The program stored in the storage medium 1630 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processor 1622 may be configured to communicate with the storage medium 1630 and execute a series of instruction operations in the storage medium 1630 on the server 1600 .

Server 1600 may also include one or more power supplies 1626, one or more wired or wireless network interfaces 1650, one or more input and output interfaces 1658, and/or, one or more operating systems 1641, such as Windows Server ^™ , Mac OS X ^TM , Unix ^TM , Linux ^TM , FreeBSD ^TM and so on.

In this embodiment, the steps performed by the central processor 1622 in the server 1600 can be implemented based on the structure shown in FIG. 16 .

According to one aspect of the present application, a computer-readable storage medium is provided. The computer-readable storage medium is used to store a computer program. The computer program is used to perform the encoding of high-frequency audio signals described in the foregoing embodiments. Decoding method.

According to one aspect of the present application, a computer program product or computer program is provided, which computer program product or computer program includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the methods provided in various optional implementations of the above embodiments.

The descriptions of the processes or structures corresponding to each of the above drawings have different emphasis. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

The terms "first", "second", "third", "fourth", etc. (if present) in the description of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe specific objects. Sequence or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances so that the embodiments of the application described herein can, for example, be practiced in sequences other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, e.g., a process, method, system, product, or apparatus that encompasses a series of steps or units and need not be limited to those explicitly listed. Those steps or elements may instead include other steps or elements not expressly listed or inherent to the process, method, product or apparatus.

In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be achieved through other means. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the coupling or direct coupling or communication connection between each other shown or discussed may be through some interfaces, and the indirect coupling or communication connection of the devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or software functional units.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to cause a computer device (which may be a computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc., which can store program code. medium.

As mentioned above, the above embodiments are only used to illustrate the technical solution of the present application, but not to limit it. Although the present application has been described in detail with reference to the foregoing embodiments, members of ordinary skill in the art should understand that they can still modify the foregoing. The technical solutions described in each embodiment may be modified, or some of the technical features may be equivalently replaced; however, these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions in each embodiment of the present application.

Claims (15)

A method for encoding high-frequency audio signals, the method is executed by a computer device, the method includes: Obtain multiple encoding methods, and obtain the original high-frequency audio signal frame decomposed from the original audio signal frame; Obtain the priorities corresponding to the multiple encoding methods, and in the order of the priorities from high to low, the number of encoding bits of the encoding method increases; According to the priority of the coding method, the coding method whose coding error is within the error preset interval is determined from the multiple coding methods as the target coding method. The coding error of the coding method is the encoding method of the original high-frequency audio signal frame. Generated by coding; The high-frequency code stream obtained by encoding the original high-frequency audio signal frame using the target encoding method is sent to the receiving end. The high-frequency code stream has an encoding identifier, and the encoding identifier is used to indicate that the encoding obtained by The encoding method used for high-frequency code streams. The method according to claim 1, wherein the coding method in which the coding error is within a preset error interval is determined from the plurality of coding methods as the target coding method according to the priority of the coding method, including: Select the undetermined encoding method from the plurality of encoding methods in order from high to low priority; Determine the coding error of the undetermined coding method; If the coding error of the undetermined encoding method is within the error preset interval, the undetermined encoding method is determined as the target encoding method, and the continued selection of the undetermined encoding method is stopped. The method according to claim 1, wherein the coding method in which the coding error is within a preset error interval is determined from the plurality of coding methods as the target coding method according to the priority of the coding method, including: Determine the coding error of each coding method in the plurality of coding methods respectively; From the coding methods with coding errors within the error preset interval, the coding method with the highest priority is determined as the target coding method. The method according to claim 1, for any one of the plurality of coding methods, the method for determining the coding error of any one of the coding methods includes: Obtain an original low-frequency audio signal frame decomposed from the original audio signal frame; According to the original low-frequency audio signal frame, use any of the encoding methods to perform high-frequency reconstruction to obtain a high-frequency reconstructed signal frame; Error analysis is performed based on the high-frequency reconstructed signal frame and the original high-frequency audio signal frame to obtain the corresponding coding error. The method according to claim 4, wherein performing error analysis based on the high-frequency reconstructed signal frame and the original high-frequency audio signal frame to obtain the encoding error includes: Calculate a difference signal between the high-frequency reconstructed signal frame and the original high-frequency audio signal frame; The coding error is determined using the difference signal. The method according to claim 5, using the difference value to determine the encoding error includes: Use the difference signal as the coding error; or, Perform auditory perception weighted energy calculation on the difference signal to obtain the difference energy, and calculate the original high-frequency audio The signal frame is subjected to auditory perception weighted energy calculation to obtain the original energy; The ratio of the difference energy to the original energy is used as the encoding error. The method according to any one of claims 4 to 6, if any coding method is an audio super-resolution method, using any coding method to perform high-frequency reconstruction based on the original low-frequency audio signal frame , obtain the high-frequency reconstructed signal frame, including: Obtain the neural network model corresponding to the audio super-resolution method; Perform feature extraction on the original low-frequency audio signal frame to obtain low-frequency features; According to the low-frequency characteristics, the high-frequency reconstructed signal frame is obtained by predicting through the neural network model. The method according to any one of claims 4 to 6, if any of the coding methods is a frequency band replication method, and using any of the coding methods to perform high-frequency reconstruction based on the original low-frequency audio signal frame, we obtain High-frequency reconstructed signal frames, including: Copy the original low-frequency audio signal frame to the high-frequency band to obtain a high-frequency copied signal frame; Extract envelope features of the original high-frequency audio signal frame; The high-frequency replica signal frame is corrected using the envelope feature to obtain the high-frequency reconstructed signal frame. The method according to any one of claims 4 to 6, if any of the coding methods is a code-excited linear prediction coding method, and based on the original low-frequency audio signal frame, using any of the coding methods to perform high-frequency Reconstruct to obtain high-frequency reconstructed signal frames, including: Obtain coding parameters from the high-frequency code stream, and obtain the pitch period of the original low-frequency audio signal frame; High-frequency reconstruction is performed according to the coding parameters and the pitch period to obtain the high-frequency reconstructed signal frame. A method for decoding high-frequency audio signals, the method is executed by a computer device, the method includes: Receive the high-frequency code stream sent by the transmitting end, the high-frequency code stream has a coding identifier, and the coding identifier is used to indicate the coding method used to encode the high-frequency code stream; Analyze to obtain the encoding identifier corresponding to the high-frequency code stream, and determine the encoding method indicated by the encoding identifier; The high-frequency code stream is decoded according to the decoding method corresponding to the encoding method to obtain a high-frequency audio signal frame. A high-frequency audio signal encoding device, the device includes an acquisition unit, a determination unit and a sending unit: The acquisition unit is used to acquire multiple encoding methods and acquire original high-frequency audio signal frames decomposed from original audio signal frames; The acquisition unit is also used to acquire the priorities corresponding to the multiple encoding methods. According to the order of the priorities from high to low, the number of encoding bits of the encoding methods increases; The determination unit is configured to determine, from the plurality of coding methods, a coding method with a coding error within a preset error interval as a target coding method according to the priority of the coding method. The coding error of the coding method is determined by using the coding method. It is generated by encoding the original high-frequency audio signal frame; The sending unit is configured to send a high-frequency code stream obtained by encoding the original high-frequency audio signal frame using the target encoding method to the receiving end. The high-frequency code stream has a coding identifier, and the coding identifier Used to indicate the encoding method used to obtain the high-frequency code stream. A decoding device for high-frequency audio signals, the device includes a receiving unit, an analysis unit and a decoding unit: The receiving unit is used to receive the high-frequency code stream sent by the transmitting end. The high-frequency code stream has a coding identifier. The encoding identifier is used to indicate the encoding method used to obtain the high-frequency code stream; The analysis unit is used to analyze and obtain the encoding identifier corresponding to the high-frequency code stream, and determine the encoding method indicated by the encoding identifier; The decoding unit is configured to decode the high-frequency code stream according to the decoding method corresponding to the encoding method to obtain a high-frequency audio signal frame. A computer device including a processor and a memory: The memory is used to store a computer program and transmit the computer program to the processor; The processor is configured to execute the method according to any one of claims 1-10 according to the computer program. A computer-readable storage medium used to store a computer program that, when executed by a processor, causes the processor to perform the method described in any one of claims 1-10. A computer program product includes a computer program that implements the method described in any one of claims 1-10 when executed by a processor.