WO2023202522A1 - Playing speed control method and electronic device - Google Patents

Abstract

The present application belongs to the technical field of audios. Disclosed are a playing speed control method and an electronic device. The specific solution comprises: acquiring target streaming media; according to a preset frame length, performing segmentation processing on a voice frame of the target streaming media which has been subjected to short-time framing processing, so as to obtain a plurality of long-time frames; respectively determining a long-time spectrum energy difference characteristic value of each long-time frame, and determining a first streaming media clip and a second streaming media clip according to the long-time spectrum energy difference characteristic value; and outputting the first streaming media clip in the target streaming media according to a first speed, and outputting the second streaming media clip in the target streaming media according to a second speed, wherein the first streaming media clip is streaming media data that contains voice information, the second streaming media clip is streaming media data that does not contain voice information, and the first speed is less than the second speed.

Description

Playback speed control method and electronic device

Cross-references to related applications

This application claims priority to the Chinese patent application with application number 202210425500.4 and the invention title "Playback Speed Control Method and Electronic Device" submitted on April 21, 2022, which is fully incorporated into this application by reference.

Technical field

This application belongs to the field of audio technology, and specifically relates to a playback speed control method and electronic equipment.

Background technique

With the development of short video platforms, video content has become more and more colorful. For short video platforms, high-quality video content is the foundation. However, the videos shot by many short video creators often have problems with delayed video content and lengthy playback rhythm. In this way, users of the short video platform may easily become bored because they find it boring. Switch videos or close platforms.

In related technologies, when users of short video platforms feel that the currently playing video clip is boring, they can speed up the playback progress of the video by dragging the playback progress bar in the video interface. However, when the user uses this method, It is difficult to grasp the dragging scale. It is usually necessary to drag the progress bar repeatedly to locate the video position you want to view. Not only does it easily make users miss key content, but it also makes the user's operation more cumbersome.

Contents of the invention

The purpose of the embodiments of the present application is to provide a playback speed control method and electronic device that can solve the problem in related technologies that the method of controlling playback speed not only easily causes users to miss key content, but also makes the user's operations more complicated.

In a first aspect, embodiments of the present application provide a playback speed control method, which method includes: Obtain the target streaming media; divide the voice frames of the target streaming media that have been processed by short-time frame division according to the preset frame length to obtain multiple long-term frames; determine the long-term spectrum energy of each of the long-term frames respectively. difference characteristic value, and determine the first streaming media segment and the second streaming media segment according to the long-term spectrum energy difference characteristic value; output the first streaming media segment in the target streaming media at the first speed, and output the first streaming media segment at the second speed. The second streaming media segment in the target streaming media; wherein, the first streaming media segment is streaming media data that contains voice information, the second streaming media segment is streaming media data that does not contain voice information, and the first streaming media segment is streaming media data that does not contain voice information. The speed is less than the second speed.

In the second aspect, embodiments of the present application provide a playback speed control device, including: an acquisition module and an output module; the acquisition module is used to acquire the target streaming media; the processing module is used to process the target streaming media according to the preset frame length. The voice frames of the target streaming media that have been processed by short-time frame division are segmented to obtain multiple long-term frames; the processing module is also used to determine the long-term spectral energy difference characteristic value of each long-term frame. , and determine the first streaming media segment and the second streaming media segment according to the long-term spectrum energy difference characteristic value; the output module is used to output the first streaming media segment in the target streaming media according to the first speed, according to the first speed. Output the second streaming media segment in the target streaming media at two speeds; wherein the first streaming media segment is streaming media data containing voice information, and the second streaming media segment is streaming media data that does not contain voice information, The first speed is less than the second speed.

In a third aspect, embodiments of the present application provide an electronic device. The electronic device includes a processor and a memory. The memory stores programs or instructions that can be run on the processor. The programs or instructions are processed by the processor. When the processor is executed, the steps of the method described in the first aspect are implemented.

In a fourth aspect, embodiments of the present application provide a readable storage medium. Programs or instructions are stored on the readable storage medium. When the programs or instructions are executed by a processor, the steps of the method described in the first aspect are implemented. .

In a fifth aspect, embodiments of the present application provide a chip. The chip includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the first aspect. the method described.

In a sixth aspect, embodiments of the present application provide a computer program product. The program product is stored in a storage medium. The program product is executed by at least one processor to implement the method described in the first aspect. method.

In a seventh aspect, embodiments of the present application provide an electronic device configured to implement the steps of the method in the first aspect.

In the embodiment of the present application, the target streaming media can be obtained; the voice frames of the target streaming media that have been processed by short-time frame division are divided according to the preset frame length to obtain multiple long-term frames; each of the said target streaming media is determined respectively. The long-term spectrum energy difference characteristic value of the long-term frame, and determine the first streaming media segment and the second streaming media segment according to the long-term spectrum energy difference characteristic value; output the first streaming media in the target streaming media at the first speed segment, outputting a second streaming media segment in the target streaming media at a second speed; wherein the first streaming media segment is streaming media data containing voice information, and the second streaming media segment is streaming media data that does not contain voice information. For streaming media data, the first speed is smaller than the second speed. Through this solution, the speech frame of the target streaming media that has been processed by short-time frame division can be divided into multiple long-term frames, and the third frame in the target streaming media can be determined by analyzing the long-term spectrum energy difference characteristic value of the long-term frame. The first-class media segment and the second streaming media segment, and the first streaming media segment is output at a first speed, and the second streaming media segment is output at a second speed. Therefore, on the one hand, since the long-term features have higher smoothness than the short-term features properties and stability, so the accuracy of the analysis results can be improved by analyzing the long-term spectrum energy difference feature values of the long-term frames; on the other hand, since the first speed is smaller than the second speed, that is, the playback speed of the second streaming media clip is greater than The playback speed of the first streaming media segment, therefore, can not only reduce the time wasted during the playback process of the streaming media segment that does not contain voice information, and avoid the user missing the key content in the second streaming media segment, but also during the output process of the target streaming media. , since the user does not need to make any input, the complexity of user operations is reduced.

Description of the drawings

Figure 1 is a schematic flowchart of a playback speed control method provided by an embodiment of the present application;

Figure 2 is one of the interface schematic diagrams of the playback speed control method provided by the embodiment of the present application;

Figure 3 is the second schematic interface diagram of the playback speed control method provided by the embodiment of the present application;

Figure 4 is the third schematic interface diagram of the playback speed control method provided by the embodiment of the present application;

Figure 5 is a schematic structural diagram of a playback speed control device provided by an embodiment of the present application;

Figure 6 is a schematic structural diagram of an electronic device provided by an embodiment of the present application;

Figure 7 is a hardware schematic diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art fall within the scope of protection of this application.

The terms "first", "second", etc. in the description and claims of this application are used to distinguish similar objects and are not used to describe a specific order or sequence. It is to be understood that the figures so used are interchangeable under appropriate circumstances so that the embodiments of the present application can be practiced in orders other than those illustrated or described herein, and that "first," "second," etc. are distinguished Objects are usually of one type, and the number of objects is not limited. For example, the first object can be one or multiple. In addition, "and/or" in the description and claims indicates at least one of the connected objects, and the character "/" generally indicates that the related objects are in an "or" relationship.

The playback speed control method provided by the embodiments of the present application will be described in detail below with reference to the accompanying drawings through specific embodiments and application scenarios.

In the playback speed control method provided by the embodiments of the present application, the execution subject of the playback speed control method may be an electronic device or a functional module or functional entity in the electronic device that can implement the playback speed control method. The electronic device mentioned in the embodiments of the present application Including but not limited to mobile phones, tablets, computers, cameras, wearable devices, etc., the playback speed control method provided by the embodiment of the present application will be described below by taking an electronic device as an execution subject as an example.

As shown in Figure 1, this embodiment of the present application provides a playback speed control method, which may include steps 101 to 104:

Step 101: Obtain the target streaming media.

Optionally, the above-mentioned target streaming media is streaming media including audio data, for example, it can be voice messages, recordings, music, audio novels, etc.; it can also be audio video.

For example, when the target streaming media is a voice message, the electronic device may obtain the target streaming media by: while displaying a chat interface with other contacts, the user may click on the target voice message received by the electronic device. Input, the electronic device can obtain the target voice message in response to the click input. When the target streaming media is a video with sound, the electronic device may obtain the target streaming media by: if the user wants to watch the target video, the user may perform a click input on the target video, and the electronic device may respond to the click input by displaying the target video. Video playback interface and obtain audio data in the target video.

Optionally, before acquiring the target streaming media, the electronic device may receive a first input from the user while displaying a playback speed setting interface; and determine the first speed and the second speed in response to the first input. The first speed is less than the second speed.

Specifically, a piece of audio may include a first streaming media segment and a second streaming media segment, where the first streaming media segment is streaming media data containing voice information, and the second streaming media segment is streaming media data not containing voice information, for example , the second streaming media segment can be a noise segment or a blank sound segment. During the audio playback process, what the user wants to obtain is the voice information in the first streaming media segment, and the second streaming media segment can be skipped. Therefore, the user can separately set the playback speed of the first streaming media segment, that is, the second streaming media segment can be skipped. One speed, and the playback speed of the second streaming segment, which is the second speed.

For example, as shown in Figure 2, if the user wants to adjust the audio playback speed of the electronic device, the electronic device can first be triggered to display a playback speed setting interface. The playback speed setting interface can include two playback speed setting options, respectively. "First streaming media segment playback speed adjustment" and "Second streaming media segment playback speed adjustment", where "first streaming media segment playback speed adjustment" corresponds to switch 111, and "second streaming media segment playback speed adjustment" corresponds to switch 112. If the user wants to adjust the playback speed of the first streaming media segment, he or she can perform a click input on the switch 111. In response to the click input, the electronic device can control the switch 111 to be on, cancel the display of the playback speed setting interface, and display The first streaming media clip playback speed adjustment interface. If the user wants to adjust the playback speed of the second streaming media clip, he or she can perform a click input on the switch 112, and the electronic device can control the switch 112 in response to the click input. Turn on the state, cancel the display of the playback speed setting interface, and display the second streaming media segment playback speed adjustment interface.

As shown in Figure 3, it is a first streaming media segment playback speed adjustment interface. The first streaming media segment playback speed adjustment interface includes a pre-stored first video playback interface 121, an acceleration adjustment control 122, a deceleration adjustment control 123 and a determination control 124. , when the first video is in the playing state, the user can adjust the playback speed of the first video by clicking the acceleration adjustment control 122 or the deceleration adjustment control 123. When the speaking speed is adjusted to a speaking speed that is comfortable for the ears, the user can confirm The control 124 performs a click input, and the electronic device can determine the adjusted final playback speed as the first speed in response to the click input.

As shown in Figure 4, it is a second streaming media clip playback speed adjustment interface. The second streaming media clip playback speed adjustment interface includes multiple speed adjustment controls and confirmation controls 131. Users can adjust multiple speeds according to their own needs for playback speed. Any control in the speed adjustment control performs click input, and the electronic device can respond to the click input and highlight the control that the user clicks to input. After that, the user can click input on the confirmation control 131, and the electronic device can respond to the click input. The second speed is determined based on the multiplier value corresponding to the highlighted control.

It should be noted that the above-mentioned first input may include multiple sub-inputs. For example, it may include an input that triggers the electronic device to display the first streaming media segment playback speed adjustment interface or the second streaming media segment playback speed adjustment interface. It may also include input to the first stream media segment playback speed adjustment interface. Touch input of controls in the media clip playback speed adjustment interface or the second streaming media clip playback speed adjustment interface.

Based on the above solution, since the first speed and the second speed can be determined according to the first input, the user can customize the two playback speeds according to their own needs, thereby meeting the diverse speed playback needs of different users.

Step 102: Divide the voice frames of the target streaming media that have undergone short-time frame division processing according to the preset frame length to obtain multiple long-duration frames.

Optionally, the electronic device can detect whether there is a human voice signal in the target streaming media through a voice activity detection (VAD) algorithm. However, the VAD algorithm in the related art may determine part of the noise in the non-voice segment. into a speech signal, and based on long-term features It has the characteristics of higher smoothness and stability than short-term features. Therefore, the electronic device can re-segment the speech frames of the target streaming media that have been processed by short-term frame segmentation through a long-term window, and analyze the re-segmented speech frames. Voice characteristics.

For example, assuming that the target streaming media includes 100 short-term frames and the length of the long-term window is 10 frames, after re-dividing these 100 short-term frames, we can obtain long-term frame 1 - long-term frame 91, where , long-term frame 1 includes short-term frame 1 to short-term frame 10, long-term frame 2 includes short-term frame 2 to short-term frame 11, and so on.

Step 103: Determine the long-term spectrum energy difference characteristic value of each long-term frame respectively, and determine the first streaming media segment and the second streaming media segment based on the long-term spectrum energy difference characteristic value.

Optionally, after re-segmenting the speech frame, the electronic device can separately determine the long-term spectral energy difference characteristic value of each long-term frame in the target streaming media.

Specifically, the electronic device can determine the long-term spectrum energy difference characteristic value of the l-th long-term frame Among them, the N-th order long-term spectrum envelope of the lth frame X(n) represents a speech segment containing noise, X(k,l+j) represents the amplitude spectrum of the l-th frame of speech at frequency k, N(k) represents the amplitude spectrum of background noise at amplitude k; NFFT represents The number of sampling points in the fast Fourier transform FFT.

According to the above formula, it can be seen that when calculating the N-order long-time spectrum envelope, electronic equipment uses the long-time principle to add a long-time window of length 2N+1 frames to the short-time amplitude spectrum for analysis. Due to the long-term The time window can expand the difference between LTE and noise amplitude spectra, so it can more accurately detect speech and noise in the target streaming media.

Optionally, the electronic device determines the first streaming media segment and the second streaming media segment based on the long-term spectrum energy difference characteristic value, which may specifically include: when the long-term spectrum energy difference characteristic value of the target long-term frame is less than the first threshold. , determine the target long-term frame as the first streaming media segment; when the long-term spectral energy difference characteristic value of the target long-term frame is greater than the first threshold, determine the target long-term frame is the second streaming media segment; wherein, the target long-term frame is For any one of the plurality of long-term frames, the first threshold is related to the noise estimate value and the signal-to-noise ratio.

Optionally, the above first threshold in, E _h (k) represents the noise estimate obtained at the current moment, that is, the latest noise estimate, and SNR represents the signal-to-noise ratio.

Based on the above solution, since it can be determined whether the target long-term frame is the first streaming media segment or the first streaming media segment based on the long-term spectrum energy difference characteristic value of the target long-term frame, the target streaming media can be determined based on the long-term spectrum energy difference characteristic value. The first streaming media segment and the second streaming media segment in , thereby providing a basis for different streaming media segments to be played at different speeds.

Optionally, the electronic device determines the first streaming media segment and the second streaming media segment based on the long-term spectrum energy difference characteristic value, which may specifically include: when the long-term spectrum energy difference characteristic value of the target long-term frame is less than the first threshold. , determine the number of frames containing pitch in the target long-term frame; when the ratio of the number of frames containing pitch to the total number of frames in the target long-term frame is greater than the second threshold, the target long-term frame is The frame is determined as the first streaming media segment; when the ratio of the number of frames containing pitch to the total number of frames of the target long-term frame is less than the second threshold, the target long-term frame is determined as The second streaming media segment; wherein the target long-term frame is any one of the multiple long-term frames.

Specifically, for sudden noises such as typing on the keyboard and bumping the microphone, the LTD characteristics of the noise are very similar to the LTD characteristics of speech, so misjudgments often occur. Therefore, gene proportion features can be introduced to assist in the discrimination. The pitch frequency refers to the vocal cord vibration frequency in speech. By detecting the pitch of the target long-term frame, the electronic device can determine the number of frames containing the _pitch M pitch. Then, the electronic device can further determine M _pitch and the total number of the target long-term frame. The ratio θ _pitch of the frame number M, in When θ v is greater than the second threshold value θ _v , the electronic device may determine the target long-term frame as the first streaming media segment; when θ _pitch is less than the second threshold value θ _v , the electronic device may determine the target long-term frame as the second streaming media segment. Streaming clips.

Based on the above solution, it can be determined whether the target long frame is the first streaming media segment or the second streaming media segment based on the ratio of the number of frames containing pitch in the target long frame to the total number of frames in the target long frame. Therefore, on the one hand, the first streaming media segment and the second streaming media segment in the target streaming media can be determined based on the frame number of the pitch, thereby providing a basis for different streaming media segments to be played at different speeds; on the other hand, the judgment can be improved accuracy of results.

Optionally, the electronic device determines the first streaming media segment and the second streaming media segment based on the long-term spectrum energy difference characteristic value, which may specifically include: when the long-term spectrum energy difference characteristic value of the target long-term frame is less than the first threshold. , convert the time domain speech signal of the target long time frame into a frequency domain energy signal to obtain the target streaming media segment; determine the average energy value and the maximum energy value of multiple frequency domain energy sampling points in the target streaming media segment; When the difference between the maximum energy value and the average energy value is within a first preset range, determine the target streaming media segment as the first streaming media segment; when the maximum energy value and the If the difference in average energy values is not within the first preset range, the target streaming media segment is determined to be the second streaming media segment; wherein the target long-term frame is the plurality of long-term frames. any time frame.

There are many types of noise, and the two most common types of noise are floor noise and background noise. The signal energy of the background noise is concentrated in the low frequency part (0~1000Hz), and there is almost no distribution in the middle and high frequency parts, while the signal energy of the background noise is evenly distributed in the entire frequency domain, and each frequency point has an energy distribution, and the energy The distribution is relatively even. The spectral characteristics of speech signals and these two types of noise are different. There is often signal energy distribution within the entire spectrum, but the distribution of energy size is irregular for different frequency points. For low noise, since the signal energy of the noise floor is basically concentrated in the low frequency band, for the same sampling point, frequency points are evenly selected across the entire frequency band to calculate the average energy and maximum energy, and the gap between the two is usually large. For background noise, since the distribution of background noise in the entire frequency band is relatively even, there is little difference in signal energy at different frequency points. Frequency points are evenly selected across the entire frequency band to calculate the average energy and maximum energy. The difference between the two is usually very small. For speech signals, since the energy of the speech signal does not follow the above rules, the difference between the average energy and the maximum energy is usually within a difference range, and this difference range is usually larger than the difference between the background noise and smaller than the background noise signal. difference. Therefore, using the noise floor, background noise, and the spectral energy characteristics of the speech signal, it is possible to determine whether a streaming media segment is speech or noise.

Specifically, electronic equipment can use the Audition tool to reduce the long-term spectrum energy difference characteristic value to a small The time-domain speech signal in the target long-time frame with the first threshold is converted into a frequency-domain energy signal to obtain the target streaming media segment. After conversion, the electronic device can display the energy size of different frequency points, where the darker the color, the greater the energy of the frequency point. The larger and darker the color, the smaller the energy of the frequency point. After that, the frequency domain energy sampling point can be selected from the target streaming media segment with a preset time length and a preset frequency domain range. For example, the preset time length is t ₀ and the preset frequency domain range is (0, f ₀ ), 40 frequency domain energy sampling points are selected from the target streaming media clip, that is, 0.1t ₀ is used as the starting time point in the time domain, and 5 points are evenly selected: t=(0.1t ₀ ,0.3t ₀ ,0.5t ₀ ,0.7t ₀ ,0.9t ₀ ), in the frequency domain as As the starting frequency, select 8 points evenly: Then, calculate the average energy value of the 8 frequency points corresponding to each time domain sampling point t and maximum energy value

After obtaining the average energy value of 5 time domain sampling points, and maximum energy value Finally, for these five average energy values and 5 maximum energy values Average the average energy value E ^mean and the maximum energy value E ^max of all frequency domain energy sampling points:

Afterwards, the electronic device can calculate the difference between the average energy value E ^mean and the maximum energy value E ^max , and determine whether the difference is within the first preset range (a, b):

If E ^max -E ^mean ≥ b, the electronic device can determine the target streaming media segment as the noise floor signal, that is, when the difference between the maximum energy value and the average energy value is not within the first preset range, the electronic device can Determine the target streaming media segment as the second streaming media segment;

If E ^max -E ^mean ≤ a, the electronic device can determine the target streaming media segment as the background noise signal No., that is, when the difference between the maximum energy value and the average energy value is not within the first preset range, the electronic device can determine the target streaming media segment as the second streaming media segment;

If a<E ^max -E ^mean <b, the electronic device can determine the target streaming media segment as the speech signal, that is, when the difference between the maximum energy value and the average energy value is within the first preset range, the electronic device can The target streaming media segment is determined as the first streaming media segment.

Based on the above solution, since the time domain speech signal of the target long time frame can be converted into a frequency domain energy signal when the long time spectrum energy difference characteristic value of the target long time frame is less than the first threshold, the target streaming media segment can be obtained. and determine the average energy value and the maximum energy value of multiple frequency domain energy sampling points in the target streaming media segment, and determine whether the target streaming media segment is the first streaming media segment or the first streaming media segment based on the difference between the maximum energy value and the average energy value, Therefore, on the one hand, the first streaming media segment and the second streaming media segment in the target streaming media can be determined based on the frequency domain energy, thereby providing a basis for different streaming media segments to be played at different speeds; on the other hand, the accuracy of the judgment results can be improved sex.

Step 104: Output the first streaming media segment in the target streaming media at the first speed, and output the second streaming media segment in the target streaming media at the second speed.

Specifically, the electronic device may output a first streaming media segment of the target streaming media at a first speed, and output a second streaming media segment of the target streaming media at a second speed. That is to say, during the process of playing the target streaming media, the electronic device can switch the playback speed of different streaming media segments.

In the embodiment of the present application, the speech frame of the target streaming media that has been processed by short-time frame division can be divided into multiple long-term frames, and the target streaming media can be determined by analyzing the long-term spectrum energy difference characteristic value of the long-term frame. The first streaming media segment and the second streaming media segment in, and the first streaming media segment is output at the first speed, and the second streaming media segment is output at the second speed. Therefore, on the one hand, since the long-term features have more advantages than the short-term features High smoothness and stability, so the accuracy of the analysis results can be improved by analyzing the long-term spectrum energy difference feature values of long-term frames; on the other hand, since the first speed is smaller than the second speed, that is, the second streaming media clip The playback speed is greater than the playback speed of the first streaming media segment. Therefore, it can not only reduce the time wasted during the playback process of streaming media segments that do not contain voice information, but also prevent the user from missing key content in the second streaming media segment. And during the output process of the target streaming media, since the user does not need to make any input, the complexity of user operations is reduced.

For the playback speed control method provided by the embodiment of the present application, the execution subject may be a playback speed control device. In the embodiment of the present application, the playback speed control device performing the playback speed control method is taken as an example to illustrate the playback speed control device provided by the embodiment of the present application.

As shown in Figure 5, the embodiment of the present application also provides a playback speed control device 500, which includes: an acquisition module 501, a processing module 502 and an output module 503; the acquisition module 501 is used to acquire target streaming media; the processing Module 502 is used to divide the voice frames of the target streaming media that have been processed by short-time frame division according to the preset frame length to obtain multiple long-time frames; the processing module 502 is also used to determine each of the required frames respectively. The long-term spectrum energy difference characteristic value of the long-term frame is determined, and the first streaming media segment and the second streaming media segment are determined according to the long-term spectrum energy difference characteristic value; the output module 503 is used to output all the streaming media segments at a first speed. The first streaming media segment in the target streaming media is output at a second speed; the first streaming media segment is streaming media data containing voice information, and the second streaming media segment is outputted at a second speed. The streaming media segment is streaming media data that does not include voice information, and the first speed is lower than the second speed.

Optionally, the processing module 502 is specifically configured to: determine the target long frame as the first streaming media segment when the long-term spectrum energy difference characteristic value of the target long-term frame is less than a first threshold. ; In the case where the long-term spectrum energy difference characteristic value of the target long-term frame is greater than the first threshold, the target long-term frame is determined as the second streaming media segment; wherein the target long-term frame The frame is any one of the plurality of long-term frames, and the first threshold is related to the noise estimate value and the signal-to-noise ratio.

Optionally, the processing module 502 is specifically configured to: when the long-term spectrum energy difference characteristic value of the target long-term frame is less than the first threshold, determine the number of frames containing pitch in the target long-term frame; When the ratio of the number of frames containing pitch to the total number of frames of the target long-term frame is greater than the second threshold, the target long-term frame is determined to be the first streaming media segment; when the number of frames containing pitch is greater than the second threshold, When the ratio of the number of frames to the total number of frames of the target long frame is less than the second threshold, the target long frame is determined to be the second streaming media segment; wherein the target long frame for Any one of the plurality of long time frames.

Optionally, the processing module 502 is specifically configured to: when the long-term spectrum energy difference characteristic value of the target long-term frame is less than the first threshold, convert the time-domain speech signal of the target long-term frame into a frequency signal. domain energy signal to obtain the target streaming media segment; determine the average energy value and the maximum energy value of multiple frequency domain energy sampling points in the target streaming media segment; when the difference between the maximum energy value and the average energy value is at the If the target streaming media segment is within a preset range, the target streaming media segment is determined to be the first streaming media segment; when the difference between the maximum energy value and the average energy value is not within the first preset range In this case, the target streaming media segment is determined as the second streaming media segment; wherein the target long-term frame is any one of the multiple long-term frames.

Optionally, continuing to refer to Figure 5, the device 500 also includes a receiving module 504; the receiving module 504 is used to receive the user's first input when the playback speed setting interface is displayed; the processing module 503, Also configured to determine the first speed and the second speed in response to the first input.

In the embodiment of the present application, the speech frame of the target streaming media that has been processed by short-time frame division can be divided into multiple long-term frames, and the target streaming media can be determined by analyzing the long-term spectrum energy difference characteristic value of the long-term frame. The first streaming media segment and the second streaming media segment in, and the first streaming media segment is output at the first speed, and the second streaming media segment is output at the second speed. Therefore, on the one hand, since the long-term features have more advantages than the short-term features High smoothness and stability, so the accuracy of the analysis results can be improved by analyzing the long-term spectrum energy difference feature values of long-term frames; on the other hand, since the first speed is smaller than the second speed, that is, the second streaming media clip The playback speed is greater than the playback speed of the first streaming media segment. Therefore, it can not only reduce the time wasted during the playback process of the streaming media segment that does not contain voice information, and avoid the user missing the key content in the second streaming media segment, but also prevent the user from missing the key content in the second streaming media segment. During the output process, since the user does not need to make any input, the complexity of the user operation is reduced.

The playback speed control device in the embodiment of the present application may be an electronic device, or may be a component of the electronic device, such as an integrated circuit or chip. The electronic device may be a terminal or other devices other than the terminal. For example, the electronic device can be a mobile phone, a tablet computer, a pen Laptops, PDAs, vehicle-mounted electronic devices, Mobile Internet Devices (MIDs), augmented reality (AR)/virtual reality (VR) devices, robots, wearable devices, super mobile individuals Computer (ultra-mobile personal computer, UMPC), netbook or personal digital assistant (PDA), etc., can also be a server, network attached storage (Network Attached Storage, NAS), personal computer (personal computer, PC), Televisions (TVs), teller machines, self-service machines, etc. are not specifically limited in the embodiments of this application.

The playback speed control device in the embodiment of the present application may be a device with an operating system. The operating system can be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiments of this application.

The playback speed control device provided by the embodiment of the present application can implement various processes implemented by the method embodiments of Figures 1 to 4. To avoid repetition, they will not be described again here.

Optionally, as shown in Figure 6, this embodiment of the present application also provides an electronic device 600, including a processor 601 and a memory 602. The memory 602 stores programs or instructions that can be run on the processor 601. When the program or instruction is executed by the processor 601, each step of the above embodiment of the playback speed control method is implemented, and the same technical effect can be achieved. To avoid repetition, the details will not be described here.

It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

FIG. 7 is a schematic diagram of the hardware structure of an electronic device that implements an embodiment of the present application.

The electronic device 1000 includes but is not limited to: radio frequency unit 1001, network module 1002, audio output unit 1003, input unit 1004, sensor 1005, display unit 1006, user input unit 1007, interface unit 1008, memory 1009, processor 1010, etc. part.

Those skilled in the art can understand that the electronic device 1000 may also include a power supply (such as a battery) that supplies power to various components. The power supply may be logically connected to the processor 1010 through a power management system, thereby managing charging, discharging, and function through the power management system. Consumption management and other functions. The structure of the electronic device shown in Figure 7 does not constitute a limitation on the electronic device, and the electronic device may include more than The figures illustrate more or fewer components, or combine certain components, or arrange different components, which will not be described again here.

Among them, the processor 1010 is used to obtain the target streaming media; the processor 1010 is used to segment the voice frames of the target streaming media that have been processed by short-time frame division according to the preset frame length to obtain multiple long-term frames; The processor 1010 is further configured to determine the long-term spectrum energy difference characteristic value of each long-term frame, and determine the first streaming media segment and the second streaming media segment according to the long-term spectrum energy difference characteristic value; audio output Unit 1003 or display unit 1006 is configured to output a first streaming media segment in the target streaming media at a first speed, and output a second streaming media segment in the target streaming media at a second speed; wherein, the first streaming media segment The media segment is streaming media data containing voice information, the second streaming media segment is streaming media data not containing voice information, and the first speed is lower than the second speed.

In the embodiment of the present application, the speech frame of the target streaming media that has been processed by short-time frame division can be divided into multiple long-term frames, and the target streaming media can be determined by analyzing the long-term spectrum energy difference characteristic value of the long-term frame. The first streaming media segment and the second streaming media segment in, and the first streaming media segment is output at the first speed, and the second streaming media segment is output at the second speed. Therefore, on the one hand, since the long-term features have more advantages than the short-term features High smoothness and stability, so the accuracy of the analysis results can be improved by analyzing the long-term spectrum energy difference feature values of long-term frames; on the other hand, since the first speed is smaller than the second speed, that is, the second streaming media clip The playback speed is greater than the playback speed of the first streaming media segment. Therefore, it can not only reduce the time wasted during the playback process of the streaming media segment that does not contain voice information, and avoid the user missing the key content in the second streaming media segment, but also prevent the user from missing the key content in the second streaming media segment. During the output process, since the user does not need to make any input, the complexity of the user operation is reduced.

Optionally, the processor 1010 is specifically configured to: when the long-term spectrum energy difference characteristic value of the target long-term frame is less than a first threshold, determine the target long-term frame as the first streaming media segment; in When the long-term spectrum energy difference characteristic value of the target long-term frame is greater than the first threshold, the target long-term frame is determined as the second streaming media segment; wherein the target long-term frame is In any one of the plurality of long-term frames, the first threshold is compared with the noise estimate value and the signal-to-noise ratio. close.

In the embodiment of the present application, since it can be determined based on the long-term spectrum energy difference characteristic value of the target long-term frame whether the target long-term frame is the first streaming media segment or the first streaming media segment, it can be determined based on the long-term spectrum energy difference characteristic value. The first streaming media segment and the second streaming media segment in the target streaming media, thereby providing a basis for different streaming media segments to be played at different speeds.

Optionally, the processor 1010 is specifically configured to: when the long-term spectrum energy difference characteristic value of the target long-term frame is less than the first threshold, determine the number of frames containing pitch in the target long-term frame; When the ratio of the number of frames containing pitch to the total number of frames of the target long-term frame is greater than the second threshold, the target long-term frame is determined as the first streaming media segment; when the number of frames containing pitch If the ratio to the total number of frames of the target long frame is less than the second threshold, the target long frame is determined as the second streaming media segment; wherein the target long frame is the Any of the multiple long time frames.

In the embodiment of the present application, since it can be determined whether the target long frame is the first streaming media segment or the second streaming media segment based on the ratio of the number of frames containing pitch in the target long frame to the total number of frames in the target long frame, therefore, On the one hand, the first streaming media segment and the second streaming media segment in the target streaming media can be determined based on the frame number of the pitch, thereby providing a basis for different streaming media segments to be played at different speeds; on the other hand, the accuracy of the judgment results can be improved .

Optionally, the processor 1010 is specifically configured to: when the long-term spectrum energy difference characteristic value of the target long-term frame is less than a first threshold, convert the time-domain speech signal of the target long-term frame into frequency-domain energy. signal to obtain the target streaming media segment; determine the average energy value and the maximum energy value of multiple frequency domain energy sampling points in the target streaming media segment; when the difference between the maximum energy value and the average energy value is at the first preset If it is within the range, determine the target streaming media segment as the first streaming media segment; if the difference between the maximum energy value and the average energy value is not within the first preset range , determining the target streaming media segment as the second streaming media segment; wherein the target long-term frame is any one of the plurality of long-term frames.

In the embodiment of the present application, since the time domain speech signal of the target long time frame can be converted into a frequency domain energy signal when the long time spectrum energy difference characteristic value of the target long time frame is less than the first threshold, number, obtain the target streaming media segment, determine the average energy value and maximum energy value of multiple frequency domain energy sampling points in the target streaming media segment, and determine the target streaming media segment as the first stream based on the difference between the maximum energy value and the average energy value. The media segment is also the first streaming media segment. Therefore, on the one hand, the first streaming media segment and the second streaming media segment in the target streaming media can be determined based on the frequency domain energy, thereby providing a basis for different streaming media segments to be played at different speeds; on the other hand, the first streaming media segment and the second streaming media segment in the target streaming media can be determined based on the frequency domain energy. In this regard, the accuracy of judgment results can be improved.

Optionally, the user input unit 1007 is configured to receive the user's first input when the playback speed setting interface is displayed; the processor 1010 is further configured to respond to the first input, determine the first speed and the second speed.

In the embodiment of the present application, since the first speed and the second speed can be determined according to the first input, the user can customize the two playback speeds according to his own needs, thereby meeting the diverse speed playback needs of different users.

It should be understood that in the embodiment of the present application, the input unit 1004 may include a graphics processor (Graphics Processing Unit, GPU) 10041 and a microphone 10042. The graphics processor 10041 is responsible for the image capture device (GPU) in the video capture mode or the image capture mode. Process the image data of still pictures or videos obtained by cameras (such as cameras). The display unit 1006 may include a display panel 10061, which may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 1007 includes at least one of a touch panel 10071 and other input devices 10072 . Touch panel 10071, also known as touch screen. The touch panel 10071 may include two parts: a touch detection device and a touch controller. Other input devices 10072 may include but are not limited to physical keyboards, function keys (such as volume control keys, switch keys, etc.), trackballs, mice, and joysticks, which will not be described again here.

Memory 1009 may be used to store software programs as well as various data. The memory 1009 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, Image playback function, etc.) etc. Additionally, memory 1009 may include volatile memory or nonvolatile memory, or memory 1009 may include both volatile and nonvolatile storage. Both devices. Among them, the non-volatile memory can be read-only memory (Read-Only Memory, ROM), programmable read-only memory (Programmable ROM, PROM), erasable programmable read-only memory (Erasable PROM, EPROM), electrically removable memory. Erase programmable read-only memory (Electrically EPROM, EEPROM) or flash memory. Volatile memory can be random access memory (Random Access Memory, RAM), static random access memory (Static RAM, SRAM), dynamic random access memory (Dynamic RAM, DRAM), synchronous dynamic random access memory (Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDRSDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (Synch link DRAM) , SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DRRAM). The memory 1009 in the embodiment of the present application includes, but is not limited to, these and any other suitable types of memory.

The processor 1010 may include one or more processing units; optionally, the processor 1010 integrates an application processor and a modem processor, where the application processor mainly handles operations related to the operating system, user interface, application programs, etc., Modem processors mainly process wireless communication signals, such as baseband processors. It can be understood that the above modem processor may not be integrated into the processor 1010.

Embodiments of the present application also provide a readable storage medium. Programs or instructions are stored on the readable storage medium. When the program or instructions are executed by a processor, each process of the above playback speed control method embodiment is implemented, and can achieve The same technical effects are not repeated here to avoid repetition.

Wherein, the processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes computer readable storage media, such as computer read-only memory ROM, random access memory RAM, magnetic disk or optical disk, etc.

An embodiment of the present application further provides a chip. The chip includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the above playback speed control method embodiment. Each process can achieve the same technical effect. To avoid repetition, we will not go into details here.

It should be understood that the chips mentioned in the embodiments of this application may also be called system-on-chip, system-on-a-chip, system-on-a-chip or system-on-chip, etc.

Embodiments of the present application provide a computer program product. The program product is stored in a storage medium. The program product is executed by at least one processor to implement each process of the above playback speed control method embodiment, and can achieve the same technology. The effect will not be described here to avoid repetition.

The embodiment of the present application also provides an electronic device. The electronic device is configured to implement each process of the above method embodiment and can achieve the same technical effect. To avoid duplication, the details will not be described here.

It should be noted that, in this document, the terms "comprising", "comprises" or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article or device that includes a series of elements not only includes those elements, It also includes other elements not expressly listed or inherent in the process, method, article or apparatus. Without further limitation, an element defined by the statement "comprises a..." does not exclude the presence of additional identical elements in a process, method, article or apparatus that includes that element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, but may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved. Functions may be performed, for example, the methods described may be performed in an order different from that described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus the necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is better. implementation. Based on this understanding, the technical solution of the present application can be embodied in the form of a computer software product that is essentially or contributes to related technologies. The computer software product is stored in a storage medium (such as ROM/RAM, disk, CD), including several instructions to cause a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the steps described in various embodiments of this application. method.

The embodiments of the present application have been described above in conjunction with the accompanying drawings. However, the present application is not limited to the above-mentioned specific implementations. The above-mentioned specific implementations are only illustrative and not restrictive. Those of ordinary skill in the art will Inspired by this application, many forms can be made without departing from the purpose of this application and the scope protected by the claims, all of which fall within the protection of this application.

Claims (15)

A playback speed control method includes: Get target streaming media; Segment the voice frames of the target streaming media that have been processed by short-time frame division according to the preset frame length to obtain multiple long-time frames; Determine the long-term spectrum energy difference characteristic value of each long-term frame respectively, and determine the first streaming media segment and the second streaming media segment based on the long-term spectrum energy difference characteristic value; Output the first streaming media segment in the target streaming media at a first speed, and output the second streaming media segment in the target streaming media at a second speed; Wherein, the first streaming media segment is streaming media data containing voice information, the second streaming media segment is streaming media data not containing voice information, and the first speed is lower than the second speed. The playback speed control method according to claim 1, wherein the determining the first streaming media segment and the second streaming media segment according to the long-term spectrum energy difference characteristic value includes: When the long-term spectrum energy difference characteristic value of the target long-term frame is less than the first threshold, determine the target long-term frame as the first streaming media segment; If the long-term spectrum energy difference characteristic value of the target long-term frame is greater than the first threshold, determine the target long-term frame as the second streaming media segment; Wherein, the target long-term frame is any one of the plurality of long-term frames, and the first threshold is related to the noise estimate value and the signal-to-noise ratio. The playback speed control method according to claim 1, wherein the determining the first streaming media segment and the second streaming media segment according to the long-term spectrum energy difference characteristic value includes: When the long-term spectral energy difference characteristic value of the target long-term frame is less than the first threshold, determine the number of frames containing pitch in the target long-term frame; When the ratio of the number of frames containing pitch to the total number of frames of the target long-term frame is greater than the second threshold, determine the target long-term frame as the first streaming media segment; When the ratio of the number of frames containing pitch to the total number of frames of the target long-term frame is less than the second In the case of a threshold, determine the target long-term frame as the second streaming media segment; Wherein, the target long-term frame is any one of the plurality of long-term frames. The playback speed control method according to claim 1, wherein the determining the first streaming media segment and the second streaming media segment according to the long-term spectrum energy difference characteristic value includes: When the long-term spectrum energy difference characteristic value of the target long-term frame is less than the first threshold, convert the time-domain speech signal of the target long-term frame into a frequency-domain energy signal to obtain the target streaming media segment; Determine the average energy value and the maximum energy value of multiple frequency domain energy sampling points in the target streaming media segment; If the difference between the maximum energy value and the average energy value is within a first preset range, determine the target streaming media segment as the first streaming media segment; If the difference between the maximum energy value and the average energy value is not within the first preset range, determine the target streaming media segment as the second streaming media segment; Wherein, the target long-term frame is any one of the plurality of long-term frames. The playback speed control method according to claim 1, wherein before obtaining the target streaming media, the method further includes: When the playback speed setting interface is displayed, receive the user's first input; The first speed and the second speed are determined in response to the first input. A playback speed control device, including: an acquisition module, a processing module and an output module; The acquisition module is used to acquire target streaming media; The processing module is configured to divide the voice frames of the target streaming media that have been processed by short-time frame division according to the preset frame length to obtain multiple long-time frames; The processing module is further configured to determine the long-term spectrum energy difference characteristic value of each long-term frame, and determine the first streaming media segment and the second streaming media segment based on the long-term spectrum energy difference characteristic value; The output module is configured to output the first streaming media segment in the target streaming media at a first speed, and output the second streaming media segment in the target streaming media at a second speed; Wherein, the first streaming media segment is streaming media data containing voice information, and the second streaming media segment The media segment is streaming media data that does not include voice information, and the first speed is lower than the second speed. The playback speed control device according to claim 6, wherein the processing module is specifically configured to: when the long-term spectrum energy difference characteristic value of the target long-term frame is less than a first threshold, the target long-term frame is The frame is determined to be the first streaming media segment; when the long-term spectrum energy difference characteristic value of the target long-term frame is greater than the first threshold, the target long-term frame is determined to be the second streaming media segment. Fragment; wherein the target long-term frame is any one of the plurality of long-term frames, and the first threshold is related to the noise estimate value and the signal-to-noise ratio. The playback speed control device according to claim 6, wherein the processing module is specifically configured to: when the long-term spectrum energy difference characteristic value of the target long-term frame is less than a first threshold, determine the target long-term frame. The number of frames containing pitch in the frame; when the ratio of the number of frames containing pitch to the total number of frames of the target long-term frame is greater than the second threshold, determine the target long-term frame as the first stream Media segment; when the ratio of the number of frames containing pitch to the total number of frames of the target long-term frame is less than the second threshold, determine the target long-term frame as the second streaming media segment ; Wherein, the target long-term frame is any one of the plurality of long-term frames. The playback speed control device according to claim 6, wherein the processing module is specifically configured to: when the long-term spectrum energy difference characteristic value of the target long-term frame is less than a first threshold, the target long-term frame is The time domain speech signal of the frame is converted into a frequency domain energy signal to obtain the target streaming media segment; the average energy value and the maximum energy value of multiple frequency domain energy sampling points in the target streaming media segment are determined; between the maximum energy value and When the difference between the average energy values is within a first preset range, the target streaming media segment is determined to be the first streaming media segment; when the difference between the maximum energy value and the average energy value is not within If it is within the first preset range, the target streaming media segment is determined to be the second streaming media segment; wherein the target long-term frame is any one of the plurality of long-term frames. The playback speed control device according to claim 6, wherein the device further includes a receiving module; The receiving module is configured to receive the user's first input when displaying the playback speed setting interface before acquiring the target streaming media; The processing module is further configured to determine the first speed and the second speed in response to the first input. An electronic device, characterized in that it includes a processor and a memory, the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the implementation of claim 1- The playback speed control method described in any one of 5. A readable storage medium, characterized in that the readable storage medium stores programs or instructions, and when the programs or instructions are executed by a processor, the playback speed control method according to any one of claims 1-5 is implemented. . A chip. The chip includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to achieve the playback speed as described in any one of claims 1-5. Control Method. A computer program product, which is executed by at least one processor to implement the playback speed control method according to any one of claims 1-5. An electronic device configured to perform the playback speed control method according to any one of claims 1-5.