WO2018107777A1 - Method and system for annotating video image - Google Patents

Abstract

The present invention relates to a method and system for annotating a video image. The method comprises: establishing a canvas, pixels of the canvas corresponding to pixels of a video image to be annotated; drawing annotation content at the appointed position of the canvas according to a received annotation command and the correspondence of pixels, and performing format conversion on the canvas to obtain a first annotated image; and assigning a pixel value of the annotation content on the first annotated image to corresponding position on the video image to be annotated according to the correspondence of pixels to obtain an annotated video image so as to achieve annotation of the video image. According to the present solution, the annotation content is drawn by using the canvas, the operation of drawing the annotation content in each video image is avoided, and repeated format conversion does not need to be carried out on the video images, so that the resource consumption of a system CPU is greatly reduced when the video images are annotated.

Description

Video image annotation method and system Technical field

The present invention relates to the field of video image processing technologies, and in particular, to a method and system for marking video images.

Background technique

Video annotation refers to superimposing lines, words, etc. on the image in order to more clearly express the meaning of the image content during video playback. For example, when analyzing a video, some key characters and items in the video need to be circled to emphasize, and even some text descriptions are attached.

In the conventional technology, when the video is marked, it is generally necessary to extract each frame image first. Since the image format extracted from the video is often in the YUV format, the YUV image needs to be converted into an RGB image, and then the annotation information is superimposed on the RGB image. , then convert the RGB image into a YUV image and encode it into a video.

In the process of labeling the video, the video image is formatted multiple times, and in the process of converting the RGB image of different video images into the YUV image, the drawing of the line or the text is required, resulting in the resources of the system CPU. High consumption.

Summary of the invention

Based on this, it is necessary to provide a method and system for labeling video images in view of the traditional method of labeling video to high resource consumption of the system CPU.

A method for labeling video images, comprising the following steps:

Creating a canvas to determine a correspondence between pixels on the canvas and pixels of the video image to be labeled;

Receiving an annotation command, and drawing the annotation content at a specified position of the canvas according to the annotation command and the correspondence relationship;

Converting the canvas into a first annotation image of the first format, the first format being an image format of the video image to be labeled;

According to the correspondence relationship, the pixel value of the labeled content on the first annotation image is assigned to the corresponding position on the video image to be labeled, and the labeled video image is obtained.

A labeling system for video images, comprising:

a canvas creating unit, configured to create a canvas, and determine a correspondence between a pixel point on the canvas and a pixel point of the video image to be labeled;

An annotation drawing unit is configured to receive the labeling command, and draw the labeling content at a specified position of the canvas according to the labeling command and the corresponding relationship;

An image conversion unit, configured to convert the canvas into a first annotation image in a first format, where the first format is an image format of the video image to be labeled;

The pixel assignment unit is configured to assign a pixel value of the labeled content on the first annotation image to a corresponding position on the video image to be labeled according to the correspondence relationship, to obtain the labeled video image.

According to the labeling method and system of the above video image, the canvas is created, and the pixel point of the canvas corresponds to the pixel point of the video image to be labeled, and the labeling content is drawn at the designated position of the canvas according to the received labeling command and the corresponding relationship of the pixel points, and the canvas is drawn. Performing format conversion to obtain a first annotation image, assigning pixel values of the annotation content on the first annotation image to corresponding positions on the video image to be labeled according to pixel correspondence, obtaining an annotated video image, thereby completing the video image Label. In this scheme, the use of the canvas to draw the annotation content avoids the operation of drawing the annotation content in each frame of the video image, and does not need to perform multiple format conversion on the video image, thereby greatly reducing the video image annotation time to the system. CPU resource consumption.

DRAWINGS

1 is a schematic flow chart of a method for marking a video image according to an embodiment of the present invention;

2 is a schematic structural diagram of an annotation system of a video image of one embodiment;

3 is a schematic structural diagram of an annotation system of a video image of one embodiment;

4 is a schematic structural diagram of an annotation system of a video image of one of the embodiments.

detailed description

The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein The invention is only intended to be illustrative, and does not limit the scope of the invention.

Referring to FIG. 1 , it is a schematic flowchart of a method for marking a video image according to the present invention. The method for marking a video image in this embodiment includes the following steps:

Step S101: Create a canvas, and determine a correspondence relationship between a pixel point on the canvas and a pixel point of the video image to be labeled;

Step S102: Receive an annotation command, and draw the annotation content at a specified position of the canvas according to the annotation command and the correspondence relationship;

In this step, the labeling command is a command for labeling the specified position in the labeled video image. Since the pixel point on the canvas corresponds to the pixel point of the video image to be labeled, the designation of the canvas can be determined according to the labeling command and the correspondence relationship. position;

Step S103: Convert the canvas into a first annotation image of the first format, where the first format is an image format of the video image to be labeled;

In this step, the canvas is converted into a first annotation image, and the image format of the first annotation image is the same as the image format of the video image to be labeled, which facilitates subsequent annotation processing;

Step S104: Assign a pixel value of the labeled content on the first annotation image to a corresponding position on the video image to be labeled according to the correspondence relationship, to obtain an annotated video image.

In this step, the pixel value of the labeled content on the first labeled image is assigned to the corresponding position on the video image to be labeled, which is equivalent to marking the corresponding position on the video image to be labeled;

In this embodiment, the canvas is created, and the pixel points of the canvas correspond to the pixel points of the video image to be labeled, and the label content is drawn at a specified position of the canvas according to the received labeling command and the corresponding relationship of the pixel points, and the canvas is format converted. The first annotation image is obtained, and the pixel value of the annotation content on the first annotation image is assigned to the corresponding position on the video image to be labeled according to the pixel point correspondence relationship, and the labeled video image is obtained, thereby completing the labeling of the video image. In this scheme, the use of the canvas to draw the annotation content avoids the operation of drawing the annotation content in each frame of the video image, and does not need to perform multiple format conversion on the video image, thereby greatly reducing the video image annotation time to the system. CPU resource consumption.

In one of the embodiments, the step of converting the canvas to the first annotation image of the first format comprises the steps of:

Converting the canvas into a second annotation image of the second format, the second format being an image format directly associated with the canvas;

The second annotation image is converted into the first annotation image, and the image of the first format and the image of the second format have a conversion relationship.

In this embodiment, since the attributes of the canvas and the video image are different, the image format directly converted by the canvas is not the image format of the video image. Therefore, after converting the canvas into the second annotation image, the second annotation image needs to be converted into The first annotation image with the same format as the video image is convenient for labeling the video image to be labeled. Although the canvas is converted twice, the effective pixel on the canvas is only the pixel of the labeled content, compared to the video image. The conversion of pixel points has low resource consumption for the system CPU.

In one of the embodiments, the size of the canvas is the same as the size of the video image to be labeled.

In this embodiment, the size of the canvas is the same as the size of the video image to be labeled, which is convenient for quickly determining the correspondence between the pixel points on the canvas and the pixel points of the video image to be labeled, and can also quickly determine the marked content on the canvas. The specified location.

In one of the embodiments, the step of converting the canvas to the second annotation image of the second format comprises the steps of:

A rectangular image is captured on the canvas, the rectangular image being the smallest rectangular image covering the content of the annotation, and the rectangular image being converted into the second annotation image.

In this embodiment, generally, the content of the annotation does not occupy the entire video image, and the annotation content on the canvas only occupies a part of the canvas. Therefore, when acquiring the second annotation image, the annotation content may be only for the annotation content, that is, on the canvas. The rectangular image is intercepted, and the rectangular image is a minimum rectangular image covering the content of the annotation, and only the rectangular image is converted into the second annotation image, thereby reducing the calculation amount of the conversion and further reducing the resource consumption of the system CPU.

In one of the embodiments, the step of creating a canvas further includes the following steps:

Sets the background pixel value of the canvas, and the background pixel value is different from the pixel value of the label content.

In this embodiment, setting a background pixel value different from the pixel value of the labeled content is advantageous for distinguishing between the background and the labeled content, and also for determining the content of the labeled content during subsequent evaluation processing.

In one of the embodiments, the step of assigning pixel values of the labeled content on the first annotation image to corresponding locations on the video image to be labeled includes the following steps:

When the pixel value of the current pixel on the first annotation image is different from the background pixel value of the first annotation image, the pixel value of the current pixel is assigned to the corresponding position on the video image to be labeled.

In this embodiment, according to the difference between the pixel value of the current pixel point on the first annotation image and the background pixel value of the first annotation image, the pixel point of the label content is determined, the determination process is simple, and the efficiency of the assignment operation can be improved.

Optionally, the pixel values of all the pixels of the first annotation image corresponding to the canvas may be compared with the background pixel values of the first annotation image, or may correspond to the smallest rectangular image covering the annotation content on the canvas. The pixel values of all the pixels of a labeled image are compared with the background pixel values of the first labeled image, respectively.

In one of the embodiments, the step of assigning pixel values of the labeled content on the first annotation image to corresponding locations on the video image to be labeled includes the following steps:

When the pixel value of the current pixel on the first annotation image is the same as the background pixel value of the first annotation image, the corresponding position on the video image to be labeled is not processed.

In this embodiment, according to the same pixel value of the current pixel point on the first annotation image and the background pixel value of the first annotation image, the background pixel on the first annotation image can be quickly excluded, thereby improving the efficiency of the assignment operation. .

In one embodiment, the method for marking a video image further includes the following steps:

Obtaining a video to be labeled, decoding the labeled video, and obtaining a video image to be labeled;

The step of obtaining the annotated video image further includes the following steps:

The annotated video image is encoded into an annotated video.

In this embodiment, the video to be labeled is decoded, and the video image to be labeled is obtained. After the labeling of the video image is completed, the labeled video image is encoded into an annotation video to complete the labeling of the video, and the whole process is coherent and orderly. Improve the efficiency of video annotation, and there are multiple frames of video images after video decoding. The content of the annotations in the multi-frame continuous video images in a certain period of time is the same, because the canvas and the video image are independent objects with corresponding relationships, in the multi-frame Continuous video images do not need to be labeled The drawing of the marked content multiple times further reduces the resource consumption of the system CPU.

In one of the embodiments, the first format is a YUV image format and the second format is an RGB image format.

In this embodiment, the content of the canvas is image data, which can be directly converted into an RGB image format. The video image obtained after video decoding is generally a YUV image format, and the conversion between the two image formats is simple, and the system CPU The resource consumption is small.

According to the above method for labeling video images, the present invention also provides an annotation system for video images. Hereinafter, an embodiment of the annotation system for video images of the present invention will be described in detail.

Referring to FIG. 2, it is a schematic structural diagram of an annotation system for a video image according to the present invention. The annotation system of the video image in this embodiment includes the following units:

a canvas creating unit 210, configured to create a canvas, and determine a correspondence between a pixel point on the canvas and a pixel point of the video image to be labeled;

An annotation drawing unit 220 is configured to receive an annotation command, and draw the annotation content at a specified position of the canvas according to the annotation command and the correspondence relationship;

The image conversion unit 230 is configured to convert the canvas into a first annotation image in a first format, where the first format is an image format of the video image to be labeled;

The pixel assignment unit 240 is configured to assign a pixel value of the labeled content on the first annotation image to a corresponding position on the video image to be labeled according to the correspondence relationship, to obtain the labeled video image.

In one embodiment, the image conversion unit 230 converts the canvas into a second annotation image in a second format, the second format is an image format directly associated with the canvas; and converting the second annotation image into a first annotation image, first The formatted image has a conversion relationship with the image of the second format.

In one of the embodiments, the size of the canvas is the same as the size of the video image to be labeled.

In one of the embodiments, the image conversion unit 230 intercepts a rectangular image on the canvas, the rectangular image being the smallest rectangular image covering the annotation content, and the rectangular image being converted into the second annotation image.

In one embodiment, as shown in FIG. 3, the annotation system of the video image further includes a background setting. The unit 250 is configured to set a background pixel value of the canvas, and the background pixel value is different from the pixel value of the labeled content.

In one embodiment, the pixel assignment unit 240 assigns the pixel value of the current pixel point to the video image to be labeled when the pixel value of the current pixel point on the first annotation image is different from the background pixel value of the first annotation image. Corresponding location.

In one embodiment, the pixel assignment unit 240 does not process the corresponding position on the video image to be labeled when the pixel value of the current pixel on the first annotation image is the same as the background pixel value of the first annotation image.

In one embodiment, as shown in FIG. 4, the labeling system of the video image further includes a video decoding unit 260 and a video encoding unit 270;

The video decoding unit 260 is configured to acquire a video to be labeled, decode the video to be labeled, and obtain a video image to be labeled;

Video encoding unit 270 is for encoding the annotated video image into an annotated video.

In one of the embodiments, the first format is a YUV image format and the second format is an RGB image format.

The labeling system of the video image of the present invention has a one-to-one correspondence with the labeling method of the video image of the present invention, and the technical features and the beneficial effects thereof described in the embodiment of the labeling method of the video image are applicable to the embodiment of the labeling system of the video image. in.

In the present invention, ordinal numbers such as "first" and "second" are merely used to distinguish the objects involved, and are not intended to limit the objects themselves.

In a specific embodiment, the specific steps of the labeling method of the video image may be:

Acquiring a video, decoding the video to obtain a decoded YUV video image;

Create a label canvas, set a background color for the canvas, and determine a correspondence between pixels on the canvas and pixels of the YUV video image to be labeled;

Taking the windows system as an example, to create a canvas, you first need to create a memory device context, and create device compatible bitmaps and drawing objects based on the context. In order to distinguish the line and background color of the label, you also need to set a color that is not used for labeling as the background color;

Receive the label command to draw the label on the canvas; after labeling, the point on the canvas that is not the background color is the marked point.

Extract the labeled RGB image from the canvas;

Converting an annotated RGB image to a labeled YUV image;

The YUV video image is processed as follows:

Determining whether the pixel value of the pixel in the marked YUV image is the background pixel value in the labeled YUV image, if not, processing the corresponding pixel point in the YUV video image, if not, marking the pixel value of the pixel in the YUV image Assigned to the corresponding pixel in the YUV video image.

The labeled YUV video image is encoded into a video, and the labeled content can be displayed on the video screen.

The specific embodiment adopts the method of labeling the canvas, avoiding the labeling on the video changed image every time, and does not need to perform the YUV→RGB→YUV format conversion on the video image, and when the label content does not change, the labeling canvas is also Without the need to redraw each time, the above method greatly reduces the resource consumption of the system CPU.

The technical features of the above-described embodiments may be arbitrarily combined. For the sake of brevity of description, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction between the combinations of these technical features, All should be considered as the scope of this manual.

The above-described embodiments are merely illustrative of several embodiments of the present invention, and the description thereof is more specific and detailed, but is not to be construed as limiting the scope of the invention. It should be noted that a number of variations and modifications may be made by those skilled in the art without departing from the spirit and scope of the invention. Therefore, the scope of the invention should be determined by the appended claims.

Claims (6)

A method for marking a video image, comprising the steps of: Creating a canvas to determine a correspondence between a pixel on the canvas and a pixel of the video image to be labeled; Receiving an annotation command, and drawing the annotation content at a specified position of the canvas according to the annotation command and the correspondence relationship; Converting the canvas into a first annotation image of a first format, where the first format is an image format of the video image to be labeled; And assigning, according to the correspondence, a pixel value of the labeled content on the first labeled image to a corresponding position on the to-be-labeled video image, to obtain an annotated video image. The method for marking a video image according to claim 1, wherein the step of converting the canvas into the first annotation image of the first format comprises the following steps: Converting the canvas into a second annotation image in a second format, the second format being an image format directly associated with the canvas; Converting the second annotation image into the first annotation image, and the image of the first format and the image of the second format have a conversion relationship. The method for marking a video image according to claim 1, wherein the size of the canvas is the same as the size of the video image to be labeled. The method for marking a video image according to claim 2, wherein the step of converting the canvas into the second annotation image of the second format comprises the following steps: A rectangular image is captured on the canvas, the rectangular image being a smallest rectangular image covering the labeled content, and the rectangular image is converted into the second labeled image. The method for marking a video image according to claim 1, wherein the step of creating a canvas further comprises the following steps: A background pixel value of the canvas is set, the background pixel value being different from a pixel value of the labeled content. The method of marking a video image according to claim 5, wherein said said

Images