CN106484722A - A kind of image procossing and searching method, device and system - Google Patents

Abstract

The invention discloses an image processing and searching method, device and system, which are used to realize automatic searching for images. The image processing method implemented on the server side includes: identifying the first image file uploaded by the first terminal, marking the recognition result on the first image file with text label information to obtain a second image file; returning the first image file to the first terminal Describe the second image file. The image search method implemented on the terminal side includes receiving an image search request, the image search request carrying search keyword information; searching for text tag information matching the search keyword information in the text tag information of each image file ; According to the image search request, return image files whose text label information matches the search keyword information.

Description

Image processing and search method, device and system

technical field

The present invention relates to the technical field of image search, in particular to an image processing and search method, device and system.

Background technique

With the development of the mobile Internet, the message content that can be sent by instant messaging applications has gradually expanded from the initial text to rich media forms such as voice, image, and video. In instant messaging applications, messages containing images (hereinafter referred to as image messages) account for an increasing proportion.

In practical applications, users have a need to search for message content. However, the current search for instant messages is only suitable for text messages, voice messages, etc. If the user wishes to search for image messages sent and received, the user can only check the historical message records one by one, manually judging whether each image sent and received is It is required by oneself, and the operation is cumbersome. It can be seen that how to realize the automatic search for image messages has become one of the technical problems to be solved urgently in the prior art.

Contents of the invention

Embodiments of the present invention provide an image processing and searching method, device and system to realize automatic searching for images.

An embodiment of the present invention provides an image processing method implemented on the server side, including:

Identifying the first image file uploaded by the first terminal, and marking the recognition result with text label information on the first image file to obtain a second image file;

returning the second image file to the first terminal.

An embodiment of the present invention provides an image processing method implemented on a terminal side, including:

Upload the first image file to the server;

receiving the second image file returned by the server, where the second image file is obtained by marking the recognition result on the first image file with text label information after the server recognizes the first image file.

An embodiment of the present invention provides an image search method based on the above image processing method implemented on the terminal side, including:

receiving an image search request, the image search request carrying search keyword information;

In the text tag information of each image file, search for the text tag information matching the search keyword information;

Image files whose text tag information matches the search keyword information are returned for the image search request.

An embodiment of the present invention provides an image processing device implemented on the server side, including:

An identification unit, configured to identify the first image file uploaded by the first terminal, and mark the identification result on the first image file with text label information to obtain a second image file;

A returning unit, configured to return the second image file to the first terminal.

An embodiment of the present invention provides an image processing device implemented on a terminal side, including:

an uploading unit, configured to upload the first image file to the server;

The first receiving unit is configured to receive the second image file returned by the server, the second image file is marked on the first image file by text label information after the server recognizes the first image file The recognition result is obtained.

An embodiment of the present invention provides an image search device, including:

The second receiving unit is configured to receive an image search request, and the image search request carries search keyword information;

A search unit, configured to search for text tag information matching the search keyword information in the text tag information of each image file;

A returning unit, configured to return image files whose text label information matches the search keyword information in response to the image search request.

An embodiment of the present invention provides an image processing system, including a terminal and a server, wherein the server is provided with the above-mentioned image processing device implemented on the server side, and the terminal is equipped with the above-mentioned image processing device implemented on the terminal side and/or Or the above-mentioned image search device.

In the image processing and search method, device, and system provided by the embodiments of the present invention, the server identifies the image files uploaded by the terminal, and feeds back the image files obtained after marking the identification results with text label information to the terminal. During image search, the search keyword can be matched with the text label information in the image file, and the matched image file can be returned. Thus, the automatic image search can be realized based on the marked text label information.

Additional features and advantages of the invention will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

Description of drawings

The accompanying drawings described here are used to provide a further understanding of the present invention, and constitute a part of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention, and do not constitute improper limitations to the present invention. In the attached picture:

FIG. 1 is a schematic diagram of an implementation process of a server implementing an image processing method in an embodiment of the present invention;

FIG. 2 is a schematic diagram of an implementation flow of an image processing method implemented by a terminal in an embodiment of the present invention;

FIG. 3 is a schematic diagram of an implementation flow of an image search method in an embodiment of the present invention;

FIG. 4a is a schematic diagram of a scene where user A transmits an image file to user B in an embodiment of the present invention;

Fig. 4b is a schematic diagram of an embodiment of an image processing and searching method applied to instant messaging in an embodiment of the present invention;

FIG. 5 is a schematic structural diagram of a first image processing device in an embodiment of the present invention;

6 is a schematic structural diagram of a second image processing device in an embodiment of the present invention;

Fig. 7 is a schematic diagram of the results of the image search device in the embodiment of the present invention;

FIG. 8 is a schematic structural diagram of an image processing system in an embodiment of the present invention.

detailed description

In order to realize automatic search of images, an image processing and search method, device and system are provided in an embodiment of the present invention.

The preferred embodiments of the present invention will be described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described here are only used to illustrate and explain the present invention, and are not intended to limit the present invention, and in the absence of conflict, the present invention The embodiments and the features in the embodiments can be combined with each other.

As shown in Figure 1, it is a schematic diagram of the implementation process of implementing the image processing method on the server side, which may include the following steps:

S11. Recognize the first image file uploaded by the first terminal, and mark the recognition result on the first image file with text label information to obtain a second image file.

Wherein the first image file may be obtained by the server in any of the following ways:

method one,

The user can use the first terminal to upload the captured image file to the server for image recognition. For example, after the user finishes capturing the image file, he uses the first terminal to upload the image file to the server for image recognition.

Method two,

The user uploads the image file locally stored in the first terminal to the server for identification.

Method three,

The image file that the user transmits to the friend through the server when chatting with the friend through the instant messaging client.

After obtaining the first image file, the server uses image recognition technology to identify the image content of the first image file, for example, to identify objects, scenes, characters, etc. included in the image. The corresponding text label information is generated according to the recognition result, and the corresponding recognition result in the image is marked with the generated text label information.

S12. Return the second image file to the first terminal.

Preferably, the text label information in the second image file is editable label information. That is, the user is allowed to modify the text label information in the second image file.

Based on this, the image processing method implemented on the server side may also include the following steps: receiving a third image file uploaded by the first terminal, wherein the third image file is obtained after modifying the text label information in the second image file; modifying and storing The text label information corresponding to the recognition result in the recognition database of the identification library is the text label information carried in the third image file.

During specific implementation, after the first terminal receives the second image file, it can display the second image file locally, and the user can confirm whether the recognition result is accurate according to the text label information marked in the second image file. The information is modified to obtain a third image file, and the third image file is uploaded to the server through the first terminal, and after the server compares the third image file with the second image file, the text label information corresponding to the recognition result is modified. For example, the server recognizes the rose in the first image file, and marks the text label information "rose" on the corresponding position of the rose in the first image file to obtain the second image file, and sends the second image file to the second image file. After a terminal, the user changes the marked text label information "rose" to "green rose" to obtain the third image file, the user sends the third image file to the server through the first terminal, and the server compares the second image file with the In the third image file, according to the comparison result, the text label information corresponding to the recognition result in the stored recognition library is corrected as "green rose".

Through the above steps, the recognition library stored in the server can be continuously revised to improve the accuracy of image recognition on the server side.

It should be noted that during specific implementation, if the server obtains the first image file in the above-mentioned method 3, the server also needs to forward the second image file or the third image file to the user's friend. Specifically, the server can pass the first image file The second terminal forwards the second image file or the third image file to the friend of the user. After receiving the second image file or the third image file, the first terminal and the second terminal display the second image file or the third image file, and store the received second image file or the third image file locally.

During specific implementation, image recognition may also be performed locally on the terminal, and the recognition result may be marked and stored using text label information.

Correspondingly, the embodiment of the present invention also provides an image processing method implemented on the terminal side, as shown in FIG. 2 , which may include the following steps:

S21. Upload the first image file to the server.

Similarly, the terminal can upload the first image file to the server in any of the following ways:

method one,

The user can use the first terminal to upload the captured image file to the server for image recognition. For example, after the user finishes capturing the image file, he uses the first terminal to upload the image file to the server for image recognition.

Method two,

The user uploads the image file locally stored in the first terminal to the server for identification.

Method three,

The image file that the user transmits to the friend through the server when chatting with the friend through the instant messaging client.

S22. Receive the second image file returned by the server.

Wherein, the second image file is obtained after the server recognizes the image content of the first image file uploaded by the terminal, and marks the recognition result on the first image file with text label information.

Specifically, after obtaining the first image file, the server uses image recognition technology to identify the image content of the first image file, such as identifying items, scenes, characters, etc. included in the image. Generate corresponding text label information according to the recognition result, and use the generated text label information to mark the corresponding recognition result in the image to obtain a second image file.

Preferably, the text label information marked by the server is editable label information. After receiving the second image file, the terminal displays the second image file to the user. If the user thinks that the text label information in the second image file is inaccurate, he can modify it to obtain the third image file, and the terminal uploads the third image file to the The server is used for the server to modify its stored recognition library to improve the accuracy of image content recognition.

Based on this, the image processing method implemented on the terminal side may further include the following step: uploading to the server a third image file obtained by modifying text label information in the second image file.

The terminal locally stores the second image file and the third image file.

Based on the second image file or the third image file locally stored in the terminal, if the user needs to search, the user may search by using a search keyword. The image search method provided by the embodiment of the present invention will be described below in conjunction with the implementation process of a user using a terminal to perform an image search.

As shown in FIG. 3, it is a schematic diagram of the implementation flow of the image search method provided by the embodiment of the present invention, which may include the following steps:

S31. Receive an image search request.

Wherein, the search keyword information is carried in the image search request. During specific implementation, when searching for an image, the user inputs a search keyword in a search box displayed on the terminal.

S32. In the text tag information of each image file, search for text tag information that matches the search keyword information.

During specific implementation, after the image search request is received, text tag information matching the search keyword is searched in locally stored text tag information according to the search keyword carried therein. The specific matching method is the same as the existing matching process for text search, and will not be repeated here.

S33. Return the image file whose text label information matches the search keyword information in response to the received image search request.

Specifically, the image file where the matched text label information is located is returned.

In order to better understand the present invention, the specific implementation process of the embodiment of the present invention will be described below by taking user A using terminal A to chat with user B using instant messaging software, and user A sending image files to user B as an example. B uses terminal B to communicate with user A. As shown in Figure 4a, it is a schematic diagram of a scene where user A transmits an image file to user B, as shown in Figure 4b, may include the following steps:

S41. User A uses terminal A to send the first image file to terminal B through the server.

S42. The server receives the first image file sent by the user A to the user.

S43. The server performs image content identification on the first image file.

For example, after the server identifies the image content of the first image file, it identifies items, scenes, and other content therein.

S44. The server generates text label information according to the recognition result and marks the first image file to obtain a second image file.

S45. The server sends the second image file to terminal A and terminal B respectively.

S46. Terminal A and terminal B respectively store the received second image file.

S47. User A submits an image search request to terminal A, which carries a search keyword.

Specifically, when user A needs to perform an image search, he may submit a search request to terminal A, and enter a search keyword in a search box displayed on terminal A.

S48. Terminal A searches the stored image files according to the search keyword for image files whose text label information matches the search keyword.

S49. Terminal A returns the searched image file.

During specific implementation, when user B needs to perform an image search, he can submit a search request to terminal B, and enter a search keyword in the search box displayed on terminal B, and terminal B will search for text in the stored image file according to the search keyword Image files whose tag information matches the search keyword and return the searched image files.

During specific implementation, the terminal can also send the search keyword input by the user to the server to assist the server in updating the recognition library and recognition algorithm, and improve the accuracy of image content recognition on the server side.

It should be noted that the terminal B can receive multiple terminals, such as the first image file sent by the terminal A1, A2, ... An, and the first image file is forwarded to the terminal by the second image file marked with text label information by the server B. Terminal B locally stores the received image file marked with text label information.

In the image processing and search method provided by the embodiment of the present invention, after the server obtains the first image file uploaded by the terminal, it performs image content recognition on the obtained image file, generates corresponding text label information according to the recognition result, and displays it in the image Use the generated text tag information to annotate the corresponding recognition result to obtain the second image file. After sending the second image file to the terminal, if the user modifies the text tag information, the terminal will use the third image file obtained after the user modifies the text tag information. The image file is sent to the server, and the server modifies the corresponding text label information stored in the recognition library according to the third image file.

Based on the same inventive concept, an image processing and searching device and an image processing system are also provided in the embodiment of the present invention respectively. Since the problem-solving principle of the above-mentioned device and system is similar to the above-mentioned method, the implementation of the above-mentioned device and system can refer to the implementation of the method , the repetitions will not be repeated.

As shown in FIG. 5, it is a schematic structural diagram of the first image processing device provided by the embodiment of the present invention, which may include:

The identification unit 51 is configured to identify the first image file uploaded by the first terminal, and mark the identification result on the first image file with text label information to obtain a second image file.

Preferably, the text label information is editable label information.

A returning unit 52, configured to return the second image file to the first terminal.

The first image processing device provided in the embodiment of the present invention may further include:

a receiving unit, configured to receive a third image file uploaded by the first terminal, where the third image file is obtained after modifying text label information in the second image file;

The correcting unit is configured to correct the text label information corresponding to the recognition result stored in the recognition library to be the text label information carried in the third image file.

Wherein, the first image file is sent by the first terminal to the second terminal; and the first image processing device provided by the embodiment of the present invention may further include:

a sending unit, configured to send the second image file to the second terminal.

For the convenience of description, the above parts are divided into modules (or units) according to their functions and described separately. Certainly, when implementing the present invention, the functions of each module (or unit) can be implemented in one or more pieces of software or hardware. The above-mentioned first image processing device can be set in a server, and the server can be an instant message server, or an image processing server dedicated to user image processing. Of course, the above-mentioned first type of image processing device may also be installed in a terminal, and the terminal performs operations of image content recognition and marking of recognition results.

As shown in FIG. 6, a schematic structural diagram of a second image processing device provided by an embodiment of the present invention may include:

uploading unit 61, configured to upload the first image file to the server;

The first receiving unit 62 is configured to receive the second image file returned by the server, and the second image file uses text label information on the first image file after the server recognizes the first image file label recognition results.

Preferably, the text label information is editable label information; and

The uploading unit 61 may also be configured to upload to the server a third image file obtained by modifying text label information in the second image file.

For the convenience of description, the above parts are divided into modules (or units) according to their functions and described separately. Certainly, when implementing the present invention, the functions of each module (or unit) can be implemented in one or more pieces of software or hardware. The above-mentioned second image processing apparatus may be set in a terminal.

As shown in FIG. 7, a schematic structural diagram of an image search device based on the above-mentioned second image processing device provided in an embodiment of the present invention may include:

The second receiving unit 71 is configured to receive an image search request, and the image search request carries search keyword information;

A search unit 72, configured to search for text tag information matching the search keyword information in the text tag information of each image file;

The returning unit 73 is configured to return image files whose text label information matches the search keyword information in response to the image search request.

For the convenience of description, the above parts are divided into modules (or units) according to their functions and described separately. Certainly, when implementing the present invention, the functions of each module (or unit) can be implemented in one or more pieces of software or hardware. The above-mentioned image search device may be installed in a terminal.

As shown in FIG. 8 , it is a schematic diagram of the results of the image processing system provided by the embodiment of the present invention, which may include a terminal 81 and a server 82, wherein the terminal 81 is provided with the above-mentioned second image processing device and/or the above-mentioned image search device , the server 82 is provided with the above-mentioned first image processing device.

In the image processing and search method, device, and system provided by the embodiments of the present invention, the server identifies the image files uploaded by the terminal, and feeds back the image files obtained after marking the identification results with text label information to the terminal. During image search, the search keyword can be matched with the text label information in the image file, and the matched image file can be returned. Thus, the automatic image search can be realized based on the marked text label information.

Those skilled in the art should understand that the embodiments of the present invention may be provided as methods, systems, or computer program products. Accordingly, the present invention can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow chart or blocks of the flowchart and/or the block or blocks of the block diagrams.

While preferred embodiments of the present invention have been described, additional changes and modifications can be made to these embodiments by those skilled in the art once the basic inventive concept is appreciated. Therefore, it is intended that the appended claims be construed to cover the preferred embodiment as well as all changes and modifications which fall within the scope of the invention.

Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention also intends to include these modifications and variations.

Claims (15)

1. An image processing method, characterized in that, comprising: Identifying the first image file uploaded by the first terminal, and marking the recognition result with text label information on the first image file to obtain a second image file; returning the second image file to the first terminal. 2. The method according to claim 1, wherein the text label information is editable label information. 3. The method of claim 2, further comprising: receiving a third image file uploaded by the first terminal, where the third image file is obtained after modifying text label information in the second image file; The text label information corresponding to the recognition result in the stored recognition library is corrected to be the text label information carried in the third image file. 4. The method according to claim 1, wherein the first image file is sent by the first terminal to the second terminal; and The method also includes: sending the second image file to the second terminal. 5. An image processing method, characterized in that, comprising: Upload the first image file to the server; receiving the second image file returned by the server, where the second image file is obtained by marking the recognition result on the first image file with text label information after the server recognizes the first image file. 6. The method according to claim 5, wherein the text label information is editable label information; and The method also includes: uploading to the server a third image file obtained by modifying the text label information in the second image file. 7. An image search method based on the method according to claim 5 or 6, characterized in that, comprising: receiving an image search request, the image search request carrying search keyword information; In the text tag information of each image file, search for the text tag information matching the search keyword information; Image files whose text tag information matches the search keyword information are returned for the image search request. 8. An image processing device, comprising: An identification unit, configured to identify the first image file uploaded by the first terminal, and mark the identification result on the first image file with text label information to obtain a second image file; A returning unit, configured to return the second image file to the first terminal. 9. The device according to claim 8, wherein the text label information is editable label information. 10. The apparatus of claim 9, further comprising: a receiving unit, configured to receive a third image file uploaded by the first terminal, where the third image file is obtained after modifying text label information in the second image file; The correcting unit is configured to correct the text label information corresponding to the recognition result stored in the recognition library to be the text label information carried in the third image file. 11. The device according to claim 8, wherein the first image file is sent by the first terminal to the second terminal; and The device also includes: A sending unit, configured to send the second image file to the second terminal. 12. An image processing device, comprising: an uploading unit, configured to upload the first image file to the server; The first receiving unit is configured to receive the second image file returned by the server, the second image file is marked on the first image file by text label information after the server recognizes the first image file The recognition result is obtained. 13. The device according to claim 12, wherein the text label information is editable label information; and The uploading unit is further configured to upload to the server a third image file obtained by modifying text label information in the second image file. 14. An image search device based on the device according to claim 12 or 13, characterized in that it comprises: The second receiving unit is configured to receive an image search request, and the image search request carries search keyword information; A search unit, configured to search for text tag information matching the search keyword information in the text tag information of each image file; A returning unit, configured to return image files whose text label information matches the search keyword information in response to the image search request. 15. An image processing system, characterized by comprising a terminal and a server, wherein the server is provided with the image processing device according to any one of claims 8 to 11, and the terminal is provided with the device according to claim 12 or the image processing device described in claim 13 and/or the image search device described in claim 14.