WO2025067005A1 - Method for picture retrieval and related apparatus - Google Patents

Abstract

A method for picture retrieval, comprising: on the basis of a first picture and first text from a client, a cloud generates a plurality of preview pictures, the first text being used for adjusting the first picture, and then sends the plurality of preview pictures to the client; on the basis of operation information from the client, the cloud retrieves a second picture associated with a target picture, the target picture being included in the plurality of preview pictures; and the cloud sends the second picture to the client.

Description

A method and related device for image retrieval

This application claims the priority of the Chinese patent application filed with the State Intellectual Property Office on September 27, 2023, with application number 202311287307.X and application name “A method for image retrieval and related devices”, all contents of which are incorporated by reference in this application.

Technical Field

The present application relates to the field of cloud computing technology, and in particular to a method and related device for image retrieval.

Background Art

Retrieval is also called search. On a search engine or shopping platform, after the user enters keywords in the search box, the search platform will return the corresponding search results.

Currently, text search is more commonly used, and some search platforms also provide image search functions. Image search is to search based on the image input by the user and return images similar to the input image, which is also called image search.

Although image retrieval can narrow the search scope compared to text retrieval, it will still retrieve a lot of images. Users need to browse the returned images one by one to find the desired images, and accurate retrieval cannot be achieved.

Summary of the invention

The present application provides a method for image retrieval, which is used to improve the accuracy of image retrieval. The present application also provides a corresponding device, system, computer-readable storage medium, and computer program product.

The first aspect of the present application provides a method for image retrieval, including: generating multiple preview images based on a first image and a first text from a client, wherein the first text is used to adjust the first image; sending the multiple preview images to the client; retrieving a second image associated with a target image based on operation information from the client, wherein the target image is included in the multiple preview images; and sending the second image to the client.

In this application, the cloud can be software or services of a cloud platform, or software or services deployed on a node in a network such as an edge node. The cloud can run on an independent physical machine or on virtualized resources.

In the present application, the client can be a terminal device or an application, for example, the application runs on the terminal device for the user to use. The application can be a search engine, a response application, or a shopping application.

In this application, the user can input the first image and the first text on the client, and use the first text to guide the cloud to adjust the first image, thereby obtaining multiple preview images. The user can select a suitable target image from the preview images, and the cloud will perform searches based on the target images, which can narrow the search scope and improve the accuracy of image retrieval.

In a possible implementation, the above steps: generating multiple preview images based on the first image and the first text from the client, include: expanding the first text to obtain multiple extended texts; generating a preview image corresponding to each extended text based on each extended text and the first image.

In this possible implementation, the cloud can expand around the content in the first text, for example, if the first text is a flower pattern configured on the first picture, the text can be expanded around the type of flower, the position of the flower on the first picture, etc. By expanding the text, the preview picture obtained can provide preview pictures that meet the user's requirements in the first text from multiple angles, thereby increasing the richness of the preview pictures within the scope of the user's requirements.

In a possible implementation, the multiple extended texts are texts in a text design database whose similarity with the first text satisfies a first condition.

In this possible implementation, the text design database can be pre-built and continuously updated, and the text in the text design database can be collected based on historical text retrieval records. In the present application, the text with high similarity to the first text can be searched in the text design database through the first text. The first condition can be a threshold value. Taking the value of the similarity in the interval of 0 to 1 as an example, the threshold value can be 0.5 or other numerical values. Taking 0.5 as an example, texts with a similarity greater than 0.5 can be understood as texts whose similarity to the first text meets the first condition. The first condition can also be a restriction condition on the number of texts, such as: the top N texts with the highest similarity to the first text are texts whose similarity to the first text meets the first condition. It can be seen from this possible implementation that a better extended text can be obtained through the text design database, thereby improving the user satisfaction of the preview image.

In a possible implementation, the multiple extended texts are texts in a text design database whose similarity and design popularity with the first text meet a second condition.

In this possible implementation, the first text can be used to search for texts with high similarity to the first text in the text design database. And the popularity of these texts with high similarity can be determined, and then the extended text can be determined by combining the similarity and popularity. The popularity of each text in the text design database can be counted in advance by the number of times it is used. The value of popularity can be a value between 0 and 1. The second condition can include one threshold or two thresholds. When it is a threshold, the similarity and popularity of the text can be weighted to obtain a weighted value (the weight of the similarity is usually greater than the popularity). If the weighted value is greater than this threshold, it means that the text meets the second condition and belongs to the extended text of the first text. If the second condition is two thresholds, one threshold can be a similarity threshold, such as: 0.5; one threshold is a threshold for the number of extended texts, such as: N, N is a positive integer. If the similarity between the found text and the first text is greater than 0.5, the extended texts with similarity greater than 0.5 can be sorted according to the size of popularity, and then the N extended texts with the largest popularity are selected as the final selected extended texts. In this application, the accuracy of the extended text can be improved by similarity and popularity.

In a possible implementation, the second image is an image retrieved from an image vector database.

In this possible implementation, the picture vector database can be pre-built and continuously updated, and the pictures in the picture vector database can be collected from the Internet and continuously updated. Of course, the second picture can also be searched from the Internet. In this application, by retrieving the second picture from the picture vector database, the speed of retrieving the second picture can be improved.

In one possible implementation, the method is applied to a response system based on a large language model (LLM); the operation information includes a target image, or the operation information includes a target image and a second text.

In this possible implementation, if the preview image contains a target image that the user is satisfied with, image retrieval can be performed based on the target image. Of course, retrieval can also be performed based on the target image and the extended text corresponding to the target image. If the preview image does not contain an image that the user is satisfied with, the target image can be used as an input image for the second image editing, and the second text can guide the cloud to edit the target image, and further provide the user with a second wave of preview images until the user selects a satisfactory image. From this possible implementation, it can be seen that in the response system, different models can be called through LLM to implement different functions such as image retrieval or image editing, thereby enriching the functions of the response system.

In one possible implementation, if the operation information includes a target image, correspondingly, the above step: retrieving a second image associated with the target image includes: calling an image editing retrieval model through LLM, and retrieving the associated second image based on the extended text corresponding to the target image and the target image.

In this possible implementation, the image editing retrieval model retrieves the second image based on the extended text corresponding to the target image and the target image, which can improve the accuracy of the retrieved second image.

In one possible implementation, if the second text indicates that the target image meets the user's expectations, correspondingly, the above step: retrieving the second image associated with the target image includes: calling the image editing retrieval model through LLM, and retrieving the associated second image based on the extended text corresponding to the target image, the second text and the target image.

In this possible implementation, the image editing retrieval model retrieves the second image based on the extended text corresponding to the target image, the second text and the target image, which can improve the accuracy of the retrieved second image.

In one possible implementation, if the second text indicates that the target image does not meet user expectations, correspondingly, the above step: retrieving the second image associated with the target image includes: calling the picture editing generation model through LLM to generate multiple preview images associated with the target image and the second text. The picture editing generation model can obtain multiple preview images associated with the target image and the second text based on the multiple extended texts obtained by expanding the second text and the target image to determine the second image.

In this possible implementation, the image editing generation model refers to a model that can use the second text and the target image to further generate the next wave of preview images. The generation process can be understood by referring to the process of generating preview images from the first text and the first image. In this way, multiple rounds of generating preview images can enable users to obtain satisfactory preview images, thereby improving the accuracy of the retrieved second image.

In a possible implementation, the LLM-based answering system also includes a text analysis model, and the text analysis model is used to analyze the second text to determine whether the target image meets the user's expectations.

In this possible implementation, the second text may instruct the LLM to call the picture editing retrieval model or the picture editing generation model. The text analysis model may learn whether the user is satisfied with the target picture by analyzing the second text. If the user is not satisfied, the text analysis model may instruct the LLM to call the picture editing generation model to continue to generate the next wave of preview pictures. If the user is satisfied, the text analysis model may instruct the LLM to call the picture editing retrieval model to generate the second picture. This can improve the efficiency of communication with users.

In a possible implementation, when receiving the second picture from the cloud, the method further includes: receiving link information associated with the second picture from the cloud.

In this possible implementation, the link information associated with the second picture may be description information of the product in the second picture and a purchase link, etc. In this way, the communication efficiency with the user can be improved.

A second aspect of the present application provides a method for image retrieval, including: sending a first image and a first text input by a user to a cloud, wherein the first text is used to adjust the first image; receiving multiple preview images from the cloud, wherein the multiple preview images are generated based on the first image and the first text; sending operation information input by the user to the cloud, wherein the operation information is used to retrieve a second image associated with a target image, wherein the target image is included in the multiple preview images; and receiving the second image from the cloud.

In this application, the user can input the first image and the first text on the client, and use the first text to guide the cloud to adjust the first image, thereby obtaining multiple preview images. The user can select a suitable target image from the preview images, and the cloud will perform searches based on the target images, which can narrow the search scope and improve the accuracy of image retrieval.

In a possible implementation, the method is applied to a response system based on a large language model (LLM); the operation information includes a target image, or the operation information includes a target image and a second text.

In this possible implementation, if the preview image contains a target image that the user is satisfied with, image retrieval can be performed based on the target image. Of course, retrieval can also be performed based on the target image and the extended text corresponding to the target image. If the preview image does not contain an image that the user is satisfied with, the target image can be used as an input image for the second image editing, and the second text can guide the cloud to edit the target image, and further provide the user with a second wave of preview images until the user selects a satisfactory image. From this possible implementation, it can be seen that in the response system, different models can be called through LLM to implement different functions such as image retrieval or image editing, thereby enriching the functions of the response system.

In a possible implementation, if the operation information includes a target image, the second image is retrieved based on the extended text of the first text and the target image by calling the image editing retrieval model through the LLM.

In this possible implementation, the image editing retrieval model is based on the extended text corresponding to the target image and the target retrieval second image, which can improve the accuracy of the retrieved second image.

In one possible implementation, if the second text indicates that the target image meets the user's expectations, the second image is retrieved by calling the image editing retrieval model through LLM based on the extended text corresponding to the target image, the second text and the target image.

In this possible implementation, the image editing retrieval model retrieves the second image based on the extended text corresponding to the target image, the second text and the target image, which can improve the accuracy of the retrieved second image.

In one possible implementation, if the second text indicates that the target image does not meet user expectations, the method also includes: receiving multiple preview images associated with the target image and the second text from the cloud, the multiple preview images associated with the target image and the second text are obtained by LLM calling a picture editing generation model, the picture editing generation model obtains the multiple preview images associated with the target image and the second text based on multiple extended texts of the second text and the target image; sending a third image to the cloud, or a third image and a third text; the third image, or the third image and the third text are used to determine the second image, and the third image is included in the multiple preview images associated with the target image and the second text.

In this possible implementation, the image editing generation model refers to a model that can use the extended text of the second text and the target image to further generate the next wave of preview images. The generation process can be understood by referring to the process of generating preview images from the first text and the first image. In this way, multiple rounds of generating preview images can enable users to obtain satisfactory preview images, thereby improving the accuracy of the retrieved second image.

In a possible implementation, when: receiving the second picture from the cloud, the method further includes: receiving link information associated with the second picture from the cloud.

In this possible implementation, the link information associated with the second picture may be description information of the product in the second picture and a purchase link, etc. In this way, the communication efficiency with the user can be improved.

In a third aspect of the present application, a cloud device is provided for executing the method in the first aspect or any possible implementation of the first aspect. Specifically, the cloud device includes a module or unit for executing the method in the first aspect or any possible implementation of the first aspect, such as a processing unit and a sending unit.

In a fourth aspect of the present application, a client is provided for executing the method in the second aspect or any possible implementation of the second aspect. Specifically, the client includes a module for executing the method in the second aspect or any possible implementation of the second aspect. Blocks or units, such as a sending unit and a receiving unit.

The fifth aspect of the present application provides a cloud device, including a transceiver, a processor and a memory, wherein the transceiver and the processor are coupled to the memory, and the memory is used to store programs or instructions. When the programs or instructions are executed by the processor, the client executes the method in the aforementioned first aspect or any possible implementation of the first aspect.

In a sixth aspect of the present application, a client is provided. The client may include at least one processor, a memory, and a communication interface. The processor is coupled to the memory and the communication interface. The memory is used to store instructions, the processor is used to execute the instructions, and the communication interface is used to communicate with other network elements under the control of the processor. When the instructions are executed by the processor, the processor executes the method in the second aspect or any possible implementation of the second aspect.

The seventh aspect of the present application provides a chip system, which includes one or more interface circuits and one or more processors; the interface circuit and the processor are interconnected by lines; the interface circuit is used to receive signals from the memory of the cloud device and send signals to the processor, and the signals include computer instructions stored in the memory; when the processor executes the computer instructions, the cloud device executes the method in the aforementioned first aspect or any possible implementation of the first aspect.

In an eighth aspect of the present application, a chip system is provided, which includes one or more interface circuits and one or more processors; the interface circuits and the processors are interconnected through lines; the interface circuits are used to receive signals from a client's memory and send signals to the processor, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, the client executes the method in the aforementioned second aspect or any possible implementation of the second aspect.

The ninth aspect of the present application provides a computer-readable storage medium on which a computer program or instruction is stored. When the computer program or instruction is executed on a computer device, the computer device executes the method in the aforementioned first aspect or any possible implementation of the first aspect.

The tenth aspect of the present application provides a computer-readable storage medium on which a computer program or instruction is stored. When the computer program or instruction is executed on a computer device, the computer device executes the method in the aforementioned second aspect or any possible implementation of the second aspect.

In an eleventh aspect of the present application, there is provided a computer device program product, which includes a computer device program code. When the computer device program code is executed on a computer device, the computer device executes the method in the aforementioned first aspect or any possible implementation of the first aspect.

The twelfth aspect of the present application provides a computer device program product, which includes a computer device program code. When the computer device program code is executed on a computer device, the computer device executes the method in the aforementioned second aspect or any possible implementation of the second aspect.

The thirteenth aspect of the present application provides a computer device cluster, comprising at least one computer device, each computer device comprising a processor and a memory; the processor of at least one computer device is used to execute instructions stored in the memory of at least one computer device, so that the computer device cluster executes the method in the aforementioned first aspect or any possible implementation of the first aspect.

In the fourteenth aspect of the present application, a retrieval system is provided, which includes a client and a cloud device, the cloud device is used to execute the method in the aforementioned first aspect or any possible implementation of the first aspect, and the client is used to execute the method in the aforementioned second aspect or any possible implementation of the second aspect.

Among them, the technical effects brought about by the third to fourteenth aspects or any possible implementation methods thereof can refer to the technical effects brought about by the first aspect or different possible implementation methods of the first aspect, and will not be repeated here.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1A is a schematic diagram of an architecture of a retrieval system provided in an embodiment of the present application;

FIG1B is a schematic diagram of image retrieval in a shopping scenario provided by an embodiment of the present application;

FIG1C is a schematic diagram of an architecture of a cloud system provided in an embodiment of the present application;

FIG2A is a schematic diagram of a structure of a terminal device provided in an embodiment of the present application;

FIG2B is a schematic diagram of a structure of a cloud device provided in an embodiment of the present application;

FIG3 is a schematic diagram of an embodiment of a method for image retrieval provided in an embodiment of the present application;

FIG4 is a schematic diagram of an example of an input interface of a client in an embodiment of the present application;

FIG5 is a schematic diagram of an example scenario of generating a preview image in an embodiment of the present application;

FIG6 is a schematic diagram of the relationship between a large language model and a text analysis model in an embodiment of the present application;

FIG7 is a schematic diagram of another embodiment of image retrieval in the embodiment of the present application;

FIG8 is a schematic diagram of an architecture of a server provided in an embodiment of the present application;

FIG9 is a schematic diagram of an architecture of a retrieval system in an embodiment of the present application;

FIG10 is a schematic diagram of a structure of a cloud device provided in an embodiment of the present application;

FIG. 11 is a schematic diagram of the structure of a client provided in an embodiment of the present application.

DETAILED DESCRIPTION

The following describes the embodiments of the present application in conjunction with the accompanying drawings. Obviously, the described embodiments are only embodiments of a part of the present application, rather than all embodiments. It is known to those skilled in the art that with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

The embodiment of the present application provides a method for image retrieval, which is used to improve the accuracy of image retrieval. The present application also provides corresponding devices, systems, computer-readable storage media, and computer program products, etc. The following are detailed descriptions.

For ease of understanding, the following briefly introduces the technical terms involved in the embodiments of the present application:

1. Text retrieval (Query): The text entered by the user in the search box is used to select the document candidate set or image candidate set that is most relevant to the text from a large-scale web document library (the document library can be a library of hundreds of billions of documents).

2. Image retrieval: Search based on the image input by the user and return images similar to the input image; also known as image search, which can be understood as searching for images by image.

3. Image editing retrieval: Image editing retrieval allows users to input images and text at the same time. The text can be used to edit and modify local information in the image. The images returned by the retrieval meet the content of the text and are as similar to the input image as possible.

4. Image editing generation: Image editing generation allows users to input images and text at the same time. The text can be used to edit and modify local information in the image. A new image is generated based on the image and text. The new image meets the text requirements, and other information is consistent with the input image.

5. Answering system: Generally refers to a system that searches for relevant documents and then understands the document information to answer user questions.

6. Conversational search and response system: A system that adds multi-round conversation capabilities to the search and response system and can understand the context, thereby realizing multi-round continuous response functions.

7. Large language model (LLM): After training, LLM internalizes knowledge into model parameters. LLM can directly generate answers to user queries. LLM can call image retrieval model, image editing retrieval model or image editing generation model to achieve different functions.

FIG. 1A is a schematic diagram of an architecture of a search provided in an embodiment of the present application.

As shown in FIG1A , the retrieval system provided in the embodiment of the present application includes a cloud and multiple clients, and the cloud can communicate with multiple clients through a network. Among them, the cloud can be software or services of a cloud platform, or software or services deployed on nodes in a network such as edge nodes. The cloud can run on an independent physical machine or on virtualized resources. The client can be a terminal device or an application, for example, the application runs on a terminal device for use by a user. The application can be a search engine, an answering application, or a shopping application, etc.

The retrieval system may be a browser search system, a search response system, or a shopping platform system, etc.

The client can send a query request to the cloud, the query request includes a first text and a first picture, and the first text is used to adjust the first picture. The cloud-side device on the cloud can generate multiple preview pictures based on the first picture and the first text from the client, and then send the multiple preview pictures to the client; then, based on the operation information from the client, retrieve a second picture associated with the target picture, the target picture is included in the multiple preview pictures; and then send the second picture to the client.

The above process takes a shopping scenario as an example. Referring to FIG. 1B , the first picture input by the user may be a picture 101 of “black short-sleeved shirt”, and the first text It could be "It would be better if there is a flower pattern" 102. The cloud can generate four preview pictures shown in Figure 1B based on the above-mentioned "black short-sleeved shirt" picture 101 and "It would be better if there is a flower pattern" 102. After the client receives the four preview pictures, the user can select one of the four preview pictures as the target picture. For example, in Figure 1B, the user selects "black short-sleeved shirt picture with evenly distributed flowers" 103 as the target picture. The cloud can retrieve a suitable second picture based on the "black short-sleeved shirt picture with evenly distributed flowers" 103. As shown in Figure 1B, the cloud returns two second pictures to the client, namely pictures 104 and 105. The flower distribution on the short-sleeved shirt in pictures 104 and 105 is highly similar to that in picture 103. Picture 104 shows a short-sleeved shirt with evenly distributed black flowers, and picture 105 shows a short-sleeved shirt with evenly distributed white flowers. Users can choose their favorite products to buy by comparing colors. The cloud can also provide users with more color options. These short-sleeved shirts with flowers in different colors can be presented in one picture.

In this application, the user can input the first image and the first text on the client, and use the first text to guide the cloud to adjust the first image, thereby obtaining multiple preview images. The user can select a suitable target image from the preview images, and the cloud will perform searches based on the target images, which can narrow the search scope and improve the accuracy of image retrieval.

The cloud-side device in the cloud in Figure 1A can be a working node or a scheduling node in the cloud system. After the scheduling node receives a query request from the client, the scheduling node can execute the corresponding retrieval process. The scheduling node can also assign the query request to one or more working nodes in the cloud system, and the corresponding retrieval process will be executed by one or more working nodes.

The function of the scheduling node can be implemented by software or hardware.

As an example of a software functional unit, the scheduling node may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the computing instance may be one or more. For example, the scheduling node may include code running on multiple hosts/virtual machines/containers. It should be noted that the multiple hosts/virtual machines/containers used to run the code may be distributed in the same region or in different regions. Furthermore, the multiple hosts/virtual machines/containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple data centers with similar geographical locations. Generally, a region may include multiple AZs.

Similarly, multiple hosts/virtual machines/containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in one region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

As an example of a hardware functional unit, the scheduling node may include at least one computing device, such as a server, etc. Alternatively, the scheduling node may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

The multiple computing devices included in the scheduling node can be distributed in the same area or in different areas. The multiple computing devices included in the scheduling node can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the scheduling node can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

A working node can be a physical machine, or a computing instance such as a virtual machine (VM) or a container. A working node can include one or more central processing units (CPUs) and graphics processing units (GPUs), etc. A working node can also be a CPU or a GPU.

The client may be a terminal device. The terminal device, also known as user equipment (UE), mobile station (MS), mobile terminal (MT), etc., is a device including a wireless communication function (providing voice/data connectivity to users), for example, a handheld device with a wireless connection function. At present, some examples of terminal devices are: mobile phones, tablet computers, laptops, PDAs, laptops, wireless routers, mobile internet devices (MID), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in the Internet of Vehicles, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, or wireless terminals in smart homes, etc. For example, the wireless terminal in the Internet of Vehicles may be a vehicle-mounted device, a vehicle-mounted device, a vehicle-mounted module, a vehicle, etc. Industrial Control The wireless terminal in the system may be a robot, etc. For example, the wireless terminal in the unmanned driving may be a drone. The terminal device may be a device running an Android system, an IOS system, a Windows system, or other systems. The terminal device may run an application that needs to render an application scene to obtain a two-dimensional image, such as a game application, a lock screen application, or a map application.

The cloud in FIG. 1A above may be located in a cloud system, and the architecture of the cloud system may be understood by referring to FIG. 1C. As shown in FIG. 1C, the cloud system includes a cloud platform and basic resources. The cloud platform includes a cloud platform manager, and the scheduling node described above may be the cloud platform manager in FIG. 1C. The basic resources may include multiple servers, each of which may be a working node, or each server may include multiple working nodes.

The working node in FIG1C may be a computing device card or a virtual machine (VM). The computing device card may be at least one of a central processing unit (CPU), a graphics processing unit (GPU), and a neural network processor (NPU).

The cloud platform manager will maintain or regularly collect information about each work node in the basic resources, such as the resource usage on each work node (resource usage rate or resource idle rate), etc. This information can be used as auxiliary decision-making information when allocating query requests.

The cloud platform manager can receive query requests from the client, and then the cloud platform manager can execute the corresponding retrieval process. The cloud platform manager can also assign the query request to one or more working nodes in the cloud system, and the one or more working nodes can execute the corresponding retrieval process.

After the working node completes the retrieval, the cloud platform manager can return the preview image and the second image to the client.

The above-mentioned client takes a terminal device as an example. The structure of the terminal device provided in the embodiment of the present application can be understood by referring to FIG. 2A below, and the structure of the cloud device in the cloud can be understood by referring to FIG. 2B below.

Please refer to Figure 2A, which is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application. As shown in Figure 2A, the terminal device may include a processor 101, a transceiver 102, a memory 103, a display 104 and a bus 105. The processor 101, the transceiver 102, the memory 103 and the display 104 are interconnected via the bus 105. In an embodiment of the present application, the processor 101 is used to control and manage the actions of the terminal device 10, for example, the processor 101 is used to respond to the process of the user inputting a query request. The transceiver 102 is used to support the communication of the terminal device 10, for example: the transceiver 102 can execute the steps of sending a query request and receiving a preview image and a second image. The memory 103 is used to store the program code and data of the terminal device 10, and the display 104 is used to display the preview image and the second image.

Among them, the processor 101 can be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, and the like. The bus 105 can be a peripheral component interconnect standard (Peripheral Component Interconnect, PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in FIG. 2A, but it does not mean that there is only one bus or one type of bus.

FIG. 2A above introduces the structure of the terminal device, and the structure of the cloud device is introduced below in conjunction with FIG. 2B .

FIG2B is a possible logical structure diagram of a cloud device provided in an embodiment of the present application. As shown in FIG2B , the cloud device 20 provided in an embodiment of the present application includes: a processor 201, a communication interface 202, a memory 203, and a bus 204. The processor 201, the communication interface 202, and the memory 203 are interconnected via the bus 204. In an embodiment of the present application, the processor 201 is used to control and manage the actions of the cloud device 20. For example, the processor 201 is used to generate multiple preview images based on a first image and a first text from a client. The communication interface 202 is used to support the cloud device 20 to communicate. For example, the communication interface 202 can receive the steps of the first image and the first text, and send the preview image and the second image. The memory 203 is used to store the program code and data of the cloud device 20.

Among them, the processor 201 can be a central processing unit, a general processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, and the like. The bus 204 can be a peripheral component interconnect standard (Peripheral Component Interconnect, PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in FIG. 2B, but it does not mean that there is only one bus or one type of bus.

The following describes the image retrieval method provided in the embodiment of the present application. The content involved in the cloud execution in the method can be executed by the cloud The content related to client execution can be executed by the client or by a component of the client (such as a processor, chip, or chip system).

FIG3 is a schematic diagram of an embodiment of a method for image retrieval provided in an embodiment of the present application.

301. The client sends the first picture and the first text input by the user to the cloud. Correspondingly, the cloud receives the first picture and the first text from the client.

In the present application, the first image and the first text may be sent through one query request or may be sent separately.

The first text is used to adjust the first picture.

The interface for inputting the first picture and the first text on the client can be understood by referring to Figure 4. As shown in Figure 4, the first text can be entered in the text input box 401, the first picture can be selected through the picture option box 402, and then the first picture and the first text can be sent by clicking the send button 403.

302. The cloud generates multiple preview images according to the first image and the first text from the client.

The process of step 302 may be: expanding the first text in the cloud to obtain a plurality of extended texts; and then generating a preview image corresponding to each extended text according to each extended text and the first image.

The cloud can expand around the content in the first text. For example, if the first text is a flower pattern configured on the first picture, the text can be expanded around the type of flower, the position of the flower on the first picture, etc.

The extended text may be generated by the cloud based on the first text, or may be a text found by the cloud from a text design database whose similarity to the first text meets the first condition. Alternatively, the extended text may be a text found by the cloud from a text design database whose similarity to the first text and design popularity meet the second condition.

The text design database may be pre-built and continuously updated. The texts in the text design database may be collected based on historical text retrieval records. Of course, the texts in the text design database may also be obtained through other means.

In the present application, the first text can be used to search for texts with high similarity to the first text in the text design database. The first condition can be a threshold value. Taking the similarity value in the range of 0 to 1 as an example, the threshold value can be 0.5 or other values. Taking 0.5 as an example, texts with a similarity greater than 0.5 can be understood as texts whose similarity to the first text meets the first condition. The first condition can also be a restriction condition on the number of texts, such as: the top N texts with the highest similarity to the first text are texts whose similarity to the first text meets the first condition. Wherein, N is an integer greater than 1.

In the present application, the first text can be used to search for texts with high similarity to the first text in the text design database. And determine the popularity of these texts with high similarity, and then, the extended text can be determined by combining the similarity and popularity. Each text in the text design database can be pre-stated by the number of times it is used. The value of popularity can be a value between 0 and 1. The second condition can include a threshold value or two threshold values. When it is a threshold value, the similarity and popularity of the text can be weighted to obtain a weighted value (the weight of the similarity is usually greater than the popularity). If the weighted value is greater than this threshold value, it means that the text meets the second condition and belongs to the extended text of the first text. If the second condition is two threshold values, one threshold value can be a similarity threshold value, such as: 0.5; one threshold value is a threshold value for the number of extended texts, such as: N, N is a positive integer. If the similarity between the text found and the first text is greater than 0.5, the extended texts with similarity greater than 0.5 can be sorted according to the size of popularity, and then the N extended texts with the largest popularity are selected as the final selected extended texts.

The above process of generating a preview image through a text design database can also be understood by referring to FIG. 5 .

As shown in Figure 5, the cloud can compare the similarity of the first text "It would be better with a flower pattern" with the text in the text design database. If the first condition requires that the similarity between the extended text and the first text reaches 0.7, the cloud can match four extended texts such as those shown in Figure 5 from the text design database, namely "multiple small flowers in the center", "small flower pattern in the upper left corner", "a red rose in the center", and "multiple blooming flowers".

The similarities between the four extended texts and the first text can be understood by referring to Table 1, as shown in Table 1:

Table 1: Similarity between the extended text and the first text

After the extended text is determined in the cloud, a picture editing generation model can be used according to each extended text and the first picture. The first picture and each extended text can be input into the picture editing generation model to generate different design schemes, and then a preview picture can be generated according to the different design schemes.

In addition, the cloud can also determine the extended text based on the similarity and popularity of the text. For example, as shown in FIG5 , according to the similarity requirement with the first text “It would be better with a flower pattern”, multiple texts with high similarity can be found from the text design database, and then the extended text can be further determined based on the popularity of each text with high similarity. For example, texts with a similarity higher than 0.7 are sorted in descending order of popularity, and then the texts ranked in the top N (N=4) are selected as extended texts, and the four extended texts in Table 2 can be obtained.

Table 2: Similarity between the extended text and the first text, and the popularity of the extended text

After obtaining the extended text, the preview image can be generated by inputting the extended text and the first image into the image editing generation model.

In an embodiment of the present application, by expanding the text, the preview images obtained can provide preview images that meet the user's requirements in the first text from multiple angles, thereby increasing the richness of the preview images within the scope of the user's requirements.

303. The cloud sends multiple preview images to the client. Correspondingly, the client receives multiple preview images from the cloud.

304. The client sends the operation information input by the user to the cloud. Correspondingly, the cloud receives the operation information from the client.

The operation information is used to retrieve a second image associated with a target image, and the target image is included in a plurality of preview images.

In an embodiment of the present application, the operation information includes a target image, or the operation information includes a target image and a second text.

In a response system based on a large language model (LLM), if the operation information includes a target image but not a second text, the LLM can call a picture editing retrieval model to retrieve the second image based on the target image and the extended text corresponding to the target image.

The answering system may also include a text analysis model. The LLM may call the text analysis model to analyze the second text. Then, based on the analysis result of the second text, the LLM may decide to call the image editing retrieval model or the image editing generation model.

The structure of the text analysis model and its relationship with the LLM can be understood by referring to FIG6. As shown in FIG6, the text analysis model can include a sentiment analysis model and an entity extraction model; wherein the sentiment analysis model can analyze the tone and feelings of the user in the text, and the entity extraction model can analyze the entities in the text, and the entities can be objects in the text. After the text analysis model analyzes the second text, the analysis result can be returned to the LLM.

305. The cloud retrieves a second image associated with the target image according to the operation information from the client, where the target image is included in the plurality of preview images.

In step 305, if the operation information does not include the second text, the cloud can call the image editing retrieval model through LLM to retrieve the associated second image based on the extended text corresponding to the target image and the target image.

If the operation information includes a second text, the process can be understood by referring to Figure 7. The text analysis model 701 analyzes the second text to determine whether the target image meets the user's expectations. If the second text is a sentence such as "pretty good", "not bad", "it's what I want", etc. that indicates user satisfaction, it means that the target image meets the user's expectations. Then LLM702 calls the picture editing retrieval model 703, and the picture editing retrieval model 703 retrieves the second image according to the extended text corresponding to the target image and the target image.

If the second text is a sentence indicating user dissatisfaction, such as "there is not what I want", "the distribution of flowers is too scattered", etc., it means that the target image does not meet the user's expectations, and LLM702 calls the picture editing generation model 704, which generates multiple preview images associated with the target image and the second text. The generation process of multiple preview images associated with the target image and the second text can be understood by referring to the process of generating preview images from the first text and the first image. Then the cloud can send this wave of preview images to the client. If the user selects a satisfactory target image from this wave of preview images, the cloud can retrieve the second image based on the target image that the user is satisfied with. If the user is still not satisfied, the above process of generating preview images can be repeated until the target image that the user is satisfied with is obtained.

The image editing retrieval model can retrieve the second image from the image vector database. The image vector database can be pre-built and continuously updated. The images in the image vector database can be collected from the Internet and can be continuously updated. The second picture may also be searched from the Internet. In the present application, by retrieving the second picture from the picture vector database, the speed of retrieving the second picture can be improved.

306. The cloud sends the second picture to the client. Correspondingly, the client receives the second picture from the cloud.

In the embodiment of the present application, when the cloud sends the second picture, it can also send the link information associated with the second picture. Correspondingly, when the client receives the second picture, it also receives the link information associated with the second picture from the cloud.

The link information associated with the second picture may be description information of the product in the second picture and a purchase link, etc. In this way, the communication efficiency with the user can be improved.

In this application, the user can input the first image and the first text on the client, and use the first text to guide the cloud to adjust the first image, thereby obtaining multiple preview images. The user can select a suitable target image from the preview images, and the cloud will perform searches based on the target images, which can narrow the search scope and improve the accuracy of image retrieval.

In the embodiment of the present application, the execution process in the cloud can be configured in the platform software of machine learning and deep learning, and can be a program code deployed on the server hardware. Taking the architecture shown in Figure 8 as an example, the program code in the cloud of the present application can exist in the runtime engine, memory management and communication management modules of the above-mentioned platform software, as well as outside the existing modules. During operation, the program code of the present application can run in the host memory and/or graphics processing unit (GPU) memory of the server. The architecture shown in Figure 8 is the implementation form in the server and platform software in the embodiment of the present application, wherein the hardware layer includes GPU and memory, and in addition, the memory will store a text design database and a picture vector database. The software layer includes LLM, picture editing generation model and picture editing retrieval model, and of course, the software layer can also include a text analysis model. The calling relationship between LLM, picture editing generation model, picture editing retrieval model and text analysis model can be understood by referring to the previous introduction. When the server executes the process of the present application, it will call the text design database in the memory through the GPU to generate extended text, and call the picture in the picture vector database to obtain the preview picture. These processes can be understood by referring to the previous introduction.

The text design database and the image vector database in the embodiment of the present application can be obtained through offline training and can be updated online. As shown in Figure 9, the retrieval system is roughly divided into an offline system, an online system and a storage module. The main task of the offline system is to construct design texts and product image feature vectors based on Internet product images and texts, and to screen the design texts and product images. The process may include: deduplication of product images, and generating image representation vectors based on the image pre-training model, constructing a product image index, and constructing a picture vector database. Based on the product text, the product attributes are classified and the specific content is extracted; after completion, deduplication and text cleaning are performed to construct a text design database.

In the embodiment of the present application, the offline system can also retrain the image editing generation model and the image editing retrieval model based on the product images and texts.

The main task of the online system is to generate preview images and retrieve products based on the image editing generation model and image editing retrieval model according to user requests. The storage module is mainly used to store design text, product image representation vectors, LLM large model, image editing retrieval model and image editing generation model for offline and online system calls.

After receiving the user input request, the online system passes it to LLM for decision-making; LLM analyzes the user input and then decides whether to start the image editing retrieval process. If started, the image editing generation module is called.

The picture editing generation module expands the text input by the user to generate multiple extended texts, and generates corresponding preview pictures based on the image editing generation model and returns them to the LLM.

LLM responds to the user's request based on the generated preview image; the user provides feedback on the generated preview image.

LLM passes user feedback to the text analysis model, which obtains key information from the user feedback and returns the analysis results to LLM.

Based on the analysis results returned by the text analysis module, LLM decides whether to continue calling the image editing generation module or the image editing retrieval module; if the user is satisfied, LLM calls the image editing retrieval module; if the user is not satisfied, LLM calls the image editing generation module to continue generating the next wave of preview images.

Once the user is satisfied with the generated preview image, the image editing retrieval model performs image editing retrieval with the target image selected by the user and the corresponding text as input, and returns the retrieved image to the LLM.

LLM combines the retrieved images to reply to the user.

After the program code of the embodiment of the present application is deployed in the cloud, it can provide external services through a remote link, such as: a user downloads an application (APP), and the user can interact with the cloud through the APP to obtain a satisfactory picture.

The solution provided by the embodiment of the present application combines the generation of preview images with product retrieval: based on the images and texts input by the user, a preview is generated through image editing generation technology, so that the user can get satisfactory images. In addition, the present application recommends expansion solutions to users through text expansion, realizes diversified recommendations, and helps to collect the user's editing intentions, solving the problem that the user inputs a short text and has a large search range. In addition, retrieval can be performed based on images that the user is satisfied with, realizing accurate retrieval of images.

The above introduces the image retrieval system and the image retrieval method. The following introduces the relevant devices in conjunction with the accompanying drawings.

As shown in FIG. 10 , a structure of a cloud device 100 provided in an embodiment of the present application includes:

The processing unit 1001 is used to generate a plurality of preview images according to a first image and a first text from a client, wherein the first text is used to adjust the first image.

The sending unit 1002 is used to send multiple preview pictures to the client.

The processing unit 1001 is further configured to retrieve a second image associated with the target image according to operation information from the client, where the target image is included in the plurality of preview images.

The sending unit 1002 is further configured to send the second image to the client.

In an embodiment of the present application, a user can input a first image and a first text on the client, and use the first text to guide the cloud to adjust the first image, thereby obtaining multiple preview images. The user can select a suitable target image from the preview images, and the cloud will perform a search based on the target image, which can narrow the search scope and improve the accuracy of image retrieval.

Optionally, the processing unit 1001 is configured to expand the first text to obtain multiple extended texts; and generate a preview image corresponding to each extended text according to each extended text and the first image.

Optionally, the multiple extended texts are texts in a text design database whose similarity with the first text satisfies a first condition.

Optionally, the multiple extended texts are texts in a text design database whose similarity and design popularity with the first text meet the second condition.

Optionally, the second picture is a picture retrieved from a picture vector database.

Optionally, the cloud device 100 is applied to a response system based on a large language model (LLM); the operation information includes a target image, or the operation information includes a target image and a second text.

Optionally, if the operation information includes a target image, the processing unit 1001 is configured to call the image editing retrieval model through the LLM, and retrieve an associated second image based on the extended text corresponding to the target image and the target image.

Optionally, if the second text indicates that the target image meets user expectations, the processing unit 1001 is used to call the image editing retrieval model through the LLM to retrieve the associated second image based on the extended text corresponding to the target image, the second text and the target image.

Optionally, if the second text indicates that the target image does not meet user expectations, the processing unit 1001 is used to call the picture editing generation model through LLM to generate multiple preview images associated with the target image and the second text. The picture editing generation model is used to obtain multiple preview images associated with the target image and the second text based on multiple extended texts of the second text and the target image to determine the second image.

Optionally, the LLM-based answering system further includes a text analysis model, and the text analysis model is used to analyze the second text to determine whether the target image meets user expectations.

Optionally, the sending unit 1002 is further configured to send link information associated with the second image to the client.

In the embodiment of the present application, the operations performed by each unit in the cloud 100 are similar to those described in the embodiments shown in Figures 3 to 9 above, and will not be repeated here.

As shown in FIG. 11 , the embodiment of the present application further provides a structure of a client 110 including:

The sending unit 1101 is used to send the first picture and the first text input by the user to the cloud, where the first text is used to adjust the first picture.

The receiving unit 1102 is used to receive multiple preview images from the cloud, where the multiple preview images are generated based on the first image and the first text.

The sending unit 1101 is used to send the operation information input by the user to the cloud, where the operation information is used to retrieve a second image associated with the target image, where the target image is included in the multiple preview images.

The receiving unit 1102 is configured to receive a second image from the cloud.

Optionally, the client 110 is applied to a response system based on a large language model (LLM); the operation information includes a target image, or the operation information includes a target image and a second text.

Optionally, if the operation information includes a target image, the second image is retrieved based on the extended text of the target image and the target image by calling the image editing retrieval model through the LLM.

Optionally, if the second text indicates that the target image meets the user's expectations, the second image is retrieved based on the extended text of the target image, the second text and the target image by calling the image editing retrieval model through LLM.

Optionally, if the second text indicates that the target image does not meet user expectations, the receiving unit 1102 is further used to receive multiple preview images associated with the target image and the second text from the cloud, and the multiple preview images associated with the target image and the second text are obtained by LLM calling the picture editing generation model, and the picture editing generation model is used to obtain multiple preview images associated with the target image and the second text based on multiple extended texts of the second text and the target image.

The sending unit 1101 is also used to send a third picture, or a third picture and a third text to the cloud. The third picture, or the third picture and the third text are used to determine the second picture, and the third picture is included in multiple preview pictures associated with the target picture and the second text.

Optionally, the receiving unit 1102 is further configured to receive link information associated with the second image from the cloud.

In the embodiment of the present application, the operations performed by each unit in the client 110 are similar to those described in the embodiments shown in the aforementioned Figures 3 to 9, and will not be repeated here.

In another embodiment of the present application, a computer-readable storage medium is also provided, in which computer execution instructions are stored. When the processor of the cloud device executes the computer execution instructions, the cloud device executes the steps executed by the cloud in Figures 3 to 9 above.

In another embodiment of the present application, a computer-readable storage medium is provided, in which computer-executable instructions are stored. When the processor of the client executes the computer-executable instructions, the client executes the steps executed by the client in Figures 3 to 9 above.

In another embodiment of the present application, a computer program product is also provided. The computer program product includes a computer program code. When the computer program code is executed on a computer, the computer device executes the steps executed by the cloud or client in Figures 3 to 9 above.

In another embodiment of the present application, a chip system is also provided, which includes one or more interface circuits and one or more processors; the interface circuit and the processor are interconnected by lines; the interface circuit is used to receive signals from the memory of the terminal and send signals to the processor, and the signals include computer instructions stored in the memory; when the processor executes the computer instructions, the terminal executes the steps performed by the cloud device or the client in the above-mentioned Figures 3 to 9. In one possible design, the chip system may also include a memory, which is used to store program instructions and data necessary for the control device. The chip system may be composed of chips, or may include chips and other discrete devices.

In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated units may be implemented in whole or in part by software, hardware, firmware, or any combination thereof.

When the integrated unit is implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

Claims (26)

A method for image retrieval, characterized by comprising: generating a plurality of preview images according to a first image and a first text from a client, wherein the first text is used to adjust the first image; Sending the plurality of preview images to the client; Retrieving a second picture associated with a target picture according to the operation information from the client, wherein the target picture is included in the plurality of preview pictures; Send the second picture to the client. The method according to claim 1, characterized in that generating a plurality of preview images according to the first image and the first text from the client comprises: Expanding the first text to obtain a plurality of expanded texts; A preview image corresponding to each extended text is generated according to each extended text and the first image. The method according to claim 2 is characterized in that the multiple extended texts are texts in a text design database whose similarity with the first text meets a first condition. The method according to claim 2 is characterized in that the multiple extended texts are texts in a text design database whose similarity and design popularity with the first text meet a second condition. The method according to any one of claims 1-4 is characterized in that the second image is a image retrieved from an image vector database. The method according to any one of claims 1 to 5, characterized in that the method is applied to a response system based on a large language model (LLM); The operation information includes the target image, or the operation information includes the target image and a second text. The method according to claim 6, characterized in that if the operation information includes the target image, correspondingly, retrieving a second image associated with the target image comprises: The picture editing retrieval model is called by the LLM to retrieve a second picture associated with the target picture based on the extended text corresponding to the target picture and the target picture. The method according to claim 6, characterized in that if the second text indicates that the target image meets the user's expectations, correspondingly, retrieving the second image associated with the target image comprises: The picture editing retrieval model is called by the LLM to retrieve the associated second picture based on the extended text corresponding to the target picture, the second text and the target picture. The method according to claim 6, characterized in that if the second text indicates that the target image does not meet the user's expectations, correspondingly, retrieving a second image associated with the target image comprises: The LLM calls a picture editing generation model to generate multiple preview pictures associated with the target picture and the second text. The picture editing generation model is used to obtain multiple preview pictures associated with the target picture and the second text based on multiple extended texts of the second text and the target picture to determine the second picture. The method according to any one of claims 6 to 9 is characterized in that the LLM-based response system also includes a text analysis model, and the text analysis model is used to analyze the second text to determine whether the target image meets user expectations. The method according to any one of claims 1 to 10, characterized in that when sending the second picture to the client, the method further comprises: Send link information associated with the second image to the client. A method for image retrieval, characterized by comprising: Sending a first picture and a first text input by a user to a cloud, where the first text is used to adjust the first picture; Receiving a plurality of preview images from the cloud, where the plurality of preview images are generated based on the first image and the first text; Sending operation information input by the user to the cloud, wherein the operation information is used to retrieve a second image associated with a target image, wherein the target image is included in the plurality of preview images; Receive the second picture from the cloud. The method according to claim 12 is characterized in that the method is applied to a response system based on a large language model (LLM). middle; The operation information includes the target image, or the operation information includes the target image and a second text. The method according to claim 13 is characterized in that if the operation information includes the target image, the second image is retrieved based on the extended text of the target image and the target image by calling the image editing retrieval model through the LLM. The method according to claim 13 is characterized in that if the second text indicates that the target image meets the user's expectations, the second image is retrieved by calling the image editing retrieval model through the LLM based on the extended text of the target image, the second text and the target image. The method according to claim 13, characterized in that if the second text indicates that the target image does not meet the user's expectations, the method further comprises: Receiving multiple preview images associated with the target image and the second text from the cloud, wherein the multiple preview images associated with the target image and the second text are obtained by the LLM calling a picture editing generation model, and the picture editing generation model is used to obtain the multiple preview images associated with the target image and the second text based on multiple extended texts of the second text and the target image; A third image, or a third image and a third text are sent to the cloud; the third image, or the third image and the third text are used to determine the second image, and the third image is included in a plurality of preview images associated with the target image and the second text. The method according to any one of claims 12 to 16, characterized in that when receiving the second picture from the cloud, the method further comprises: Receive link information associated with the second image from the cloud. A cloud device, comprising: A processing unit, configured to generate a plurality of preview images according to a first image and a first text from a client, wherein the first text is used to adjust the first image; A sending unit, used for sending a plurality of preview images generated by the processing unit to the client; The processing unit is further configured to retrieve a second image associated with a target image according to the operation information from the client, wherein the target image is included in the plurality of preview images; The sending unit is further used to send the second picture retrieved by the processing unit to the client. A client, characterized by comprising: A sending unit, configured to send a first image and a first text input by a user to a cloud, wherein the first text is used to adjust the first image; A receiving unit, configured to receive a plurality of preview images from the cloud, where the plurality of preview images are generated based on the first image and the first text; The sending unit is further used to send operation information input by the user to the cloud, where the operation information is used to retrieve a second image associated with a target image, where the target image is included in the plurality of preview images; The receiving unit is further configured to receive the second image from the cloud. A cloud device, characterized in that it includes: a communication interface, a processor and a memory, wherein the communication interface and the processor are coupled to the memory, and the memory is used to store programs or instructions. When the program or instructions are executed by the processor, the cloud device executes the method as described in any one of claims 1-11. A client, characterized in that it includes: a transceiver, a processor and a memory, wherein the transceiver and the processor are coupled to the memory, and the memory is used to store programs or instructions. When the program or instructions are executed by the processor, the client executes the method described in any one of claims 12-17. A computer-readable storage medium, characterized in that instructions are stored in the computer-readable storage medium, and when the instructions are executed on a computer device, the computer device executes the method according to any one of claims 1 to 17. A computer program product, characterized in that the computer program product comprises computer program code, and when the computer program code is run on a computer device, the computer device is caused to execute the method according to any one of claims 1 to 17. A chip system, characterized in that the chip system includes one or more interface circuits and one or more processors; the interface circuit and the processor are interconnected by a line; the interface circuit is used to receive a signal from a memory of a cloud device and send a signal to the processor, the signal including a computer instruction stored in the memory; when the processor executes the computer instruction, the cloud device executes the method according to any one of claims 1 to 11; or the interface circuit is used to receive a signal from a memory of a client and send a signal to the processor, the signal including a computer instruction stored in the memory Computer instructions stored in; when the processor executes the computer instructions, the client executes the method as described in any one of claims 12-17. A computer device cluster, characterized in that it comprises at least one computer device, each of which comprises a processor and a memory; The processor of the at least one computer device is used to execute instructions stored in the memory of the at least one computer device, so that the computer device cluster executes the method according to any one of claims 1 to 11. A retrieval system, characterized in that the retrieval system includes a client and a cloud device; The cloud device is used to execute the method as described in any one of claims 1 to 11 above; The client is used to execute the method as described in any one of claims 12 to 17 above.