WO2025023530A1 - Method, computer device and computer program for generating video snapshots by using image hashing - Google Patents

Abstract

A method, a computer device and a computer program for generating a video snapshot by using image hashing are disclosed. The method for generating video snapshots comprises a step of removing redundant frames from among frames included in a video and generating snapshots by using remaining frames, the snapshots being generated by maintaining frames that include target information defined by a user.

Description

Method for generating video snapshots using image hash, computer device, and computer program

The description below describes the technique for creating a video snapshot.

When analyzing a video, if a method that requires a lot of computation, such as a deep learning model, is used for every frame, video analysis consumes a significant amount of time and resources.

One technique to solve this problem is to remove some of the frames contained in the video and create a snapshot consisting of only the remaining frames, which can then be used for video analysis.

In addition to video analysis, snapshot generation technology can be used to provide highlight reels for videos.

For example, Korean Patent Registration No. 10-1174176 (registration date August 8, 2012) discloses a technology for generating a thumbnail that summarizes the entire content of a video file by sampling and selecting snapshots at multiple points in the entire running time of the video file.

By removing redundant frames, it is possible to create a snapshot in which key frames containing target information are not missed when creating a snapshot.

A zero-shot technique can be used to determine whether a frame contains a user-defined class using a hash.

A method for generating a video snapshot executed on a computer device, the computer device including at least one processor configured to execute computer-readable instructions contained in a memory, the method for generating a video snapshot including the step of removing, by the at least one processor, redundant frames from frames included in a video and generating a snapshot with the remaining frames, while maintaining frames including target information defined by a user to generate the snapshot.

According to one aspect, the generating step may generate a snapshot of the video using a hash generated through a machine learning model.

According to another aspect, the generating step can generate a snapshot of the video through a multi-modal model learned by expressing embeddings of different modalities in a single vector space.

According to another aspect, the generating step may include: a step of calculating a first similarity indicating image similarity between frames included in the video; a step of calculating a second similarity indicating a similarity relationship with a prompt registered as the target information for each frame included in the video; and a step of selecting some frames included in the video as the duplicate frames based on the first similarity and the second similarity.

According to another aspect, the step of calculating the first similarity calculates the first similarity by using a multi-modal model learned by expressing embeddings of different modalities in a single vector space, and the first similarity can be calculated through the vector distance between image embeddings that embed features of images corresponding to each frame.

According to another aspect, the step of calculating the second similarity may include a step of registering the prompt as content in any one of text, image, and audio forms.

According to another aspect, the step of calculating the second similarity calculates the second similarity by using a multi-modal model learned by expressing embeddings of different modalities in a single vector space, and the second similarity can be calculated through the vector distance between the prompt embedding that embeds the features of the prompt registered as the target information and the image embedding that embeds the features of the image corresponding to each frame.

According to another aspect, the selecting step may include: a step of clustering frames having a similarity of a certain level or higher based on the first similarity; a step of selecting frames remaining after excluding a certain number of frames per cluster as candidate frames of the duplicate frame; and a step of excluding frames having the second similarity of a certain level or higher among the candidate frames from the selection target of the duplicate frame.

According to another aspect, the multi-modal model may be a model in which the similarity of embeddings for data pairs is learned using contrastive learning.

According to another aspect, the multi-modal model may be a vision-language pre-trained (VLP) model learned through contrastive learning.

A computer program stored in a non-transitory computer-readable recording medium is provided for executing the above video snapshot generation method on the computer device.

In a computer device, at least one processor configured to execute computer-readable instructions contained in a memory, the at least one processor processing a process of removing duplicate frames from frames contained in a video and generating a snapshot with the remaining frames, while maintaining frames containing target information defined by a user to generate the snapshot.

FIG. 1 is a diagram illustrating an example of a network environment according to one embodiment of the present invention.

FIG. 2 is a block diagram illustrating an example of a computer device according to one embodiment of the present invention.

Figure 3 illustrates an example of frames that make up a video.

Figure 4 illustrates an example of creating a video snapshot.

FIG. 5 is a flowchart illustrating an example of a method that a computer device according to an embodiment of the present invention can perform.

FIGS. 6 and 7 are exemplary drawings for explaining a searchable multi-modal model according to one embodiment of the present invention.

FIG. 8 is a flowchart illustrating an example of a duplicate frame filtering process according to one embodiment of the present invention.

Figures 9 and 10 illustrate examples of snapshot creation using a prompt according to one embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

Embodiments of the present invention relate to techniques for generating video snapshots.

Embodiments including those specifically disclosed herein can generate a snapshot in a form that retains key frames containing target information when generating a video snapshot.

A video snapshot generation system according to embodiments of the present invention can be implemented by at least one computer device, and a video snapshot generation method according to embodiments of the present invention can be performed by at least one computer device included in the video snapshot generation system. At this time, a computer program according to an embodiment of the present invention can be installed and run on the computer device, and the computer device can perform the video snapshot generation method according to embodiments of the present invention under the control of the run computer program. The above-described computer program can be stored in a computer-readable recording medium so as to be combined with the computer device and cause the computer to execute the video snapshot generation method.

FIG. 1 is a diagram illustrating an example of a network environment according to an embodiment of the present invention. The network environment of FIG. 1 represents an example including a plurality of electronic devices (110, 120, 130, 140), a plurality of servers (150, 160), and a network (170). FIG. 1 is an example for explaining the invention, and the number of electronic devices or servers is not limited to FIG. 1. In addition, the network environment of FIG. 1 merely describes one example of environments applicable to the present embodiments, and the environment applicable to the present embodiments is not limited to the network environment of FIG. 1.

The plurality of electronic devices (110, 120, 130, 140) may be fixed terminals or mobile terminals implemented as computer devices. Examples of the plurality of electronic devices (110, 120, 130, 140) include smart phones, mobile phones, navigation devices, computers, laptops, digital broadcasting terminals, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), tablet PCs, etc. For example, in FIG. 1, the shape of a smart phone is shown as an example of the electronic device (110), but in the embodiments of the present invention, the electronic device (110) may actually mean one of various physical computer devices that can communicate with other electronic devices (120, 130, 140) and/or servers (150, 160) via a network (170) using a wireless or wired communication method.

The communication method is not limited, and may include not only a communication method that utilizes a communication network (for example, a mobile communication network, a wired Internet, a wireless Internet, a broadcasting network) that the network (170) may include, but also a short-range wireless communication between devices. For example, the network (170) may include any one or more of a network such as a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), and the Internet. In addition, the network (170) may include any one or more of a network topology including, but not limited to, a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree, or a hierarchical network.

Each of the servers (150, 160) may be implemented as a computer device or multiple computer devices that communicate with multiple electronic devices (110, 120, 130, 140) through a network (170) to provide commands, codes, files, contents, services, etc. For example, the server (150) may be a system that provides a service (e.g., a video analysis service, etc.) to multiple electronic devices (110, 120, 130, 140) connected through a network (170).

FIG. 2 is a block diagram illustrating an example of a computer device according to an embodiment of the present invention. Each of the plurality of electronic devices (110, 120, 130, 140) or servers (150, 160) described above can be implemented by the computer device (200) illustrated in FIG. 2.

The computer device (200) may include a memory (210), a processor (220), a communication interface (230), and an input/output interface (240), as illustrated in FIG. 2. The memory (210) may be a computer-readable storage medium, and may include a random access memory (RAM), a read only memory (ROM), and a permanent mass storage device such as a disk drive. Here, the permanent mass storage devices such as the ROM and the disk drive may be included in the computer device (200) as a separate permanent storage device distinct from the memory (210). In addition, the memory (210) may store an operating system and at least one program code. These software components may be loaded into the memory (210) from a computer-readable storage medium separate from the memory (210). The separate computer-readable storage medium may include a computer-readable storage medium such as a floppy drive, a disk, a tape, a DVD/CD-ROM drive, a memory card, etc. In another embodiment, the software components may be loaded into the memory (210) via a communication interface (230) other than a computer-readable recording medium. For example, the software components may be loaded into the memory (210) of the computer device (200) based on a computer program that is installed by files received over a network (170).

The processor (220) may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input/output operations. The instructions may be provided to the processor (220) by the memory (210) or the communication interface (230). For example, the processor (220) may be configured to execute instructions received according to program code stored in a storage device such as the memory (210).

The communication interface (230) may provide a function for the computer device (200) to communicate with other devices (for example, the storage devices described above) via the network (170). For example, requests, commands, data, files, etc. generated by the processor (220) of the computer device (200) according to program codes stored in a recording device such as the memory (210) may be transmitted to other devices via the network (170) under the control of the communication interface (230). Conversely, signals, commands, data, files, etc. from other devices may be received by the computer device (200) via the communication interface (230) of the computer device (200) via the network (170). Signals, commands, data, etc. received via the communication interface (230) may be transmitted to the processor (220) or the memory (210), and files, etc. may be stored in a storage medium (the permanent storage device described above) that the computer device (200) may further include.

The input/output interface (240) may be a means for interfacing with an input/output device (250). For example, the input device may include a device such as a microphone, a keyboard, or a mouse, and the output device may include a device such as a display or a speaker. As another example, the input/output interface (240) may be a means for interfacing with a device that integrates input and output functions into one, such as a touch screen. The input/output device (250) may be configured as a single device with the computer device (200).

Also, in other embodiments, the computer device (200) may include fewer or more components than the components of FIG. 2. However, it is not necessary to explicitly illustrate most of the conventional components. For example, the computer device (200) may be implemented to include at least some of the input/output devices (250) described above, or may further include other components such as a transceiver, a database, etc.

Below, specific embodiments of a method and device for generating a video snapshot using an image hash are described.

Video snapshots can be used to analyze videos, including video files such as MP4 or AVI, and animated images such as GIF or WEBP.

For example, one could apply a uniform sampler that detects one frame at a specific time to generate a video snapshot.

As another example, a snapshot can be generated by calculating the sum of absolute difference (SAD), which represents the difference in pixel values between neighboring frames, and then detecting frames where the SAD is greater than a user-defined value.

In addition to the above methods, an algorithm for removing duplicate frames through image processing based hash technology can be applied to generate video snapshots.

In this specification, a duplicate frame may mean not only a perfectly matching frame, but also a frame with a certain degree of similarity. For example, a frame with a slight deformation such as translation or rotation may be recognized as a duplicate frame. In reality, most similar frames can be said to be near duplicates.

The present embodiments can remove some of the frames contained in a given video and create a snapshot using only the remaining frames.

In the process of removing some frames, such as duplicate frames, from the frames included in the video to create a video snapshot, there may be an issue where key frames are missed. For example, when creating a video snapshot for the purpose of content monitoring, frames containing target information for monitoring may be missed in the snapshot because they are removed.

Figure 3 illustrates an example of video frames for snapshot generation. As illustrated in Figure 3, it is assumed that a given video (30) consists of 10 frames.

Based on the image similarity between the frames included in the video (30), duplicate frames can be removed and a snapshot can be generated with the remaining frames. Among the 10 frames included in the video (30), there is a high probability that one of the frames (1) and (2) will be removed due to the image similarity. For example, with respect to the video (30) of FIG. 3, as shown in FIG. 4, frames (1), (4), (5), (7), and (10) are selected as duplicate frames and deleted, and then a snapshot (41) can be generated with frames (2), (3), (6), (8), and (9).

For example, among the QR (quick response code) used on messengers, there are QRs for various purposes, such as QRs that connect to OA (official accounts) and QRs shared for personal meetings. When monitoring content for videos, some QRs may require censorship as they are prohibited from being shared.

However, in an algorithm that removes duplicate frames based on image similarity, it is difficult to distinguish between frames (1) and (2), so frames that require censorship may be detected as duplicate frames and deleted.

Therefore, a technology is needed that can retain frames containing target information, such as key monitoring content, when creating snapshots for video analysis, highlight production, etc.

The computer device (200) according to the present embodiment can provide a video analysis service to a client through a dedicated application installed on the client or a web/mobile site access related to the computer device (200). The computer device (200) can be configured with a video snapshot generation system implemented by a computer. For example, the video snapshot generation system can be implemented in the form of an independently operating program, or can be configured in the form of an in-app of a specific application and implemented so as to be operable on the specific application.

The processor (220) of the computer device (200) may be implemented as a component for performing the following video snapshot generation method. Depending on the embodiment, the components of the processor (220) may be selectively included in or excluded from the processor (220). In addition, depending on the embodiment, the components of the processor (220) may be separated or merged to express the function of the processor (220).

These processors (220) and components of the processor (220) can control the computer device (200) to perform the steps included in the following video snapshot generation method. For example, the processor (220) and components of the processor (220) can be implemented to execute instructions according to the code of the operating system included in the memory (210) and the code of at least one program.

Here, the components of the processor (220) may be representations of different functions performed by the processor (220) according to instructions provided by the program code stored in the computer device (200).

The processor (220) can read necessary commands from the memory (210) loaded with commands related to the control of the computer device (200). In this case, the read commands may include commands for controlling the processor (220) to execute steps to be described later.

The steps included in the method for generating a video snapshot to be described hereinafter may be performed in a different order than the illustrated order, and some of the steps may be omitted or additional processes may be included.

The steps included in the method for generating a video snapshot may be performed on the server (150), and in some embodiments, at least some of the steps may also be performed on the client.

FIG. 5 is a flowchart illustrating an example of a method that a computer device can perform according to an embodiment of the present invention.

Referring to FIG. 5, in step S510, the processor (220) can calculate the image similarity (hereinafter, referred to as 'first similarity') between frames included in the video. For example, the processor (220) can calculate the degree of similarity between frames using a hash based on machine learning. The processor (220) can calculate the first similarity that represents the image similarity through the distance between the embedding vectors by embedding the features of each image corresponding to the frame.

In step (S520), the processor (220) may calculate the similarity (hereinafter, referred to as 'second similarity') between each frame included in the video and the prompt when a prompt defined by the user is given. Similarly, the processor (220) may calculate the degree of similarity between the frame and the prompt using a machine learning-based hash. In the present embodiment, target information to be included in a snapshot may be defined using a prompt, and at this time, the prompt may be input by the user as various types of content such as text, image, and audio. The processor (220) may embed the features of the target information defined through the prompt and calculate the second similarity through the vector distance between the prompt embedding and the image embedding of each frame. In the present embodiment, the hash may be used to determine the similarity between the class defined by the user for the image frame using a zero-shot technique.

In step (S530), the processor (220) may select duplicate frames to be removed based on the first similarity and the second similarity among frames included in the video. The processor (220) may cluster frames having a similarity level higher than a certain level based on the first similarity, and then select the remaining frames, excluding a set number of frames (for example, 1) for each cluster, as candidate frames. At this time, the processor (220) may finally select a frame among the candidate frames having a second similarity lower than a certain level as a duplicate frame. In other words, the processor (220) may exclude a frame among the candidate frames having a second similarity level higher than a certain level from the target of selecting a duplicate frame.

In step (S540), the processor (220) can remove frames finally selected as duplicate frames from among the frames included in the video and create a snapshot with the remaining frames. After clustering the frames included in the video based on the first similarity, a snapshot can be created while maintaining the maximum total frames of a specific cluster corresponding to the second similarity. In this embodiment, duplicate frames are removed based on image similarity and a snapshot can be created with the remaining frames, while including as many frames as possible that are similar to a class defined by the user.

FIGS. 6 and 7 are exemplary drawings for explaining a searchable multi-modal model according to one embodiment of the present invention.

The present invention can use a multi-modal model that enables retrieval using clustering and various types of prompts.

Referring to FIG. 6, a multi-modal model according to the present invention includes a method of extracting embeddings for various modalities such as text (601), video (or image) (602), and audio (603), while learning so that embeddings of different modalities can be shared in a single vector space (600).

The above-mentioned model learning method can utilize contrastive learning, etc., and a representative method is a large-scale VLP (vision-language pre-trained) model such as CLIP (contrastive language-image pre-training model).

Referring to FIG. 7, in CLIP, learning is performed so that the similarity of the embeddings extracted from the text encoder (701) and the image encoder (702) for image-text pairs corresponding to contexts with the same meaning increases, and the similarity of the embeddings extracted from the text encoder (701) and the image encoder (702) for image-text pairs corresponding to contexts with different meanings decreases.

In the case of an image encoder (702) trained together with a text encoder (701), it can be used for dual purposes as it can represent the visual characteristics of an image well while simultaneously representing the semantic characteristics of a text.

Image-text embedding is a method for embedding and combining features of images and texts. It can well express the relationship between embedded classes and can be mainly used for image-text retrieval, image captioning, and VQA (visual question answering).

FIG. 8 is a flowchart illustrating an example of a duplicate frame filtering process according to one embodiment of the present invention.

In the present invention, filtering using a prompt defined by a user and filtering using image similarity can be performed simultaneously by using an embedding extracted through a multi-modal model.

Referring to FIG. 8, the processor (220) can extract image embedding for each frame included in a video through an image encoder (702) learned together with a text encoder (701), and at this time, image similarity between frames can be calculated using the frame-by-frame embedding (S801).

The processor (220) can input a text-type prompt (80) by a user, and at this time, a text embedding for the prompt (80) can be extracted through a text encoder (701) trained together with an image encoder (702).

For example, if you want to censor a video that contains a QR code during the video analysis process, the user can input a prompt such as "qr", "qr image", "photo of qr code", or "qr code image" as text representing the target information.

The processor (220) can compare the image embedding extracted through the image encoder (702) for each frame included in the video with the text embedding extracted through the text encoder (701) to calculate the similarity with the prompt (S802).

In other words, the processor (220) can determine whether each frame includes a user-defined class based on the similarity with the prompt through a zero-shot technique using a hash generated based on machine learning.

The processor (220) can select a frame among the frames included in the video that has an image similarity level higher than a certain level with other frames as a duplicate frame, and at this time, among the frames selected as duplicate frames, a frame that has a similarity level higher than a certain level with a prompt can be excluded from the selection of duplicate frames (S803).

Accordingly, the processor (220) can remove frames finally selected as duplicate frames from among the frames included in the video and create a snapshot with the remaining frames, and at this time, frames having a similarity level with the prompt above a certain level can be included in the snapshot.

In the case of an algorithm that removes duplicate frames based on image similarity, one of frame (1) and frame (2) where a QR code exists may be selected as a duplicate frame and deleted, but in this embodiment, a user may specify desired target information through a prompt (80) to maintain the frame containing the target information and generate a video snapshot.

Figures 9 and 10 illustrate examples of snapshot creation using a prompt according to one embodiment of the present invention.

Figures 9 and 10 illustrate examples of snapshot generation of a video (30) consisting of 10 frames from (1) to (10).

As illustrated in FIG. 9, when a user inputs the text 'QR' as a prompt to define desired target information, the processor (220) can exclude frames (1) and (2) containing QR codes among 10 frames included in the video (30) from the selection of duplicate frames.

For example, a snapshot (91) for a video (30) may be composed of frames (1), (2), (3), (6), (8), and (9) in which frames (4), (5), (7), and (10) are selected as duplicate frames and deleted. That is, the processor (220) may create a snapshot (91) in a form in which frames (1) and (2) in which a QR code exists as target information defined through a prompt are maintained.

In the above embodiment, a text prompt is described as an example, but it is not limited thereto, and filtering using a prompt in the form of audio or image can also be applied.

Embeddings of different modalities are represented in a single vector space, and the learned model can be used with zero-shot techniques to generate snapshots that do not miss frames containing target information using not only text prompts but also other forms of modality prompts.

For example, as illustrated in FIG. 10, if a user inputs audio 'sky' as a prompt for defining desired target information, the features of the audio 'sky' can be embedded to maintain frames having a similarity level with the prompt embedding that is higher than a certain level. The processor (220) can create a snapshot (101) in a form of maintaining frames (7) and (8) corresponding to the 'sky' information as target information defined through the audio prompt. In other words, the snapshot (101) for the video (30) can be composed of frames (2), (3), (6), (7), (8), and (9) with frames (1), (4), (5), and (10) deleted as duplicate frames.

By applying a methodology using machine learning-based hashing, it is possible to search for frames related to specific keywords entered in the form of text, audio, images, etc., as well as frames containing specific sounds (e.g., cheering sounds, car crash sounds, wave sounds, alarm sounds, etc.).

These embodiments include zero-shot features, so that all information of pre-learned modalities can be targeted and applied. Specific keywords that can be entered in the form of text, audio, or images can be registered as target information, and in addition, various sounds, adult content, or violent content can be registered through prompts so that users can search for frames containing the information they want.

According to embodiments of the present invention, when generating a video snapshot by removing duplicate frames, it is possible to generate a snapshot in which key frames containing target information are not omitted. Furthermore, according to embodiments of the present invention, by determining whether a frame includes a class defined by a user through a zero-shot technique using a hash generated based on machine learning, it is possible to generate a video snapshot by including as many frames as possible that are similar to a class defined by a user.

The devices described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, the devices and components described in the embodiments may be implemented using one or more general-purpose computers or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding to them. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. In addition, the processing device may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device is sometimes described as being used alone, but those skilled in the art will appreciate that the processing device may include multiple processing elements and/or multiple types of processing elements. For example, the processing device may include multiple processors, or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.

The software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing device to perform a desired operation or may independently or collectively command the processing device. The software and/or data may be embodied in any type of machine, component, physical device, computer storage medium, or device for interpretation by the processing device or for providing instructions or data to the processing device. The software may be distributed over network-connected computer systems and stored or executed in a distributed manner. The software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. At this time, the medium may be one that continuously stores a program executable by a computer, or one that temporarily stores it for execution or downloading. In addition, the medium may be various recording means or storage means in the form of a single or multiple hardware combinations, and is not limited to a medium directly connected to a computer system, and may also be distributed on a network. Examples of the medium may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and ROMs, RAMs, flash memories, etc., configured to store program commands. In addition, examples of other media may include recording media or storage media managed by app stores that distribute applications, sites that supply or distribute various software, servers, etc.

Although the embodiments have been described above by way of limited examples and drawings, those skilled in the art can make various modifications and variations from the above description. For example, appropriate results can be achieved even if the described techniques are performed in a different order than the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form than the described method, or are replaced or substituted by other components or equivalents.

Therefore, other implementations, other embodiments, and equivalents to the claims are also included in the scope of the claims described below.

Claims (20)

A method for creating a video snapshot running on a computer device, The computer device comprises at least one processor configured to execute computer-readable instructions contained in a memory, The above video snapshot creation method is: A step of generating a snapshot by removing redundant frames from frames included in a video and using the remaining frames, by at least one processor, while maintaining frames containing target information defined by a user. A method for creating a video snapshot including: In the first paragraph, The above generating steps are: Generating a snapshot of the video using a hash generated by a machine learning model. A method for creating a video snapshot, comprising: In the first paragraph, The above generating steps are: Generating snapshots of the video using a multi-modal model learned by representing embeddings of different modalities in a single vector space. A method for creating a video snapshot, comprising: In the first paragraph, The above generating steps are: A step of calculating a first similarity indicating image similarity between frames included in the above video; A step of calculating a second similarity indicating a similarity relationship with a prompt registered as the target information for each frame included in the video; and A step of selecting some frames included in the video as the duplicate frames based on the first similarity and the second similarity. A method for creating a video snapshot including: In paragraph 4, The step of calculating the first similarity is as follows: The first similarity is calculated by using a multi-modal model learned by expressing embeddings of different modalities in a single vector space. Calculating the first similarity through the vector distance between image embeddings that embed the features of the images corresponding to each frame. A method for creating a video snapshot, comprising: In paragraph 4, The step of calculating the second similarity is as follows: Step for registering the above prompt as content in any form of text, image, or audio A method for creating a video snapshot including: In paragraph 4, The step of calculating the second similarity is as follows: The second similarity is calculated by using a multi-modal model learned by expressing embeddings of different modalities in a single vector space. Calculating the second similarity through the vector distance between the prompt embedding that embeds the features of the prompt registered as the above target information and the image embedding that embeds the features of the image corresponding to each frame. A method for creating a video snapshot, comprising: In paragraph 4, The above selection steps are: A step of clustering frames having a similarity level higher than a certain level based on the first similarity; A step of selecting the remaining frames, excluding a certain number of frames per cluster, as candidate frames for the duplicate frames; and A step of excluding frames among the above candidate frames whose second similarity is above a certain level from the selection target of the above duplicate frames. A method for creating a video snapshot including: In the third paragraph, The above multi-modal model is a model that learns the similarity of embeddings for data pairs using contrastive learning. A method for creating a video snapshot, comprising: In the third paragraph, The above multi-modal model is a VLP (vision-language pre-trained) model learned through contrastive learning. A method for creating a video snapshot, comprising: A computer program stored on a non-transitory computer-readable recording medium for causing a computer device to execute the method for generating a video snapshot according to any one of claims 1 to 10. In computer devices, At least one processor configured to execute computer-readable instructions contained in memory Including, At least one processor of the above, A process of removing duplicate frames from frames included in a video and creating a snapshot with the remaining frames, while maintaining frames containing target information defined by the user. A computer device that processes data. In Article 12, At least one processor of the above, Generating a snapshot of the above video using a hash generated by a machine learning model. A computer device characterized by: In Article 12, At least one processor of the above, Generating snapshots of the above video through a multi-modal model learned by representing embeddings of different modalities in a single vector space. A computer device characterized by: In Article 12, At least one processor of the above, Compute a first similarity indicating the image similarity between frames included in the above video, For each frame included in the above video, a second similarity is calculated indicating a similarity relationship with the prompt registered as the target information, Selecting some frames included in the video as the duplicate frames based on the first similarity and the second similarity A computer device characterized by: In Article 15, At least one processor of the above, The first similarity is calculated by using a multi-modal model learned by expressing embeddings of different modalities in a single vector space. Calculating the first similarity through the vector distance between image embeddings that embed the features of the images corresponding to each frame. A computer device characterized by: In Article 15, At least one processor of the above, Registering the above prompt as content in any form of text, image, or audio A computer device characterized by: In Article 15, At least one processor of the above, The second similarity is calculated by using a multi-modal model learned by expressing embeddings of different modalities in a single vector space. Calculating the second similarity through the vector distance between the prompt embedding that embeds the features of the prompt registered as the above target information and the image embedding that embeds the features of the image corresponding to each frame. A computer device characterized by: In Article 15, At least one processor of the above, Clustering frames with a similarity level higher than a certain level based on the first similarity above, The remaining frames, excluding a certain number of frames per cluster, are selected as candidate frames for the above duplicate frames. Among the above candidate frames, frames with a second similarity level above a certain level are excluded from the selection target of the above duplicate frames. A computer device characterized by: In Article 14, The above multi-modal model is a VLP model that learns the similarity of embeddings for data pairs using contrastive learning. A computer device characterized by:

Images