WO2025154220A1 - Information processing system and game image sharing method - Google Patents

Abstract

A game image acquisition unit 210 acquires an image in a game played by a user in the past or an image in a game being played by the user. A content identification unit 212 performs image analysis on the acquired game image to determine whether or not the game image is related to content shared with another user in the past. A sharing processing part 214 causes the game image, which has been determined to be related to the shared content in the past, to be shared with another user.

Description

Information processing system and game image sharing method

The present disclosure relates to a technology for saving images of a game that a user is currently playing or has played in the past as content and/or a technology for sharing the content with other users.

Patent Document 1 discloses an information processing device that executes a game program based on user operations, displays game images on an output device, and records the game images in a ring buffer in the background. On an editing screen, the user specifies the start and end points for cutting out the game video recorded in the ring buffer, and uploads the cut-out game video to a content sharing server.

JP 2020-870 A

Traditionally, users generate their own original content by manually capturing images of the game they are currently playing. Generally, content generated by users is called User Generated Content, and users share User Generated Content with other users by uploading it to a shared server or providing it to specific friends.

The purpose of this disclosure is to provide a mechanism for efficiently realizing the process of generating content and sharing content with other users.

An information processing system according to one aspect of the present disclosure includes a game image acquisition unit that acquires images of a game that the user has played in the past or images of a game that the user is currently playing, a content identification unit that performs image analysis on the acquired game images to determine whether the game images are related to content that has been shared with other users in the past, and a sharing processing unit that allows game images determined to be related to previously shared content to be shared with other users.

A game image sharing method according to another aspect of the present disclosure includes the steps of acquiring an image of a game that the user has played in the past or an image of a game that the user is currently playing, analyzing the acquired game image to determine whether the game image is related to content that has been shared with another user in the past, and allowing the game image determined to be related to the previously shared content to be shared with another user.

In addition, any combination of the above components, and conversions of the expressions of this disclosure between methods, devices, systems, recording media, computer programs, etc., are also valid aspects of this disclosure.

FIG. 1 is a diagram illustrating an information processing system according to an embodiment. FIG. 2 is a diagram illustrating a hardware configuration of an information processing device. FIG. 2 is a diagram illustrating an example of functional blocks of an information processing device. FIG. 11 is a diagram showing an example of a displayed game image. 1 is a flowchart for automatically generating content. FIG. 11 is a diagram showing an example of a displayed game image. FIG. 13 is a diagram for explaining a method for extracting game video images. FIG. 11 is a diagram illustrating another example of functional blocks of the information processing device. 13 is a flowchart showing a process for sharing content. FIG. 13 is a diagram showing an example of a screen on which content is presented. FIG. 11 is a diagram showing another example of a screen presenting content.

In the information processing system of the embodiment, the information processing device executes a game program based on the user's operation, outputs game images and game sounds to an output device, and may have a function of automatically generating user-original content including game images and/or game sounds without user instructions. The automatically generated content may be still images (screenshots) of the game or video images of the game. The information processing device may also have a function of automatically performing a process of sharing content with another user without user instructions. The process of sharing content includes providing the content to other users (e.g., friends) and uploading the content to a distribution server so that the content can be viewed by other users. The information processing device may automatically perform a sharing process of determining a sharing destination and providing the content to the sharing destination, but may also present the content and the sharing destination to the user before performing the sharing process and perform the sharing process after obtaining the user's consent.

FIG. 1 shows an information processing system 1 according to an embodiment of the present disclosure. The information processing system 1 of the embodiment includes an information processing device 10 operated by a user. The information processing system 1 may include a management server 5 that provides network services to users, and a content sharing server 9 that distributes content. The information processing system 1 may further include another server, such as a cloud game server.

The access point (hereinafter referred to as "AP") 8 has the functions of a wireless access point and a router, and the information processing device 10 connects to the AP 8 wirelessly or via a wired connection to be able to communicate with the management server 5 and the content sharing server 9 on the network 3 such as the Internet.

The information processing device 10 is connected wirelessly or by wire to an input device 6 having multiple operating members, and the input device 6 outputs information of the user operating the operating members to the information processing device 10. When the information processing device 10 receives operation information from the input device 6, it reflects the information in the processing of the system software and game software, and causes the output device 4 to output the processing results. In the information processing system 1, the information processing device 10 is a game device (game console) that executes a game, and the input device 6 is an operating device such as a game controller, which supplies information of the user operating the operating members to the information processing device 10. The input device 6 may also be an input interface such as a keyboard or mouse.

In a modified example, the information processing system 1 may include a cloud game server (not shown) that executes a game program based on user operations. In this modified example, the information processing device 10 transmits information (game operation information) of the user operating the input device 6 to the game server. The game server generates game images and game sounds based on the game operation information, and streams the game images and game sounds to the information processing device 10. In this modified example, the information processing device 10 does not need to have a function for executing a game program, and may be a terminal device that outputs game images and game sounds from the output device 4.

The auxiliary storage device 2 is a large-capacity recording device such as an HDD (hard disk drive) or SSD (solid state drive), and may be an internal recording device, or may be an external recording device connected to the information processing device 10 via a USB (Universal Serial Bus) or the like. The output device 4 may be a television having a display for outputting images and a speaker for outputting sound. The output device 4 may be connected to the information processing device 10 via a wired cable, or may be connected wirelessly.

Camera 7, which is an imaging device, is provided near output device 4 and captures an image of the space around output device 4. While FIG. 1 shows an example in which camera 7 is attached to the top of output device 4, it may be disposed on the side or bottom of output device 4, and in either case, it is disposed in a position where it can capture an image of a user positioned in front of output device 4. Camera 7 may be a stereo camera.

The management server 5 provides network services to users of the information processing system 1. The management server 5 manages user accounts that identify each user, and each user signs in to the network services provided by the management server 5 using their user account. By signing in to the network services from the information processing device 10, users can register game save data and trophies, which are virtual rewards acquired during game play, in the management server 5. By registering the save data and trophies in the management server 5, the save data and trophies can be synchronized even if the user uses an information processing device other than the information processing device 10. Account information such as being a paid member of a specified network service is associated with the user account.

The content sharing server 9 makes the content generated by the information processing device 10 available for viewing by specific or unspecified users. Users participating in the information processing system 1 can view the desired content by accessing the content sharing server 9 and selecting the content. In the embodiment, the information processing device 10 automatically identifies content suitable for sharing with other users from among multiple stored contents, and uploads the content to the content sharing server 9. At this time, the information processing device 10 may automatically upload the content to the content sharing server 9, or may upload the content to the content sharing server 9 on the condition that the user's consent is obtained. Note that the process of uploading content to the content sharing server 9 is one form of sharing content with other users, and the information processing device 10 may send the content to a specific user (e.g., a friend) and share the content with the specific user.

The information processing device 10 has a content generation model that has learned from content that users have manually generated in the past (hereinafter also referred to as "user-generated content"). The content generation model performs machine learning on multiple user-generated contents to extract regularities (features) of the user-generated content. User-generated content is generated according to the preferences of the user, and has different characteristics for each user. The more user-generated content is used for machine learning, the more accurately features that accurately capture the regularities of the user-generated content can be extracted, and the more reliable the content generation model can be.

The content generation model may be trained to input an image of the game that a user is currently playing, and output the degree to which the game image reflects the characteristics of the user-generated content (similarity) as a score value. The similarity indicates the degree to which the game image matches the user-generated content, and the score value may be expressed on a scale of 100 points.

The content generation model (machine learning model) may be composed of a CNN (Convolutional Neural Network), which is equivalent to a multi-layer neural network including an input layer, one or more convolutional layers, and an output layer. The information processing device 10 may train each coupling coefficient (weight) in the CNN using a learning method such as deep learning. Note that the information processing device 10 may train the content generation model using other AI training methods.

The information processing device 10 also has a shared content identification model that learns content that a user has shared with other users in the past (hereinafter also referred to as "shared content"). The shared content identification model performs machine learning on multiple shared contents to extract regularities (features) of the shared content. Shared content is content that a user selects from among user-generated content in order to share it with other users, and has different characteristics for each user. The more shared contents are used for machine learning, the more accurately features that accurately capture the regularities of the shared content can be extracted, and the more reliable the shared content identification model can be.

The shared content identification model may be trained so that when content, which is an image of a game that the user has played in the past, is input, the model outputs the degree to which the content reflects the characteristics of the shared content (similarity) as a score value. The similarity indicates the degree to which the content matches the shared content, and the score value may be expressed on a scale of 100 points.

The shared content identification model (machine learning model) may be configured as a CNN (Convolutional Neural Network), which is equivalent to a multi-layer neural network including an input layer, one or more convolutional layers, and an output layer. The information processing device 10 may train each coupling coefficient (weight) in the CNN using a learning method such as deep learning. Note that the information processing device 10 may train the shared content identification model using other AI training methods.

FIG. 2 shows the hardware configuration of the information processing device 10. The information processing device 10 is configured to include a main power button 20, a power ON LED 21, a standby LED 22, a system controller 24, a clock 26, a device controller 30, a media drive 32, a USB module 34, a flash memory 36, a wireless communication module 38, a wired communication module 40, a subsystem 50, and a main system 60.

The main system 60 comprises a main CPU (Central Processing Unit), a memory and memory controller serving as the main storage device, a GPU (Graphics Processing Unit), etc. The GPU is primarily used for the calculation and processing of game programs. The main CPU has the function of starting up the system software and executing the game program installed in the auxiliary storage device 2 in the environment provided by the system software. The subsystem 50 comprises a sub-CPU, a memory and memory controller serving as the main storage device, etc., but does not comprise a GPU.

While the main CPU has the function of executing game programs installed in the auxiliary storage device 2, the sub-CPU does not have such a function. However, the sub-CPU has the function of accessing the auxiliary storage device 2 and the function of sending and receiving data with the management server 5. The sub-CPU is configured with only these limited processing functions, and therefore can operate with less power consumption compared to the main CPU. These functions of the sub-CPU are executed when the main CPU is in standby mode.

The main power button 20 is an input unit through which the user inputs operations, is provided on the front of the housing of the information processing device 10, and is operated to turn on or off the power supply to the main system 60 of the information processing device 10. The power ON LED 21 lights up when the main power button 20 is turned on, and the standby LED 22 lights up when the main power button 20 is turned off. The system controller 24 detects when the main power button 20 is pressed by the user.

The clock 26 is a real-time clock that generates current date and time information and supplies it to the system controller 24, the subsystem 50, and the main system 60.

The device controller 30 is configured as an LSI (Large-Scale Integrated Circuit) that transfers information between devices like a south bridge. As shown in the figure, devices such as the system controller 24, media drive 32, USB module 34, flash memory 36, wireless communication module 38, wired communication module 40, subsystem 50, and main system 60 are connected to the device controller 30. The device controller 30 absorbs differences in the electrical characteristics and data transfer speeds of each device and controls the timing of data transfer.

The media drive 32 is a drive device that operates by mounting a ROM medium 44 on which application software such as games and license information are recorded, and reads programs, data, and the like from the ROM medium 44. The ROM medium 44 is a read-only recording medium such as an optical disk, magneto-optical disk, or Blu-ray disk.

The USB module 34 is a module that connects to an external device via a USB cable. The USB module 34 may also be connected to the auxiliary storage device 2 and the camera 7 via a USB cable. The flash memory 36 is an auxiliary storage device that constitutes the internal storage. The wireless communication module 38 wirelessly communicates with the input device 6 using a communication protocol such as the Bluetooth (registered trademark) protocol or the IEEE802.11 protocol. The wired communication module 40 communicates with an external device via a wired connection and connects to the network 3 via the AP 8.

The information processing device 10 of the embodiment executes a game program based on user operations, outputs game images and game sounds to the output device 4, and has a function of automatically generating user original content including game images and/or game sounds. The information processing device 10 also has a function of automatically performing a process of sharing content with another user. Below, the function of automatically generating content of the information processing device 10 and the function of automatically performing a process of sharing content are described.

<Automatic content generation function>
3 shows an example of functional blocks of the information processing device 10. The information processing device 10 of the embodiment includes a processing unit 100 and a communication unit 102, and the processing unit 100 has a function of automatically generating user-original content including game images and/or game sounds. The processing unit 100 includes a game execution unit 110, a game image and sound generation unit 112, an output processing unit 114, a game image acquisition unit 120, a game image specification unit 122, a content storage unit 124, a content presentation unit 126, a learning unit 130, and a user voice acquisition unit 140. The storage device 150 has a user-generated content storage unit 152 that stores content (user-generated content) that the user manually generated in the past, and a content storage unit 154 that stores content that the processing unit 100 automatically generates. The storage device 150 may be an auxiliary storage device 2.

The functions of the components in the information processing device 10 may be implemented in circuitry or processing circuitry, including general purpose processors, application specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), a CPU (a Central Processing Unit), conventional circuits, and/or combinations thereof, configured or programmed to implement the functions described herein. A processor is considered to be a circuitry or processing circuitry that includes transistors and other circuits. A processor may be a programmed processor that executes a program stored in a memory.

In this specification, a circuit, unit, or means is hardware that is programmed to realize or executes the described functions. The hardware may be any hardware disclosed in this specification or any hardware that is programmed to realize or known to execute the described functions.

If the hardware is a processor that is considered to be a type of circuitry, the circuitry, means, or unit may be a combination of hardware and software used to configure the hardware and/or processor.

The communication unit 102 has the functions of both the wireless communication module 38 and the wired communication module 40. While the user is playing a game, the communication unit 102 receives information (game operation information) about the user's operation of the operation members of the input device 6, and provides this information to the game execution unit 110.

The game execution unit 110 executes game software based on game operation information. The game software includes at least a game program, image data, and sound data. While the user is playing the game, the game execution unit 110 performs calculation processing to move a player character in a virtual space based on the user's operation information. The game image and sound generation unit 112 includes a GPU (Graphics Processing Unit), and receives the results of the calculation processing in the virtual space, and generates a game image from a viewpoint position (virtual camera) in the virtual space. The game image and sound generation unit 112 also generates game sound from a viewpoint position in the virtual space. The output processing unit 114 outputs the game images and game sounds generated by the game image and sound generation unit 112 from the output device 4.

FIG. 4 shows an example of a game image displayed on the output device 4. The user operates the input device 6 while viewing the game images and game sounds output from the output device 4 to progress through the game.

The user-generated content storage unit 152 stores content that users have manually generated in the past (user-generated content). In an embodiment, the user-generated content is used as learning data for training a content generation model, which is a machine learning model (or an artificial intelligence model). The user-generated content may be used as training data in supervised learning. The learning unit 130 may train the content generation model using user-generated content generated by one user as training data.

The content generation model is trained to input an image of the game currently being played by a user and output the degree to which the game image reflects the characteristics of the user-generated content (similarity) as a score value. The similarity indicates the degree to which the game image matches the user-generated content, and the score value may be expressed on a scale of 100 points. The closer the score value is to 100 points, the higher the probability that the game image matches the user-generated content, and the closer the score value is to 0 points, the lower the probability that the game image matches the user-generated content.

If the user-generated content includes still images (screenshots) and moving images, the learning unit 130 may train a content generation model for still images and a content generation model for moving images separately. In other words, the content generation model for still images learns the still images, which are user-generated content, to extract the characteristics of the user-generated content, and the content generation model for moving images learns the moving images, which are user-generated content, to extract the characteristics of the user-generated content. In a modified example, the learning unit 130 may input the still images and moving images, which are user-generated content, to the content generation model without distinguishing between them, and train the content generation model.

The learning unit 130 may train a content generation model for each game title or game genre. Since the type of game image differs for each game title or game genre, it is preferable that the learning unit 130 trains a content generation model for each game title or game genre.

During game play, the game image acquisition unit 120 acquires game images and game sounds generated by the game image and sound generation unit 112 and supplies them to the game image identification unit 122. The game image identification unit 122 performs image analysis of the game images acquired by the game image acquisition unit 120 to identify game images related to past user-generated content. In the embodiment, the game image identification unit 122 performs AI image analysis of the game images using a trained content generation model. When an image of the game being played by the user is input, the trained content generation model outputs the degree to which the game image reflects the characteristics of the user-generated content (similarity) as a score value. At this time, the content generation model may output the score value together with information that identifies the game image. The information that identifies the game image may be the game image itself, or may be metadata such as information indicating the date and time of generation of the game image.

The game image identification unit 122 determines whether or not a game image is suitable as content based on the score value output by the content generation model. For example, the game image identification unit 122 may determine that a game image for which a score value equal to or greater than a predetermined score threshold (e.g., 80 points) is output is suitable as user-generated content, and may determine that a game image for which a score value less than the score threshold is output is inappropriate as user-generated content. The content generation model may be configured to output information indicating whether the game image is suitable or inappropriate as content, and information identifying the game image.

A case will now be described in which the game image specification unit 122 specifies a still image (screenshot) appropriate as content.
5 shows an example of a flowchart for automatically generating content. The automatic content generation process is carried out while the user is playing the game (Y in S10).
6 shows an example of a game image displayed on the output device 4. The game image acquisition unit 120 acquires the game image generated by the game image and sound generation unit 112, and supplies it to the game image specification unit 122 (S12). The game image acquisition unit 120 may include a buffer memory for temporarily storing the acquired game image. The game image specification unit 122 inputs the supplied game image into a content generation model for still images, performs AI image analysis, and acquires a score value of the game image.

If the score value of the game image is less than a predetermined score threshold (N in S14), the game image identification unit 122 determines that the game image is not related to past user-generated content, i.e., that the game image does not have characteristics in common with the user-generated content and is inappropriate as content. On the other hand, if the score value of the game image is equal to or greater than a predetermined score threshold (Y in S14), the game image identification unit 122 determines that the game image is related to past user-generated content, i.e., that the game image has characteristics in common with the user-generated content and is appropriate as content.

When the game image identification unit 122 determines that the image of the game the user is currently playing is related to past user-generated content, the content saving unit 124 saves the identified game image in the content storage unit 154 as new content (S16). For example, if the score value of the game image shown in FIG. 6 is equal to or greater than a predetermined score threshold, the content saving unit 124 automatically saves the game image shown in FIG. 6 as new content in the content storage unit 154. In this way, the game image identification unit 122 identifies a game image that has common characteristics with past screenshots, so that screenshots that reflect the user's preferences are automatically saved in the content storage unit 154. Since the screenshots are generated without the user having to give an instruction to take a screenshot, the user can concentrate on playing the game. The automatic content generation process may be performed until the user finishes playing the game (N in S10).

The game image identification unit 122 may identify multiple game images that are similar to each other as game images related to past screenshots. For example, the game images displayed before and after the game image shown in FIG. 6 are similar to each other and often have common features with past screenshots. When multiple game images that are similar to each other are identified, it is preferable for the content storage unit 124 to store only a portion of the multiple game images in the content storage unit 154. For example, the content storage unit 124 may store only one of the multiple game images in the content storage unit 154. This makes it possible to avoid storing a large number of similar game images as content, and allows the capacity of the content storage unit 154 to be used efficiently.

For this purpose, the content storage unit 124 may have a function for evaluating the similarity of game images successively identified by the game image identification unit 122. It is known that the similarity of two images is evaluated by calculating the distance between the two images, and the content storage unit 124 may use a known evaluation method to identify similar game images. If the content storage unit 124 stores a game image in the content memory unit 154 as new content each time the game image identification unit 122 identifies the game image, the content storage unit 124 may, after storing the game images in the content memory unit 154, identify multiple similar game images, keep only some of the game images, and delete the remaining game images from the content memory unit 154.

In the above example, the game image identification unit 122 inputs all generated game images (frame images) into a content generation model for still images and performs AI image analysis. In a modified example, the game image identification unit 122 may input game images into a content generation model for still images at a predetermined cycle and perform AI image analysis. In other words, the game image identification unit 122 may input game images into a content generation model for still images at a frequency of, for example, one image every few seconds and perform AI image analysis.

When performing AI image analysis, the game image identification unit 122 may reduce the amount of calculations by lowering the resolution of the game image. The game image identification unit 122 determines whether or not it is necessary to reduce the amount of calculations based on the available computational resources, and if it is necessary, may take measures such as lowering the resolution of the game image or canceling the automatic content generation process. When canceling the automatic content generation process, it is preferable for the game image identification unit 122 to notify the user of the reason.

Next, a case will be described in which the game image specification unit 122 specifies an appropriate moving image as content. The automatic content generation process is carried out while the user is playing the game (Y in S10).
The game image acquisition unit 120 acquires the game images generated by the game image and sound generation unit 112 and stores them in a buffer memory (S12). The buffer memory may be configured as a ring buffer that stores a predetermined upper limit of game video images (for example, one hour). The game image specification unit 122 reads out only the predetermined amount of game video images temporarily stored in the buffer memory, inputs the game video images to a content generation model for video images, performs AI image analysis, and acquires a score value of the game video images.

If the score value of the game video is less than a predetermined score threshold (N in S14), the game image identification unit 122 determines that the game video is not related to past user-generated content, i.e., the game video does not have characteristics in common with the user-generated content and is inappropriate as content. On the other hand, if the score value of the game video is equal to or greater than a predetermined score threshold (Y in S14), the game image identification unit 122 determines that the game video is related to past user-generated content, i.e., the game video has characteristics in common with the user-generated content and is appropriate as content.

When the game image identification unit 122 determines that the game video temporarily stored in the buffer memory is related to past user-generated content, the content saving unit 124 saves the identified game video in the content storage unit 154 as new content (S16). In this way, the game image identification unit 122 identifies game videos that have common characteristics with past video content, so that game videos that reflect the user's preferences are automatically saved in the content storage unit 154. Since the game video is generated without the user issuing an instruction to capture the game video, the user can concentrate on playing the game. The automatic content generation process may be performed until the user finishes playing the game (N in S10).

In the above example, the game image identification unit 122 reads out a predetermined amount of game video temporarily stored in the buffer memory and inputs it into the content generation model for video. In another example, the game image identification unit 122 may refer to metadata included in the game video temporarily stored in the buffer memory to determine the start and end points of the game video to be cut out, and read out the game video between the start and end points. For example, if metadata indicating the start and end of an event (e.g., a boss battle) set in the game video is set, the game image identification unit 122 can determine the start and end points of the game video to be cut out by referring to the metadata. Also, if it is known from the metadata that the player character is using a rare weapon, the game image identification unit 122 may read out the game video when the rare weapon is being used.

The game image identification unit 122 may detect the display content and/or display mode included in the game video temporarily stored in the buffer memory, determine the start point and end point of the game video to be cut out, and read out the game video between the start point and the end point. For example, if the game video displays "Mission Completed", the game image identification unit 122 may set the time when the content is displayed as the end point, set a time a predetermined time before the end point as the start point, and read out the game video between the start point and the end point. In addition, if the game video includes a letterbox (black bars displayed at the top and bottom of the screen), the game image identification unit 122 may read out the game video including the letterbox. The game image identification unit 122 may use known pattern recognition technology or scene detection technology to determine the start point and end point of the game video to be cut out.

In addition, after reading out a predetermined amount of game video from the buffer memory, the game image identification unit 122 may input various sections of the game video for the predetermined amount of time into a content generation model for video to perform AI image analysis and obtain a score value for the game video for each section.

FIG. 7(a) shows a schematic diagram of a game video for a predetermined time T. The game image identification unit 122 cuts out this game video at various start and end points, inputs the cut-out game video into a content generation model, and obtains a score value. The game image identification unit 122 compares the score values of the game video cut out at various intervals, and identifies the game video with the highest score value.

7B shows an example of a section of a game video showing the highest score value. In this example, the game video between time _t1 and time _t2 shows the highest score value. From this result, the game image specification unit 122 may determine time _t1 as the start point and time _t2 as the end point, and the content saving unit 124 may save the game video between time _t1 and time _t2 in the content storage unit 154 as new content.

In an embodiment, the learning unit 130 may periodically train the content generation model using user-generated content that has not yet been learned. For example, the learning unit 130 may train the content generation model once on the last day of a month using user-generated content generated in that month. The learning unit 130 may train the content generation model every time new user-generated content is generated, or every time a predetermined number of new user-generated content is generated. The learning unit 130 may train the content generation model using content automatically generated by the game image identification unit 122, that is, content stored in the content storage unit 154. In this case, the learning unit 130 may train the content generation model using content that is highly rated by the user and/or content that is shared with friends among the automatically generated content. Information indicating whether the content is user-generated content or automatically generated content is added to the content as metadata, and the learning unit 130 may train the content generation model so that the learning results of the user-generated content are reflected with a higher weight than the learning results of the automatically generated content.

In an embodiment, the game image specification unit 122 automatically generates game image content that reflects the user's preferences using a trained content generation model that has learned from user-generated content. In a modified example, the game image specification unit 122 may automatically generate game image content using a trained content generation model that has learned from content previously generated by a user different from the user in question. The learning unit 130 collects content previously generated by another user (player) via the network 3 and trains the content generation model.

The learning unit 130 may train the content generation model using content manually captured by a friend. The learning unit 130 may train the content generation model using content manually captured by players who frequently play a game together. The learning unit 130 may train the content generation model using content manually captured by players who frequently share content. The learning unit 130 may train the content generation model using content manually captured by players who play the same game title. The learning unit 130 may train the content generation model using content manually captured by players who play the same game genre.

The learning unit 130 trains the content generation model using content generated by another user, which enables the content generation model to generate content that the other user prefers. Therefore, the game image identification unit 122 will identify game images related to content previously generated by a user other than the user from among images of a game being played by the user. The user may be able to set what type of content the learning unit 130 will learn.

When a game assigns metadata to game video images, the metadata may include, for example, data indicating enemy characters, data indicating the weapon used by the player character, the current position on the map, etc. The learning unit 130 may generate a machine learning model for each matching combination of metadata (e.g., enemy character, weapon used, current position), and when the game image identification unit 122 analyzes game images being played by the user, it may perform AI image analysis using a machine learning model with a matching or similar combination of metadata.

In the embodiment, the game image identification unit 122 performs image analysis on the image of the game that the user is currently playing to identify game images related to past user-generated content, but it may also perform image analysis on images of games that the user has played in the past to identify game images related to past user-generated content. For example, if the storage device 150 stores images of games that the user has played in the past (note that the images are not user-generated content), the game image acquisition unit 120 may acquire images of the game from the storage device 150 and supply them to the game image identification unit 122, and the game image identification unit 122 may perform image analysis on the game images to identify game images related to past user-generated content.

The game image identification unit 122 may set the usage authority of the automatic content generation function according to the user's account information in the information processing system 1. For example, if the user is a paid member of a network service, the game image identification unit 122 allows the user to use the automatic content generation function, and on the other hand, if the user is not a paid member, the game image identification unit 122 does not allow the user to use the automatic content generation function. Note that if a grade is set for paid members, some of the usage of the automatic content generation function may be restricted if the grade is low.

In the embodiment, the game image identification unit 122 performs image analysis on the game image to identify new content, but the game image identification unit 122 may also analyze user voice or game sounds to identify game images related to past content. In the information processing device 10, the user voice acquisition unit 140 acquires voice spoken by the user and supplies it to the game image identification unit 122. When the game image identification unit 122 analyzes the user voice or game sounds to detect excitement, it may cut out a game video image with the timing before the section in which the excitement was detected as the start point and the timing after the section as the end point. For example, when the user is yelling loudly, it is highly likely that the user is excited, so the game image identification unit 122 may cut out a game video image taking the user voice into account.

The content presenting unit 126 may present the game image identified by the game image identifying unit 122 to the user. For example, the content presenting unit 126 provides the output processing unit 114 with a question message such as "Do you want to save this game image?" along with the game image, and the output processing unit 114 displays the game image identified by the game image identifying unit 122 and the question message superimposed on the currently displayed game image. The user can select "Yes" or "No" on the screen in response to the question message. If the user selects "Yes", the game image is saved as new content in the content storage unit 154, and if the user selects "No", the game image may be discarded without being saved in the content storage unit 154. At this time, the user may select whether to save the game image as new content in the content storage unit 154 or discard the game image by uttering "Yes" or "No". Note that the game image for which "Yes" was selected is one that the user has determined to be appropriate as content, and the learning unit 130 may cause the content generation model to learn the game image as an appropriate game image for user-generated content. On the other hand, a game image for which "No" was selected is one that the user judged to be inappropriate as content, and the learning unit 130 may cause the content generation model to learn that game image as a game image that is inappropriate as user-generated content.

In the information processing system 1, the initial learning model may be provided, for example, by the management server 5, and the learning unit 130 may customize the content generation model by learning user-generated content.

<Automatic content sharing function>
FIG. 8 shows another example of the functional blocks of the information processing device 10. The information processing device 10 of the embodiment includes a processing unit 100 and a communication unit 102, and the processing unit 100 has a function of automatically performing a sharing process of content including game images and/or game sounds. The processing unit 100 includes a game image acquisition unit 210, a content identification unit 212, a sharing processing unit 214, a learning unit 220, and an output processing unit 114. The processing unit 100 of the embodiment further includes a functional block (see FIG. 3) that realizes a function of automatically generating content. The storage device 150 has a user-generated content storage unit 152 that stores content (user-generated content) that the user manually generated in the past, and a content storage unit 154 that stores content automatically generated by the processing unit 100 shown in FIG. 3. The storage device 150 may be the auxiliary storage device 2. The communication unit 102 has the functions of both the wireless communication module 38 and the wired communication module 40.

The functions of the components in the information processing device 10 may be implemented in circuitry or processing circuitry, including general purpose processors, application specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), a CPU (a Central Processing Unit), conventional circuits, and/or combinations thereof, configured or programmed to implement the functions described herein. A processor is considered to be a circuitry or processing circuitry that includes transistors and other circuits. A processor may be a programmed processor that executes a program stored in a memory.

In this specification, a circuit, unit, or means is hardware that is programmed to realize or executes the described functions. The hardware may be any hardware disclosed in this specification or any hardware that is programmed to realize or known to execute the described functions.

If the hardware is a processor that is considered to be a type of circuitry, the circuitry, means, or unit may be a combination of hardware and software used to configure the hardware and/or processor.

A user selects content that he or she wishes to share with other users from the storage device 150 and shares the content with other users by uploading it to the content sharing server 9 or providing it to a specific friend. Metadata indicating that the content has been shared is added to the shared content (hereinafter sometimes referred to as "shared content"). Note that the storage device 150 may separately store information that identifies the shared content (identification information) in a storage unit (not shown) of the storage device 150 and manage the identification information of the shared content and the date and time of sharing.

In an embodiment, the shared content is used as learning data for training a shared content identification model, which is a machine learning model (or an artificial intelligence model). The shared content may be used as training data in supervised learning. The learning unit 220 may train the shared content identification model using the shared content shared by one user as training data.

The shared content identification model is trained to input content that is an image of a game that the user has played in the past, and output the degree to which the content reflects the characteristics of the shared content (similarity) as a score value. The similarity indicates the degree to which the content matches the shared content, and the score value may be expressed on a scale of 100 points. The closer the score value is to 100 points, the higher the probability that the content matches the shared content, and the closer the score value is to 0 points, the lower the probability that the content matches the shared content.

If the shared content includes still images (screenshots) and moving images, the learning unit 220 may separately train a shared content identification model for still images and a shared content identification model for moving images. In other words, the shared content identification model for still images learns the still images that are the shared content and extracts the features of the shared content, and the shared content identification model for moving images learns the moving images that are the shared content and extracts the features of the shared content. In a modified example, the learning unit 220 may input the still images and moving images that are the shared content into the shared content identification model without distinguishing between them, and train the shared content identification model.

The learning unit 220 may train a shared content identification model for each game title or game genre. Since the type of game image varies depending on the game title or game genre, it is preferable that the learning unit 220 generates a shared content identification model for each game title or game genre.

The game image acquisition unit 210 acquires content (game images) stored in the storage device 150 that has never been shared, and supplies the content to the content identification unit 212. If metadata indicating that the shared content has been shared is attached, the game image acquisition unit 210 acquires content to which the metadata is not attached. The game image acquisition unit 210 may acquire screenshots or game video images that have never been shared.

The content identification unit 212 performs image analysis of the content acquired by the game image acquisition unit 210 to determine whether the content is related to content previously shared with other users. In the embodiment, the content identification unit 212 performs AI image analysis of the content using a trained shared content identification model. When content is input, the trained shared content identification model outputs the degree to which the content reflects the characteristics of the shared content (similarity) as a score value. At this time, the shared content identification model may output the score value together with information that identifies the content. The information that identifies the content may be the content itself, or may be information indicating the file name of the content and/or an address in the storage device 150.

The content identification unit 212 determines whether or not the content is suitable as shared content based on the score value output by the shared content identification model. For example, the content identification unit 212 may determine that content for which a score value equal to or greater than a predetermined score threshold (e.g., 80 points) is output is suitable as shared content, and may determine that content for which a score value less than the score threshold is output is inappropriate as shared content. The shared content identification model may be configured to output information indicating whether the content is suitable or inappropriate as shared content, and information identifying the content.

FIG. 9 shows an example of a flowchart for processing content sharing. If there is unshared content in the storage device 150 (Y in S20), the content sharing process shown in FIG. 9 is performed. Note that the content sharing process may be performed on the condition that the unshared content has never been evaluated by this flow. The processing unit 100 may perform the content sharing process each time new content is stored in the storage device 150, or may perform the content sharing process each time a predetermined number of new contents are stored in the storage device 150.

The game image acquisition unit 210 acquires unshared content from the storage device 150 and supplies it to the content identification unit 212 (S22). This content is an image of a game that the user has played in the past. The content identification unit 212 inputs the supplied content into a shared content identification model, performs AI image analysis, and acquires a score value for the content. If there are a shared content identification model for still images and a shared content identification model for moving images, if the supplied content is a still image, the content identification unit 212 inputs the content into the shared content identification model for still images to acquire a score value for the content, and if the supplied content is a moving image, the content identification unit 212 inputs the content into the shared content identification model for moving images to acquire a score value for the content.

If the score value of the content is less than a predetermined score threshold (N in S24), the content identification unit 212 determines that the content is not related to past shared content, i.e., the content does not have characteristics in common with past shared content, and is unsuitable as shared content. On the other hand, if the score value of the content is equal to or greater than a predetermined score threshold (Y in S24), the content identification unit 212 determines that the content is related to past shared content, i.e., the content has characteristics in common with past shared content, and is suitable as shared content.

When the content identification unit 212 determines that the content read from the storage device 150 is related to past shared content, the sharing processing unit 214 starts processing to allow the content to be shared with other users. Specifically, the sharing processing unit 214 starts processing to provide the content directly to other users or upload the content to the content sharing server 9 so that other users can view the content. The sharing processing unit 214 may present the content determined to be related to past shared content to the user from the output processing unit 114 (S26).

FIG. 10 shows an example of a screen presenting candidate contents for sharing. The sharing processing unit 214 may present the user with options for candidate contents for sharing and sharing destinations, and inquire of the user whether or not the content can be shared. Note that it is preferable for the sharing processing unit 214 to present candidate contents for sharing to the user when the user is not playing a game. Therefore, while the user is playing a game, the sharing processing unit 214 does not present candidate contents for sharing to the user so as not to disturb the gameplay, and presents candidate contents for sharing to the user after the gameplay has ended. Candidate contents for sharing may be screenshots or game video images.

If the user wishes to share the content with other users, the user selects "Yes", and if not, the user selects "No". In this example, the user operates the input device 6 to select the "Yes" or "No" display area, but the user may be able to choose whether to share the content with other users or not by saying "Yes" or "No". When sharing, it is preferable that the user can specify the sharing destination. In this example, "content sharing server" is selected. When sending content directly to a friend, checking "Select friends" displays a friend list and the user can select one or more friends with whom to share the content. Note that multiple options for "content sharing server" may be provided, and the user may be able to select one or more content sharing servers. The user may also be able to select one or more friends and one or more content sharing servers as sharing destinations.

When the user selects "Yes", the sharing processing unit 214 determines that the user has agreed to the sharing of the content (Y in S28) and transmits the content to the specified sharing destination (S30). This allows the user to share the content with other users without having to manually select the content. Furthermore, the content in question is content that the user has missed out on sharing, and the sharing processing of the embodiment can provide the user with an opportunity to share the content.

On the other hand, if the user selects "No", the sharing processing unit 214 determines that the user has not consented to the content sharing process (N in S28) and does not share the content with other users. The content sharing process ends when it has been performed on all unshared content (N in S20).

In the flowchart shown in FIG. 9, the sharing processing unit 214 presents candidate content to be shared to the user to obtain the user's approval. In a modified example, the sharing processing unit 214 may share content without presenting candidate content to the user. In this case, the sharing processing unit 214 autonomously determines a sharing destination and transmits the content to the sharing destination. The sharing processing unit 214 may determine the sharing destination that has been selected most frequently in the past as the sharing destination of the content. For example, if past statistics show a tendency that screenshots are shared with specific users and videos are uploaded to a content sharing server, the sharing processing unit 214 may determine the sharing destination of the content based on the past statistical data.

In the example screen shown in FIG. 10, the sharing processing unit 214 presents one piece of content that is a candidate for sharing to the user, but a plurality of pieces of content that are candidates for sharing may be presented to the user.
FIG. 11 shows another example of a screen presenting contents that are candidates for sharing. The sharing processing unit 214 may present a plurality of contents that are candidates for sharing to the user and inquire of the user whether or not the contents are to be shared. Although five image frames are shown in FIG. 11, contents are actually displayed in each image frame. The contents that are candidates for sharing may be screenshots or game moving images. The user checks the checkbox of the content that the user wishes to share to select the content to be shared. After selecting the content, the user can share the content with other users by selecting "Yes". In this example, the user operates the input device 6 to select a display area of "Yes" or "No", but the user may be able to select whether or not to share the content with other users by uttering "Yes" or "No".

In an embodiment, the learning unit 220 may periodically train the shared content identification model using shared content that has not yet been learned from among the content (game images) stored in the storage device 150. For example, the learning unit 220 may train the shared content identification model once on the last day of a month using the content shared in that month. The learning unit 220 may train the shared content identification model each time new content is shared, or may train the shared content identification model each time a predetermined number of new contents are shared.

In the embodiment, the content identification unit 212 automatically identifies the content to be shared using a trained shared content identification model that has learned about content shared by a user. In a modified example, the content identification unit 212 may automatically identify the content to be shared using a trained shared content identification model that has learned about content shared by a user different from the user in question. The learning unit 220 collects content shared by other users (players) via the network 3 and trains the shared content identification model.

The learning unit 220 may train the shared content identification model using content shared by friends. The learning unit 220 may train the shared content identification model using content shared by players who frequently play a game together. The learning unit 220 may train the shared content identification model using content shared by players who frequently share content. The learning unit 220 may train the shared content identification model using content shared by players who play the same game title. The learning unit 220 may train the shared content identification model using content shared by players who play the same game genre. The learning unit 220 may also train the shared content identification model using content shared by the same sharing destination. In particular, when there are multiple content sharing servers as sharing destinations and each content sharing server has its own characteristics in the type of distributed content, the learning unit 220 may train the shared content identification model for each content sharing server.

The learning unit 220 trains the shared content identification model using content shared by another user, which enables the shared content identification model to identify content that the other user prefers to share. Therefore, the content identification unit 212 becomes able to identify, from among the user's content, content related to content shared by a user other than the user. The user may be able to set what type of content the learning unit 220 is to learn.

When a game assigns metadata to game video images, the metadata may include, for example, data indicating enemy characters, data indicating the weapon used by the player character, the current position on the map, etc. The learning unit 220 may generate a machine learning model for each matching combination of metadata (for example, enemy character, weapon used, current position), and when the content identification unit 212 analyzes the content, it may perform AI image analysis using a machine learning model with a matching or similar combination of metadata.

In the embodiment, the content identification unit 212 performs image analysis on an image of a game that the user has played in the past to determine whether the game image is related to content that was shared with other users in the past. However, the content identification unit 212 may also perform image analysis on an image of a game that the user is currently playing to determine whether the game image is related to content that was shared with other users in the past. For example, the content identification unit 212 may perform image analysis on a game image acquired by the game image acquisition unit 120 shown in FIG. 3 to determine whether the game image is related to content that was shared with other users in the past.

In the embodiment, the content identification unit 212 performs image analysis of the content to determine whether or not the content is related to the shared content, but the content identification unit 212 may also analyze user voice or game sounds to determine whether or not the content is related to the shared content. In the information processing device 10, the user voice acquisition unit 140 acquires voice spoken by the user and supplies it to the content identification unit 212. When the content identification unit 212 analyzes the user voice or game sounds and detects excitement, it may determine that content as a candidate for sharing.

The present disclosure has been described above based on an embodiment. This embodiment is merely an example, and it will be understood by those skilled in the art that various modifications are possible in the combination of each of the components and each of the processing processes, and that such modifications are also within the scope of the present disclosure.

In the embodiment, the information processing device 10 includes a processing unit 100 that realizes a content generation function and a content sharing function, but the content generation function and the content sharing function may be realized by the information processing device 10 and the management server 5 working together. For example, if the information processing system 1 includes a cloud game server, the content generation function and the content sharing function may be realized by the cloud game server alone, may be realized by the cloud game server and the management server 5 working together, may be realized by the cloud game server and the information processing device 10 working together, or may be realized by the cloud game server, the information processing device 10, and the management server 5 working together.

In the embodiment, in the flow shown in FIG. 9, unshared content stored in the storage device 150 is the subject of the sharing process, but shared content may also be the subject of the sharing process. For example, when the number of sharing destinations increases due to the addition of a new friend or the availability of a new content sharing server in the information processing system 1, the shared content may be reevaluated according to the flow shown in FIG. 9.

In the embodiment, the learning unit 130 trains the content generation model using user-generated content generated by one user as training data. In a modified example, the management server 5 pre-trains the content generation model using user-generated content generated by many users participating in the information processing system 1 as training data, and generates a trained content generation model common to all users (hereinafter also referred to as a "common content generation model"). The management server 5 provides the common content generation model to the information processing device 10, and the learning unit 130 may fine-tune the common content generation model using user-generated content generated by the user operating the information processing device 10. The learning unit 130 may fine-tune the common content generation model for each game title or game genre, or may fine-tune using user-generated content generated by another user. In a modified example, by generating the common content generation model by pre-learning, a highly reliable content generation model can be generated even when there is little user training data.

In the embodiment, the learning unit 220 trains the shared content identification model using the shared content shared by one user as training data. In a modified example, the management server 5 pre-trains the shared content identification model using the shared content shared by many users participating in the information processing system 1 as training data, and generates a trained shared content identification model (hereinafter also referred to as a "common content identification model") common to all users. The management server 5 provides the common content identification model to the information processing device 10, and the learning unit 220 may fine-tune the common content identification model using the shared content shared by the user operating the information processing device 10. The learning unit 220 may fine-tune the common content identification model for each game title or game genre, or may fine-tune using the shared content generated by another user. In a modified example, by generating the common content identification model by pre-training, a highly reliable shared content identification model can be generated even when there is little user training data.

The present disclosure may include the following aspects.
[Item 1]
An information processing system, comprising:
The circuitry includes:
Acquire images of games that the user has played in the past or images of games that the user is currently playing;
performing image analysis on the acquired game image to determine whether the game image is related to content previously shared with other users;
sharing with another user a game image determined to be related to the previously shared content;
Information processing system.
[Item 2]
The circuitry determines whether the acquired game image has common characteristics with content previously shared with other users.
Item 2. An information processing system according to item 1.
[Item 3]
The circuit determines a destination to which the game image is to be shared.
Item 2. An information processing system according to item 1.
[Item 4]
the circuitry presents to the user game images determined to be associated with the previously shared content.
Item 2. An information processing system according to item 1.
[Item 5]
the circuitry presents the user with options for sharing the game image together with the game image;
5. An information processing system according to item 4.
[Item 6]
the circuitry presents game images to the user when the user is not playing a game.
5. An information processing system according to item 4.
[Item 7]
The circuit acquires a screenshot or a game video image as the game image.
Item 2. An information processing system according to item 1.
[Item 8]
1. A method for sharing game images, comprising:
Acquire images of games that the user has played in the past or images of games that the user is currently playing;
performing image analysis on the acquired game image to determine whether the game image is related to content previously shared with other users;
The game image determined to be related to the previously shared content is shared with another user.
[Item 9]
A recording medium on which a program executed in a computer of an information processing device is recorded, the program being recorded on the computer,
A function for acquiring images of games that the user has played in the past or images of games that the user is currently playing;
A function of analyzing the acquired game image and determining whether the game image is related to content previously shared with other users;
A function for allowing game images determined to be related to previously shared content to be shared with another user;
Make this a reality.

The present disclosure can be used in the technical field of storing images of a game that a user is currently playing or has played in the past as content, and/or in the technical field of sharing content with other users.

1: Information processing system, 10: Information processing device, 100: Processing unit, 102: Communication unit, 110: Game execution unit, 112: Game image and sound generation unit, 114: Output processing unit, 120: Game image acquisition unit, 122: Game image identification unit, 124: Content storage unit, 126: Content presentation unit, 130: Learning unit, 140: User voice acquisition unit, 150: Storage device, 152: User-generated content storage unit, 154: Content storage unit, 210: Game image acquisition unit, 212: Content identification unit, 214: Sharing processing unit, 220: Learning unit.

Claims (9)

An information processing system,
a game image acquisition unit that acquires images of a game that the user has played in the past or an image of a game that the user is currently playing;
a content identification unit that performs image analysis on the acquired game image and determines whether the game image is related to content previously shared with other users;
a sharing processing unit that causes a game image determined to be related to the past shared content to be shared with another user;
An information processing system comprising: the content identification unit determines whether the acquired game image has a common feature with content previously shared with another user;
2. The information processing system according to claim 1 . The sharing processing unit determines a sharing destination of the game image.
2. The information processing system according to claim 1 . the sharing processing unit presents to a user a game image determined to be related to the past shared content;
2. The information processing system according to claim 1 . the sharing processing unit presents to the user options for sharing destinations together with the game image;
5. The information processing system according to claim 4. the sharing processing unit presents a game image to the user when the user is not playing the game;
5. The information processing system according to claim 4. the game image acquisition unit acquires a screenshot or a game video image as the game image;
2. The information processing system according to claim 1 . A method for sharing a game image in an information processing system, comprising:
acquiring an image of a game that the user has played in the past or an image of a game that the user is currently playing;
performing image analysis on the acquired game image to determine whether the game image is related to content previously shared with other users;
causing the game images determined to be related to the previously shared content to be shared with another user;
A game image sharing method comprising: On the computer,
A function for acquiring images of games that the user has played in the past or images of games that the user is currently playing;
A function of analyzing the acquired game image and determining whether the game image is related to content previously shared with other users;
A function for allowing game images determined to be related to previously shared content to be shared with another user;
A program to achieve this.