[go: up one dir, main page]

WO2025079731A1 - Appareil de relais d'informations et système de relais d'informations - Google Patents

Appareil de relais d'informations et système de relais d'informations Download PDF

Info

Publication number
WO2025079731A1
WO2025079731A1 PCT/JP2024/036664 JP2024036664W WO2025079731A1 WO 2025079731 A1 WO2025079731 A1 WO 2025079731A1 JP 2024036664 W JP2024036664 W JP 2024036664W WO 2025079731 A1 WO2025079731 A1 WO 2025079731A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
response
video
release
odor
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/JP2024/036664
Other languages
English (en)
Japanese (ja)
Inventor
賢治 久保
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Komi Hakko Corp
Original Assignee
Komi Hakko Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Komi Hakko Corp filed Critical Komi Hakko Corp
Publication of WO2025079731A1 publication Critical patent/WO2025079731A1/fr
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/235—Processing of additional data, e.g. scrambling of additional data or processing content descriptors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/262—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists

Definitions

  • the present invention relates to an information relay device and an information relay system that relays odor and taste information via communication.
  • Patent Document 1 discloses a technology in which a specific ligand is applied to a receptor array in which specific receptors are comprehensively arranged, and the activity of each receptor is measured. Patent Document 1 also discloses a method of reproducing the odor of a target substance by combining two or more standard substances (fragrances) based on the activation levels of each receptor measured for a target substance and a standard substance.
  • Patent Document 1 discloses a method for constructing the odor of a target substance, in which the degree of activation of each mammal-derived olfactory receptor by the target substance is configured based on the degree of activation of each mammal-derived olfactory receptor by two or more standard substances, by combining the two or more standard substances to construct the odor of the target substance.
  • the present invention was made in consideration of such circumstances, and aims to provide an information relay device and information relay system that can synchronize the timing of video playback and odor playback when relaying video information (including audio information) and playing associated odors on the receiving side.
  • the playback information generating means includes in the emission information synchronized with the video information a playback preparation instruction for controlling the timing of the start of preparation for emission corresponding to the next scene from a timing that is a predetermined time earlier.
  • the video information includes a time series of frame images to be played back, and the emission information and playback preparation information are included in a preamble of the frame images.
  • the storage means stores in advance avoidance information that should be avoided from being reproduced on the receiving side, and when the response information corresponds to the avoidance information, the release information determination means determines the release information so that the release corresponding to the avoidance information is not carried out on the receiver side.
  • an information relay system for relaying video information from a transmitter to a receiver and transmitting smell information or taste information that is reproduced in association with the video information on the receiver side
  • the information relay system comprising a transmitter for acquiring and transmitting video information, and an information relay device for relaying the video information
  • the information relay device comprising a response information acquisition means for acquiring response information indicating responses of a plurality of types of olfactory receptors or gustatory receptors, a video information acquisition means for acquiring the video information transmitted from the transmitter, a storage means for storing the response information and the video information, and a system for generating timing information that is detected as a scene change in the video information.
  • the device includes a scene discrimination processing means, a release information determination means for determining release information related to the release of substances related to the reception of multiple types of smells or tastes according to the scene and response information, a playback information generation means for generating the video information and the release information as playback information based on the timing information so that the timing of the start of preparation for release by the release information is advanced by a predetermined time from the point of switching in the video information, and a playback information transmission means for transmitting the playback information to a receiver, and a receiver that receives the playback information, plays an image corresponding to the video information, and releases substances to reproduce the smell or taste according to the release information in the playback information.
  • the transmitter generates response information by analyzing the captured odor or taste and transmits it to the information relay device.
  • the storage means records the registration of response information in advance in association with the image of the object
  • the scene discrimination means detects whether or not an image of the object is present in the video information
  • the emission information determination means reads out the corresponding response information from the storage means and determines the emission information in response to the image of the object being detected in the video information.
  • the transmitter transmits the transmitter's position information in synchronization with the video information
  • the storage means records the registration of the response information in advance in association with the predetermined position information
  • the scene discrimination means detects whether the predetermined position information is present in the video information
  • the emission information determination means reads out the corresponding response information from the storage means and determines the emission information in response to the detection of the predetermined position information in the video information.
  • the information relay device of the present invention makes it possible to synchronize the timing of video playback and odor playback on the receiver side.
  • FIG. 1 is a diagram showing an overview of an odor reproduction method to which an information relay device according to a first embodiment is applied; 2 is a functional block diagram for explaining the configuration of a conversion device U1 functioning as a receptor information determination unit 220 of the first embodiment.
  • FIG. 10 is a conceptual diagram for explaining response information and spray information.
  • FIG. 1 is a functional block diagram of a system configuration of an information relay system (odor reproduction system) according to a first embodiment.
  • 4 is a conceptual diagram for explaining the contents of playback information according to the first embodiment;
  • FIG. FIG. 2 is a block diagram showing a hardware configuration of a server 100 according to the first embodiment.
  • 5A to 5C are diagrams illustrating an example of video relay and spray control processing according to the first embodiment.
  • FIG. 11 is a conceptual diagram showing an overview of the signal flow in each functional block in the video relay and spray control processing.
  • 13A to 13C are diagrams illustrating an example of video relay and spray control processing according to the second embodiment.
  • FIG. 11 is a functional block diagram of a system configuration of an information relay system (odor reproduction system) according to a third embodiment.
  • FIG. 13 is a conceptual diagram for explaining the contents of reproduction information according to the third embodiment.
  • FIG. 11 is a diagram illustrating an example of response information.
  • FIG. 1 is a diagram showing an overview of an odor reproduction method to which an information relay device according to a first embodiment is applied.
  • the odor spray device 400 is provided with multiple types of odor cartridges CA, and the fragrance (e.g., liquid or powder) extracted from the multiple types of odor cartridges CA is sprayed (ejected, sprayed) through a variable spray nozzle (nozzle NO).
  • the fragrance e.g., liquid or powder
  • the receptor information determination unit 220 (conversion device) in FIG. 1 is disclosed in, for example, the following document.
  • Publicly known document 1 JP 2023-81909
  • Publicly known document 2 Japanese Patent No. 7208697
  • Publicly known document 3 JP 2022-11869 A
  • the following provides an overview of the configuration of the receptor information determination unit 220.
  • FIG. 2 is a functional block diagram for explaining the configuration of a conversion device U1 that functions as the receptor information determination unit 220 in the first embodiment.
  • the conversion device U1 is a device that converts output information output from a device (hereinafter also referred to as gas detection device U100) having a predetermined gas detection function for a predetermined odor molecule into response information indicating the response of the olfactory receptor.
  • gas detection device U100 a device having a predetermined gas detection function for a predetermined odor molecule into response information indicating the response of the olfactory receptor.
  • the conversion device U1 converts output information of different values output from each of the gas detection devices U100 using different analysis methods for the same odor molecule into one response information that is a response of the olfactory receptor U200 that obtains the same result.
  • the conversion device U1 can link the output information of the gas detection device U100 to one response information regardless of the difference in the analysis method of the gas detection device U100.
  • the conversion device U1 can link output information of different values and units for each of multiple gas detection devices U100 for one odor molecule to one response information as a standard, so to speak, for the so-called "meter prototype" for the odor.
  • the conversion device U1 converts output information into response information using a learning model that has been machine-learned in advance.
  • the gas detection device U100 is, for example, a gas sensor or a mass spectrometer (gas chromatograph mass spectrometer).
  • the "gas sensor” outputs, as output information, a predetermined electrical signal (e.g., voltage value, resistance value, or current value), a change in frequency, a change in wavelength of light, or the degree of activation due to exposure of the odor molecule to the olfactory receptors, depending on the type and concentration of the odor molecule.
  • the gas detection device U100 outputs an electrical signal with a larger voltage value, for example, the greater the concentration of the odor molecule it comes into contact with.
  • the mass spectrometer outputs, for example, graph data in the form of a plot showing the mass-to-charge relationship and its intensity with respect to the type and concentration of the odor molecule.
  • the olfactory receptor U200 can be realized, for example, by using a known olfactory receptor as shown in the above-mentioned Patent Document 1 and the like.
  • the olfactory receptor U200 is, for example, a nucleic acid mounted in contact with a substrate (not shown).
  • the nucleic acid includes a nucleic acid containing a gene encoding a specific receptor.
  • There are multiple types of nucleic acid in contact with the substrate and the various nucleic acids are arranged on the substrate at a distance from each other.
  • the receptor can cause a change in the cell state by contacting the cell with a test substance. Specifically, a change can occur in the intracellular calcium concentration or the intracellular cAMP concentration. Such a change can be measured using a cAMP-sensitive dye, a cAMP-sensitive fluorescent protein, a calcium-sensitive dye, or a calcium-sensitive fluorescent protein.
  • the degree of receptor activation can be quantitatively calculated by measuring the change in brightness caused by the cAMP-sensitive dye or the cAMP-sensitive fluorescent protein. The degree of activation is used as response information or response information indicating the response of the olfactory receptor U200.
  • the olfactory receptor may be an olfactory receptor of a human, mammal, insect, or nematode.
  • the olfactory receptor is not limited to a liquid test substance, but may be in a form that responds to contact with a gaseous (gaseous) test substance.
  • An olfactory receptor in the latter form can be used as either a gas detection device U100 or an olfactory receptor U200.
  • the olfactory receptor is not limited to the form expressed in the cells described above, but may also be in an acellular form.
  • liposomes formed of a lipid bilayer membrane such as a cell membrane, in which various olfactory receptors are present may be placed on a substrate in a mutually isolated manner.
  • the size of the liposomes is not particularly limited, and may typically be about 100 nm in diameter.
  • the method for producing liposomes is not particularly limited, but may include a process of separating cells expressing olfactory receptors into a cell membrane fraction (which may contain intracellular signaling proteins such as G proteins, adenylate cyclase, and cyclic nucleotide-gated channels) and a cytoplasmic fraction (which may contain intracellular signaling substances such as GDP, GTP, ATP, and cAMP), and mixing and stirring the two to fuse them.
  • a cell membrane fraction which may contain intracellular signaling proteins such as G proteins, adenylate cyclase, and cyclic nucleotide-gated channels
  • a cytoplasmic fraction which may contain intracellular signaling substances such as GDP, GTP, ATP, and cAMP
  • the degree of receptor activation can also be quantitatively evaluated by using these acellular olfactory receptors to measure changes in current, voltage, impedance, etc. caused by the test substance.
  • the conversion unit U12 is realized, for example, by the CPU operating based on a program and data stored in a memory (not shown).
  • the conversion unit U12 converts the acquired output information into response information based on the output signals output from the multiple gas detection devices U100 for each of the multiple odor molecules and the response information indicating the response of the olfactory receptor U200 for each of the multiple odor molecules.
  • the conversion unit U12 converts the acquired output information into response information indicating the response of the olfactory receptor U200 using a machine-learned prediction model with the output information output from the multiple gas detection devices U100 for each of the multiple odor molecules as an explanatory variable and the response information indicating the response of the olfactory receptor U200 for each of the multiple odor molecules as an objective variable.
  • the conversion unit U12 uses a prediction model trained by machine learning using, for example, a numerical value, a function, a spatial or time series index calculated using a mathematical, statistical or machine learning method of the output information output from the gas detection device U100, or a variable newly created by feature engineering, as an explanatory variable.
  • the conversion unit U12 also uses a prediction model trained by machine learning using, for example, a numerical value, a function, a spatial or time series index calculated using a mathematical, statistical or machine learning method of the response information of the olfactory receptor U200, or a variable newly created by feature engineering, as an objective variable.
  • the conversion unit U12 converts the acquired output information into response information using a machine-learned prediction model that uses, for example, the time series data and feature patterns of the output information output from the multiple gas detection devices U100 as explanatory variables, and the time series data and feature patterns of the response information of the multiple olfactory receptors U200 as objective variables.
  • the conversion unit U12 uses a prediction model trained by machine learning using the presence or absence of output information from multiple gas detection devices U100 as an explanatory variable. Also, in the first embodiment, the conversion unit U12 uses a prediction model trained by machine learning using the presence or absence of reaction of multiple receptors included in the olfactory receptor U200 as an objective variable. Also, in the first embodiment, the conversion unit U12 uses a prediction model trained by machine learning using the mass-charge and intensity of the output information output from the gas detection device U100 as explanatory variables. In this way, the conversion unit U12 converts a conversion voltage value indicated by a voltage value, resistance value, current value, frequency change, or light wavelength change into response information indicating the degree of activation of the olfactory receptor U200.
  • the output unit U13 is realized, for example, by the operation of the CPU.
  • the output unit U13 outputs the converted response information.
  • the output unit U13 may be configured, for example, to convert the activation level into a numerical value and output it as the response information.
  • Figure 3 is a conceptual diagram for explaining response information and spray information.
  • the odor spray device 400 is provided with multiple types of odor cartridges CA, and the fragrance (e.g., gas, liquid, or powder, etc.) extracted from the multiple types of odor cartridges CA is sprayed (ejected, sprayed) through a variable spray nozzle (nozzle NO).
  • the fragrance e.g., gas, liquid, or powder, etc.
  • a fragrance is released from the release device (scent spray device 400), but the released object is not limited to fragrance, and any substance related to the reception of smells or tastes may be used.
  • substances related to the reception of smells or tastes include agonists, antagonists, modulators, etc.
  • An agonist is a substance that enhances or exerts an effect when it binds to a receptor.
  • An antagonist is a substance that acts inhibitoryly when it binds to a receptor or prevents other agonists from binding.
  • a modulator is a substance that does not itself respond to a stimulus, but amplifies or decreases the stimulus response of an agonist or antagonist.
  • a modulator is a substance that binds to a site other than the ligand binding site and increases or decreases the action of an olfactory or gustatory receptor.
  • Figure 3 shows an example of the process from extracting features from the captured odor, generating response information, and finally generating spray information.
  • the receptor information determination unit 220 extracts the features of the target odor based on the measured activity of each receptor.
  • the method for extracting the features is not particularly limited, but for example, the method described in Patent Document 1 can be used.
  • the response information is a characteristic amount for odor molecules in multiple types of receptors, and is information indicating at least one characteristic amount among, for example, response intensity, area of response intensity, response persistence, response speed, peak time, response onset, and number of peaks.
  • the response information is not limited to the above, and may be various parameters for odor molecules in multiple types of receptors.
  • the two-dimensional code comprises a data section in which a bit string corresponding to target data is represented as a pattern in which white or black cells, which are the constituent units of the symbol, are arranged in a matrix, and one or more finder patterns arranged separately from the data section.
  • the data section includes information (e.g., the response characteristics and response information described above) based on receptor response information obtained using olfactory receptors or gustatory receptors that are responsive to extracellular substances (e.g., odor molecules).
  • the server 100 converts the response information into spray information for spraying the reproduced odor.
  • spray information for cartridges CA1 to CA4 corresponding to receptors OR1 to OR4 is shown.
  • cartridge CA1 is filled with a fragrance for stimulating receptor OR1.
  • FIG. 3 illustrates an example in which receptors and cartridges correspond one-to-one, this is not limiting, and more or fewer types of cartridges may be used than the number of types (of) receptors.
  • one cartridge CA may be able to provide (agonize) or adjust (antagonize or modulate) responses from multiple receptors. In this way, by using a cartridge CA that stimulates multiple receptors, the size of the odor spray device 400 can be reduced and costs can be kept down.
  • the spray information is information for reproducing a target scent by spraying one or a combination of two or more types of fragrances, and examples of this include the fragrance spray time, spray amount, spray temperature, number of sprays, amount of nozzle narrowing, spray direction, spray flow rate, etc.
  • the odor spraying device 400 sprays fragrance based on the above-mentioned spray information, thereby providing the viewer of the content with an odor (aroma) that reproduces the target odor.
  • the odor spray device 400 can be a display device (television), a smartphone, a projection device (projector), an HMD (head-mounted display), or any other device.
  • a specific odor specified by the user may be sprayed.
  • the odor items and the response information or spray information corresponding to the odor items are stored in advance in a specific memory unit.
  • an N-dimensional code that replaces the target odor with electronic data is used as the digital odor information, but for example, the extracted feature amount (code ST2) or secondary receptor response information (code ST3) may be used as the digital odor information.
  • the feature amount, secondary receptor response information, or N-dimensional code may each be considered to indicate the data structure of the receptor response information used in the server 100 (information relay device).
  • a substance related to odor reception e.g., a fragrance
  • a substance related to taste reception e.g., a seasoning
  • FIG. 4 is a functional block diagram of the system configuration of the information relay system (odor reproduction system) according to the first embodiment.
  • the information relay system is based on the premise that the odor information to be relayed is collected by a collection device CO as described in FIG. 1, and is composed of a data acquisition transmitter 200 on the transmitting side, a server 100 as an information relay device, and a content playback device 300, an image display device and audio playback device 500, and an odor spray device 400 on the receiving side, all connected to each other via a predetermined network 2 such as broadcast waves or the Internet.
  • a predetermined network 2 such as broadcast waves or the Internet.
  • the data acquisition transmitter 200 includes an odor analysis unit 210 (gas detection device U100) and a receptor information determination unit 220 (receptor information determination device 2).
  • the server 100 cooperates with the receptor information determination unit 220, the content playback device 300, the image display device and audio playback device 500, and the odor spray device 400 to carry out various processes for relaying.
  • the image display device and audio playback device 500 will be described as a television receiver 500.
  • the odor collection device CO collects the odor corresponding to the content. As shown in FIG. 1, the odor may be collected in real time when the video related to the content is shot, or the odor may be collected at a time separate from the time when the video related to the content is shot.
  • the odor analysis unit 210 analyzes the odor captured by the odor capture device CO described above.
  • the receptor information determination unit 220 may analyze the components contained in the odor (e.g., gas or liquid), but in the first embodiment, it analyzes receptor response information (the results of measuring the activity of each receptor by acting a specific ligand on a receptor array in which receptors are comprehensively arranged) obtained using olfactory receptors that are responsive to the above-mentioned odors (odor molecules). This makes it possible to reproduce a target odor using components different from the target odor.
  • the receptor information determination unit 220 may analyze the response characteristics using receptors as described in Patent Document 1, or may analyze the response characteristics from components contained in odor molecules.
  • the content playback device 300 displays (plays) video content, etc. on the television receiver 500.
  • the content playback device 300 also transmits spray information to the scent spray device 400 so that fragrance is sprayed from the scent spray device 400 at a predetermined timing in the video content.
  • a content playback device 300 is provided because a television receiver 500 such as digital signage is also assumed, but content may be sent directly from the server 100 to the television receiver 500, and spray information may be sent directly from the server 100 to the odor spray device 400.
  • the television set 500 displays content.
  • the television set 500 and the odor spraying device 400 are provided independently, but the television set 500 and the odor spraying device 400 may be an integrated device.
  • the television set 500 is a display device, but this is not limited thereto, and the television set 500 may be, for example, a projection device such as a projector.
  • the odor spraying device 400 reproduces the target odor by spraying fragrance based on the spray information.
  • the odor spraying device 400 is provided with an attachment portion (not shown) for attaching a cartridge CA (odor cartridge) and a nozzle NO (variable spray nozzle), but is not limited to this.
  • the cartridge CA is a cartridge for filling with fragrance to be sprayed in response to processing by a specific information relay system (server 100, content playback device 300, etc.), and is a cartridge filled with a composition that selectively causes only a specific receptor to respond.
  • the composition is not limited to one that selectively causes only a specific receptor to respond, but may be a composition that causes multiple receptors to respond. Note that multiple types of cartridges CA with different compositions are attached to the odor spray device 400.
  • Nozzle NO is a spray nozzle that sprays a predetermined fragrance based on spray control (spray information) by a predetermined information relay system (server 100, content playback device 300, etc.).
  • a predetermined information relay system server 100, content playback device 300, etc.
  • the CPU (Central Processing Unit) 11 of the server 100 when it operates, it processes a program, and thereby an information receiving unit 30, a model information acquiring unit 33, an image scene determining unit 34, an audio scene determining unit 35, a spray information determining unit 36, a playback information generating unit 37, and a playback information transmitting unit 38 function. These are implemented, for example, as program modules, but are not limited thereto.
  • the information receiving unit 30 includes a response information acquiring unit 31 and a video information acquiring unit 32.
  • "Video information" includes information for playing the video to be relayed (for example, information on time-series image frames) as well as audio information synchronized therewith.
  • playback information is information that integrates and organizes both “time-series video information (including audio information as necessary)” and “spray information” so that the playback timing of the two can be synchronized, and is transmitted to the receiving side (playback side).
  • the response information acquisition unit 31 acquires receptor response information obtained using multiple types of receptors, including olfactory receptors (or gustatory receptors) that are responsive to extracellular substances. Specifically, the response information acquisition unit 31 acquires the above-mentioned response information analyzed using the receptor information determination unit 220.
  • the acquired response information (receptor response information) is stored in a response information database (hereinafter, response information DB) 41.
  • response information DB response information database
  • the response information is stored in association with the above-mentioned content (or time information of the content) as described below. In this way, for example, it is possible to reproduce the target odor at an appropriate timing in the content.
  • the model information acquisition unit 33 may also acquire information on the fragrance (cartridge). This makes it possible to refer to a table (or learning model) corresponding to the cartridge (type) when associating receptor information with spray information in the process of determining spray information, which will be described later.
  • the image scene discrimination processing unit 34 detects the timing of switching between image scenes, as described below. Then, for each image scene, it associates it with the response information for reproducing the smell (or taste) acquired by the response information acquisition unit 31, and stores it in the video information DB 43.
  • Information about the detected scene changes is stored in the video information DB 43 together with the video information in association with each scene.
  • the spray information determination unit 36 determines spray information regarding the scent spray of multiple types of fragrances according to the response information.
  • the correspondence table is set so that the response information and the spray information are associated with each other in the following correlation relationship.
  • response intensity refers to the maximum peak value of the strength of the response of the olfactory receptor
  • response speed refers to, for example, the speed at which the first response peak rises
  • number of peaks refers to the number of peaks
  • response durability refers to the time from when the response intensity first rises above a predetermined value until the response intensity reaches a steady value below a predetermined value.
  • the relationship between the "response intensity” and the amount of sprayed "fragrance” corresponding to the "response intensity” is, for example, as follows.
  • the "fragrance” corresponding to the response of the olfactory receptor 01 can be a combination of multiple types of fragrances in a predetermined ratio, and the following explanation is an example of the conditions for "spraying" one of the multiple types of fragrance.
  • the higher the “response duration" the longer the "fragrance spray time”.
  • response information and spray information are just examples, and are not limiting. These pieces of information may be combined. For example, a combination such as the higher the "response speed” and “response rise time,” the higher the “fragrance spray temperature” may be.
  • the spray information determination unit 36 uses a learning model to determine spray information based on the response information.
  • the learning model is a machine learning model that uses learning data (teacher data) that is a set of receptor response information and spray information corresponding to the receptor response information to learn the correspondence between the two, and outputs spray information from the response information.
  • learning data teacher data
  • the above-mentioned learning model is stored in the correspondence DB 44.
  • the learning method in machine learning is not particularly limited, and may be unsupervised learning, reinforcement learning, deep learning, etc. Also, multiple learning methods may be combined.
  • the spray information determination unit 36 may determine the spray information taking into account the above-mentioned model information. That is, the spray information determination unit 36 may determine the spray information using the above-mentioned correspondence table or learning model according to the number (and type) of cartridges CA included in the model information.
  • the spray information determination unit 36 may also determine spray information regarding the odor spray in accordance with spatial parameters in addition to the response information described above.
  • the spatial parameters are parameters related to the state of the space into which the odor is sprayed or the positional relationship with the subject (subject or object) onto which the odor is sprayed, and specifically include various parameters such as the size of the room or space into which the odor is sprayed, the distance between the odor spray device 400 and the nose, air volume, wind direction, temperature, humidity, etc.
  • the spatial parameters can be configured so that a specified acquisition unit (not shown) acquires the spatial parameters sensed by sensors provided on the odor spray device 400 or other external devices.
  • the spray information determination unit 36 can control the spray direction, spray amount, spray concentration, etc., based on the distance from the nose, temperature, humidity, etc.
  • the spray information determination unit 36 may control the spray direction, spray amount, spray concentration, etc. based on the size of the space, air volume, wind direction, temperature, humidity, etc.
  • the correspondence between the spatial parameters and the spray information may be established using a correspondence table such as that described above, or may be established using a learning model.
  • the playback information generating unit 37 (a means of emission control) generates playback information for the video information stored in the video information DB 43 by embedding, in the video information, instructions to prepare spray information for reproducing the scent of the next scene at a timing shifted forward a predetermined time from the associated scene change information.
  • FIG. 5 is a conceptual diagram for explaining the contents of the playback information in the first embodiment.
  • the image signal and audio signal are signals for each frame, and Figure 5 shows the concept of a situation where scene 1 switches to scene 2.
  • the image scene discrimination processing unit 34 and the audio scene discrimination processing unit 35 detect the timing of switching from scene 1 to scene 2.
  • spray information corresponding to odor 1 in scene 1 is associated with the reproduction of scene 1
  • spray information corresponding to odor 2 in scene 2 is associated with the reproduction of scene 2.
  • the configuration can be such that the spray information for odor 1 or the spray information for odor 2 is stored in an empty area in the preamble of this frame signal.
  • the odor reproduction control signal includes a signal (reproduction preparation signal) that instructs preparation for reproduction of odor 2.
  • the spraying process for reproducing the next odor is advanced by a certain time and a reproduction preparation signal is included, and advance preparation is started in the odor reproduction process on the odor spraying device 200 side, thereby suppressing the time delay in the production of the odor of the next scene.
  • “advance preparation” is assumed to be, for example, in a configuration in which the odor spraying device 200 has a mixing chamber for the fragrance for the currently reproduced odor and a mixing chamber for the fragrance for the next odor independently provided, in which advance mixing of the fragrance for the next odor is started in advance in a mixing chamber different from the current odor.
  • “advance preparation” is not limited to such a configuration, and the timing at which the reproduction preparation signal starts to be included in the reproduction information may be changed according to a parameter (not shown) of a table set in advance depending on the model of the odor spraying device 200, the configuration of the odor reproduction process, the type of odor to be reproduced, the reproduction conditions, etc.
  • the spray information is information for controlling the spraying of at least one of the multiple types of fragrances (multiple types of cartridges CA).
  • the playback information generated by the playback information generating unit 37 is transmitted by the playback information transmitting unit 38 to the odor spraying device 400 (television receiver 500) via the communication unit 19. ⁇ Hardware Configuration>
  • FIG. 6 is a block diagram showing the hardware configuration of the server 100 according to the first embodiment.
  • the server 100 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an input/output interface 15, an output unit 16, an input unit 17, a memory unit 18, a communication unit 19, and a drive 20.
  • a CPU Central Processing Unit
  • ROM Read Only Memory
  • RAM Random Access Memory
  • the storage unit 18 includes, but is not limited to, non-volatile storage devices such as a hard disk drive (HD), a solid state drive (SSD), and flash memory.
  • non-volatile storage devices such as a hard disk drive (HD), a solid state drive (SSD), and flash memory.
  • the CPU 11 executes various processes according to programs recorded in the ROM 12 or programs loaded from the storage unit 18 to the RAM 13.
  • the RAM 13 also stores data and the like required for the CPU 11 to execute various processes as appropriate.
  • the RAM 13 is composed of, for example, a DRAM (Dynamic Random Access Memory) and stores various data.
  • the CPU 11, ROM 12, and RAM 13 are interconnected via a bus 14.
  • An input/output interface 15 is also connected to this bus 14.
  • the input/output interface 15 is connected to an output unit 16, an input unit 17, a memory unit 18, a communication unit 19, and a drive 20.
  • the output unit 16 is made up of a display, a speaker, etc., and outputs various information as images and sounds.
  • the input unit 17 is made up of a keyboard, a mouse, etc., and inputs various information.
  • the communication unit 19 communicates with other devices via a network N, including the Internet.
  • the drive 20 is fitted with a removable medium 21, which may be a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, as appropriate. Programs read from the removable medium 21 by the drive 20 are installed in the storage unit 18 as necessary.
  • the removable medium 21 can also store various data stored in the storage unit 18 in the same way as the storage unit 18.
  • the collection device CO, receptor information determination unit 220, content playback device 300, television receiver 500, and odor spray device 400 have basically the same hardware configuration as that shown in Figure 6 for their control. ⁇ Odor spray device 400>
  • the odor spraying device 400 has an attachment part and an emission outlet (nozzle NO), and the CPU (not shown) of the odor spraying device 400 may be configured to function as an emission control part (emission control means) when the odor spraying device 400 is in operation.
  • the release control unit is a functional unit that releases at least one of the substances related to the reception of multiple types of odors or tastes from the cartridge CA based on release information related to the release of the substance related to the reception of multiple types of odors or tastes.
  • the release control unit may also determine whether the cartridge CA meets a predetermined standard (authenticity determination). For example, the release control unit may make the above-mentioned determination by acquiring information in an IC chip provided in the cartridge CA.
  • the cartridge CA is detachably mountable to the mounting unit of the odor spray device 400, and has a case for storing substances related to the reception of multiple types of odors or tastes, an IC chip, and an interface for transmitting and receiving data between the IC chip and the release control unit.
  • the IC chip includes a non-volatile memory for storing data and a calculation device for control processing.
  • This configuration is used because, when reproducing an odor in the odor spray device 400, there is a risk that the appropriate odor (or taste) cannot be reproduced if the quality is not as intended, such as if the fragrance in the cartridge CA has deteriorated over time or has been reused and filled with a fragrance that does not meet the predetermined standard.
  • the release control unit may issue an alert indicating deterioration due to aging, output a message encouraging the cartridge CA to be replaced, or restrict the use of the odor spray device 400.
  • the timing at which the cartridge CA was first used is stored in an IC chip provided in the cartridge CA, as described above.
  • non-volatile memory of the IC chip provided in the cartridge CA stores identification information for the cartridge CA, and the identification information and information associated with the identification information, such as the start timing of use, may be stored in the memory unit 18 in the server 100 described above.
  • the release control unit may update information such as the above-mentioned start timing of use when the cartridge CA meets certain criteria.
  • the release control unit may also adjust the release information according to the usage state of the cartridge CA. For example, when using a cartridge that has been in use for a predetermined period of time, the release control unit may spray more fragrance from the cartridge CA than usual or may extend the spray time. ⁇ Processing details>
  • FIG. 7 shows an example of video relay and spray control processing according to the first embodiment.
  • the spray control process is performed by the odor spray device 400, but this is not limited thereto, and for example, the spray control process may be performed from the server 100 side.
  • Figure 8 is a conceptual diagram showing an overview of the signal flow in each functional block in the video relay and spray control process in Figure 7.
  • the camera 2 and microphone 4 acquire image and audio information at the relay source location (S100), and the odor collected by the collector CO is analyzed by the odor analyzer 210 (S102).
  • the camera position may be determined by a positioning device (e.g., GPS: Global Positioning System) 6 in conjunction with the shooting of the video, and the video information may include the position information.
  • the captured image information and audio information are converted into digital data by the video/audio encoding unit 8 and transmitted from the data transmission/reception unit 10 to the server 100.
  • the signal from the odor analysis unit 210 is converted into response information by the receptor information determination unit 220 and transmitted to the server 100 as relay information in synchronization with the image information (S104).
  • the information acquisition unit 30 receives the video information and response information (S200), and the model information acquisition unit 33 acquires model information about the odor spray device (S202).
  • the image scene discrimination processing unit 34 and the audio scene discrimination processing unit 35 detect the timing of the scene change (S204) as described in FIG. 5, and the spray information determination unit 36 determines the spray information based on the model information and response information (S206).
  • the playback information generating unit 37 generates the playback information as shown in FIG. 5 by encoding it into a format that compiles the video data in accordance with a specified standard (S208), and the playback information transmitting unit 38 relays and transmits the playback information to the content playback device 300 via the communication unit 19 (S210).
  • VR goggles can function as the above-mentioned scent spray device, further enhancing the sense of immersion.
  • a display type equipped with a smell spraying function can be prepared, and by having the customer select a specific menu (by pressing a button), a smell suited to the food can be instantly reconstructed and sprayed along with the video of the food. By changing the pattern of this reconstruction, it is possible to accommodate the smells of various foods.
  • the above-described embodiment can also be used for presentations. For example, when developing a new product, it may be necessary to explain the product in an online meeting. In particular, scent can be an important factor for food and cosmetics. However, it is difficult for people in remote locations to smell the scent directly. By applying the above-described embodiment, it becomes possible to share the scent with people in remote locations on the other side of the screen in a similar way to sharing a screen, which is expected to increase the chances of successful business negotiations.
  • odors are collected in real time by the odor collection device CO in response to each image scene, and response information is generated for each image scene in the receptor information determination unit 220.
  • the response information of the "corresponding odor” is stored in advance in the response information DB 41, and it is also possible to configure it so that, for example, when the image scene discrimination processing unit 34 detects that the "object" specified in advance has appeared in the image information, it is associated with that information and stored in the video information DB 43.
  • the spray information determination unit 36 generates spray information from the model information, etc.
  • scene 1 is an image of a street
  • scene 2 is an image of the inside of a ramen restaurant.
  • response information of a "certain ramen smell" that is set in advance in association with the image of the specified ramen is stored in the response information DB 41, and spray information and further playback information are generated based on this.
  • the specified odor may be configured to be sprayed at a predetermined timing.
  • an image of the object corresponding to the odor item and response information or spray information corresponding to the odor item are assumed to be stored in a predetermined memory unit in advance.
  • FIG. 9 is a conceptual diagram showing an overview of the signal flow in each functional block in the video relay and spray control processing of embodiment 2.
  • FIG. 9 is a diagram to be compared with FIG. 8 of the first embodiment, and the same parts are given the same reference numerals and their description will not be repeated.
  • FIG. 9 as well, the main components of the configuration shown in FIG. 4 are extracted and shown, and the configuration for capturing and transmitting moving images is also added.
  • FIG. 10 shows an example of video relay and spray control processing according to the second embodiment.
  • the spray control process is performed by the odor spray device 400, but this is not limited thereto, and for example, the spray control process may be performed from the server 100 side.
  • the camera 2 and microphone 4 acquire image and audio information at the relay source location (S100).
  • the camera position may be determined by a positioning device (e.g., GPS: Global Positioning System) 6 in conjunction with the shooting of the video, and the video information may include the position information.
  • a positioning device e.g., GPS: Global Positioning System
  • the captured image information and audio information are converted into digital data by the video/audio encoding unit 8 and transmitted as relay information from the data transmission/reception unit 10 to the server 100 (S104).
  • the server 100 associates response information corresponding to the object with the image of the object and stores it in advance in the response information DB 41 of the storage device 18.
  • response information corresponding to the location information is associated with previously registered location information and stores it in advance in the response information DB 41 of the storage device 18.
  • the information acquisition unit 30 receives the video information and response information (S200), and the model information acquisition unit 33 acquires model information of the odor spray device (S202).
  • the image scene discrimination processing unit 34 and the audio scene discrimination processing unit 35 detect the timing of scene changes as described in FIG. 5 (S204). At this time, for example, the image scene discrimination processing unit 34 also detects whether an image of a previously registered object is included in the received image information. Alternatively, the image scene discrimination processing unit 34 also detects whether the location information included in the received image information corresponds to the previously registered location information. Note that the audio scene discrimination processing unit 35 may be configured to detect whether the location information included in the received audio information corresponds to the previously registered location information.
  • the spray information determination unit 36 reads out response information 41.1 corresponding to the object or response information 41.2 corresponding to the position information from the response information DB 41 (S205).
  • the spray information determination unit 36 determines the spray information based on the model information and the response information (S206).
  • the playback information generating unit 37 generates the playback information as shown in FIG. 5 by encoding it into a format that compiles the video data in accordance with a specified standard (S208), and the playback information transmitting unit 38 relays and transmits the playback information to the content playback device 300 via the communication unit 19 (S210).
  • the content playback device 300 receives the playback information (S300), decodes it (S302), the display device/audio playback device 500 plays the image and audio (S304), and the odor spray device 400 executes the odor playback process (S306).
  • the server 100 of this embodiment functions as an information relay device for video information that includes information for reproducing odors.
  • the smell of a hot spring will be played on the receiving side in sync with the playback of the video.
  • response information for smells or tastes
  • response information it is not necessarily necessary that response information be prepared in advance for all target images.
  • target images and their corresponding response information are successively compiled as a database in accordance with the operation of the information relay system, for example, when image information and target location information of a target object are preregistered, the smell (or taste) may be reproduced on the receiving side from the registered response information within the range selected by the registrant.
  • the spray information determination unit 36 unconditionally determines the spray information based on the response information sent from the sender (or the response information registered in advance) as long as the model information and response information exist.
  • FIG. 11 is a functional block diagram of the system configuration of an information relay system (odor reproduction system) according to the third embodiment.
  • response information for odors whose reproduction should be avoided is registered in the avoidance information DB 46 in accordance with registration from the administrator terminal 600.
  • the spray information determination unit 36 when converting the response information into spray information, may determine whether the converted spray information corresponds to the reproduction of the same odor (or taste) even if it is not the response information registered in the avoidance information DB 46, and if it corresponds to the same odor (or taste), determine the spray information to prohibit the reproduction of such odor (or taste).
  • FIG. 12 is a conceptual diagram for explaining the contents of the playback information in the third embodiment.
  • Figure 12 shows the concept of a situation in which scene 1 switches to scene 2.
  • the image scene discrimination processing unit 34 and the audio scene discrimination processing unit 35 detect the timing of switching from scene 1 to scene 2.
  • the spray information corresponding to odor 1 in scene 1 is set to prohibit reproduction of the odor.
  • reproduction prohibition information is added to the spray information for odor 1, and the spray information corresponding to odor 2 in scene 2 is associated with the reproduction of scene 2.
  • this is not particularly limited, but for example, if the video information is time-series data of a frame signal for each scene, it is possible to configure it so that the spray information for odor 1 and reproduction prohibition information, or the spray information for odor 2, are stored in the free space in the preamble of this frame signal.
  • the spray information determined by the spray information determination unit 36 is configured to include information indicating prohibition of regeneration. However, for response information for which regeneration is prohibited, the spray information determination unit 36 may not generate the spray information itself.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

L'invention concerne un appareil de relais d'informations qui est apte à synchroniser une synchronisation de reproduction vidéo et une synchronisation de reproduction d'odeur. Un serveur 100 reçoit, conjointement avec des informations d'une odeur ou des informations d'un goût, au moins des informations vidéo associées provenant d'un émetteur et les transmet à un récepteur. Dans le serveur (100), une unité de traitement de détermination de scène d'image 34 génère des informations de synchronisation qui doivent être détectées en tant que commutation d'une scène dans les informations vidéo, et une unité de détermination d'informations de pulvérisation 36 détermine, conformément à des informations de réponse de récepteur correspondant à la scène et à l'odeur, des informations d'émission concernant l'émission de substances associées à la réception d'une pluralité de types d'odeurs ou de goûts. Une unité de génération d'informations de reproduction 37 génère, sur la base des informations de synchronisation, les informations vidéo et les informations d'émission en tant qu'informations de reproduction de sorte que la synchronisation du début de préparation pour l'émission selon les informations d'émission est définie plus tôt selon une période prescrite par rapport à la synchronisation de la commutation dans les informations vidéo.
PCT/JP2024/036664 2023-10-13 2024-10-15 Appareil de relais d'informations et système de relais d'informations Pending WO2025079731A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2023177863 2023-10-13
JP2023-177863 2023-10-13

Publications (1)

Publication Number Publication Date
WO2025079731A1 true WO2025079731A1 (fr) 2025-04-17

Family

ID=95395950

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2024/036664 Pending WO2025079731A1 (fr) 2023-10-13 2024-10-15 Appareil de relais d'informations et système de relais d'informations

Country Status (1)

Country Link
WO (1) WO2025079731A1 (fr)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2001089590A1 (fr) * 2000-05-26 2001-11-29 Yukinobu Tajima Procede et appareil de generation d'odeur
WO2005048541A1 (fr) * 2003-11-12 2005-05-26 Electronics And Telecommunications Research Institute Dispositif et procede destines a transmettre et synchroniser des informations multisensorielles avec des donnees audio/video
JP2006295321A (ja) * 2005-04-06 2006-10-26 Tokai Univ 匂い提示情報付き映像コンテンツ及びその上映システム
JP2009049746A (ja) * 2007-08-21 2009-03-05 Pioneer Electronic Corp 芳香伝達装置及び芳香伝達方法
JP2009265453A (ja) * 2008-04-28 2009-11-12 Hitachi Ltd 視覚情報と融合した嗅覚情報提示装置
WO2016072120A1 (fr) * 2014-11-07 2016-05-12 ソニー株式会社 Système de traitement d'informations, procédé de commande et support de stockage
JP2020005168A (ja) * 2018-06-29 2020-01-09 株式会社東芝 カメラ装置、カメラシステム及びサーバ装置
WO2020095530A1 (fr) * 2018-11-06 2020-05-14 ソニー株式会社 Dispositif de traitement d'informations, procédé de traitement d'informations et programme

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2001089590A1 (fr) * 2000-05-26 2001-11-29 Yukinobu Tajima Procede et appareil de generation d'odeur
WO2005048541A1 (fr) * 2003-11-12 2005-05-26 Electronics And Telecommunications Research Institute Dispositif et procede destines a transmettre et synchroniser des informations multisensorielles avec des donnees audio/video
JP2006295321A (ja) * 2005-04-06 2006-10-26 Tokai Univ 匂い提示情報付き映像コンテンツ及びその上映システム
JP2009049746A (ja) * 2007-08-21 2009-03-05 Pioneer Electronic Corp 芳香伝達装置及び芳香伝達方法
JP2009265453A (ja) * 2008-04-28 2009-11-12 Hitachi Ltd 視覚情報と融合した嗅覚情報提示装置
WO2016072120A1 (fr) * 2014-11-07 2016-05-12 ソニー株式会社 Système de traitement d'informations, procédé de commande et support de stockage
JP2020005168A (ja) * 2018-06-29 2020-01-09 株式会社東芝 カメラ装置、カメラシステム及びサーバ装置
WO2020095530A1 (fr) * 2018-11-06 2020-05-14 ソニー株式会社 Dispositif de traitement d'informations, procédé de traitement d'informations et programme

Similar Documents

Publication Publication Date Title
JP5299018B2 (ja) 情報処理装置、コンテンツ処理方法及びプログラム
US7095450B1 (en) Method and apparatus for generating a display signal
Freeman et al. Focus group exploration of presence through advanced broadcast services
Doumanoglou et al. Quality of experience for 3-D immersive media streaming
CN103024293B (zh) 动态地替换及插入节目内容中的电影舞台道具
US20150199995A1 (en) Modular content generation, modification, and delivery system
JP2011086283A (ja) 動的レンダリング用３次元ビデオキャプチャ・ワークフロー用のシステム及び方法
JP2012198694A (ja) 匂いを伴う音声付動画の提供システム及び匂い情報データベース
KR20100033954A (ko) 실감 효과 표현 방법 및 장치
KR20220062591A (ko) 장면의 이미지 캡처의 품질을 평가하기 위한 장치 및 방법
US8767067B2 (en) Broadcasting system, sending apparatus and sending method, receiving apparatus and receiving method, and program
WO2025079731A1 (fr) Appareil de relais d'informations et système de relais d'informations
EP4488678A1 (fr) Dispositif de traitement d'informations, cartouche, dispositif de libération, code à n dimensions, procédé de commande et programme informatique
US20230031160A1 (en) Information processing apparatus, information processing method, and computer program
US20010017671A1 (en) "Midlink" virtual insertion system and methods
Brkic et al. Cross-modal affects of smell on the real-time rendering of grass
JP4250814B2 (ja) ３次元映像の送受信システム及びその送受信方法
JP4742283B2 (ja) 匂い提示情報を付加する方法及び上映システム
WO2023234348A1 (fr) Système, appareil de gestion, dispositif de reproduction et cartouche
JP2000184353A (ja) 撮影装置及び情報付加装置
CN110244582A (zh) 一种数字化气味数据关联平台
CN112528050A (zh) 一种多媒体互动系统及方法
CN101017696B (zh) 标签信息产生装置和标签信息产生方法
KR20220147239A (ko) Ott 영상 분석용 관리서버 및 그 관리방법
KR20160092299A (ko) 영상의 발향 정보를 추출하는 영상 분석 장치 및 발향 추출 방법

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24877325

Country of ref document: EP

Kind code of ref document: A1