[go: up one dir, main page]

WO2017155126A1 - Système de transmission d'informations, dispositif d'envoi d'informations, dispositif de réception d'informations, et programme informatique - Google Patents

Système de transmission d'informations, dispositif d'envoi d'informations, dispositif de réception d'informations, et programme informatique Download PDF

Info

Publication number
WO2017155126A1
WO2017155126A1 PCT/JP2017/010290 JP2017010290W WO2017155126A1 WO 2017155126 A1 WO2017155126 A1 WO 2017155126A1 JP 2017010290 W JP2017010290 W JP 2017010290W WO 2017155126 A1 WO2017155126 A1 WO 2017155126A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
image
person
avatar
camera
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2017/010290
Other languages
English (en)
Japanese (ja)
Inventor
靖和 本玉
寛紀 山内
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
JAPAN IMAGE ANALYSIS ASSOCIATION
Original Assignee
JAPAN IMAGE ANALYSIS ASSOCIATION
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by JAPAN IMAGE ANALYSIS ASSOCIATION filed Critical JAPAN IMAGE ANALYSIS ASSOCIATION
Priority to JP2017564647A priority Critical patent/JP6357595B2/ja
Publication of WO2017155126A1 publication Critical patent/WO2017155126A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering

Definitions

  • the present invention relates to a system for transmitting video information acquired by a camera.
  • the present invention provides a technology that enables effective use of a network communication band when transmitting video from a camera.
  • an information transmission system includes: A feature point extraction unit that extracts feature points from a subject in an image captured by at least one camera and outputs the feature points as feature information; A coordinate information adding unit for acquiring coordinate information of the subject in the shooting range of the camera; An information transmission unit for transmitting the feature information and the coordinate information to the network; An information receiver for receiving feature information and coordinate information from the network; A dynamic generation unit that generates an avatar image of the subject based on the feature information; And an image composition unit that generates a composite image by compositing an avatar image based on the coordinate information with an image representing the background of the shooting range of the camera.
  • the feature point feature information extracted from the subject and the subject coordinate information in the shooting range of the camera are transmitted to the network.
  • an avatar image of the subject is generated based on the feature information, and the avatar image is combined with an image representing the background of the shooting range of the camera based on the coordinate information.
  • a composite image is generated from the background image and the avatar image on the receiving side without sending the video signal of the camera. Therefore, the network communication band is made effective compared to the case of sending the video signal of the camera. It can be used.
  • an avatar image is used instead of the actual video of the subject, there is an advantage that privacy is not infringed even when a large number of unspecified persons are photographed.
  • the coordinate information adding unit sets the position identified as the ground contact point of the person who makes the subject appearing on the photographing screen as the photographing ground position, and displays the person image area appearing at the photographing installation position on the photographing screen.
  • the coordinate information adding unit uses a camera two-dimensional coordinate system set on the camera shooting screen with the walking plane in the real space when the subject is a person as a reference in the height direction.
  • Actual grounding position coordinate information which is the grounding position coordinates of the person in the real space based on the position / height conversion relation information on the ground position / height specifying means and the information of the identified shooting grounding position coordinates and shooting height
  • real person coordinate / height information generating means for converting and generating the height of the person in the real space into real person height information as information.
  • the dynamic generation unit includes an avatar height determining unit that determines the height dimension of the avatar image based on the generated real person height information, and the image composition unit includes the background image based on the actual ground position coordinate information.
  • An avatar composition position determining means for determining the composition position of the avatar image to the image can be provided.
  • the spatial existence range of the person in the area to be photographed is a horizontal plane (similarly, x ⁇ direction) such as a floor surface or the ground where the height direction (assuming to be the z axis direction of the real space orthogonal coordinate system) is constant. It is almost limited to the y plane), and the z coordinate of the contact point (foot position) can always be regarded as constant (for example, 0). That is, the coordinates of the contact point of the person walking in the area can be substantially described in an xy two-dimensional system, and can be uniquely associated with the camera two-dimensional coordinate system.
  • the camera two-dimensional coordinate system corresponds to a real-space three-dimensional coordinate system obtained by projective transformation, and an object separated from the camera is projected with a reduced size.
  • This transformation is mathematically described as a matrix, but if a reference object with a known height is placed at various known positions in the real space coordinate system on the floor or on the ground, the image is taken with a camera.
  • the position and height of the reference body image on the shooting screen By comparing the position and height of the reference body image on the shooting screen with the position and actual size in the real space, the position and height of the person on the camera screen are changed to the position and height in the real space.
  • Position / height conversion related information that is information to be converted can be obtained.
  • the dynamic generation unit can easily determine the height of the avatar image to be synthesized on the background image, and the image synthesis unit reasonably and easily determines the synthesis position of the avatar image to the background image. be able to.
  • the feature point extraction unit analyzes the motion or orientation of the subject and outputs it as motion analysis information
  • the information transmission unit sends the person attribute information to the network
  • the image composition unit operates. It can be configured to adjust the movement or orientation of the avatar image based on the analysis information. According to this, since the movement and direction of the avatar image are adjusted based on the movement and direction of the subject at the time of shooting, for example, the moving speed and moving direction of the subject can be reflected in the avatar image.
  • the camera can capture moving images
  • the coordinate information adding unit acquires coordinate information of a person who makes a subject for each frame of the captured moving image
  • the feature point extracting unit stores the coordinate information of the person. It may be configured to output movement trajectory information between frames as motion analysis information. If the movement trajectory information of the previous frame is analyzed for the current frame, it becomes particularly easy to grasp the movement of the person image that reaches the current frame. For example, a person usually walks with their face and torso facing forward unless they move irregularly, such as walking sideways or backwards, so the movement trajectory of the representative point of the person image (for example, the ground contact point) is If it is found out, the orientation of the body according to the walking motion can be grasped sequentially.
  • the image composition unit can be configured to adjust the orientation of the avatar image to be synthesized on the background image based on the movement trajectory information.
  • the dynamic generation unit can be configured to generate different avatar images according to the movement direction of the person in the real space so that the appearance of the person from the viewpoint of the camera is reflected.
  • the reality of the avatar image expression can be increased by changing the avatar image in accordance with the way (angle) the camera is reflected in the walking direction.
  • the dynamic generation unit includes a direction-specific two-dimensional avatar image data storage unit that stores a plurality of two-dimensional avatar image data having different representation forms according to a plurality of predetermined movement directions of a person in real space.
  • the dynamic generation unit includes a three-dimensional avatar image data storage unit that stores the data of the avatar image as the three-dimensional avatar image data, and generates a three-dimensional avatar object based on the three-dimensional avatar image data.
  • the image composition unit generates a two-dimensional avatar image data by projecting and transforming a three-dimensional avatar object in the real space whose arrangement direction is determined to a two-dimensional coordinate system of a background image, and based on the two-dimensional avatar image data
  • the avatar image can be configured to be combined with the background image. In this case, the data capacity is increased by making the avatar image data three-dimensional.
  • the direction in which the avatar image is pasted on the background image can be made stepless, and more realistic expression can be realized.
  • the image composition unit can also generate an image representing a person's flow line based on the movement trajectory information. According to this configuration, it is easy to visually grasp how a specific subject has moved on the background image. For example, it can be used effectively for crime prevention purposes, and it can be clarified by statistical trend analysis of flow line images where there are places where individual people are interested in exhibition halls and public facilities. Benefit from the benefits.
  • the information transmission system of the present invention can be configured such that the feature point extraction unit analyzes the person attribute of the subject and outputs it as person attribute information, and the information transmission unit sends the person attribute information to the network. According to this configuration, various analysis / statistical processes and the like can be performed using the person attribute information on the receiving side.
  • the dynamic generation unit can be configured to generate the avatar image as reflecting the person attribute information.
  • the attribute of a corresponding person can be easily grasped even after being converted into an avatar image.
  • the attributes of the people can be simplified or emphasized by the avatar image, and there is an advantage that the tendency on the image can be easily grasped.
  • the person attribute information can be configured to include gender information that reflects the gender of the person and age information that reflects the age of the person, but is not limited thereto, for example, the appearance of the face, etc.
  • nationality for example, Japanese or Westerners
  • the feature point extraction unit can analyze the appearance of the subject person and output it as appearance feature information
  • the information transmission unit can be configured to send the appearance feature information to the network.
  • the appearance of the subject is important information that leads to the identification of individual persons following the person attributes, and is useful in analysis and statistical processing.
  • the dynamics generation unit can be configured to generate the avatar image as a reflection of the appearance feature information, so that the features of the corresponding person can be further understood after being converted into the avatar image. . Examples of elements that most reflect the characteristics of the appearance of a person include hair, clothing, and belongings.
  • the appearance characteristic information includes hair information that reflects one or both of the form and color of the person's hair, clothing information that reflects one or both of the form and color of the person's clothing, and the form and color of the person's belongings.
  • the body shape of a person is also useful information.
  • the appearance feature information can be configured to include body shape information that reflects the body shape of a person.
  • gaits features of walking
  • the appearance feature information can be configured to include gait information reflecting a person's gait.
  • the information for specifying the gait is, for example, the stride (or the frequency of movement linked to the walking speed), the swing angle of the arm, the walking speed, the upper body angle at the time of walking, the vertical shaking, etc.
  • the dynamic generation unit can be configured to use avatar animation data composed of frame data obtained by subdividing a person walking action, and can realistically represent an avatar image as an animation of a walking action on a background image.
  • the dynamic generation unit performs image correction processing for correcting each frame of the frame data based on the gait information
  • the image composition unit reflects the gait feature on the avatar image based on the corrected frame data. It can be configured to be combined with the background image in the form of animation. The movement of the avatar image reflecting the gait information of the corresponding person can be easily realized by the correction processing of each frame of the avatar animation data.
  • the imaging range can be covered by a plurality of cameras in the form of sharing real space coordinates.
  • Each camera shoots a common area with different camera coordinate systems, but if the real space of the area to be jointly monitored is stretched in the same coordinate system, if you want to integrate the shooting information of each camera later,
  • the integration process can be completed immediately only by performing the process of converting the coordinates of the person into the common real space (for example, a global coordinate system that can be acquired by GPS or the like).
  • the image composition unit can be configured to generate a composite image as a bird's-eye view image including shooting ranges of a plurality of cameras. According to this, the whole imaging range of a plurality of cameras can be grasped at a glance.
  • the coordinate information adding unit may be configured as follows. That is, the position identified as the ground contact point of the person appearing on the shooting screens of the plurality of cameras is set as the shooting ground position, and the height of the person image area appearing at the shooting setting position on the shooting screen is set as the person shooting height.
  • the information adding unit includes a plane coordinate in a camera two-dimensional coordinate system set on a camera shooting screen and a real space three-dimensional coordinate system with a walking plane in the real space when the subject is a person as a reference in the height direction.
  • a position / height conversion relationship information acquisition means for acquiring height conversion relation information
  • a shooting contact position / height specifying means for specifying a shooting contact position and a shooting height of a person image on the shooting screen
  • Them Based on the position / height conversion relation information, the information on the shadow contact position coordinates and the shooting height is the actual contact position coordinate information that is the contact position coordinates of the person in the real space and the actual height of the person in the real space.
  • the coordinate information adding unit is configured as having real person coordinate / height information generating means for converting / generating person height information.
  • the dynamic generation unit includes avatar height determining means for determining the height dimension of the avatar image based on the generated real person height information, and the image composition unit is a person photographed by a plurality of cameras in the real space coordinate system.
  • the actual ground position coordinate information is converted from the viewpoint of the bird's-eye view image, and the avatar composition position determining means for determining the composition position of the avatar image to the bird's-eye view image is provided.
  • the feature point extraction unit divides the image of the subject into a plurality of parts corresponding to parts of the human body, and extracts feature points from each part. According to this structure, the feature point of each part can be detected effectively.
  • the dynamic generation unit includes an avatar image data storage unit that divides and stores the data of the avatar image into a plurality of avatar fragments corresponding to the parts, and uses the feature point information extracted for the corresponding parts of the person. After correcting the avatar fragment of the avatar image based on the basis, the corrected avatar fragment can be integrated to generate an avatar image. In this way, it is possible to make fine corrections that reflect feature points for each avatar fragment (that is, a person's part), and it is not necessary to prepare a large number of image data of the entire avatar for each feature. Reduction can be achieved.
  • the information transmitting apparatus of the present invention is A feature point extraction unit that extracts feature points from a subject in an image captured by at least one camera and outputs the feature points as feature information;
  • a coordinate information adding unit for acquiring coordinate information of the subject in the shooting range of the camera;
  • An information transmission device comprising an information transmission unit for transmitting feature information and coordinate information to a network,
  • the feature information is associated with the constituent elements of the avatar image of the subject displayed at the transmission destination,
  • the coordinate information is used to specify a position where the avatar image is to be combined in an image representing the background of the shooting range of the camera at the transmission destination.
  • the information receiving apparatus of the present invention is An information receiving unit that receives, via a network, feature information representing feature points extracted from a subject in an image captured by at least one camera, and coordinate information of the subject in a shooting range of the camera; A dynamic generation unit that generates an avatar image of the subject based on the feature information; An image composition unit that generates an image by combining an avatar image with an image representing a background of a shooting range of a camera based on coordinate information is provided.
  • the computer program applied to the information transmission side of the present invention is: A feature point extraction process for extracting feature points from a subject in an image captured by at least one camera and outputting them as feature information; Coordinate information addition processing for acquiring coordinate information of the subject in the shooting range of the camera; A computer program for causing a computer to execute information transmission processing for transmitting feature information and coordinate information to a network,
  • the feature information is associated with the constituent elements of the avatar image of the subject displayed at the transmission destination,
  • the coordinate information is used to specify a position where the avatar image is to be combined in an image representing the background of the shooting range of the camera at the transmission destination.
  • the computer program applied to the information receiving side of the present invention is: A reception process for receiving, via a network, feature information representing feature points extracted from a subject in an image captured by at least one camera, and coordinate information of the subject in a shooting range of the camera; Dynamic generation processing for generating an avatar image of a subject based on feature information; A computer is caused to perform an image composition process for generating a composite image by compositing the avatar image based on coordinate information with an image representing the background of the shooting range of the camera.
  • the present invention it is possible to provide a transmission method that does not hinder the effective use of the communication band of the network when transmitting video from the camera.
  • FIG. 1 is a block diagram showing a schematic configuration of an information transmission system according to the first embodiment of the present invention.
  • FIG. 2 is a flowchart showing the processing procedure of the feature point extraction unit.
  • FIG. 3 is a schematic diagram showing how the feature point extraction unit extracts features by dividing the human body into parts.
  • FIG. 4 is a flowchart showing a flow of processing in which the feature point extraction unit extracts person attribute information.
  • FIG. 5 is a schematic diagram illustrating an example of a coordinate system set in the shooting range of the camera.
  • FIG. 6 is a schematic diagram illustrating a display example in which an avatar image is combined with a background image.
  • FIG. 7 is a schematic diagram showing an application example of the present invention.
  • FIG. 1 is a block diagram showing a schematic configuration of an information transmission system according to the first embodiment of the present invention.
  • FIG. 2 is a flowchart showing the processing procedure of the feature point extraction unit.
  • FIG. 3 is a schematic diagram showing how the feature point extraction unit
  • FIG. 8 is a schematic diagram illustrating an expression example of an avatar when there is no continuity of the transmitting camera.
  • FIG. 9 is a block diagram illustrating a schematic configuration of an information transmission system according to the second embodiment.
  • FIG. 10 is a schematic diagram showing a conventional transmission method.
  • FIG. 11 is a diagram for explaining the concept of extracting a difference in a person image area.
  • FIG. 12 is a conceptual diagram of a background image.
  • FIG. 13 is an explanatory diagram of the coordinate information addition process.
  • FIG. 14 is an explanatory diagram following FIG. 13.
  • FIG. 15 is an explanatory diagram of lens distortion correction.
  • FIG. 16 is a flowchart showing the flow of the coordinate information addition process.
  • FIG. 17 is a diagram showing an example of a person image region extraction state on the screen.
  • FIG. 17 is a diagram showing an example of a person image region extraction state on the screen.
  • FIG. 18 is an explanatory diagram for converting the height h of the person image area into the actual height H using the conversion coefficient ⁇ .
  • FIG. 19 is a flowchart showing the flow of the person area detection process.
  • FIG. 20 is a diagram showing a concept of extracting gait feature information.
  • FIG. 21 is a diagram illustrating a concept of extracting movement trajectory information.
  • FIG. 22 is a diagram showing the concept of information storage on the receiving side.
  • FIG. 23 is a diagram illustrating the concept of an avatar image database.
  • FIG. 24 is a diagram illustrating the concept of the person moving direction used for determining the direction of the avatar image.
  • FIG. 25 is a diagram illustrating an example of avatar fragment graphic data.
  • FIG. 26 is an explanatory diagram illustrating an example in which an avatar image is obtained by combining avatar fragment graphics.
  • FIG. 27 is a diagram illustrating an example in which avatar image data is configured as avatar animation data.
  • FIG. 28 is a diagram illustrating an example in which avatar fragment image data is configured as two-dimensional vector graphic data.
  • FIG. 29 is a flowchart showing a flow of processing on the reception unit side.
  • FIG. 30 is a flowchart showing a flow of new avatar creation processing.
  • FIG. 31 is a flowchart showing the flow of the avatar background composition process.
  • FIG. 32 is a flowchart showing the flow of the integrated mode display process.
  • FIG. 33 is a diagram illustrating an example of a planar display form in the integrated display mode.
  • FIG. 34 is a diagram showing an example of a bird's eye view display form.
  • FIG. 35 is an image showing an example of displaying a three-dimensional avatar image.
  • FIG. 1 is a block diagram showing a schematic configuration of the information transmission system 1.
  • the information transmission system 1 includes an information transmission system transmission unit 12 (information transmission device) and an information transmission system reception unit 13 (information reception device).
  • the information transmission system transmission unit 12 and the information transmission system reception unit 13 are connected via a network 15.
  • the network 15 is a public network such as the Internet, but may be a private network such as a local network.
  • the information transmission system transmission unit 12 receives video signals from a plurality of cameras 11 (11a, 11b%) Installed in various places, performs pre-transmission processing (described in detail later), and then performs the network 15. To send. In FIG. 1, only two cameras 11 are shown, but the number of cameras is arbitrary. Communication between the camera 11 and the information transmission system transmission unit 12 may be wired communication or wireless communication.
  • the information transmission system reception unit 13 receives the video signal transmitted from the information transmission system transmission unit 12 via the network 15, performs post-reception processing (described in detail later), and then displays it on the monitor 14. Or recording to a video recording device (not shown) as necessary.
  • the information transmission system transmission unit 12 includes a coordinate information addition unit 121, a feature point extraction unit 122, a multiple camera linkage unit 123, and an information transmission unit 124.
  • One set of coordinate information adding unit 121 and feature point extracting unit 122 is provided for each camera 11.
  • a coordinate information adding unit 121a and a feature point extracting unit 122a are provided for the camera 11a
  • a coordinate information adding unit 121b and a feature point extracting unit 122b are provided for the camera 11b.
  • the feature point extraction unit 122 detects a person area from the video signal photographed by the camera 11, and further extracts features regarding the appearance (for example, clothing, hairstyle, body shape, belongings, etc.) of each person.
  • the coordinate information adding unit 121 detects the position of a person in an area photographed by the camera 11 as coordinate information.
  • the information transmission system 1 differs from the conventional information transmission system in which the video signal photographed by the camera is compressed and transmitted as it is, and is obtained by the feature information obtained by the feature point extraction unit 122 and the coordinate information addition unit 121. Only the received coordinate information is transmitted via the network 15.
  • the information transmission system receiving unit 13 that has received the feature information and the coordinate information records a background image of the shooting range of each camera 11 in advance and accurately identifies each person based on the feature information. An avatar image is generated, and an avatar image is synthesized at an appropriate position of the background image according to the coordinate information.
  • each of the cameras 11 includes the coordinate information addition unit 121 and the feature point extraction unit 122.
  • the multi-camera cooperation unit 123 uses the video signal of any of the plurality of cameras 11 from the coordinate information obtained by the coordinate information addition unit 121 and the feature information obtained by the feature point extraction unit 122.
  • Tag information indicating whether the information is obtained is attached and sent to the information transmission unit 124.
  • the information transmission unit 124 encodes information obtained from the multi-camera cooperation unit 123 according to a predetermined standard, and transmits the encoded information to the network 15.
  • the information transmission system reception unit 13 includes an information reception unit 131, a dynamic generation unit 132, and an image composition unit 133.
  • the information receiving unit 131 decodes the information received from the network 15 and sends it to the dynamic generation unit 132.
  • the dynamic generation unit 132 generates an avatar image representing a photographed person based on the feature information included in the received information.
  • the avatar image generated by the dynamic generation unit 132 is sent to the image composition unit 133 together with the coordinate information.
  • the image composition unit 133 Based on the avatar image and the coordinate information, the image composition unit 133 generates a composite image of the background image of the shooting range of each camera 11 and the avatar image and displays the composite image on the monitor 14. At this time, the tag information indicating which camera 11 is the information obtained from the video signal is used to specify the background image.
  • the coordinate information adding unit 121 specifies the coordinates of the position of the person in the coordinate system set for the shooting range of each camera 11. For example, as shown in FIG. 5, an xy coordinate system 51 is set in the shooting range of one camera 11.
  • the coordinate information adding unit 121 detects the coordinates of the person area specified by the feature point extracting unit 122 in the xy coordinate system 51.
  • the coordinates detected here are sent to the information transmission system receiving unit 13 together with the feature information as coordinate information representing the position of the person.
  • the subject targeted by the present invention is a person who moves around in the area photographed by the camera 11, and considering the spatial geometric movement characteristics, the person on the screen of the single camera 11 shown in FIG.
  • the position and height of the person in the real space can be specified from the information of the image area PA.
  • the spatial existence range of the person in the area to be photographed is the floor surface or the ground, in the case of FIG.
  • the road surface RS on which the person walks, and the position in the height direction (z-axis direction) is constant. Note that it is almost confined to the horizontal plane.
  • This road surface RS is an xy plane whose z-axis coordinate is always 0 in an orthogonal coordinate system, and the coordinates of the contact point of a person walking on the road surface RS can be substantially described in two dimensions of xy. Although it is a point in the three-dimensional space, it can be uniquely associated with the camera two-dimensional coordinate system set on the photographing screen.
  • the camera two-dimensional coordinate system corresponds to a projective transformation of the real space three-dimensional coordinate system, and a subject that is separated in the camera optical axis direction is projected with a reduced size.
  • the reference points p1 to p3 are read by the ⁇ - ⁇ coordinate system which is a camera two-dimensional coordinate system set on the screen, and stored as screen coordinate data p ( ⁇ , ⁇ ) of the reference point (S503).
  • image on the shooting screen is affected by distortion of the camera lens, it is not a strict projective transformation image in real space, and the image may be distorted depending on the position in the field of view. is there. As shown on the left of FIG. 15, the distortion is larger in the region closer to the edge of the screen, and the coordinate system becomes nonlinear.
  • a lens having a large viewing angle such as a wide-angle lens has an outward convex distortion
  • a lens having a small viewing angle such as a telephoto lens has a concave distortion. Therefore, this distortion is eliminated and conversion correction is performed so as to be a point in the orthogonal plane coordinate system (S504).
  • the correction coefficient at this time is determined by an optimization operation that linearizes the shape of a figure that is known to be straight in real space, such as the white line WL appearing on the screen in FIG. Can do. Note that the size of the edge of the screen expands as the edge of the screen is eliminated by this correction, and thus the corrected screen shape SA ′ protrudes outside the original screen SA.
  • the real space coordinates P (x, y, 0) of the ground contact point of the reference body SC can be obtained.
  • the coordinates may be directly specified by a satellite positioning system (GPS).
  • GPS satellite positioning system
  • the real space coordinate system used here may be an independent coordinate system set within the shooting range of each camera, or linked to a global coordinate system provided by a satellite positioning system (GPS). May be.
  • GPS satellite positioning system
  • the height h on the screen of the reference body image SCI is read (S506).
  • FIG. 13 is a flowchart showing a processing procedure of the feature point extraction unit 122.
  • FIG. 3 is a schematic diagram showing how the feature point extraction unit 122 extracts features by dividing the human body into parts.
  • the feature point extraction unit 122 calculates the moving object MO reflected in the video signal by taking the difference between the frames FM as shown in FIG. 11. It detects (step S11 of FIG. 2). Specifically, the image area MO ′ of the preceding frame and the image area MO of the succeeding frame have different positions and shapes between frames if the image area is a moving object, but the background does not change. The image area MO of the moving object can be extracted by taking the image difference between them. On the other hand, if an image is taken in the absence of a moving object, a background image BP is obtained as shown in FIG. The background image BP is taken for each camera, transmitted to the receiving unit 13 in FIG.
  • the feature point extraction unit 122 extracts a person region by performing segmentation, edge detection, pattern matching, and the like on the moving object image detected in step S11, and determines whether or not the moving object is a person. Judgment is made (step S12).
  • Various methods can be used for the moving object detection process and the person extraction process from the video signal, and the method is not limited to a specific method. Also, among the moving objects detected from the video signal, those having a relatively small size are likely to be noise, so they are determined not to be humans, and those having a relatively large size are determined to be humans.
  • the detected position of the lower end edge of the person area PA is regarded as the ground point p, and the coordinates p ( ⁇ , ⁇ ) on the screen are read (S1201), and the above-described position / height conversion is performed. Since the dimension in the height direction of the person area changes depending on the posture of the object with reference to the relationship information, all the images of the person area that seems to be closest to the upright state are searched and specified in the frame (S1203).
  • the feature point extraction unit 122 further performs a process (step S15) of analyzing the operation of each part. For example, for the head p1, head movement (movement and orientation) is detected.
  • the head is the easiest to recognize. If the orientation of the head is known by first extracting the head p1, it is easy to specify the state of other parts, the moving direction, and the like. In addition, for example, when the head is pointing to the right, it is possible to work on the assumption that the left hand and the left foot may be hidden and invisible in the parting described later. For example, if a person is walking, the movement is analyzed and acquired as gait information. In this case, as the operation of the trunk p2, for example, as shown in FIG. 20, the posture such as the upper body angle ⁇ , whether or not it is a stoop, etc. are detected.
  • the movements of the right hand p3 and the left hand p4 are detected as, for example, each hand swing ⁇ .
  • movements of the right foot p5 and the left foot p6 for example, a walking speed, a stride WL, a knee bending angle, and the like are detected.
  • the motion analysis information such as a gait detected here is sent to the information transmission system receiving unit 13 as motion analysis information, and is reflected in the movement and orientation of the avatar representing the person. What is important as motion analysis information is the moving direction of the person. As shown in FIG. 21, when the coordinate information P1, P2,... Pn of the person is specified for each frame of the captured moving image, the set of the coordinate information P1, P2,. The movement trajectory information between them will be configured.
  • a difference Vn ⁇ Vn ⁇ 1 between the position vectors Vn and Vn ⁇ 1 of the coordinates Pn and Pn ⁇ 1 between the adjacent frames can be used as an index representing the moving direction of the person at the position Pn, and the direction of the avatar image described later It is also used effectively in decisions.
  • the feature point extraction unit 122 uses the human region P extracted in step S12 (see FIG. 3A) as six parts: a head p1, a torso p2, a right hand p3, a left hand p4, a right foot p5, and a left foot p6. (Refer to (b) of FIG. 3) to form parts (step S13). Then, an external feature analysis is performed for each of the six parts that have been made into parts (step S14).
  • the hairstyle, hair color, presence / absence of a hat, and the like are extracted as feature points.
  • the body shape, the shape of clothing, the color of clothing, the presence or absence of specific belongings such as a rucksack, and the like are extracted as feature points.
  • the characteristic points regarding the right hand p3 and the left hand p4 are, for example, the body shape, the shape (or type) of clothing, the color of clothing, and the belongings.
  • the characteristic points regarding the right foot p5 and the left foot p6 are, for example, body shape, clothing shape (or type), clothing color, shoes, and the like.
  • the number of parts at the time of making into parts is not limited to six.
  • the feature points listed here are merely examples, and various elements may be extracted as feature points.
  • the hairstyle and hair color, and the clothing shape and clothing color are extracted as independent feature points, but the “hair color” and “clothing color” It may be treated as additional data of “hairstyle” and “clothing shape”.
  • the extracted feature points for each part are output as feature data and sent to the information transmission system receiver 13.
  • the variation of the extracted feature point (feature data) corresponds to the variation of the component (partial image) of each part in the avatar of the person generated by the information transmission system receiving unit 13 as described later.
  • a partial image of “long hair” is used as the head hair of the avatar.
  • a thick body is used as a partial image of the body of an avatar.
  • the feature point extraction unit 122 may further extract information (person attribute information) that specifies the person to some extent, such as the age and sex of the person.
  • the feature point extraction unit 122 determines the age and sex of the person based on the feature amount extracted from the image of the part that has been partized (step S ⁇ b> 23). For example, if the head p1 can be captured, it is possible to discriminate age and sex using face recognition technology.
  • the age may be output as age data in increments of 1 year, or may be output as data representing the age zone, for example, “20 years old”.
  • gender and age are exemplified as the person attribute information, but any information other than this can be used as information for specifying a person to some extent. For example, it may be possible to discriminate between “adult” and “child”.
  • a person database in which images such as faces and personal information (names, etc.) are registered in advance is used instead of uniquely identifying a person but by gender and age. If it can be used, it is possible to uniquely identify an individual by collating the image of the head p1 with the face image registered in the person database as necessary (step S24).
  • the information receiving unit 131 receives information from the network 15 and decodes it.
  • the decoded information includes information (feature information and coordinate information) obtained from the video signals of a plurality of cameras 11 (cameras 11a, 11b,). Stored and accumulated in the statistical processing unit 135.
  • FIG. 22 shows an example of accumulated information.
  • a detection ID is assigned to a person who is determined to be the same from the degree of coincidence of the feature information described above, the time and date of reception, and the position (x coordinate) And y coordinate), how to walk (gait), physique, height, hair color, upper body clothing color, lower body clothing color, facial feature information, gender, age, etc., sequentially stored in association with each other It has been done.
  • the date and ID part are abbreviated to # 1, # 2, etc., but information such as the type (form) of the upper and lower body clothes, the presence or absence of a hat, and belongings are also associated.
  • the gait data includes a stride WL, an arm swing angle ⁇ , an upper body angle ⁇ , a knee bending angle ⁇ , a one-step cycle ⁇ , and the like.
  • the dynamic generation unit 132 generates an avatar image of each person based on the received feature information. That is, as described above, the feature information includes feature data representing the feature of each part of the person.
  • the dynamic generation unit 132 is a database (in the information accumulation / statistical processing unit 135 in FIG. 1) that stores in advance a partial image of an avatar corresponding to each of the feature data. It may be a storage device or a server).
  • FIG. 23 conceptually shows a construction example of the database.
  • the database includes avatar fragment graphic data, which is an avatar constituent element, including upper and lower body clothes, hairstyles, belongings, and the like when the height and body shape are set as standard.
  • Each avatar fragment graphic data for each avatar component is configured in different representations according to the moving direction of the person in the real space so that the appearance of the person from the viewpoint from the camera (direction with respect to the camera) is reflected. Yes.
  • the direction of the person P with respect to the camera 11 is determined in eight directions (J1 to J8), and the avatar fragments divided in a form corresponding to the human body parts p1 to p6 described in FIG.
  • Graphic data (p2 to p4 for upper body clothes, p5 and p6 for lower body clothes) is prepared for each of the eight directions (v1 to v8: arguments correspond to J1 to J8).
  • shoes, hair, and belongings are not divided, but they are also prepared in eight ways.
  • the left of FIG. 25 shows an example of selection of avatar fragment graphic data when the direction v7 in FIG. 24 is designated, and the right shows an example of selection of avatar fragment graphic data when the direction v1 is designated.
  • FIG. 26 shows avatar images AV7 and AV1 obtained by combining them.
  • contours and human phases that reflect the extracted facial feature information are synthesized for each direction, but the standard face (or head) of the face depends on gender and age.
  • An image may be prepared.
  • the avatar image data (or avatar fragment graphic data) is configured as avatar animation data including a set of frame data obtained by subdividing the object walking motion.
  • one frame for two steps of a plurality of frames (four frames AFM1 to AFM4 in this case) until the landing with the right foot and four frames (AFM5 to 8) until the landing with the left foot is performed.
  • AFM1 to AFM4 for at least the lower body clothes and the upper body clothes, data of these eight frames is prepared for each type of avatar fragment graphic data.
  • the image data of each avatar fragment is configured as two-dimensional vector graphic data as shown in FIG.
  • the vector graphic data is obtained by circularly concatenating vertex coordinates that specify a graphic outline with a vector.
  • the vertex coordinates are also moved according to a matrix operation representing the primary transformation, and the point after the movement is moved.
  • FIG. 29 is a flowchart showing the flow of processing on the receiving side.
  • the person ID, operation information (coordinate points), and feature information sent via the network are received (S601).
  • the received coordinate information P is plotted on real space coordinates shared by a plurality of cameras (S602).
  • the person's walking direction vector is calculated from the position change of the person's coordinate P in the preceding and following frames, and the avatar is selected from the eight directions J1 to J8 in FIG. It determines as an image arrangement direction (S603).
  • an avatar image may have already been created and an avatar image may have been created. (S604). If there is no avatar creation history in S605, a search is made as to whether or not there is a person on the database whose time, position, and feature match under a predetermined condition (S606). If there is no corresponding person in S607, a process for creating a new avatar image is performed (S610).
  • step S601 the hairstyle, clothes, belongings and their colors included in the feature data are specified.
  • step S6102 among the avatar fragment graphics corresponding to the identified feature, the one corresponding to the determined avatar image arrangement direction (any of J1 to J8) (any of v1 to v8 in FIG. 23). read out.
  • step S6103 the avatar fragment graphic is corrected based on the height / body shape / gait information included in the feature data, and the avatar fragment graphic is colored in the color designated in step S6104.
  • the avatar image data is completed by synthesizing each avatar fragment in S6105.
  • the process proceeds to S609, where the avatar image data of the corresponding ID is read from the database and reused.
  • the received person ID is updated with the ID of the corresponding person, and the process proceeds to S609 and the same processing is performed.
  • the avatar image of each person generated by the dynamic generation unit 132 is sent to the image composition unit 133 together with the coordinate information of each person, and the avatar / background composition process is performed (S611).
  • the image composition unit 133 can access a database (in the information accumulation / statistical processing unit 135) in which background images of the photographing ranges of the respective cameras 11 are stored in advance.
  • the image composition unit 133 acquires the background image of each camera 11 from the database, and composes it with the avatar image generated by the dynamic generation unit 132.
  • the composite image is output to the monitor 14.
  • the position where the avatar image is arranged is based on the coordinate information of the person of the avatar.
  • the image composition unit 133 changes the avatar direction or adjusts the speed at which the avatar moves based on the motion analysis information (data representing the movement and direction of the person) obtained by the feature point extraction unit 122. can do.
  • transmission from the information transmission system transmission unit 12 is performed at a frame rate as high as possible within a range that can be processed by the feature point extraction unit 122, the coordinate information addition unit 121, the dynamic generation unit 132, and the image synthesis unit 133.
  • FIG. 31 shows the flow of the avatar background / compositing process.
  • avatar image data corresponding to the specified ID and direction is read.
  • This avatar image data is a set of frame data constituting an animation (FIG. 27), and avatar animation frames are allocated to frames for moving image reproduction in accordance with the speed and stride of the moving coordinate point P (S61102).
  • this is a composite video display mode, in this embodiment, a display mode (camera actual video mode) in the same field of view as the shooting screen of the camera 11 or an integrated display mode of a plurality of cameras can be selected. Yes.
  • This mode selection can be switched by an input unit (configured by a keyboard or a touch panel) connected to the information transmission system reception unit 13 in FIG.
  • the process proceeds to S61104, and the position coordinates P (x, y, 0) of all avatars to be displayed simultaneously are plotted in the real space visual field region of the corresponding camera.
  • the real space visual field area is projected and converted to the corresponding coordinate system of the camera together with the plotted position coordinates P.
  • the camera two-dimensional coordinate system used for determining the position coordinate P is temporarily corrected from the left state in FIG. 15 to the right state in consideration of lens distortion.
  • the entire field of view fits on the output screen in the coordinate system before correction, but after correction, the region at the end of the field of view extends beyond the screen of the monitor (FIG. 1: reference numeral 14).
  • the image changes by the amount of distortion correction, creating a sense of incongruity and displaying the avatar image of the person reflected at the edge of the field of view. It may disappear. Therefore, in S61106 of FIG. 31, reverse distortion correction for restoring the influence of the original lens distortion is performed on the projective transformation image, and the shape of the field of view is restored. As a result, the above problem is solved.
  • step S61107 the selected background image is superimposed on the output plane returned to the camera two-dimensional coordinate system through projective transformation and reverse distortion correction together with the mapped human coordinate position P (S61107)
  • the avatar image data adjusted in size and orientation as described above is pasted and synthesized at each position p (according to the camera two-dimensional coordinate system).
  • the screen of the monitor 14 in FIG. 1 may be divided to display the video signals of the plurality of cameras 11 at the same time, or any one of the plurality of cameras 11 by switching the screen of the monitor 14. Only the video signal may be displayed.
  • the integrated display mode is selected in S61104, the process proceeds to S1000 to display in the integrated mode.
  • step S1001 the positions P (x, y, 0) and directions of all avatars to be displayed simultaneously are plotted in a real space shared by a plurality of cameras.
  • step S1002 flow line trajectory data is created by superimposing the person position coordinates P in the previous and next frames.
  • an avatar image for plane view may be prepared separately, or the avatar may be displayed in the horizontal direction so that the feature information can be easily grasped.
  • the flow line display is designated, the flow line image ML of the corresponding avatar image AV is displayed based on the flow line locus data described above.
  • the process proceeds to S1006, where the real space position, direction, and flow line data of the avatar are projectively transformed according to the bird's-eye view angle and direction, and the background image in the case of bird's-eye view is superimposed in S100. .
  • a captured image for overhead view may be prepared and used, or three-dimensional background image data (for example, three-dimensional computer graphics (CG) data) may be prepared and converted to an overhead view by projective transformation.
  • three-dimensional background image data for example, three-dimensional computer graphics (CG) data
  • CG computer graphics
  • the avatar image corresponding to the direction of the avatar after the projective transformation is read, and the pasted avatar image AVS is pasted on the overhead view background image PBPS as shown in FIG.
  • the flow line image MLS of the corresponding avatar image AVS is displayed based on the above-mentioned flow line locus data.
  • the avatar image data may be 3D avatar image data, and the avatar image may be displayed as a 3D CG image as shown in FIG.
  • the image composition unit 133 (FIG. 1) generates two-dimensional avatar image data by projectively transforming the three-dimensional avatar object in the real space whose arrangement direction is determined into the two-dimensional coordinate system of the background image.
  • the avatar image based on the two-dimensional avatar image data is combined with the background image.
  • the image of the person is not displayed as it is on the monitor 14 but is displayed in an anthropomorphic (avatarized) state, privacy can be used when shooting an unspecified number of persons such as a security camera on the street.
  • anthropomorphic avatarized
  • the person in the captured image shown in FIG. 5 is displayed on the monitor 14 as an avatar as shown in FIG.
  • each avatar is designed to represent the characteristics of each person based on the feature information extracted from the video, so it is possible to grasp what person is in the shooting range.
  • the feature information and coordinate information of the person is acquired from the video signal of the camera 11, and further, the motion analysis information indicating the motion and direction of the person and the person attribute information such as age and sex are obtained.
  • the example to acquire was demonstrated.
  • Various applications can be considered using such information.
  • the above information may be processed and processed by the image composition unit 133 and a plurality of screens may be displayed on the monitor 14.
  • an actual video space screen 81, a feature amount reproduction screen 82, a statistical space screen 83, a flow line analysis space screen 84, and a personal identification space screen 85 are displayed side by side on the monitor 14.
  • the real video space screen 81 is a screen that displays video signals from the plurality of cameras 11 in a state where a person is replaced with an avatar.
  • the actual video space screen 81 is divided into four, and the video signals from the four cameras 11 are displayed simultaneously, but the number of cameras is not limited to this.
  • the feature amount reproduction screen 82 is a screen for displaying videos from a plurality of cameras 11, in which a person is replaced with an avatar and a background image is also displayed in a graphic display.
  • the feature amount reproduction screen 82 is generated by three-dimensionally integrating the images from the plurality of cameras 11. That is, the feature amount reproduction screen 82 is configured as a bird's-eye view image by combining videos taken by a plurality of cameras installed at a plurality of locations.
  • the feature amount reproduction screen 82 illustrated in FIG. 7 is a screen representing the state of the station premises (the vicinity of the platform and the ticket gate) and the surrounding stores.
  • video signals respectively obtained from an installation camera at a station platform, an installation camera around a ticket gate, and a camera installed at each of a plurality of stores are used. Although it is impossible to shoot all of these areas with a single camera, such a bird's-eye view image can be obtained by three-dimensionally combining images taken with multiple cameras installed at multiple locations.
  • a simple screen can be configured.
  • the motion analysis information extracted from the video signal of the camera includes information about the direction of the person and the direction in which the person is moving. By using this information and arranging the avatars so as to match the direction of the actual person, there is an advantage that the movement of the crowd can be easily grasped on the feature amount reproduction screen 82.
  • the statistical space screen 83 is a screen that displays various statistical results. For example, the transition of the number of people within the shooting range of a certain camera can be represented by a graph.
  • the analysis space screen 84 pays attention to a certain person (avatar), and displays how the person has moved in the shooting range of the camera by a flow line. This is possible by acquiring the coordinate information of a certain person (avatar) in time series.
  • the personal identification space screen 85 displays the person attribute information of the person in the shooting range. In the example of FIG. 7, the face part of each person's avatar image, gender, and age are displayed.
  • the real image space screen 81, the feature amount reproduction screen 82, the statistical space screen 83, and the flow line analysis space 84 preferably have a GUI (graphical user interface) function.
  • the person attribute of the person represented by the avatar is displayed.
  • Information is highlighted on the personal identification space screen 85.
  • “male, 35 years old” which is the personal attribute information of the avatar 82 a is highlighted on the personal identification space screen 85.
  • an avatar image corresponding to the selected person attribute information is highlighted on the feature amount reproduction screen 82.
  • the moving path of the avatar may be displayed on the flow line analysis space screen 84.
  • FIG. 8 is a schematic diagram illustrating an expression example of an avatar when there is no continuity of the transmitting camera.
  • the received avatar is colored when the feature amount captured by the camera A can be confirmed by the camera B at the destination,
  • the received avatar is not colored and the avatar (default avatar) is left as it is and the camera is used in each case while distinguishing between the two.
  • FIG. 9 is a block diagram illustrating a schematic configuration of the information transmission system 2. As shown in FIG.
  • the information transmission system 2 is different from the information transmission system 1 (first embodiment) including a plurality of cameras 11a, 11b,. .
  • the information transmission system 2 includes only one set of the coordinate information addition unit 121 and the feature point extraction unit 122 and does not include the multiple camera cooperation unit 123.
  • the operations of the coordinate information adding unit 121, the feature point extracting unit 122, and other processing units are the same as those in the first embodiment.
  • a system that extracts and transmits information only from the video signal of one camera 11 is also included in one embodiment of the present invention.
  • each of the information transmission system transmission unit 12 and the information transmission system reception unit 13 can be realized as an independent device (camera controller), a computer, or a server.
  • Each unit such as the coordinate information adding unit 121 shown in the block diagram can be realized by the processor executing the program recorded in the memory in these devices.
  • the information transmission system transmission unit 22 according to the second embodiment can be realized as an apparatus integrated with the camera 11.
  • the present invention can also be implemented as a program executed by a general-purpose computer or server, or a medium recording the program, in addition to the embodiment in which the present invention is implemented as hardware.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Processing Or Creating Images (AREA)

Abstract

L'invention concerne un système de transmission d'informations qui permet une utilisation efficace d'une bande de communication de réseau lors de la transmission de signaux vidéo de caméras. Le système de transmission d'informations (1) comprend : une unité d'extraction de points caractéristiques (122) pour extraire des points caractéristiques d'un sujet dans une vidéo prise avec au moins une caméra (11), et délivrer lesdits points caractéristiques à titre d'informations de caractéristiques ; une unité d'ajout d'informations de coordonnées (121) pour acquérir des informations de coordonnées pour un sujet dans la plage d'imagerie de la caméra ; une unité d'envoi d'informations (124) pour envoyer, à un réseau, les informations de caractéristiques et les informations de coordonnées ; une unité de réception d'informations (131) pour recevoir les informations de caractéristiques et les informations de coordonnées en provenance du réseau ; une unité de génération de mouvement (132) pour générer une image d'avatar d'un sujet sur la base des informations de caractéristiques ; et une unité de composition d'image (133) pour générer une image composite par combinaison, avec une image montrant l'arrière-plan dans la plage d'imagerie de la caméra, l'image d'avatar sur la base des informations de coordonnées.
PCT/JP2017/010290 2016-03-08 2017-03-08 Système de transmission d'informations, dispositif d'envoi d'informations, dispositif de réception d'informations, et programme informatique Ceased WO2017155126A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2017564647A JP6357595B2 (ja) 2016-03-08 2017-03-08 情報伝送システム、情報受信装置、およびコンピュータプログラム

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2016-044793 2016-03-08
JP2016044793 2016-03-08

Publications (1)

Publication Number Publication Date
WO2017155126A1 true WO2017155126A1 (fr) 2017-09-14

Family

ID=59790440

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2017/010290 Ceased WO2017155126A1 (fr) 2016-03-08 2017-03-08 Système de transmission d'informations, dispositif d'envoi d'informations, dispositif de réception d'informations, et programme informatique

Country Status (2)

Country Link
JP (1) JP6357595B2 (fr)
WO (1) WO2017155126A1 (fr)

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPWO2017141454A1 (ja) * 2016-05-13 2018-02-22 株式会社日立製作所 混雑状況可視化装置、混雑状況可視化システム、混雑状況可視化方法、および混雑状況可視化プログラム
CN110619807A (zh) * 2018-06-20 2019-12-27 北京京东尚科信息技术有限公司 生成全局热力图的方法和装置
JP2020160988A (ja) * 2019-03-27 2020-10-01 オムロン株式会社 報知システム、報知装置
JP2021150735A (ja) * 2020-03-17 2021-09-27 本田技研工業株式会社 情報処理装置、情報処理システム、情報処理方法およびプログラム
CN115004715A (zh) * 2020-02-14 2022-09-02 欧姆龙株式会社 图像处理装置、图像传感器、图像处理装置的控制方法
JP2022551660A (ja) * 2020-01-16 2022-12-12 ▲騰▼▲訊▼科技（深▲セン▼）有限公司 シーンのインタラクション方法及び装置、電子機器並びにコンピュータプログラム
JP2023076633A (ja) * 2017-12-14 2023-06-01 マジック リープ， インコーポレイテッド 仮想アバタのコンテキストベースのレンダリング
CN116403140A (zh) * 2023-03-30 2023-07-07 武汉智凯科技有限公司 视频流中可形变对象的律动节拍辩识方法及装置
JP7319637B1 (ja) 2022-05-30 2023-08-02 株式会社セルシス 情報処理システム、情報処理方法および情報処理プログラム
US20230274580A1 (en) * 2020-08-14 2023-08-31 Intel Corporation Method and system of image processing for action classification
JP7653746B1 (ja) * 2024-12-27 2025-03-31 株式会社PocketRD 合成映像生成システム、合成映像生成方法及び合成映像生成プログラム

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH09307868A (ja) * 1996-03-15 1997-11-28 Toshiba Corp コミュニケーション装置及びコミュニケーション方法
JP2015149557A (ja) * 2014-02-05 2015-08-20 パナソニックＩｐマネジメント株式会社 モニタリング装置、モニタリングシステムおよびモニタリング方法

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5040844B2 (ja) * 2008-07-31 2012-10-03 カシオ計算機株式会社 撮影装置、撮影方法及びプログラム
JP5783629B2 (ja) * 2011-07-08 2015-09-24 株式会社ドワンゴ 映像表示システム、映像表示方法、映像表示制御プログラム、動作情報送信プログラム
WO2015136796A1 (fr) * 2014-03-10 2015-09-17 ソニー株式会社 Appareil de traitement d'informations, support d'informations et procédé de commande
JP6312512B2 (ja) * 2014-04-23 2018-04-18 博司 佐久田 遠隔見守りシステム

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH09307868A (ja) * 1996-03-15 1997-11-28 Toshiba Corp コミュニケーション装置及びコミュニケーション方法
JP2015149557A (ja) * 2014-02-05 2015-08-20 パナソニックＩｐマネジメント株式会社 モニタリング装置、モニタリングシステムおよびモニタリング方法

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10607356B2 (en) 2016-05-13 2020-03-31 Hitachi, Ltd. Congestion analysis device, congestion analysis method, and congestion analysis program
JPWO2017141454A1 (ja) * 2016-05-13 2018-02-22 株式会社日立製作所 混雑状況可視化装置、混雑状況可視化システム、混雑状況可視化方法、および混雑状況可視化プログラム
JP2023076633A (ja) * 2017-12-14 2023-06-01 マジック リープ， インコーポレイテッド 仮想アバタのコンテキストベースのレンダリング
JP7678020B2 (ja) 2017-12-14 2025-05-15 マジック リープ， インコーポレイテッド 仮想アバタのコンテキストベースのレンダリング
US12340477B2 (en) 2017-12-14 2025-06-24 Magic Leap, Inc. Contextual-based rendering of virtual avatars
CN110619807A (zh) * 2018-06-20 2019-12-27 北京京东尚科信息技术有限公司 生成全局热力图的方法和装置
JP2020160988A (ja) * 2019-03-27 2020-10-01 オムロン株式会社 報知システム、報知装置
JP7127592B2 (ja) 2019-03-27 2022-08-30 オムロン株式会社 報知システム
US11967183B2 (en) 2019-03-27 2024-04-23 Omron Corporation Notification system and notification device
JP2022551660A (ja) * 2020-01-16 2022-12-12 ▲騰▼▲訊▼科技（深▲セン▼）有限公司 シーンのインタラクション方法及び装置、電子機器並びにコンピュータプログラム
JP7408792B2 (ja) 2020-01-16 2024-01-05 ▲騰▼▲訊▼科技（深▲セン▼）有限公司 シーンのインタラクション方法及び装置、電子機器並びにコンピュータプログラム
US12033241B2 (en) 2020-01-16 2024-07-09 Tencent Technology (Shenzhen) Company Limited Scene interaction method and apparatus, electronic device, and computer storage medium
CN115004715A (zh) * 2020-02-14 2022-09-02 欧姆龙株式会社 图像处理装置、图像传感器、图像处理装置的控制方法
US12167052B2 (en) 2020-02-14 2024-12-10 Omron Corporation Image processing device, image sensor, and image processing device control method
JP7017596B2 (ja) 2020-03-17 2022-02-08 本田技研工業株式会社 情報処理装置、情報処理システム、情報処理方法およびプログラム
JP2021150735A (ja) * 2020-03-17 2021-09-27 本田技研工業株式会社 情報処理装置、情報処理システム、情報処理方法およびプログラム
US20230274580A1 (en) * 2020-08-14 2023-08-31 Intel Corporation Method and system of image processing for action classification
JP2023175084A (ja) * 2022-05-30 2023-12-12 株式会社セルシス 情報処理システム、情報処理方法および情報処理プログラム
JP7319637B1 (ja) 2022-05-30 2023-08-02 株式会社セルシス 情報処理システム、情報処理方法および情報処理プログラム
CN116403140A (zh) * 2023-03-30 2023-07-07 武汉智凯科技有限公司 视频流中可形变对象的律动节拍辩识方法及装置
JP7653746B1 (ja) * 2024-12-27 2025-03-31 株式会社PocketRD 合成映像生成システム、合成映像生成方法及び合成映像生成プログラム

Also Published As

Publication number Publication date
JPWO2017155126A1 (ja) 2018-06-14
JP6357595B2 (ja) 2018-07-11

Similar Documents

Publication Publication Date Title
JP6357595B2 (ja) 情報伝送システム、情報受信装置、およびコンピュータプログラム
JP2019009752A (ja) 画像処理装置
US12244970B2 (en) Method and system for providing at least one image captured by a scene camera of a vehicle
US11217006B2 (en) Methods and systems for performing 3D simulation based on a 2D video image
JP4473754B2 (ja) 仮想試着装置
Fuchs et al. Virtual space teleconferencing using a sea of cameras
JP3512992B2 (ja) 画像処理装置および画像処理方法
KR101947619B1 (ko) 모바일 디바이스에서의 slam
US20160342861A1 (en) Method for Training Classifiers to Detect Objects Represented in Images of Target Environments
US20200257121A1 (en) Information processing method, information processing terminal, and computer-readable non-transitory storage medium storing program
US11494963B2 (en) Methods and systems for generating a resolved threedimensional (R3D) avatar
JPH0877356A (ja) 三次元多眼画像の処理方法及び処理装置
JP4695275B2 (ja) 動画像生成システム
JP5833526B2 (ja) 映像コミュニケーションシステム及び映像コミュニケーション方法
JP2023539865A (ja) リアルタイムなクロススペクトルオブジェクトの関連付け及び深度推定
CN107016730A (zh) 一种虚拟现实与真实场景融合的装置
US11076097B2 (en) Method for street view service and apparatus for performing same method
CN106981100A (zh) 一种虚拟现实与真实场景融合的装置
KR20220124490A (ko) 동작 유사도 평가 장치 및 동작 유사도 평가 방법
JP6154759B2 (ja) カメラパラメータ推定装置、カメラパラメータ推定方法及びカメラパラメータ推定プログラム
Dijk et al. Image processing in aerial surveillance and reconnaissance: from pixels to understanding
Malerczyk 3d reconstruction of sports events for digital tv
Ghasemzadeh et al. MPL: Lifting 3D Human Pose from Multi-view 2D Poses
Ikeuchi et al. Visualization/AR/VR/MR Systems
Sankaranarayanan et al. Modeling and visualization of human activities for multicamera networks

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2017564647

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17763458

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 17763458

Country of ref document: EP

Kind code of ref document: A1