WO2016152288A1 - Material detection device, material detection method, and program - Google Patents

Abstract

Disclosed is a material detection device that is provided with: a shape acquiring unit that acquires the three-dimensional shape of an object; a picked-up image information acquiring unit that acquires picked-up image information obtained by picking up an image of the object; a region-dividing unit that divides the three-dimensional shape into regions on the basis of the three-dimensional shape and the picked-up image information; and a material acquiring unit that acquires, by each of the regions, material of the object on the basis of information of material detection waves with which each of the regions of the object is irradiated, and information of material detection waves reflected by the object. With such configuration, material of each element of the three-dimensional object can be detected with the simple processing.

Description

Material detection device, material detection method and program

The present disclosure relates to a material detection device, a material detection method, and a program.

Recently, the market expansion and popularization of 3D printers is progressing. Particularly recently, simultaneous printing of a plurality of colors and a plurality of materials has become possible. At the same time, various data format standards for 3D printers are evolving. The basic data format has point group coordinate position, polygon size, and distance information between point groups in a three-dimensional object. Recently, in addition to these basic information, a standard that can hold color information, shadow information, and material information for each point cloud is becoming widespread.

On the other hand, the current 3D scanner detects three-dimensional shapes, but none has the function of detecting even materials. That is, in order to hold material information for each point group, the 3D data format detected by the 3D scanner must be once imported into the data editing environment, and the material must be set for each point group.

In this regard, the following Patent Document 1 describes a material recognition technique that provides material information of an object included in an image captured by a user.

JP 2013-250263 A

However, the technique described in the above-mentioned patent document receives an incident wave generated by the transmitter of the exploration radar unit in the direction of the object and receives a surface reflected wave having an amplitude lower than that of the incident wave and having the same phase. Thus, the physical property information of the object is detected, and the physical property information is compared with the reference physical property information corresponding to the material of the object to recognize what the object is. In such a method, since physical property information is detected at a minimum specific point, it is difficult to acquire the physical properties of the entire three-dimensional object.

Especially, as the number of point groups of 3D scanned data increases, the accuracy increases and finer object representation becomes possible. For this reason, in order to detect the three-dimensional object material for every point group, and to set a material for every point group and to make a database, an enormous time and an enormous process are required.

Therefore, it has been desired to detect the material of each element with a simple process for a three-dimensional object.

According to the present disclosure, a shape acquisition unit that acquires a three-dimensional shape of a target object, an imaging information acquisition unit that acquires imaging information obtained by imaging the target object, the three-dimensional shape and the imaging information Based on the information of the area detection unit that divides the three-dimensional shape for each area, the material detection wave irradiated for each area of the target object, and the material detection wave reflected by the target object And a material acquisition unit that acquires the material for each region.

Further, according to the present disclosure, acquiring the three-dimensional shape of the target object, acquiring imaging information obtained by imaging the target object, and based on the three-dimensional shape and the imaging information, Dividing the three-dimensional shape for each region, and determining the material of the target object based on information of the material detection wave irradiated to the region of the target object and the material detection wave reflected by the target object. A material detection method comprising: acquiring each time.

Further, according to the present disclosure, means for acquiring a three-dimensional shape of a target object, means for acquiring imaging information obtained by imaging the target object, the tertiary based on the three-dimensional shape and the imaging information Means for dividing the original shape into regions, and obtaining the material of the target object for each region based on information of the material detection wave irradiated to the region of the target object and the material detection wave reflected by the target object A program for causing a computer to function as a means for performing the above is provided.

As described above, according to the present disclosure, the material of each element can be detected with a simple process for a three-dimensional object.
Note that the above effects are not necessarily limited, and any of the effects shown in the present specification, or other effects that can be grasped from the present specification, together with or in place of the above effects. May be played.

It is a mimetic diagram showing the composition of the system concerning one embodiment of this indication. It is a schematic diagram which shows the example which took in the scissors with the system of this embodiment. It is a schematic diagram which shows a mode that the system which concerns on this embodiment detects the material of scissors for every area | region A, B, and C. FIG. It is a flowchart which shows the process of this embodiment. It is a schematic diagram which shows the example which implement | achieved the system with mobile devices, such as a smart phone. It is a schematic diagram which shows the structure of the mobile device of FIG. It is a schematic diagram which shows the example which implement | achieved the system with mobile devices, such as a smart phone. It is a schematic diagram which shows the structure of the mobile device of FIG.

Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

The description will be made in the following order.
1. 1. Configuration of material detection system for 3D objects 2. Shape analysis and region segmentation of 3D objects 3. Material detection at specific points in each region 4. Add material information to 3D data format Process flow in this embodiment 6. Application to mobile devices

1. Configuration of 3D Object Material Detection System First, a schematic configuration of a 3D object material detection system 1000 according to an embodiment of the present disclosure will be described with reference to FIG. FIG. 1 is a schematic diagram showing a configuration of a system 1000 according to the present embodiment. As shown in FIG. 1, a system 1000 according to this embodiment includes a recording hardware 100 and an analysis unit (software) 200. The recording hardware 100 includes a shape recognizing infrared light emitting unit (projector) 102, a shape recognizing infrared light receiving unit (sensor) 104, an RGB color recognition camera 106, a material recognizing infrared light emitting unit (projector) 108, and a material recognizing infrared light. A light receiving unit (sensor) 110 is included.

The analysis unit 200 includes a target object shape detection unit 202, a specific point material detection unit 204, a material determination library 206, and a region-specific material allocation unit 208. Each component of the analysis unit 200 shown in FIG. 1 can be composed of a central processing unit such as a CPU and software (program) for causing it to function. In this case, the program can be stored in a memory (not shown) included in the recording hardware 100 or a recording medium connected from the outside. In the system 1000 according to the present embodiment, the recording hardware 100 and the analysis unit 200 may be configured as an integrated device, or may be configured as separate devices.

2. Detecting the shape of a three-dimensional object The 3D scanner includes a contact type and a non-contact type, but the system 1000 of this embodiment uses a non-contact type as an example. In the non-contact type, the shape detection unit 202 detects the shape and surface of the target object by irradiating the target object (three-dimensional object) with infrared light from the shape recognition infrared light emitting unit 102 and receiving the infrared light received by the shape recognition infrared light receiving unit 104. Detect irregularities in As a shape recognition method, there are a type that scans 360 ° around the target object while moving the laser beam, and a type that irradiates the target object with a regular pattern and the sensor receives reflected infrared rays. In the case of measuring the shape while moving the laser beam, the shape is estimated by obtaining the time phase difference difference between the irradiation light and the reflected light. Specifically, light is received at the timing of infrared light emission, the phase difference between the light-emitting part and the light-receiving part is calculated from the received light signal intensity, and the shape is estimated by calculating the flight time (distance) from the phase difference. To do. On the other hand, in the pattern irradiation type, the distance to the object is estimated by triangulation from the difference between the irradiation pattern and the reflection pattern, and the shape and depth information is acquired. Specifically, the light is received in accordance with the light emission timing, the light emission pattern and the light reception pattern are collated, and the distance is calculated from the projection angle and the incident angle based on the principle of three-stroke surveying. In general, the pattern irradiation type is considered to perform measurement at a higher speed than the type in which the laser beam is moved. Note that the type of scanning by moving the laser beam is called a ToF (Time OF Flight) method, and the type of irradiating a pattern is also called an active stereo method. Note that light other than infrared light may be used for shape recognition.

In addition, there is also a technique for obtaining shape / depth information by triangulation from the difference between images of visible light cameras. Specifically, light is received simultaneously by two cameras, feature points of two images are collated, and a distance is calculated by triangulation from two incident angles. In addition, move one visible light camera to receive light at multiple locations, collate feature points of the obtained images, estimate the camera movement distance, and calculate the distance by triangulation from two incident angles You can also Furthermore, there is also a technique for obtaining shape and depth information from the degree of blurring of the image, called Depth From Defocus. In the present embodiment, the shape detection method is not particularly limited, and it is also possible to detect the shape using a method other than the above.

2. Shape analysis / region segmentation of 3D object Once the shape of the 3D object is detected by infrared rays, the shape of the 3D object is analyzed. The target object shape detection unit 202 analyzes shape information from an image / video captured by the recording hardware 100. Specifically, the RGB color recognition camera 106 images a target object, analyzes shape information of the target object from information obtained by the imaging, and performs region division. By performing shape analysis at this stage, it is possible to speed up the processing when material information is later dropped into the 3D data format.

As an example, FIG. 2 is a schematic diagram showing an example in which the scissors 300 are captured by the system 1000 of the present embodiment. At this time, the shape detection unit 202 analyzes the shape information for segmenting the metal portion and the handle portion of the scissors 300 and the color of the handle. For this reason, the shape detection unit 202 includes an imaging information acquisition unit 202 a that acquires imaging information obtained by imaging from the RGB color recognition camera 106. In addition, the shape detection unit 202 uses color information / edge information, luminance / saturation information, contrast, depth information, background information, light reflectance, flat part (region with less texture) information, temperature, and the like as reference information. An area dividing unit 202b that divides (segments) the three-dimensional shape of the target object for each area is provided. The shape detection unit 202 extracts edge information of the target three-dimensional object subjected to background separation and segmentation, and performs region division based on the edge information.

The region dividing unit 202b divides the three-dimensional shape for each region by providing a boundary line in the three-dimensional information including the mesh information or the depth information indicating the three-dimensional shape. At this time, the boundary line is set based on imaging information such as color information, edge information, luminance / saturation information, and contrast. Imaging information such as color information, edge information, luminance / saturation information, and contrast used for area division is acquired by the RGB color recognition camera 106 and acquired by the imaging information acquisition unit 202a. It is also possible for the user to input auxiliary information such as an object / background and perform segmentation by the graph cut method. For example, the target object is specified from the background by selecting the target object with a touch operation at the time of shooting. As a result, the background can be separated with high accuracy.

As a result of shape detection by the shape detection unit 202, as shown in FIG. 2, the scissors 300 are segmented into a metal portion (region A), a handle portion (region B), and a resin portion (region C) other than the handle portion. . Here, both the region B and the region C are made of resin and have different colors.

3. Next, the material of the specific point is measured. Here, an infrared ray for detecting the material irradiated by the infrared ray emitting unit for material recognition 108 and an infrared ray receiving unit for material recognition 110 for receiving the infrared ray are used. Here, the material is estimated by irradiating a specific point on the three-dimensional object with the infrared rays emitted by the material recognition infrared light emitting unit 108 using a known method. Infrared rays emitted from the material recognizing infrared light emitting unit 108 are applied to a specific point of the target object and received by the material recognizing infrared light receiving unit 110. The material detection unit 204 obtains the reflectance of the infrared light emitted by the material recognition infrared light emitting unit 108 based on the infrared light emitted by the material recognition infrared light emitting unit 108 and the infrared light received by the material recognition infrared light receiving unit 110. Then, by comparing with the reflectance held in the material determination library 206, the material of the specific point of the target object is predicted.

In conventional object material detection, signals (material detection waves) such as infrared rays, ultrasonic waves, and electromagnetic waves are irradiated on the object, the reflectance (response) is measured, and the material of the object is predicted by comparing the coefficients of the reflection components. Things are generally used. Such material detection is realized by a system that irradiates a part of a target object with an infrared laser of a point light source and detects the intensity, transfer function, and attenuation degree of the reflected infrared light. However, in the material detection by the point light source irradiation, the material of only the region (specific point) that can be irradiated with the infrared laser is detected, so it takes a lot of time to measure the material of the entire three-dimensional object.

Here, the time required to detect the material of a specific object is proportional to the size of the object. As an example, it is known that it takes about 20 ms to detect the material of an area corresponding to 256 pixels by a conventional method. This detection time is the time required to specify an object by irradiating an object with infrared rays, measuring the reflectance, and comparing how close the material coefficient is to the reflectance.

In order to suppress such an enormous amount of processing time, in this embodiment, the target object is divided (segmented) into regions, and material determination is performed for each region, thereby greatly reducing processing time and load. . In the example of the scissors 300 described above, since the scissors 300 are roughly divided into a metal part and a handle part, first, the metal part is irradiated with infrared rays as an example of a material detection wave to determine that the material is metal. Next, the handle portion is irradiated with infrared rays to determine that the material is plastic or resin. Thereby, the material can be determined for each of the regions A, B, and C by performing material recognition once for each of the regions A, B, and C shown in FIG. Each material determined at this time is held in association with the world coordinate system. In this way, only a specific part in a limited area is recognized as a material, and is used when it is dropped into a later 3D data format. Therefore, the material detection is not performed on all of the point groups showing the three-dimensional shape, and the processing can be greatly simplified.

4). Giving material information to the three-dimensional data format In this embodiment, the target object is estimated by material for each region, and the process of dropping into the 3D data format is performed. As described above, in the conventional material prediction of a specific point, it is possible to measure the material of a very small area of a 3D object. I can't have information with me. Here, if it is going to measure the material of all the point groups, enormous measurement time will be needed.

In particular, if it is assumed that a 3D printer prints out based on 3D shape information, materials must be assigned to all 3D shape point clouds. In the present embodiment, the material is determined for each region, and the material prediction unit for each region is introduced to speed up the material prediction of the three-dimensional object.

The segmentation result by the shape detection unit 202 is used for this speeding up. As described above, the shape detection unit 202 extracts edge information of the target three-dimensional object subjected to background separation and segmentation. As a means for extracting edge information, there is a method in which Sobel, Laplacian, and Canny operators are applied to the first derivative result of an image. Further, it is possible to extract edge information such as pixel luminance value difference, saturation value difference, template matching, and the like.

[The edge of the 3D object extracted here produces a continuous closed area when the background is separated. For example, it means that a line starting from point A has returned to point A again. In this case, it can be determined that the closed region surrounded by this line is likely to be one of the components of a certain object. Here, the material allocation unit 208 by region allocates material information of a specific point detected by the material detection unit 204 to the closed region. When assigning, a paint routine algorithm can be used. The paint routine algorithm is an algorithm for painting a color in a certain closed space, and is used in a paint application of Windows (registered trademark) accessories. This makes it possible to have material information as if all point cloud data is painted.

3D data format generally has world coordinate information, polygon size and position, texture information, etc. In recent 3D formats, formats that can have material information such as 3DS, FBX format, and OBJ format have been born. Since the world coordinates for each object (areas A, B, and C shown in FIG. 2) divided by segmentation can be specified, material information can be assigned to each object. A format such as an AMF (Additive Manufacturing File) format that can describe information about colors, materials, and internal structures may be used based on the STL used in 3D printers.

FIG. 3 is a schematic diagram showing how the system 1000 according to this embodiment detects the material of the scissors 300 for each of the areas A, B, and C. After the segmentation is performed by the shape detection unit 202 and divided into regions A, B, and C, infrared light is emitted from the material recognition infrared light emitting unit 108 for each of the regions A, B, and C, and the infrared light is received by the material recognition infrared light receiving unit 110. , The material detection unit 204 detects the material for each of the regions A, B, and C. At this time, the material is detected by comparing with the material coefficient of each material (ABS resin, PLA resin, nylon, plastic, gypsum, rubber, metal, etc.) held in the material determination library 206.

In order to detect the material for each of the regions A, B, and C, each region A, B, and C emits light once. The material is detected by emitting light only once at a specific point in each of the areas A, B, and C, and the detected material is assigned to all of the areas. As a result, the processing time and load can be significantly reduced.

5. Processing Flow in the Present Embodiment FIG. 4 is a flowchart showing processing in the present embodiment. First, in step S <b> 10, infrared irradiation is performed by the shape recognition infrared light emitting unit 102 and light is received by the shape recognition infrared light receiving unit 104, so that the shape detection unit 202 detects the shape of the target object. In the next step S12, imaging information by the RGB color recognition camera 106 is acquired. In the next step S14, the shape area and the classification are performed based on the imaging information. In the next step S16, background separation is performed. In the next step S18, the region and the segment of the shape based on the edge information are divided. As described above, the region division by the region dividing unit 202b is performed in steps S14 to S18.

In the next step S20, the material determination library 206 is collated, and material detection is performed for each region. Therefore, in steps S22, S24, and S26, one of the areas A, B, and C is determined. If the area is A, the process proceeds to step S28, and the material of the area A is recognized by a paint routine. In the case of the area B, the process proceeds to step S30, and the material of the area B is recognized by the paint routine. In the case of the area C, the process proceeds to step S32, and the material of the area C is recognized by the paint routine. After steps S28, S30, and S32, the process proceeds to step S34 to create a 3D data format.

6). Application to Mobile Device FIG. 5 is a schematic diagram illustrating an example in which the system 1000 is realized by a mobile device 1010 such as a smartphone. FIG. 6 is a schematic diagram showing the configuration of the mobile device 1010. As illustrated in FIG. 6, the mobile device 1010 includes a display unit 114, a touch sensor 116, and an irradiation position guide unit 210 in addition to the system 1000 illustrated in FIG. 1. The mobile device 1010 includes a touch panel configured by providing a touch sensor 116 on the display unit 114. A captured image of the scissors 300 captured by the RGB color recognition camera 106 is displayed on the display unit 114. The irradiation position guiding unit 210 is an application for guiding the irradiation position of the sensor signal to an area where material detection is desired. In the example shown in FIG. 5, the infrared irradiation position (mark 400 indicated by a cross in FIG. 5) by the material recognizing infrared light emitting unit 108 is displayed so as to be superimposed on the captured image displayed on the display unit 114. The position of the mark 400 can be calculated from, for example, the infrared irradiation direction and the distance to the target object (scissors 300). Accordingly, the user can detect the material of the region including the position by aligning the mark 400 with an arbitrary position of the scissors 300 in the captured image. The user can inform the mobile device 1010 that the area to be detected is irradiated by operating the touch sensor 116.

In the example shown in FIG. 6, the mobile device 1010 includes a shape recognition infrared light emitting unit 102, a shape recognition infrared light receiving unit 104, and an RGB color recognition camera 106. The mobile device 1010 does not have to include the shape recognition infrared light emitting unit 102, the shape recognition infrared light receiving unit 104, and the RGB color recognition camera 106.

FIG. 7 is a schematic diagram illustrating an example in which the system 1000 is realized by a mobile device 1020 such as a smartphone. The material recognition infrared light emitting unit 108 and the material recognition infrared light receiving unit 110 are used for detecting a material. The case where 1020 is not provided is shown. FIG. 8 is a schematic diagram showing the configuration of the mobile device 1020. As shown in FIG. 8, unlike the mobile device 1010 of FIG. 6, the mobile device 1020 does not include the material recognizing infrared light emitting unit 108 and the material recognizing infrared light receiving unit 110. The mobile device 1020 includes a communication unit 118 in order to communicate with the infrared sensor unit 1030 and Bluetooth (registered trademark).

The infrared sensor unit 1030 includes a material recognizing infrared light emitting unit 1032, a material recognizing infrared light receiving unit 1034, and a communication unit 1036. The material recognizing infrared light emitting unit 1032 and the material recognizing infrared light receiving unit 1034 correspond to the material recognizing infrared light emitting unit 108 and the material recognizing infrared light receiving unit 110 of FIGS. The communication unit 1036 communicates with the mobile device 1020 via Bluetooth (registered trademark) or the like. The infrared sensor unit 1030 sends information about the infrared rays irradiated by the material recognition infrared light emitting unit 1032 and the infrared rays received by the material recognition infrared light receiving unit 1034 from the communication unit 1036 to the mobile device 1020. The material detection unit 204 of the mobile device 1020 acquires the infrared reflectance based on the information sent from the infrared sensor unit 1030 and compares it with the reflectance held in the material determination library 206. The material of the object is detected for each region. Thus, even if the mobile device 1020 does not have an infrared sensor, if communication with the infrared sensor unit 1030 is possible, by sending a signal from the infrared sensor unit 1030 side to the mobile device 1020 side, It is possible to estimate the material.

As described above, according to the present embodiment, the three-dimensional shape obtained by shape detection is divided into regions based on the imaging information obtained by the RGB color recognition camera 106, and infrared rays are irradiated to specific points in the divided regions. Thus, material detection is performed for each region. The material obtained by detecting the material at the specific point is assigned to the 3D data format as the material of the entire region. Therefore, it is not necessary to detect the material for each point group in the 3D data format, and the processing time and processing load can be greatly reduced.

The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, but the technical scope of the present disclosure is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field of the present disclosure can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that it belongs to the technical scope of the present disclosure.

In addition, the effects described in this specification are merely illustrative or illustrative, and are not limited. That is, the technology according to the present disclosure can exhibit other effects that are apparent to those skilled in the art from the description of the present specification in addition to or instead of the above effects.

The following configurations also belong to the technical scope of the present disclosure.
(1) a shape acquisition unit that acquires the three-dimensional shape of the target object;
An imaging information acquisition unit that acquires imaging information obtained by imaging the target object;
Based on the three-dimensional shape and the imaging information, a region dividing unit that divides the three-dimensional shape into regions,
A material acquisition unit that acquires the material of the target object for each region based on the information of the material detection wave irradiated for each region of the target object and the material detection wave reflected by the target object;
A material detection device comprising:
(2) The material detection device according to (1), wherein the material detection wave is infrared light.
(3) The region dividing unit divides the three-dimensional shape for each region by providing boundary lines in three-dimensional information including mesh information or depth information indicating the three-dimensional shape. Material detection device.
(4) The material detection device according to (3), wherein the region dividing unit provides the boundary line based on the imaging information.
(5) The material acquisition unit further includes a region-specific material allocation unit that allocates the material of the target object acquired for each region to the three-dimensional shape data acquired by the shape acquisition unit for each region. The material detection apparatus as described in 1).
(6) A material recognizing infrared light emitting unit for irradiating infrared light for acquiring the material of the target object;
Infrared light receiving part for material recognition for receiving infrared light reflected by the target object, which is irradiated by the infrared light emitting part for material recognition;
The material detection device according to (1), further comprising:
(7) The material detection device according to (2), wherein the material acquisition unit acquires the material based on a reflectance of the infrared light irradiated on the target object in the target object.
(8) Infrared reflectance for each material is held in advance, and the material acquisition unit is configured to reflect the reflectance of the infrared light applied to the target object in the target object, and the infrared reflectance for each material held in advance. The material detection device according to (7), wherein the material is obtained by comparing the two.
(9) The material detection device according to (1), wherein the shape acquisition unit acquires the three-dimensional shape based on information on a light beam applied to the target object and a light beam reflected by the target object.
(10) The light beam is infrared;
An infrared light emitting part for shape recognition that emits infrared light for acquiring the three-dimensional shape of the target object;
Infrared light receiving unit for shape recognition for receiving infrared light reflected by the target object, which is irradiated by the infrared light emitting unit for shape recognition;
The material detection device according to (9), further comprising:
(11) The material detection device according to (10), wherein the shape acquisition unit acquires the three-dimensional shape based on a phase difference between infrared irradiation light and reflected light.
(12) The infrared light emitting unit for shape recognition irradiates infrared rays in a predetermined pattern,
The material detection device according to (10), wherein the shape acquisition unit acquires the three-dimensional shape by triangulation from a difference between an infrared irradiation pattern and a reflection pattern.
(13) The material according to (1), wherein the shape acquisition unit collates feature points of images obtained by imaging the target object at different positions, and acquires the three-dimensional shape by triangulation based on an incident angle. Detection device.
(14) an imaging unit that images the target object;
A display unit that superimposes and displays a captured image captured by the imaging unit and an infrared irradiation position for acquiring the material of the target object on the captured image;
The material detection device according to (1), comprising:
(15) The material detection device according to (1), further including a communication unit that receives information on infrared rays irradiated on the target object and infrared rays reflected on the target object from an external infrared sensor unit.
(16) acquiring the three-dimensional shape of the target object;
Obtaining imaging information obtained by imaging the target object;
Dividing the three-dimensional shape into regions based on the three-dimensional shape and the imaging information;
Obtaining the material of the target object for each region based on the information of the material detection wave irradiated for each region of the target object and the material detection wave reflected by the target object;
A material detection method comprising:
(17) means for acquiring the three-dimensional shape of the target object;
Means for acquiring imaging information obtained by imaging the target object;
Means for dividing the three-dimensional shape into regions based on the three-dimensional shape and the imaging information;
Means for acquiring the material of the target object for each region based on the information of the material detection wave irradiated for each region of the target object and the material detection wave reflected by the target object;
As a program to make the computer function as.

102 Infrared light emitting unit for shape recognition 104 Infrared light receiving unit for shape recognition 106 RGB color recognition camera 108 Infrared light emitting unit for material recognition 110 Infrared light receiving unit for material recognition 114 Display unit 202 Shape detection unit 202a Imaging information acquisition unit 202b Area division unit 204 Material detection unit 208 Area-specific material allocation unit 1000 System 1010, 1020 Mobile device

Claims (17)

A shape acquisition unit that acquires the three-dimensional shape of the target object;
An imaging information acquisition unit that acquires imaging information obtained by imaging the target object;
Based on the three-dimensional shape and the imaging information, a region dividing unit that divides the three-dimensional shape into regions,
A material acquisition unit that acquires the material of the target object for each region based on the information of the material detection wave irradiated for each region of the target object and the material detection wave reflected by the target object;
A material detection device comprising: The material detection device according to claim 1, wherein the material detection wave is infrared rays. The material detection device according to claim 1, wherein the region dividing unit divides the three-dimensional shape for each region by providing a boundary line in three-dimensional information including mesh information or depth information indicating the three-dimensional shape. . The material detection device according to claim 3, wherein the region dividing unit provides the boundary line based on the imaging information. The material acquisition unit according to claim 1, further comprising a region-specific material allocation unit that allocates the material of the target object acquired for each region by the material acquisition unit to the data of the three-dimensional shape acquired by the shape acquisition unit for each region. Material detection device. Infrared light emitting part for material recognition that irradiates infrared rays for acquiring the material of the target object;
Infrared light receiving part for material recognition for receiving infrared light reflected by the target object, which is irradiated by the infrared light emitting part for material recognition;
The material detection device according to claim 1, further comprising: The material detection device according to claim 2, wherein the material acquisition unit acquires the material based on a reflectance of the target object of infrared rays irradiated on the target object. The infrared reflectance for each material is held in advance, and the material acquisition unit compares the reflectance of the infrared light irradiated to the target object with respect to the target object, and the infrared reflectance for each material held in advance. The material detection device according to claim 7, wherein the material is acquired. The material detection device according to claim 1, wherein the shape acquisition unit acquires the three-dimensional shape based on information on a light beam irradiated on the target object and a light beam reflected by the target object. The ray is infrared;
An infrared light emitting part for shape recognition that emits infrared light for acquiring the three-dimensional shape of the target object;
Infrared light receiving unit for shape recognition for receiving infrared light reflected by the target object, which is irradiated by the infrared light emitting unit for shape recognition;
The material detection device according to claim 9, further comprising: The material detection device according to claim 10, wherein the shape acquisition unit acquires the three-dimensional shape based on a phase difference between infrared irradiation light and reflected light. The infrared light emitting unit for shape recognition irradiates infrared rays in a predetermined pattern,
The material detection device according to claim 10, wherein the shape acquisition unit acquires the three-dimensional shape by triangulation from a difference between an infrared irradiation pattern and a reflection pattern. The material detection device according to claim 1, wherein the shape acquisition unit collates feature points of images obtained by capturing the target object at different positions, and acquires the three-dimensional shape by triangulation based on an incident angle. An imaging unit for imaging the target object;
A display unit that superimposes and displays a captured image captured by the imaging unit and an infrared irradiation position for acquiring the material of the target object on the captured image;
The material detection device according to claim 1, comprising: The material detection device according to claim 1, further comprising a communication unit that receives information on infrared rays irradiated to the target object and infrared rays reflected by the target object from an external infrared sensor unit. Obtaining the three-dimensional shape of the target object;
Obtaining imaging information obtained by imaging the target object;
Dividing the three-dimensional shape into regions based on the three-dimensional shape and the imaging information;
Obtaining the material of the target object for each region based on the information of the material detection wave irradiated for each region of the target object and the material detection wave reflected by the target object;
A material detection method comprising: Means for obtaining a three-dimensional shape of a target object;
Means for acquiring imaging information obtained by imaging the target object;
Means for dividing the three-dimensional shape into regions based on the three-dimensional shape and the imaging information;
Means for acquiring the material of the target object for each region based on the information of the material detection wave irradiated for each region of the target object and the material detection wave reflected by the target object;
As a program to make the computer function as.

Images