WO2015075894A1 - Imaging device, pupil imaging device, pupil-diameter measurement device, pupil-state detection device, and pupil imaging method - Google Patents

Abstract

[Problem] To provide an imaging device which solves the problems of device size and cost increases. [Solution] This imaging device has one imaging element and an image formation means for forming an image, on the one imaging element, of a plurality of images from different perspectives of a shared site on an imaging subject irradiated by light from a light source.

Description

Imaging device, pupil imaging device, pupil diameter measuring device, pupil state detection device, and pupil imaging method

The present invention relates to an imaging device, a pupil imaging device, a pupil diameter measuring device, a pupil state detecting device, and a pupil imaging method, and in particular, an imaging device having a single imaging element, a pupil imaging device, a pupil diameter measuring device, a pupil state detecting device, And a pupil imaging method.

Patent Documents 1 to 3 describe a method of preparing two sets of imaging devices and measuring a position and the like based on triangulation in order to measure accurate three-dimensional coordinates when an object moves back and forth. .

JP 2007-144113 A JP 2012-210394 A Japanese Patent Laid-Open No. 11-063927

However, the techniques described in Patent Documents 1 to 3 require at least two cameras, which increases the size and cost of the apparatus.

An object of the present invention is to provide an imaging apparatus that solves the above-described problems.

The image pickup apparatus of the present invention includes one image pickup device and an image forming unit that forms a plurality of images at different viewpoints at a common portion of the subject irradiated with light from the light source on the one image pickup device, Have

According to the present invention, since an image capable of triangulation can be taken with a single camera, an imaging apparatus capable of suppressing an increase in size and cost can be provided.

The top view of the imaging device in a first embodiment is shown. The top view in case the imaging device in 1st embodiment has arbitrary light sources is shown. The top view in case the imaging device in 1st embodiment has arbitrary division | segmentation means is shown. The top view in case the imaging device in 1st embodiment has an imaging means is shown. The top view of the pupil imaging device in a second embodiment is shown. The top view of the pupil diameter measuring apparatus in 3rd embodiment is shown. The top view in case the pupil diameter measuring apparatus in 3rd embodiment is provided with a pupil extraction means and a pupil diameter calculation means as a pupil diameter estimation means is shown. In the pupil diameter measuring apparatus in 3rd embodiment, the flowchart showing the flow until it calculates the three-dimensional coordinate and dimension of one or more pupils from one imaged image is shown. In the pupil diameter measuring apparatus according to the third embodiment, (a) an example of an image captured by the imaging device 8, (b) an intensity distribution of an image in a cross section of the line segment PQ in (a), (c) an extracted pupil An example of obtaining the circumference of the portion 21 and the coordinates of the circumference is shown. It is a figure explaining the calculation method of the three-dimensional coordinate of the pupil in 3rd embodiment. The top view of the pupil state detection apparatus in 3rd embodiment is shown. It is a flowchart which shows a process in case it has a pupil state determination means in 3rd embodiment. The top view of the pupil imaging device in a third embodiment is shown. It is the figure which showed an example of the specific structure of the pupil imaging device in 4th embodiment. It is a figure which shows the optical path of the light of the 1st wavelength of the pupil imaging device in 4th embodiment. It is a figure which shows the optical path of the light of the 2nd wavelength of the pupil imaging device in 4th embodiment. It is a figure which shows the structure of the prism body provided with the total reflection surface and the wavelength selection film | membrane. The top view of the pupil diameter measuring apparatus in 5th embodiment is shown. It is a flowchart in case a pupil diameter estimation means is provided with a pupil extraction means and a pupil diameter calculation means in 5th embodiment. It is a flowchart which shows operation | movement of the pupil diameter measuring apparatus in 5th embodiment. (A) of the pupil diameter measuring apparatus in 5th embodiment is a figure which shows intensity distribution of the image imaged with the 1st image pick-up element 44, and (b) intensity distribution of the image imaged with the 2nd image pick-up element 45. It is. It is a figure explaining the effect | action of the feature extraction filter of the pupil diameter measuring apparatus in 5th embodiment. It is a figure explaining the effect | action of the feature extraction filter of the pupil diameter measuring apparatus in 5th embodiment. It is a figure explaining the effect | action of the feature extraction filter of the pupil diameter measuring apparatus in 5th embodiment. It is a figure explaining the effect | action of the feature extraction filter of the pupil diameter measuring apparatus in 5th embodiment. It is a figure explaining the effect | action of the feature extraction filter of the pupil diameter measuring apparatus in 5th embodiment. It is a figure explaining the effect | action of the feature extraction filter of the pupil diameter measuring apparatus in 5th embodiment. It is a figure explaining the effect | action of the feature extraction filter of the pupil diameter measuring apparatus in 5th embodiment. It is a figure explaining the effect | action of the feature extraction filter of the pupil diameter measuring apparatus in 5th embodiment. It is a flowchart which shows operation | movement of the pupil state detection apparatus in 5th embodiment. It is a perspective view in case the pupil state detection display apparatus in 6th Embodiment has an optical filter as a display apparatus. It is a perspective view in case the pupil state detection display apparatus in 6th Embodiment has the optical filter 61a and the display part 61b as a display apparatus. It is a perspective view in case the pupil state detection display apparatus in 6th Embodiment has the optical filter 61a, the display part 61b, and the half mirror 61d as a display apparatus.

[First Embodiment] A preferred embodiment for carrying out the present invention will be described below with reference to the drawings. However, the preferred embodiments described below are technically preferable for carrying out the present invention, but the scope of the invention is not limited to the following.

[Description of Configuration] FIG. 1 is a top view of the imaging apparatus 100 according to the present embodiment. As shown in FIG. 1, the imaging apparatus 100 according to the present embodiment includes one light source 2a, half mirror 2ba, mirrors 2bb to 2bd, one imaging element 5, and a lens 6b.

The light source 2a emits incident light to the half mirror 2ba. The half mirror 2ba divides incident light from the light source 2a into light that passes through the half mirror 2ba and travels in the direction of the mirror 2bb, and light that reflects off the half mirror 2ba and travels in the direction of the mirror 2bc. The mirror 2bc reflects incident light from the half mirror 2ba in the direction of the mirror 2bd. The mirrors 2bb and 2bd allow incident light to enter the subject 1 from different positions. Here, FIG. 1 shows an optical axis A and an optical axis B as two optical axes when incident light is incident on the subject 1 from these mirrors 2bb and 2bd. In this embodiment, the optical axis A and the optical axis B are parallel, but if the optical axis A and the optical axis B do not intersect and incident light can be incident on different positions of the subject 1, It does not necessarily have to be parallel. The light from the subject 1 on the optical axis A is reflected by the mirror 2bb and the half mirror 2ba and is incident on the lens 6b, and an image on the optical axis A (for example, the first viewpoint 3) is formed on the image sensor 5. Let The mirror 2bd and the mirror 2bc reflect the light from the subject 1 on the optical axis B and make the light incident on the half mirror 2ba. The half mirror 2ba makes the light from the mirror 2bc incident on the lens 6b. Then, the half mirror 2ba forms an image on the image sensor 5 on the optical axis B (for example, the second viewpoint 4). On the image sensor 5, at least light from the subject at the first viewpoint 3 and the second viewpoint 4 which are two different viewpoints is simultaneously imaged by the lens 6b.

[Description of Operation] Next, the operation of the imaging apparatus 100 in the present embodiment will be described. First, the process until the incident light from the light source 2a enters the subject 1 will be described. The light emitted from the light source 2a enters the half mirror 2ba. Incident light at the half mirror 2ba is split into light that passes through the half mirror 2ba and travels in the direction of the mirror 2bb, and light that reflects off the half mirror 2ba and travels in the direction of the mirror 2bc. The light reflected by the half mirror 2ba and traveling in the direction of the mirror 2bc is reflected by the mirror 2bc and incident on the mirror 2bd, is parallel to the light transmitted through the half mirror 2ba and traveled in the direction of the mirror 2bb, and is at a different position on the subject 1 Is incident on.

Next, the process until the light from the subject 1 forms an image on the image sensor 5 will be described. First, light from the subject 1 on the optical axis A is reflected by the mirror 2bb, reflected by the half mirror 2ba, enters the lens 6b, and an image on the optical axis A (for example, the first viewpoint 3) is imaged. 5 is imaged. On the other hand, the light from the subject 1 in the direction of the optical axis B is reflected by the mirror 2bd, further reflected by the mirror 2bc, passes through the half mirror 2ba, enters the lens 6b, and together with the light from the subject at the first viewpoint 3 An image on the optical axis B (for example, the second viewpoint 4) is formed on the image sensor 5. Here, the images at the first viewpoint 3 and the second viewpoint 4 include at least the image of the feature point 1a that is a common part of the subject 1 at each viewpoint. That is, at least an image of the feature point 1 a that is a common part of the subject 1 is formed on the image sensor 5.

[Explanation of Effects] If the configuration of the imaging apparatus 100 according to the present embodiment is adopted, the light of the subject 1 from different viewpoints can be imaged and imaged on one image element 5 by the optical system, so that the configuration of the apparatus is simple. Thus, the imaging device 100 capable of reducing the size can be obtained. Further, with the above-described configuration, the exact position of the feature point 1a can be obtained from the captured image by triangulation.

In the above description, the example using the single light source 2a, the half mirror 2ba, the mirror 2bb, the mirror 2bc, and the mirror 2bd has been described, but the present embodiment is not necessarily limited to each of these configurations. . As shown in FIG. 2, the light source 2a shown in FIG. 1 may be replaced by an arbitrary light source 2 having any position / angle such as natural light or indoor lighting. Further, as shown in FIG. 3, an arbitrary dividing means 2b may be used instead of the half mirror 2ba of FIG.

In the above description, the light from the subject 1 at the first viewpoint 3 and the second viewpoint 4 is imaged on the image sensor 5, but the light from the subject 1 at least at two different viewpoints is combined. You can image it. If light from two or more viewpoints is imaged, it is possible to calculate three-dimensional coordinates by triangulation.

The imaging device 5 may be a known device such as a CMOS (complementary metal oxide semiconductor) or a CCD (charge-coupled device).

In FIG. 1, the optical path length between the half mirror 2ba and the mirror 2bb is L, the optical path length between the half mirror 2ba and the mirror 2bc, and the optical path length between the mirror 2bc and the mirror 2bd are respectively L / 2. That is, the sum of the optical path length between the half mirror 2ba and the mirror 2bc and the optical path length between the mirror 2bc and the mirror 2bd is L. This is the path from the light source 2a to the mirror 2bb, the path from the mirror 2bb to the optical axis A exiting in the direction of the subject 1, the light source 2a to the mirror 2bc, the mirror 2bc to the 2bd, and the mirror 2bd The optical path lengths of the light traveling along the path reaching the optical axis B emitted in the direction of the subject 1 are arranged to be equal. Thus, the light from the subject 1 from the direction of the optical axis A and the light from the subject 1 from the direction of the optical axis B are imaged on the image sensor 5 with the same optical path length. In this way, each light is imaged with the same optical path length, so that accurate three-dimensional coordinates can be calculated from the imaged image by triangulation. In the imaging apparatus 100 according to the present embodiment, processing for obtaining three-dimensional coordinates from an image formed is not performed. The triangulation will be described in detail in the description of the configuration and operation in the third embodiment to be described later. If the sum is L, the optical path length between the half mirror 2ba and the mirror 2bc and the optical path length between the mirror 2bc and the mirror 2bd may be set to arbitrary values.

In addition, the configuration in which the light from the subject 1 at the first viewpoint 3 and the second viewpoint 4 is imaged on the image sensor 5 uses the above-described number of mirrors, half mirrors, and lenses. Not necessarily. As shown in FIG. 4, the above-described configuration has an arbitrary mirror, half mirror, lens, and the like that can image light from the subject 1 at the first viewpoint 3 and the second viewpoint 4 on the image sensor 5. An imaging means 6 comprising an optical system consisting of can be substituted. In the configuration shown in FIG. 3, the arbitrary light source 2 shown in FIG. 2 is configured by an optical system such as a light source 2a, a half mirror 2ba, and a mirror 2aa. The imaging apparatus 100 can be made smaller if the configuration of the optical path shared with the optical system constituting the imaging means 6 is configured using these optical system configurations. Furthermore, since the internal configuration of the apparatus can be simplified, the imaging apparatus 100 can be manufactured at a lower cost.

[Second Embodiment] Next, a second embodiment will be described. FIG. 5 is a top view showing a configuration of the pupil imaging device 200 according to the present embodiment.

[Description of Configuration] The pupil imaging device 200 of the present embodiment includes a light source 11, a wavelength selection unit 12, and an imaging unit 20, as shown in FIG. The light source 11 emits light having a predetermined wavelength. The wavelength selection unit 12 selectively transmits light having a predetermined wavelength emitted from the light source 11. The image forming means 20 has images at different viewpoints for at least a part of the face including at least one pupil 14a of the detected person 14 irradiated with light of a predetermined wavelength from the light source 11 via the wavelength selecting means 12. Is imaged on one image pickup device 13 via the wavelength selecting means 12. Here, the different viewpoints are on different optical axes substantially parallel to the direction in which the light source 11 emits light, and the different optical axes are parallel to each other.

Other configurations and configuration variations are the same as those in the first embodiment. That is, the pupil imaging device 200 of the second embodiment includes a light source 11, a wavelength selection unit 12, a single imaging device 13, a first viewpoint 15, a second viewpoint 16, and an imaging unit 20. Have In the present embodiment, an example in which the imaging unit 20 includes a half mirror 20a, mirrors 20b to 20d, and a lens 20e will be described.

[Description of Operation] Next, the operation of the pupil imaging apparatus 200 of the present embodiment will be described. First, the process until the incident light from the light source 11 irradiates the detection subject 14 will be described. As shown in FIG. 5, the light emitted from the light source 11 enters the half mirror 20a. Incident light at the half mirror 20a is split into light that passes through the half mirror 20a and travels in the direction of the mirror 20b, and light that reflects off the half mirror 20a and travels in the direction of the mirror 20c. The light reflected by the half mirror 20a and traveling in the direction of the mirror 20c is reflected by the mirror 20c and incident on the mirror 20d, and is parallel to the light transmitted through the half mirror 20a and proceeding in the direction of the mirror 20b. The light enters the different positions of the person 14.

Next, the process until the light from the person to be detected 14 forms an image on the image sensor 13 will be described. First, light from the person 14 to be detected in the direction of the optical axis A is reflected by the mirror 20b, reflected by the half mirror 20a, and incident on the lens 20e, and an image at the first viewpoint 15 is formed on the image sensor 13. To do. On the other hand, the light from the detected person 14 incident from the direction of the optical axis B is reflected by the mirror 20d, further reflected by the mirror 20c, passes through the half mirror 20a, enters the lens 20e, and is detected at the first viewpoint 15. An image at the second viewpoint 16 together with the light from the detector 14 is formed on the image sensor 13. Here, the light from the detected person 14 at the first viewpoint 15 and the second viewpoint 16 includes light from at least a part of the face including at least one pupil 14a of the detected person 14 at each viewpoint. It is. That is, an image of at least a part of the face including one or more pupils 14 a of the person to be detected 14 is formed on the image sensor 13. Since the optical axes A and B are substantially parallel to the light incident from the person 14 to be detected, fundus reflection light is captured at the pupil portion of the image to be formed.

[Explanation of Effects] In the pupil imaging apparatus 200 according to the present embodiment, at least a part of the face including one or more pupils 14a of the person 14 to be detected at two different viewpoints is applied to one imaging element 13 by the optical system. Since an image can be formed and imaged, the configuration of the apparatus is simple, and the imaging apparatus 200 that can be reduced in size can be obtained. Further, with such an apparatus, an image of the pupil 14a that can accurately obtain the three-dimensional coordinates around the pupil and the pupil diameter using triangulation can be captured.

The pupil imaging apparatus 200 according to the present embodiment may be used to calculate and measure a human feature amount such as the intensity of the pupil image of the detected person and the reflected light distribution of the face in addition to the three-dimensional coordinates around the pupil.

[Third Embodiment] In addition to the configuration of the second embodiment, at least two regions corresponding to one or more pupils 14a in one image formed on the image sensor 13 are used. By having the pupil diameter estimating means 21 that estimates the pupil diameter by triangulation, the pupil diameter measuring apparatus 300 as shown in FIG. 6 can be obtained.

As shown in FIG. 7, the pupil diameter estimating means 21 includes a pupil extracting means 21a and a pupil diameter calculating means 21b. The pupil extracting means 21a extracts at least two regions corresponding to one or more pupils 14a from one image formed on the image sensor 13. The pupil diameter calculating means 21b calculates the three-dimensional coordinates and dimensions of one or more pupils 14a from the at least two regions described above. Since there are usually two pupils 14a, a total of four regions are extracted by the pupil extraction means 21a.

[Description of Configuration / Operation] Next, the configuration and operation of the difference from the second embodiment will be described. The configuration and operation will be described by taking as an example the case where the pupil diameter estimating means 21 includes a pupil extracting means 21a and a pupil diameter calculating means 21b. The pupil extraction means 21a extracts at least two regions corresponding to one or more pupils 14a from one image formed on one image sensor 13. From the two regions extracted by the pupil extracting means 21a, the pupil diameter calculating means 21b calculates the three-dimensional coordinates and dimensions of one or more pupils 14a. As described above, since there are usually two pupils 14a, a total of four regions are extracted by the pupil extracting means 21a.

Below, the pupil extraction means 21a extracts two regions from one image imaged on one image sensor 13, and the pupil diameter calculation means 21b determines the three-dimensional coordinates and dimensions of one or more pupils 14a. The flow until the calculation is performed will be described with reference to the flowchart of FIG.

As shown in FIG. 8, first, the pupil extracting means 21a captures one image formed on the image sensor 13 (S2; image capture). Then, the pupil extracting unit 21a performs binarization processing on the captured image, extracts four regions corresponding to the pupil 14a in the image, and outputs them to the pupil diameter calculating unit 21b (S3; pupil part extraction). From the four regions corresponding to the pupil 14a in the image extracted by the pupil extracting means 21a, the pupil diameter calculating means 21b has coordinates (X and Y coordinates) in the surface of the image sensor 13 around the pupil 14a in the image. Is acquired (S4; coordinate acquisition). Further, the pupil diameter calculating means 21b generates a real image including the pupil 14a imaged on the image sensor 13 by triangulation using the calculation methods shown in Equations 1 to 3 described later in addition to the obtained X and Y coordinates. On the other hand, a vertical distance (referred to as Z coordinate) from the image sensor 13 is obtained (S5; acquisition of three-dimensional coordinates of the pupil). Further, the pupil diameter calculating means 21b calculates the pupil diameter from the obtained three-dimensional coordinates (S6; pupil diameter calculation), and records the calculated pupil diameter (S7; pupil diameter recording). In the above-described pupil diameter measuring apparatus 300, when imaging is performed a plurality of times, such as when imaging in time series, the process returns to step S1 after the process of S7 is completed in the flowchart of FIG.

Next, the details of the above steps will be described. The image captured in S2 in FIG. 8, that is, the image captured by the image sensor 8 is a facial image of the detected person 14 as shown in FIG. 9A as an example. This image is an image in which the facial images of the detection subject 14 viewed from the directions of the optical axis A and the optical axis B in FIG. FIG. 9B shows the intensity distribution of the image in the cross section of the line segment PQ in FIG. 9A, and the pupil portions 22 of the detection subject 14 viewed from the directions of the optical axis A and the optical axis B are high. Images are taken with intensity values.

Next, when the binarized image processing is performed with the threshold shown in FIG. 9B by the pupil extracting means 21a in S3, the optical axis A and the optical axis B are viewed from the respective directions as shown in FIG. 9C. Only the pupil portion 22 of the detected person 14 is extracted. In step S4, the pupil extracting unit 21a acquires and records the coordinates of the circumference of the extracted pupil portion 21 in FIG. 9C.

Here, for example, as shown in FIG. 9C, the coordinates to be recorded are the light of the X component of the coordinates recorded by the pupil extracting means 21a (the direction of the line segment PQ in FIG. 9A). The minimum coordinate XARmin and the maximum coordinate XARmax of the right eye image in the axis A direction are recorded. The recorded coordinates are recorded as the minimum coordinate XALmin and the maximum coordinate XALmax of the left eye image in the optical axis A direction. The recorded coordinates are recorded as the minimum coordinate XBRmin and the maximum coordinate XBRmax of the right eye image in the optical axis B direction. The recorded coordinates are recorded as the minimum coordinate XBLmin and the maximum coordinate XBLmax of the left eye image in the optical axis B direction. Similarly, the Y coordinate can be recorded.

Next, a method for calculating the three-dimensional coordinates of the pupil 14a will be described with a specific example. FIG. 10 is a diagram for explaining the operation of the pupil diameter calculating means 21 of the pupil diameter measuring apparatus 300 of the present embodiment. In FIG. 10, the position of the principal point of the lens 20e is the origin, and the optical axis A direction of the lens 20e is the Z direction. Further, it is perpendicular to the optical axis A, and the horizontal direction on the surface of FIG. Further, the depth direction in FIG. Here, the optical axes A and B in FIG. 10 and the optical axes A and B in FIG. 5 are the same. In other words, FIG. 10 shows the optical axes A and B in FIG. 5 re-expressed linearly for each optical axis, the lens 20e, and the image sensor 13. In FIG. The images viewed from the respective optical axes in FIG. 5 are the viewpoints A and B (corresponding to the first viewpoint 15 and the second viewpoint 16) on the imaging element 13 and the optically equivalent imaging element 13a in FIG. The image at the position. Accordingly, in FIG. 7, the minimum X coordinate XARmin of the right end of the right eye pupil of the viewpoint A and the minimum X coordinate XBRmin of the right end of the right eye pupil of the viewpoint B recorded by the pupil extracting means 21a in FIG. The positional relationship is as shown in FIG. 10 with respect to the distance. Hereinafter, a method of measuring the three-dimensional coordinates E (XRmin, YRmin, ZRmin) will be described by taking the right end of the right eye pupil 14a shown in FIG. 10 as an example.

10, the distance d between the optical axes A and B is L + L / 2 from FIG. F is the focal length of the lens 20e. The lens 20e and the image sensor 13 with respect to the optical axis B in FIG. 5 are expressed as an optically equivalent lens 20ea and an optically equivalent image sensor 13a in FIG. Hereinafter, description will be given using this optical equivalent diagram.

In FIG. 10, the detected person 14 and the optical axis A and optical axis B, and the viewpoints A and B have the same relationship as that of triangulation (stereo measurement), and therefore, the three-dimensional coordinates E (XRmin) of the right end of the right eye pupil 14a. , YRmin, ZRmin) are expressed by Equations 1 to 3.
[Formula 1]
ZRmin = f · d / (XBRmin−XARmin)
[Formula 2]
XRmin = ZRmin · (XARmin) / f
[Formula 3]
YRmin = ZRmin · (YARmin) / f
Similarly, three-dimensional coordinates of the left end of the right eye pupil, the right end of the left eye pupil, and the left end of the left eye pupil can be obtained. Furthermore, the diameters of the left and right pupils can be obtained from the distance between the left and right ends of the right eye pupil and the distance between the left and right ends of the left eye pupil. This makes it possible to accurately detect the pupil diameter of the pupil 14a even if the subject 14 moves in the optical axis direction with respect to the lens 20e (even if ZRmin varies).

In the present embodiment, the pupil diameter measuring device 300 is shown, but by further including the pupil state determining means 24, the pupil state detecting device 400 can be obtained. Below, the difference with the pupil diameter measuring apparatus 300 among the structures of the pupil state detection apparatus 400 which has the pupil state determination means 24 is demonstrated using FIG. In addition, processing when the pupil state determination unit 24 is provided will be described with reference to the flowchart of FIG.

As shown in FIG. 11, the pupil state determination means 24 determines the psychological state and physiological state of the detected person 14 from the three-dimensional coordinates and dimensions of one or more pupils 14a output from the pupil diameter calculation means 21b.

FIG. 12 shows a flowchart when the pupil state determination means 24 performs frequency analysis on the three-dimensional coordinates and dimensions of one or more pupils 14a and determines the psychological state and physiological state of the detected person 14 from the result. Show. As shown in FIG. 12, the pupil diameter calculation means 21b calculates the pupil diameter in a predetermined cycle in S6. The pupil diameter calculated in S7 is recorded, the pupil diameter data collected in time series in S8 is analyzed by the pupil state determination means 24, and the psychological state and physiological state of the detected person 14 are determined from the analysis result. In the above pupil state detection apparatus 400, as shown in the flowchart of FIG. 12, after the completion of S7, the process returns to the step of S1. Steps S8 and after may be performed in parallel with the processing of S1 to S7, or may be performed separately.

Hereinafter, the difference step from the pupil diameter measuring apparatus 300 will be described with a specific example. In S7, the pupil diameter calculation means 21b calculates the pupil diameter data as time-series data every 1/60 seconds. The calculated pupil diameter data can be recorded by the pupil state determination means 24. The time-series data recorded in the pupil state determination unit 24 can be subjected to wavelet transform as frequency analysis by the pupil state determination unit 24.

As a method of determining the psychological state and physiological state of the person 14 to be detected from the result of the wavelet transform, first, the frequency band of the wavelet transform is a low frequency band 0.04 to 0.15 Hz with 0.15 Hz as a boundary, and a high frequency band. It is divided into 0.16 Hz to 0.5 Hz. Then, the psychological state can be determined from the amplitude ratio between the low frequency band amplitude and the high frequency band amplitude. In general, the parasympathetic nerve is dominant in the low frequency band, and the sympathetic nerve is dominant in the high frequency band. For example, if the value obtained by dividing the power in the low frequency band by the power in the high frequency band is 4 or more, the parasympathetic nerve is dominant and the detected person is relaxed. It is determined that the detector is excited.

[Description of Effects] As described above, the pupil diameter measuring apparatus 300 according to the third embodiment has a simple configuration and can be reduced in size. In addition, the pupil diameter can be accurately obtained using triangulation. It is. In the pupil state detection device 400, it is possible to provide a small and low-cost device that objectively and quantitatively evaluates the mental state in a free posture without restraining the body of the person 14 to be detected.

The frequency band of the wavelet transform may be other than 0.04 to 0.5 Hz as long as the psychological state can be distinguished, and the frequency dividing the low frequency band and the high frequency band may be other than 0.15 Hz. For frequency analysis, in addition to wavelet transform, Fourier transform or other methods may be used.

In the experiment, the image pickup device 13 was a CMOS sensor having horizontal 1920 pixels and vertical 1080 pixels, and the pixel pitch in the X and Y directions in this case was 3 μm. Other pixel numbers may be applied to the image definition.

The image update frequency of the image sensor 13 was 60 times per second. That is, the pupil diameter data is calculated by the pupil diameter calculation means 21b as time series data every 1/60 seconds, but may be other time intervals.

When the CMOS sensor having the number of pixels described above is used and the focal length f of the lens 20e is 15 mm and d is 50 mm, the measurement resolution of the pupil 14a in the X and Y directions when the imaging distance (Z coordinate) is 500 mm is 0.1 mm. It becomes. Further, in the Z direction, the imaging distance could be corrected with a resolution of 1 mm.

Further, although an LED having a wavelength of 850 nm is used as the light source 1, a light having a different wavelength may be used as long as the same effect can be obtained. In addition, a sensor for measuring ambient light may be provided to correct the pupillary light reflection. “LED” is an abbreviation for “Light Emitting Diode”.

In the present embodiment, the case where the optical axis A and the optical axis B are parallel has been described. However, even in the condition of convergence imaging in which the angle between the optical axis A and the optical axis B is changed by changing the angle of each mirror. Good.

[Fourth Embodiment] Next, a fourth embodiment will be described. FIG. 13 is a top view showing the configuration of the pupil imaging device 500 according to this embodiment. As shown in FIG. 13, the pupil imaging apparatus 500 of the present embodiment is different from the second embodiment in that it includes a second light source 42 and a second imaging means 49. Other configurations and configuration variations are the same as those in the second embodiment. That is, the pupil imaging device 500 of the fourth embodiment includes a first light source 41, a second light source 42, a wavelength selection unit 43, a first imaging element 44, a second imaging element 45, The first imaging means 48 and the second imaging means 49 are provided.

[Description of Configuration] Next, the configuration of the pupil imaging apparatus 500 of this embodiment will be described with reference to FIG. The first light source 41 emits light having a first wavelength. The second light source 42 emits light of the second wavelength from a position different from that of the first light source 41 at a different angle. The wavelength selection unit 43 selectively transmits the light having the first wavelength and the light having the second wavelength. The first imaging device 44 selects the light of the first wavelength at the first viewpoint 46 and the second viewpoint 47 from at least a part of the face including the one or more pupils 14a of the person 14 to be detected. Light is received through the means 43. The second imaging element 45 receives, via the wavelength selection means 43, the light having the second wavelength at the first viewpoint 46 of at least a part of the face including the one or more pupils 14a of the person 14 to be detected. . The first image forming unit 48 selects the light from the detected person 14 in wavelength selection means substantially parallel to the optical axis A and the optical axis B when the light from the first light source 41 is emitted to the detected person 14. Light is received through 43. Further, the first imaging means 48 is configured so that each of the first viewpoint 46 and the second viewpoint 47 from at least a part of the face including one or more pupils 14a of the subject 14 to be irradiated is first. An image of light having a wavelength of 1 is formed on the first image sensor 44 via the wavelength selection means 43. The second imaging unit 49 generates an image of the second wavelength light at the first viewpoint of at least a part of the face including at least one or more pupils of the subject via the wavelength selection unit 43. An image is formed on the second image sensor 45.
The first viewpoint 46 and the second viewpoint 47 are on the optical axis A and the optical axis B, which are two optical axes that are substantially parallel to the direction in which the first light source 41 emits light. , Parallel to each other.

[Description of Operation] Next, the operation of the pupil imaging apparatus 500 of the present embodiment will be described with reference to FIG. First, the process until the incident light from the light source 41 irradiates the detection subject 14 will be described. The first light source 41 emits light having a first wavelength. The light of the first wavelength is irradiated to the detected person 14 via the wavelength selection means 43. On the other hand, the second light source 42 emits light having the second wavelength at a different angle from a position different from that of the first light source 41. The light of the second wavelength is irradiated to the person to be detected 14.

Next, a description will be given of how light from the person 14 to be detected forms an image on the first image sensor 44 and the second image sensor 45. The light of the first wavelength from at least a part of the face including one or more pupils 14a of the detected person 14 is emitted from the first light source 41 to the detected person 14 via the wavelength selecting means 43. Is incident on the first imaging means 48 on the first optical axis (optical axis A in FIG. 13) and the second optical axis (optical axis B in FIG. 13) which are coaxial with the optical axis when To do. The incident light having the first wavelength forms an image on the first image sensor 44 by the first imaging means 48. On the other hand, the light having the second wavelength from at least a part of the face including one pupil 14 a of the person to be detected 14 enters the second imaging unit 49 via the wavelength selection unit 43. The incident light of the second wavelength is imaged on the second image sensor 45 by the second imaging means 49.

Hereinafter, details will be described with specific configurations of the first imaging means 48 and the second imaging means 49. FIG. 14 is a diagram illustrating an example of a specific configuration of the pupil imaging device 500. FIG. 15A is a diagram showing an optical system of the optical path of the first wavelength from the first light source 41. FIG. 15B is a diagram showing an optical system of the optical path of the second wavelength from the second light source 42. Here, the configuration of FIG. 15A is an optical system that optically has the same action as that of the third embodiment, and each of the optical axes A and B is included in the first image sensor 44 as in FIG. 9A. The face images of the person to be detected 14 corresponding to the direction are captured in an overlapping manner.

As shown in FIG. 14, the first imaging means 48 includes a first wavelength selection mirror 48a, a second wavelength selection mirror 48b, a third wavelength selection mirror 48c, a first mirror 48d, and a second mirror. 48e, a half mirror 48f, and a lens 48g. The second imaging means 49 includes a first wavelength selection mirror 48a, a mirror 49a, a second wavelength selection mirror 48b, a lens 48g, and a third wavelength selection mirror 48c. Here, the first wavelength selection mirror 48a, the second wavelength selection mirror 48b, and the third wavelength selection mirror 48c selectively reflect the light of the first wavelength emitted from the first light source 41, and the second wavelength selection mirror 48b. The light is selectively transmitted to the second wavelength emitted by the light source 42.

Next, a flow until incident light emitted from the first light source 41 is irradiated on the detected person 14 will be described with reference to FIG. 15A. The light emitted from the first light source 41 enters the half mirror 48f. Incident light that has entered the half mirror 48f is split into light that passes through the half mirror 48f and travels in the direction of the first wavelength selection mirror 48a, and light that reflects by the half mirror 48f and travels in the direction of the second mirror 48e. The light reflected by the half mirror 48f is reflected by the second mirror 48e, reflected by the first mirror 48d, and travels in the direction of the person to be detected 14 via the wavelength selection means. On the other hand, the light transmitted through the half mirror 48f is reflected by the first wavelength selection mirror 48a and travels in the direction of the detected person 14 through the wavelength selection means. That is, the light transmitted through the half mirror 48f is parallel to the light reflected by the half mirror 48f and is incident on different positions of the detection subject 14.

On the other hand, as shown in FIG. 15B, the second light source 42 emits light of the second wavelength from the position different from that of the first light source 41 in the direction of the person to be detected 14.

Next, a flow until light from the person 14 to be detected forms an image on the first image sensor 44 and the second image sensor 45 will be described. As shown in FIG. 15A, first, light from at least a part of the face including one or more pupils 14a of the person to be detected 14 is parallel to the optical axis A and is selected via the wavelength selection unit 43. The light enters the mirror 48a. And the light of the 1st wavelength among the lights which entered via the wavelength selection means 43 is reflected by the 1st wavelength selection mirror 48a. Further, the first wavelength light is partially reflected by the half mirror 48f, reflected by the second wavelength selection mirror 48b, incident on the lens 48g, and reflected by the third wavelength selection mirror 48c. The image is formed on the image sensor 44.

Similarly, light from at least a part of the face including one or more pupils 14a of the person 14 to be detected is incident on the first mirror 48d via the wavelength selection means 43 in parallel with the optical axis B. And the light of the 1st wavelength among the lights which entered via the wavelength selection means 43 is reflected by the 1st mirror 48d. Further, the light is reflected by the second mirror 48e, partially transmitted through the half mirror 48f, and thereafter imaged on the first image sensor 44 through the same path as the light incident parallel to the optical axis A. That is, the image of the person 14 to be detected from the viewpoints in both the optical axis A direction and the optical axis B direction is formed on the first image sensor 44. Of the image formed, the pupil 14a has an optical axis (optical axis A and optical axis B) on which the light emitted from the light source 41 is irradiated to the detected person 14 and light reflected from the detected person 14. Since the optical axes (optical axis A and optical axis B) incident on the image sensor 44 are coaxial, fundus reflection light is imaged. Therefore, a so-called bright pupil image is captured.

Note that light transmitted through the half mirror 48f with light from the optical axis A direction and light reflected through the half mirror 48f with light from the optical axis B direction return to the light source 41 direction. Since the return light is only the light reflected from the face of the person 14 to be detected among the incident light from the light source 41, the return light has a much higher intensity than the intensity of the incident light emitted from the light source 41. small. For this reason, even if it is reflected again by the surface of the light source 41 or the like, there is no particular problem in this embodiment.

Further, as shown in FIG. 15B, light from at least a part of the face including one or more pupils 14 a of the person to be detected 14 enters the wavelength selection unit 43. Of the light incident through the wavelength selection means 43, the second wavelength light is transmitted through the first wavelength selection mirror 48a. The transmitted light is reflected by the mirror 49a, transmitted by the second wavelength selection mirror 48b, enters the lens 48g, passes through the third wavelength selection mirror 48c, and is coupled onto the second image sensor 45. Imaged. That is, unlike the first image sensor 44, only an image of at least a part of the face including one or more pupils 14 a of the person 14 to be detected from the direction of the optical axis A is formed on the image sensor 45. Of the image formed, the pupil 14 a has an optical axis on which the light emitted from the light source 42 is applied to the detected person 14, and an optical axis on which the light reflected from the detected person 14 enters the image sensor 44 ( Since the optical axis A and the optical axis B) are non-coaxial, fundus reflection light is not imaged. For this reason, a so-called dark pupil image is captured. The light of the second wavelength from the direction of the optical axis B is transmitted through the second wavelength selection mirror 48b in FIG.

[Explanation of Effects] In the pupil imaging device 500 according to the present embodiment, light sources having different wavelengths are arranged coaxially with the imaging element, and the other is arranged at a non-coaxial position / angle, and on different imaging elements for each wavelength. Since the image of the pupil 14a is formed on the light, the images of the bright pupil and the dark pupil can be captured.

Since the light incident from both the optical axis A and the optical axis B is imaged on the image sensor 44, the half mirror 48f has a first wavelength transmittance of 50% (a reflectance of 50%). It is preferable to use one.

In order to simplify the alignment process of the optical system when manufacturing the pupil imaging device 500, for example, the first wavelength selection mirror 48a, the second wavelength selection mirror 48b, and the third wavelength selection mirror 48c are: An optically equivalent configuration such as the prism body 52 shown in FIG. 15C may be used. The prism body 52 includes a total reflection surface 51 and a wavelength selection film 52. In this case, it arrange | positions at the same angle as a wavelength selection mirror so that the wavelength selection film | membrane 51 may be directed in the direction in which the light reflected on the face of the to-be-detected person 14 injects into a wavelength selection mirror. Similarly, other mirrors may have other optically equivalent configurations. When the mirror that totally reflects is replaced with the prism body 52, the total reflection surface 50 of the prism body 52 is used as the surface of the above-described mirror. Use.

[Fifth Embodiment] FIG. 16 is a top view showing a pupil diameter measuring apparatus 600 of this embodiment. In the present embodiment, in addition to the configuration of the fourth embodiment, a region obtained by imaging the pupil 14a in one image formed on the first image sensor 44 and a second image sensor 45 are connected. A pupil diameter estimating means 53 is provided for estimating the pupil diameter by triangulation using the same imaged area of the pupil 14a in one image.

The pupil diameter estimating means 53 includes a pupil extracting means 53a and a pupil diameter calculating means 53b as shown in FIG. The pupil extracting means 53a images the pupil 14a in the image imaged on the first image sensor 44 and the same pupil 14a in the image imaged on the second image sensor 45. Extract regions. Usually, since there are two pupils, a total of four regions are extracted. The pupil diameter calculating means 53b calculates the three-dimensional coordinates and dimensions of the pupil 14a from the extracted four regions.

[Description of Configuration / Operation] In the following, the pupil extracting means 53a uses a single image formed on the first image sensor 44 and a single image formed on the second image sensor 45. The flow from extraction of two regions until the pupil diameter calculation means 53b calculates the three-dimensional coordinates and dimensions of one or more pupil extraction means 53a will be described with reference to the flowchart of FIG.

As shown in FIG. 18, first, the pupil extracting means 53a captures the respective images formed on the first image sensor 44 and the second image sensor 45 (S102; image capture). Next, the pupil extracting means 53a calculates the difference between the image formed on the first image sensor 44 and the image formed on the second image sensor 45 (S103; image difference). Further, the pupil extracting means 53a performs binarization processing on the image after the difference calculation, and extracts two regions in which the pupil 14a is imaged (S104; optical axis A image pupil partial extraction). The pupil diameter calculating unit 53b acquires coordinates (X and Y coordinates) in the imaging element plane around the pupil 14a from the two extracted regions, and calculates at least one pupil diameter (S105: viewpoint) A image pupil partial pupil diameter calculation). The pupil diameter calculating means 53b sets a differential filter from the obtained pupil diameter (S106; differential filter setting). Further, the pupil diameter calculating means 53b applies the above-described differential filter to the image formed on the first image sensor 44 (S107; image differential processing). The pupil extracting unit 53a performs binarization processing on the image to which the above-described image differentiation processing is applied, extracts four regions corresponding to the pupil portion 22 in the image, and outputs them to the pupil diameter calculating unit 53b (S108; Pupil part extraction). The pupil diameter calculating means 53b calculates the X and Y coordinates in the first imaging element 44 plane of the four regions corresponding to the pupil portion 22 in the image (S109; coordinate acquisition). In addition to the obtained X and Y coordinates, the pupil diameter calculation unit 53b converts the pupil image 22 into a real image including the pupil portion 22 imaged on the first image sensor 44 by triangulation using the calculation methods shown in Equations 1 to 3. On the other hand, a distance in the vertical direction (Z coordinate) is obtained from the first image sensor 44 (S110; acquisition of three-dimensional coordinates of the pupil). The pupil diameter calculating means 53b calculates the pupil diameters of the four pupil portions 22 from the obtained three-dimensional coordinates (S111; pupil diameter calculation), and records the calculated pupil diameter (S112; pupil diameter recording). In the above pupil diameter measuring apparatus 600, when imaging a plurality of times, such as when imaging in time series, the process returns to step S101 after the process of S112 is completed in the flowchart of FIG.

Next, among the steps described above, S104 to S108, which are differences from the flowchart of FIG. 8 in the third embodiment, will be described in detail. First, S104 will be described. FIG. 19A is an intensity distribution of an image captured by the first image sensor 44, and FIG. 19B is a diagram illustrating an intensity distribution of an image captured by the second image sensor 45.

In FIG. 19A, as in FIG. 9B, the intensity of the pupil part 22 increases with respect to the face part 23, but when the difference in intensity level between the face part 23 and the pupil part 22 is small, binarization processing is performed. Becomes difficult.

In such a case, the difference between FIG. 19A, which is an image of a bright pupil, and FIG. 19B, which is an image of a dark pupil, is taken (subtraction processing is performed on FIG. 19B from FIG. 19A). Thus, the component of reflection from the face portion 23 is canceled. As a result, as shown in FIG. 19C, only the pupil portion 22 when viewed in the direction of the optical axis A can be extracted. After the extraction of the pupil portion 22, as shown in FIG. 19C, the right and left pupil diameters DR and DL of the pupil portion 22 in the optical axis A direction can be obtained from the extracted pupil portion 22 as S105.

From the binarized image, the minimum coordinate of the right eye image in the optical axis A direction is recorded as XARmin and the maximum coordinate is recorded as XARmax, as in FIG. 9C. The minimum coordinate of the left eye image in the direction of the optical axis A is recorded as XALmin, and the maximum coordinate is recorded as XALmax. The minimum coordinate of the right eye image in the optical axis B direction is recorded as XBRmin, and the maximum coordinate is recorded as XBRmax. The minimum coordinate of the left eye image in the optical axis B direction is recorded as XBLmin, and the maximum coordinate as XBLmax. Similarly, the Y component can be recorded.

Next, S106 will be described. FIG. 20 is a diagram for explaining the operation of the feature extraction filter. In the present embodiment, a second-order differential filter represented by Expression 4 is exemplified as the feature extraction filter. Here, the differential interval DR is the right pupil diameter of the pupil portion in the optical axis A direction described above. As S106, any one of the right and left pupil diameters DR and DL of the pupil portion 22 in the direction of the optical axis A is applied to Equation 4. Then, the image of FIG. 20A is subjected to the second-order differential filter of FIG. 20B as S107, and the binarization process is performed as S108, so that the one having the diameter DR can be selectively extracted, as shown in FIG. 20C. The pupil portion 22 when viewed in the direction of the optical axis B can also be extracted. The extracted pupil portion 22 is output to the pupil diameter calculating means 53b.
[Formula 4]
I (x, y) = 2 · I (x, y) −I (x−DR, y) −I (x + DR, y)
Here, since the DR value can always be obtained from the extracted image of the pupil 14a in the direction of the optical axis A, even when the imaging distance varies and the DR value increases as shown in FIG. 20E, for example. Since the time value DR is the differential interval, the secondary differential filter can be optimized as shown in FIG. 20F, and the pupil 14a can be selectively extracted as shown in FIG. 20G.

Although the pupil diameter measuring apparatus 600 has been described above, the pupil diameter measuring apparatus 600 may further include pupil state determining means 54 as shown in FIG. FIG. 21 shows a flowchart when the pupil state determination means 54 is provided. From the value of the pupil diameter (S111; pupil diameter calculation) calculated in FIG. 18 (S112; pupil diameter recording) and the analysis result of the frequency analysis (S8 in FIG. 12) shown in the third embodiment, the detected person 14 The psychological state and physiological state are determined. Note that, as described in the third embodiment, as a method for determining the psychological state and physiological state of the person 14 to be detected, first, as described in the third embodiment, the frequency band of wavelet transform is set to a low frequency band 0. It is divided into 04 to 0.15 Hz and high frequency band 0.16 Hz to 0.5 Hz. Then, the psychological state can be determined from the amplitude ratio between the low frequency band amplitude and the high frequency band amplitude. That is, when the pupil state determination means 54 is provided, the flowchart of FIG. 21 returns to the step of S201 after the processing of S211 is completed. The steps after S212 may be performed in parallel with the processing of S201 to S211 or may be performed separately.

[Description of Effects] In the pupil imaging apparatus 600 according to the present embodiment, even when the difference in intensity level between the face portion 23 and the pupil portion 22 is small, an accurate pupil diameter can be calculated by binarization processing. Moreover, the pupil state detection apparatus 700 in this embodiment can provide a small and low-cost apparatus that objectively and quantitatively evaluates the mental state in a free posture without restraining the body of the person 14 to be detected. Is possible.

Note that a feature extraction filter may be a two-dimensional distribution represented by Equation 5 instead of a second-order differential filter. In this case, the filtering process is as shown in FIGS. 20D and 20H.
[Formula 5]
I (x, y) = 4 · I (x, y) −I (x−D, y) −I (x + D, y) −I (x, y−D) −I (x, y + D)
For extraction of the pupil 14a, the left pupil diameter DL may be used, or the right and left pupil diameters DR and DL may be used alternately.

As long as the feature extraction filter is based on the values DR and DL, various known image processing filters may be used.

An LED having a wavelength of 850 nm was used as the light source 1. An LED with a wavelength of 940 nm was used as the light source 67. The light sources 1 and 2 may have different wavelengths as long as the same effect can be obtained.

As the wavelength selection filter, a dichroic mirror using a dielectric multilayer film can be used, but other known wavelength selection filters may be used.

In the present embodiment, the optical axis A and the optical axis B are parallel, but it may be a condition for convergence imaging in which the angle between the optical axis A and the optical axis B is changed by changing the angle of each mirror.

[Sixth Embodiment] Next, the sixth embodiment will be described. FIG. 22 is a top view showing a configuration of a pupil state detection / display apparatus 800 according to the present embodiment. As shown in FIG. 22, the pupil state detection display device 800 of the present embodiment includes a display device 61 between the detected person 14 and the wavelength selection means 43 as compared with the third or fifth embodiment. Is different. Other configurations and configuration variations are the same as those in the third and fifth embodiments. 22A shows a case where the display device 61 is an optical filter 61a, FIG. 22B shows an optical filter 61a and a display unit 61b, an information processing device 61c, and FIG. 22C shows an optical filter 61a, a display unit 61b, and a half mirror 61d. .

[Description of Configuration] In FIG. 22A, the pupil state detection device 101 is the pupil state detection device 400 or the pupil state detection device 700 of the third and fifth embodiments of the present invention. When the optical filter 61a in FIG. 22A is visible light total reflection and infrared light total transmission, when it sees from the to-be-detected person 14, it looks like a mirror. Further, if the optical filter 61a is made to absorb visible light and transmit infrared light, it will appear as a black plane when viewed from the person 14 to be detected.

FIG. 22B shows a configuration in which the result determined by the information processing device 61c is displayed on the display unit 61b based on the determination result of the pupil state detection device 101. In this configuration, by displaying on the display unit 61b, the subject 14 is notified of the fatigue state and the mental state, or a visual stimulus for measuring the reaction of the subject 14 such as light reflection is given. Also good. Note that, as described in the third embodiment, as a method for determining the psychological state and physiological state of the person 14 to be detected, first, as described in the third embodiment, the frequency band of wavelet transform is set to a low frequency band 0. It is divided into 04 to 0.15 Hz and high frequency band 0.16 Hz to 0.5 Hz. Then, the psychological state can be determined from the amplitude ratio between the low frequency band amplitude and the high frequency band amplitude.

FIG. 22C is a configuration provided with a half mirror 61c in the configuration shown in FIG. 22B.
When there is no display on the display unit 61b, it acts as a mirror, and when there is a display on the display unit 61b, the mirror image of the detected person 14 and the display image can be optically fused.

[Explanation of Effects] With the pupil state detection display device 800 of the sixth embodiment, in the case of the configuration of FIG. 22A, the detected person 14 can evaluate a more accurate psychological state without feeling that it is being measured. Furthermore, in the case of the configuration of FIG. 22B, a visual stimulus is given to the detected person 14 according to the evaluation result by the psychological state of the detected person 14, the pupil state detecting device 400 of the physiological state, and the pupil state detecting device 700, It has become possible to have further effects on stress management and treatment of mental illness. In the case of the configuration of FIG. 22C, the effect of the configuration of FIG. 22A and FIG. 22B described above can be obtained according to the display content.

A liquid crystal display can be used as the display unit 61b. In addition to the liquid crystal display, an organic EL display or other display may be used. Note that “EL” is an abbreviation for “Electroluminescence”. The display unit 61b may be a stereoscopic display using a lenticular lens or a parallax barrier, in addition to a normal two-dimensional display. When a stereoscopic display is used, the left and right pupil positions obtained by the pupil state detection device 101, the convergence angle obtained from the pupil position, and the focus adjustment state of the eyeball obtained from the pupil diameter are used to observe the stereoscopic display. An appropriate parallax image may be displayed.

A part or all of the above-described embodiment can be described as in the following supplementary notes, but is not limited thereto.
[Appendix 1]
One image sensor,
An imaging apparatus, comprising: an imaging unit configured to form a plurality of images at different viewpoints of a common portion of a subject irradiated with light from a light source on the one imaging element.
[Appendix 2]
The at least two different viewpoints are a first viewpoint and a second viewpoint,
The imaging means reflects the light from the subject at the first viewpoint by a mirror and enters the lens to form an image on the one image sensor, and the light from the subject at the second viewpoint. The imaging apparatus according to appendix 1, wherein the image is reflected by a mirror and incident on the lens so as to form an image of the subject at the first viewpoint and the second viewpoint at the same time on the one imaging element.
[Appendix 3]
The arbitrary light source includes one light source, a dividing unit that divides incident light emitted from the one light source, and a subject direction from two directions that are different depending on an angle at which the incident light divided by the dividing unit is parallel or converged. The imaging apparatus according to appendix 1 or 2, wherein the imaging apparatus is a mirror that is made incident on the lens.
[Appendix 4]
The imaging apparatus according to appendix 3, wherein the dividing unit is a half mirror.
[Appendix 5]
A light source that emits light of a predetermined wavelength;
Wavelength selection means for selectively transmitting light of the predetermined wavelength;
One image sensor,
Respective images at different viewpoints of at least a part of the face including one or more pupils of the detection subject irradiated with light of the predetermined wavelength from the light source via the wavelength selection unit, the wavelength selection unit An image forming means for forming an image on the one image sensor via
The different viewpoints are on different optical axes substantially parallel to the direction in which the light source emits light;
The different optical axes are arranged parallel or converging to each other,
Pupil imaging device.
[Appendix 6]
The pupil diameter is determined by triangulation using the pupil imaging device of Appendix 5 and at least two regions corresponding to the one or more pupils in one image formed on the one imaging element. A pupil diameter measuring device comprising: a pupil diameter estimating means for estimating.
[Appendix 7]
The pupil diameter estimating means, the pupil extracting means for extracting the at least two regions from one image formed on the one image sensor;
The pupil diameter measuring device according to appendix 6, further comprising pupil diameter calculating means for calculating and outputting three-dimensional coordinates and dimensions of the one or more pupils from the at least two regions.
[Appendix 8]
The pupil diameter measuring device according to appendix 6 or 7, and pupil state determining means for determining the psychological state and physiological state of the detected person from the three-dimensional coordinates and dimensions of the one or more pupils output from the pupil diameter calculating means And a pupillary state detection device.
[Appendix 9]
A first light source that emits light of a first wavelength;
A second light source that emits light of a second wavelength at an angle different from that of the first light source;
Wavelength selection means for selectively transmitting the light of the first wavelength and the light of the second wavelength;
A first image sensor;
A second imaging device;
When the light from the first light source is emitted to the detected person, the optical axis is coaxial, and one of the detected persons irradiated with the light of the first wavelength through the wavelength selecting means Images of the light of the first wavelength at the first viewpoint and the second viewpoint of at least a part of the face including the pupil are connected onto the first imaging element via the wavelength selection unit. First imaging means for imaging,
The image by the light of the second wavelength at the first viewpoint of at least a part of the face including at least one pupil of the subject irradiated with the light of the second wavelength is passed through the wavelength selection means. And second image forming means for forming an image on the second image sensor,
The first viewpoint and the second viewpoint are on two optical axes substantially parallel to a direction in which the first light source emits light,
The two optical axes are arranged parallel to each other or converging.
Pupil imaging device.
[Appendix 10]
One or more of the pupil imaging device according to appendix 9, one image imaged on the first image sensor, and one image imaged on the second image sensor And a pupil diameter estimating means for estimating a pupil diameter by triangulation using at least two regions corresponding to the pupil of the pupil diameter.
[Appendix 11]
The pupil diameter estimating means determines the region of the one or more pupils from one image imaged on the first image sensor and one image imaged on the second image sensor. Pupil extracting means for extracting;
The pupil diameter measuring device according to appendix 10, further comprising pupil diameter calculating means for calculating and outputting three-dimensional coordinates and dimensions of the one or more pupils.
[Appendix 12]
The pupil diameter measuring device according to appendix 11, wherein each of the images formed on the first imaging element and the second imaging element is an image including a bright pupil and an image including a dark pupil.
[Appendix 13]
The pupil diameter measuring device according to appendix 11, wherein the pupil extracting means performs a subtraction process on the respective images.
[Appendix 14]
The first imaging means is a first wavelength selection mirror that selectively reflects the first wavelength of light from at least a part of the face including the one or more pupils in the first viewpoint. Reflect, further reflect by a mirror, further reflect by the second wavelength selective mirror that selectively reflects the first wavelength, and enter the lens to form an image on the first imaging element, The light of at least a part of the face including the one or more pupils in the second viewpoint is reflected by a mirror and further incident on the lens to form an image on the first image sensor. Pupil imaging device.
[Appendix 15]
The second imaging means reflects light from at least a part of the face including the one or more pupils at the first viewpoint with a mirror and makes it incident on a lens to be coupled onto the second imaging element. The pupil image pickup device according to appendix 9, wherein an image is formed.
[Appendix 16]
A pupil state detection display device comprising: the pupil imaging device according to attachment 15; and a display device further provided between the detected person and the wavelength selection unit.
[Appendix 17]
Emitting light of a predetermined wavelength from the light source to at least a part of the face including one or more pupils of the subject;
Light from at least a part of the face including the one or more pupils on at least two different viewpoints on each of the two optical axes substantially parallel to the direction of emitting the light on one image sensor Image,
Pupil imaging method.
[Appendix 18]
The first light source emits light of the first wavelength to at least a part of the face including one pupil of the detected person,
The second light source emits light of the second wavelength to at least a part of the face including one pupil of the detected person at a different angle from a position different from the first light source,
The light of the first wavelength and the light of the second wavelength that enter from at least a part of the face including one pupil of the detected person selects the light of the first wavelength and the light of the second wavelength Is incident on the first imaging means and the second imaging means via the wavelength selection means that transmits the light,
When the light from the first light source is emitted to the detected person, the first imaging means whose optical axis is coaxial is one pupil of the detected person at the first viewpoint and the second viewpoint. Imaging light of the first wavelength of each of at least a portion of the face including a first imaging element;
A second imaging means images the light of the second wavelength of at least a part of the face including at least one pupil of the subject at the first viewpoint on the second image sensor;
Pupil imaging method.
While the present invention has been described with reference to the embodiments (and examples), the present invention is not limited to the above embodiments (and examples). Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.
This application claims the priority on the basis of Japanese application Japanese Patent Application No. 2013-238493 for which it applied on November 19, 2013, and takes in those the indications of all here.

DESCRIPTION OF SYMBOLS 1a Feature point 2 Arbitrary light source 2a Light source 2aa Mirror 2b Dividing means 2ba Half mirror 2bb-2bd Mirror 2b Lens 3 First viewpoint 4 Second viewpoint 5 Imaging element 6 Imaging means 6a Mirror 6b Lens 100 Imaging device 11 Light source 12 Wavelength selection means 13 Imaging element 13a Optically equivalent imaging element 14 Detected person 14a Pupil 15 First viewpoint 16 Second viewpoint 20 Imaging means 20a Half mirror 20b-d Mirror 20e Lens 20ea Optically equivalent lens 21 pupil diameter estimation means 21a pupil extraction means 21b pupil diameter calculation means 22 pupil part 23 face part 24 pupil state determination means 200 pupil imaging device 300 pupil diameter measurement device 400 pupil state detection device 41 first light source 42 second light source 43 Wavelength selection means 44 First imaging element 45 Second imaging element 46 First viewpoint 47 Second viewpoint 48 First imaging means 48a First wavelength selection mirror 48b Second wavelength selection mirror 48c Third wavelength selection mirror 48d First mirror 48e Second mirror 48f Half mirror 48g Lens 49 Second imaging means 49a Mirror 50 Total reflection surface 51 Wavelength selection film 52 Prism body 53 Pupil diameter estimation means 53a Pupil extraction means 53b Pupil diameter calculation means 54 Pupil State determining means 500 Pupil imaging device 600 Pupil diameter measuring device 700 Pupil state detecting device 61 Display device 61b Display unit 101 Pupil state detecting device 103 Optical filter 104 Half mirror 800 Pupil state detecting display device

Claims (10)

One image sensor,
An imaging apparatus, comprising: an imaging unit configured to form a plurality of images at different viewpoints of a common portion of a subject irradiated with light from a light source on the one imaging element. The at least two different viewpoints are a first viewpoint and a second viewpoint,
The imaging means reflects the light from the subject at the first viewpoint by a mirror and enters the lens to form an image on the one image sensor, and the light from the subject at the second viewpoint. The imaging apparatus according to claim 1, wherein an image of the subject at the first viewpoint and the second viewpoint is simultaneously formed on the one imaging element by reflecting the light from a mirror and entering the lens. . A light source that emits light of a predetermined wavelength;
Wavelength selection means for selectively transmitting light of the predetermined wavelength;
One image sensor,
Respective images at different viewpoints of at least a part of the face including one or more pupils of the detection subject irradiated with light of the predetermined wavelength from the light source via the wavelength selection unit, the wavelength selection unit Imaging means for forming an image on the one image sensor via
The different viewpoints are on different optical axes substantially parallel to the direction in which the light source emits light;
The different optical axes are arranged parallel or converging to each other,
Pupil imaging device. A pupil diameter by triangulation using the pupil imaging device according to claim 3 and at least two regions corresponding to the one or more pupils in one image formed on the one image sensor. A pupil diameter measuring means for estimating the pupil diameter. The pupil diameter estimating means, the pupil extracting means for extracting the at least two regions from one image formed on the one image sensor;
The pupil diameter measuring device according to claim 4, further comprising a pupil diameter calculating unit that calculates and outputs three-dimensional coordinates and dimensions of the one or more pupils from the at least two regions. A first light source that emits light of a first wavelength;
A second light source that emits light of a second wavelength at an angle different from that of the first light source;
Wavelength selection means for selectively transmitting the light of the first wavelength and the light of the second wavelength;
A first image sensor;
A second imaging device;
When the light from the first light source is emitted to the detected person, the optical axis is coaxial, and one of the detected persons irradiated with the light of the first wavelength through the wavelength selecting means Images of the light of the first wavelength at the first viewpoint and the second viewpoint of at least a part of the face including the pupil are connected onto the first imaging element via the wavelength selection unit. First imaging means for imaging,
The image by the light of the second wavelength at the first viewpoint of at least a part of the face including at least one pupil of the subject irradiated with the light of the second wavelength is passed through the wavelength selection means. And second image forming means for forming an image on the second image sensor,
The first viewpoint and the second viewpoint are on two optical axes substantially parallel to a direction in which the first light source emits light,
The two optical axes are arranged parallel to each other or converging.
Pupil imaging device. 7. The pupil imaging device according to claim 6, and one of the one image formed on the first image sensor and the one image formed on the second image sensor. A pupil diameter measuring device comprising: pupil diameter estimating means for estimating a pupil diameter by triangulation using at least two regions corresponding to the above pupils. The pupil diameter estimating means determines the region of the one or more pupils from one image imaged on the first image sensor and one image imaged on the second image sensor. Pupil extracting means for extracting;
The pupil diameter measuring device according to claim 7, further comprising pupil diameter calculating means for calculating and outputting three-dimensional coordinates and dimensions of the one or more pupils. Each image formed on the first image sensor and the second image sensor is an image including a bright pupil and an image including a dark pupil,
9. The pupil diameter measuring device according to claim 8, wherein the pupil extracting means performs a subtraction process on the respective images. Emitting light of a predetermined wavelength from the light source to at least a part of the face including one or more pupils of the subject;
Light from at least a part of the face including the one or more pupils on at least two different viewpoints on each of the two optical axes substantially parallel to the direction of emitting the light on one image sensor Image,
Pupil imaging method.

Images