JP2000322660A - Customer image database creation device, imaging device, imaging method, and recording medium - Google Patents

Abstract

(57) [Summary] [Purpose] To construct a database that can extract and browse customer group image data under specific conditions such as sales products and sales hours without losing privacy. [Structure] A POS register capable of acquiring information on the sale of a product and a photograph of the customer at the time of purchase of the product. An imaging device having personal identification difficulties as data, and image data of the imaging device;
A customer image database creating apparatus comprising: an arithmetic unit that creates a database as sales information with customer images using the sales information acquired from the POS register.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION The present invention relates to a method of protecting customer's privacy by using digitized data such as time and numerals in sales information of products (so-called POS information).
The present invention relates to a technology for creating a database to which customer video information is added.

[0002]

[Prior Art] (Current state of POS information technology) So-called POS
Information is input into the register of information on products purchased by customers and information on customers to be purchased, and is converted into a database, so that merchandising can be verified and analyzed, and subsequent merchandising can be planned. It is useful data.

(Type of Data Input When One Product is Purchased) Normally, when one product is purchased, JAN
A single item code, quantity, time, and the like of a product called a code are recorded as data. The product code is input by barcode scanning, and price information is called from the product master. Further, when data such as the age group of the customer and the classification such as gender are input by the clerk's subjective judgment, the data is also recorded.

(Specific usage of accumulated data) Normally, the sales quantity of each single product is totaled to determine the order quantity. In this way, the opportunity loss due to shortage is prevented. It is also used as data for determining the adoption / non-adoption of a product, such as cutting a dead product while leaving a hot selling product. (Potential demand for video information from customers) By the way, the amount of information obtained from image data must be information that directly appeals to the sight, although the large amount of recording required by the recording medium is often technically taken up. Therefore, there is much information that can be obtained by the information recipient. For example, for a customer who has purchased a certain product, "20"
Woman ". Assuming that the video information of the customer exists, it can be obtained from the video information that "women in their twenties are dressed like a company employee or housewife with children".

[0005] As described above, image data of a customer group under specific conditions such as sales products and sales hours can be extracted and browsed without impairing privacy, and a merchandising strategy for retailers, merchandise development, and product development. It has the potential to become extremely important data in marketing strategies for manufacturers and the like, and the potential demand for obtaining customer video information for commercial use is considered to be extremely high.

On the other hand, digital camera and digital video technologies for capturing images as digital data are evolving. For example, the digital camera product cycle is 3
Companies are competing in a development competition in which the number of pixels increases and prices decrease in about a month. Therefore, the technical and economic barriers to capturing customer image data are becoming increasingly lower.

(Present State of Privacy Protection) With the progress of image information technology, when captured image data is used for commercial purposes, the privacy of people in the image must be protected. In Japan, laws protecting privacy protection are not enough compared with those in the West, but if image data is commercialized with insufficient protection of privacy in image data, the company will be socially sanctioned. It will be. Therefore, when providing image data for commercial use, it is customary for a company to pay close attention.

[0008]

The prior art has the following problems. (Accuracy of customer information) Of the data taken in by the POS system, the merchandise information is input with a barcode, so there is almost no erroneous input. However, since the customer information is input by a person who operates a register, The accuracy of the customer information input depends on the operator.

To improve the accuracy of inputting customer information, there are prior measures such as education for raising the awareness of the operator, but there is no rational means as a technique for post-verification. In other words, although the input data remains in the database, the customer to be input only exists at the time of passing through the register.Therefore, a means for verifying the correspondence with the data input by the register operator after the fact is required. Did not.

[0010] Here, if image information of the customer's figure to be settled at the register can be acquired and associated with product information, it can be assumed that this problem can be solved. However, if the image of the customer is acquired as image information, it is expected that the customer will feel discomfort that privacy has been violated and may avoid shopping at the store.

(Technical Problem of Privacy Protection) The essence of privacy protection is to make it impossible to identify an individual. For example, in image data for commercial use, for an image that can be identified by an individual, the eye and its surroundings are subjected to mask processing or mosaic processing, but these processing steps are performed artificially. ing.

However, from the standpoint of a customer when an image is acquired during shopping, from the viewpoint of the customer, image data acquisition means that cannot be personally identified at the same time as the acquisition of the image data and without human intervention. If not, it is expected that the aforementioned “discomfort of privacy infringement” cannot be resolved. However, such a technique has not been established.

(Problem to be Solved by the Present Invention) The problem to be solved by the present invention is to extract image data of a customer group under specific conditions such as sales product and sales time without impairing privacy.
It is to provide browsable technology. Here, an object of the invention described in claim 1 is to provide a database creating apparatus capable of extracting and browsing image data of a customer group under specific conditions such as sales products and sales hours without impairing privacy. .

It is another object of the present invention to provide an imaging apparatus capable of capturing a commercial image without losing privacy. It is another object of the present invention to provide an imaging method capable of capturing a commercial image without impairing privacy. It is still another object of the present invention to provide a program for causing a computer to implement an imaging method capable of capturing a commercial image without impairing privacy.

[0015]

SUMMARY OF THE INVENTION The present invention is to achieve the above object. (Claim 1) The invention according to claim 1 has a POS register capable of acquiring information at the time of sale of a product and a photograph of a customer at the time of purchase of a product, and at the same time, it is difficult to perform personal authentication on the photographed customer image data. And an image pickup device provided with means for making personal identification difficult to obtain data for making personal identification difficult, and sales with customer images using the image data of the image pickup device and the point-of-sale information obtained from the POS register. The present invention relates to a customer image database creation device including a calculation device that creates a database as time information.

(Definition of terms) "Difficulty in personal recognition" means means for changing the image to a certain extent that it is difficult to recognize and identify "who". Specifically, so-called mosaic processing, mask processing and so on. For example, a procedure from input of customer image data to conversion to personal identification difficult data is provided in one chip and provided, and is incorporated in the “imaging device”.

(Operation) First, the customer goes to the POS register for payment of the purchased product. Then, the POS register acquires the point-of-sale information. On the other hand, the imaging device takes an image of the customer. At the same time, the individual recognition difficulting means performs processing for making the personal authentication difficult on the customer image data.

The image data thus produced and the sales information obtained by the POS register are made into a database by the arithmetic unit as sales information with customer images. The sales information with customer images captured in the database can be used as image data to understand the overall image and mood of the customer,
Since personal authentication is difficult, there is a very small risk of violating privacy, such as in what store the customer purchased what kind of product.

(Claim 2) According to the invention of claim 2, an individual is photographed, and at the same time, a process for making personal authentication difficult is performed on the photographed customer image data to make personal identification difficult data. An image pickup apparatus provided with identification difficulty means, comprising: a flesh color extracting means for extracting a flesh color part in a photographed image; and a predetermined area extracting means for extracting an area larger than a predetermined area from the flesh color part extracted by the flesh color extracting means. The present invention also relates to an imaging apparatus comprising: a conversion unit that randomly converts pixel data of an eye-color part existing in an area extracted by the predetermined area extraction unit and a peripheral part thereof.

(Explanation of Terms) The "skin color" in the "skin color extraction means" can be adjusted or defined according to the environment in which the imaging apparatus according to the present invention is installed.
The “eye color” in the “conversion means” can also be adjusted or defined according to the environment in which the imaging device according to the present invention is installed.

The random conversion performed by the "converting means" includes so-called mosaic processing and mask processing. (Operation) First, the imaging device photographs a human figure. The skin color extracting unit extracts a skin color portion from the captured image thus captured. At this time, faces, hands, and the like are extracted from the captured image.

Next, the predetermined region extracting means extracts a skin color region occupying an area equal to or larger than the predetermined region from the extracted skin color portion, and ignores a portion of the skin color area not reaching the predetermined region. At this time, “hands” in the image data that do not require random conversion and “small faces” in which the need for random conversion is not high are discarded. Next, the conversion unit recognizes the eye-color portion of the extracted region, and randomly converts the pixel data of the eye-color portion and the peripheral portion. Then, the images of the eyes and the peripheral portions of the face in the image data are disturbed, and it is extremely difficult to perform personal authentication of the converted face.

(Claim 3) The invention according to claim 3 limits the image pickup apparatus according to claim 2, and comprises an output restricting means for making it impossible to output image data before conversion by the converting means. Related to an imaging device. For example, when the procedure from the input of customer image data to the personal identification difficult data is provided in one chip and provided in an "imaging device", the process including the output restriction means is made one chip.

(Operation) According to the present invention, video data which has not been converted is not output, and privacy protection can be enhanced. (Claim 4) The invention according to claim 4 is an imaging method comprising a face modification step of modifying a face portion in image data of a photographed human figure while photographing a human figure as image data. There is a skin color extraction procedure for extracting a skin color part in a photographed image, a constant area extraction procedure for extracting an area larger than a certain area from the skin color part extracted by the skin color extraction procedure, and a constant area extraction procedure. And a conversion procedure for randomly converting pixel data of an eye-colored portion present in a region and a peripheral portion thereof.

(Claim 5) The invention according to claim 5 is a limitation of the imaging method according to claim 4, and includes an output restriction procedure for disabling output of image data before going through the conversion procedure. The present invention relates to an imaging method. (Claim 6) The invention according to claim 6 is for realizing a computer with a face modification function of modifying a face portion in image data of a photographed human figure while photographing a human figure as image data. The present invention relates to a recording medium on which the above program is recorded. That is, the program includes a skin color extraction procedure for extracting a skin color part in a captured image, a constant area extraction procedure for extracting an area larger than a certain area from the skin color part extracted by the skin color extraction procedure, and a constant area extraction procedure. And a conversion procedure for randomly converting the pixel data of the eye-colored portion existing in the region extracted by the above and the peripheral portion thereof. Then, it is installed and used in a computer via a computer-readable recording medium.

Here, the recording medium is a floppy disk, CD-ROM, hard disk, magneto-optical disk,
DVD-ROM and the like.

[0027]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in more detail with reference to embodiments and drawings. The drawings used here are FIGS. 1 to 4. FIG. 1 is an overall view showing installation positions of a POS register and a digital camera which constitute an embodiment of the present invention. FIG. 2 is a conceptual diagram illustrating a process of processing image data captured by a digital camera. FIG.
FIG. 4 is a conceptual diagram for describing one process in a process of processing image data in more detail. FIG. 4 is a conceptual diagram illustrating a construction process of a customer image database. FIG. 5 is a conceptual diagram showing a utilization state of the constructed customer image database.

(Description Based on FIG. 1) FIG. 1 shows a state near a register in a certain store. The registers are
Here, two units (a, b) are installed, each of which is P
Since it is an OS register, data such as the type and price of a product can be read by reading a barcode attached to the product. In each POS register (a, b), each cashier A, B inputs the attributes (sex, age group, etc.) of the customers C, D who came to the settlement.

In the POS register, the image of the customer at the time of product purchase is photographed, and at the same time, the process of making the personal authentication difficult is performed on the photographed customer image data to make the personal identification difficult data. There is provided a digital camera provided with a conversion unit. (Explanation based on FIG. 2) A method in the individual identification difficulting means will be described with reference to FIG.

When a customer stands in front of the POS register, the digital camera takes a picture. The photographing may be performed by an operation of a cashier, but in this embodiment, an automatic photographing method in which a command to release a shutter is issued immediately after the total payment amount is displayed. When the shutter is released, the image is stored in RAM (random access memory). No recording for fixing to a recording medium is performed, and no output is possible. It is assumed that the image data at this time is (1) in FIG. Here, the customer C, the cashier B next, and the customers E and F are behind the customer C.

The image data stored in the RAM undergoes a skin color extraction procedure for extracting a predetermined skin color portion from the photographed image, and becomes data as shown in (2) in FIG. By this procedure, the face and hand of the customer C, the face and hand of the cashier B, and the faces of the customers E and F are extracted. Next, through a certain area extraction procedure for extracting an area part of a certain area or more from the skin color part extracted by the skin color extraction procedure, and a procedure for painting a non-skin color part surrounded by the skin color part, (3) in FIG.
It becomes data like Here, the faces of customers E and F are discarded. Details will be described with reference to FIG.

Subsequently, by extracting a predetermined eye color portion existing in the area extracted by the fixed area extraction procedure, data as shown in (4) in FIG. 2 is obtained. Here, the reason why the eyebrows, the eyeglass frame, and the clock were extracted together with the eyes is that they were recognized as being the colors defined as “eye colors”. Finally, through a conversion procedure of randomly converting the pixel data of the eye color portion and its peripheral portion, data is obtained as shown in (5) in FIG. Here, personal authentication of the customer C and the cashier B is almost impossible.

(Explanation Based on FIG. 3) FIG. 3 is a more detailed explanation of the aforementioned “fixed region extraction procedure”. That is, the pixel portion recognized as “skin color” (the black circle in the above figure) is once expanded, for example, by an eight-neighbor expansion process (center diagram), and then subjected to an eight-neighbor contraction process to separate the pixels. Are connected, and the non-skin color part in the skin color is set as the skin color. As a result, one pixel surrounded by flesh-colored pixels is recognized as flesh-color, and the region surrounded in the figure below becomes flesh-color.

Although illustration is omitted, on the other hand, a plurality of pixels may be shrunk in the eight-neighbor shrinking process in order to discard a region where the random conversion procedure is unnecessary, that is, a flesh-color portion having a certain area or less. (Description based on FIG. 4) FIG. 4 is a diagram showing the customer image database after storing the converted image data obtained by the procedure described in FIG. 2 and the register information obtained from the POS register in respective databases. Is shown.

From the converted image data obtained by the procedure described with reference to FIG. 2, only the largest customer C is cut out, or data in an area other than the customer C is thinned out before being transferred to the image database. It is possible to reduce the amount of data. To match the image data with the registration information, a means such as adding a receipt number in a register as attribute information of the image data is used.

(Explanation Based on FIG. 5) FIG. 5 is a conceptual diagram showing the utilization state of the constructed customer image database.
For example, suppose you want to see what customers purchased a certain product. In that case, the product code of the product is input. Then, the customer images are arranged on one screen. Here, the entire customer segment can be recognized by the video on the screen.

Further, using facial feature recognition software,
Digitize (vectorize) facial features. By the digitization, a similar numerical value is assumed to be the same person, and a fixed customer rate is output. It can also be used to verify the input accuracy of customer attribute information entered by the cashier.

[0038]

According to the first aspect of the present invention, there is provided a database creation apparatus capable of extracting and browsing image data of a customer group under specific conditions such as sales products and sales hours without impairing privacy. Was completed. According to the second and third aspects of the present invention, it is possible to provide an imaging apparatus capable of capturing a commercial image without impairing privacy.

According to the fourth and fifth aspects of the present invention, it is possible to provide an imaging method capable of capturing a commercial image without losing privacy. Furthermore, according to the sixth aspect of the present invention, it is possible to provide a program for causing a computer to realize an imaging method capable of capturing a commercial image without impairing privacy.

[Brief description of the drawings]

FIG. 1 shows a POS register constituting an embodiment of the present invention;
FIG. 2 is an overall view showing an installation position of the digital camera.

FIG. 2 is a conceptual diagram illustrating a process of processing image data captured by a digital camera.

FIG. 3 is a conceptual diagram for describing one process in a process of processing image data in more detail;

FIG. 4 is a conceptual diagram illustrating a construction process of a customer image database.

FIG. 5 is a conceptual diagram showing a utilization state of a constructed customer image database.

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 3E042 BA17 CC10 CD04 5B049 BB11 CC08 CC31 DD02 DD05 EE02 EE03 EE05 EE07 EE10 FF02 FF09 GG01 GG05 5B075 KK07 KK13 KK33 ND03 ND08 ND23 ND34 PP05 U03P01 JJ58 KK57 LL03

Claims (6)

[Claims] 1. A POS register capable of acquiring information on the sale of a product and a photograph of a customer at the time of purchase of the product. An imaging device provided with personal identification difficulty means for making difficult data, and an arithmetic unit for creating a database as sales information with customer images using image data of the imaging device and sales information obtained from the POS register And a customer image database creating device. 2. An image pickup apparatus comprising: means for photographing a human figure, and at the same time, processing for making photographed customer image data difficult for personal authentication to make personal identification data difficult to obtain. Means for extracting a flesh color portion in the photographed image, a predetermined region extracting device for extracting an area larger than a predetermined region from the flesh color portion extracted by the flesh color extracting device, and a predetermined region extracted by the predetermined region extracting device. An imaging apparatus comprising: a conversion unit that randomly converts pixel data of an eye color part existing in a region and a peripheral part thereof. 3. The imaging apparatus according to claim 2, further comprising an output restricting unit that disables output of the image data before the conversion by the converting unit. 4. An imaging method comprising: photographing a human figure to form image data; and a face modification procedure for modifying a face portion in the photographed human figure image data. A skin color extraction procedure for extracting a portion, a constant area extraction procedure for extracting an area larger than a certain area from the skin color part extracted by the skin color extraction procedure, and an eye color part existing in the area extracted by the constant area extraction procedure And a conversion procedure for randomly converting pixel data of a peripheral portion thereof. 5. The imaging method according to claim 4, further comprising an output restriction procedure for disabling output of the image data before going through the conversion procedure. 6. A recording medium storing a program for causing a computer to realize a face modification function for modifying a face portion in image data of a photographed human figure while photographing a human figure to generate image data. The program includes a skin color extraction procedure for extracting a skin color part in a captured image, a constant area extraction procedure for extracting an area larger than a certain area from the skin color part extracted by the skin color extraction procedure, and the constant area extraction. A computer-readable recording medium for causing a computer to realize a conversion procedure for randomly converting pixel data of an eye color part existing in an area extracted by the procedure and a peripheral part thereof.