JP2008152810A - Customer information collection and management system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To allow personal features of customers to be accurately decided as personal information and to provide an immediate sales promotion service suitable for a customer on the basis of the personal information. <P>SOLUTION: An image processing part 51 extracts sex, ages, features such as occupations, stay places, stay periods, interesting noticed commodities shown by directions of faces, customers trail courses, etc. of individual customers from a plurality of screens of commodity-purchasing customers, which are consecutively photographed by a plurality of cameras 11 to 18, and specifies them and registers them in a visiting customer list 52. A customer management part 53 takes out feature data from the visiting customer list 52 and takes out preheld voice guide information/electronic bulletin information corresponding to the feature data from a notification information DB 70 and announces/electronically posts the taken-out information from an output part 40, and data of leaving customers are recorded in a customer DB 60. Trail courses of customers recorded in the customer DB 60 can be effectively used for improvement of commodity exhibition. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a customer information collection management method and system for acquiring customer image information with a camera and utilizing it for marketing, and more particularly to a customer characteristic based on personal information that can be more accurately determined as customer personal characteristics. The present invention relates to a customer information collection and management system that can provide a sales promotion service corresponding to the demographic without losing time.

  Conventionally, this type of customer information collection management system reduces the input work and prevents variations in judgments about customer attributes, so that the system can create an accurate database for each camera at each doorway. Based on the photographed customer image, the customer attribute is automatically determined by image processing (see, for example, Patent Document 1).

  This customer information collection management system will be described with reference to FIG.

  In each in-store information analysis system 100, the control unit 108 determines gender, age group, and the like as attributes of the customer from the image information of the cashier camera 102 that images the purchase customer at the register. Next, the store clerk accurately inputs the data of the product purchased by the customer into the register using, for example, a barcode. As a result, product data and customer attributes are recorded in association with each other.

  That is, since the attribute of the customer is determined based on the image information of the product purchase customer, not only the input work by the store clerk is reduced, but also the variation in judgment from the uniform reference from the image information is prevented. In addition, these data at each store are sent from the in-store information analysis system 100, are centrally stored in the information analysis center 110, and are used for marketing.

  In addition, there is a customer data processing system that acquires and analyzes customer data at a terminal device to determine the type of customer, specifies product guidance data based on the determination result, and provides the customer with the product data (for example, , See Patent Document 2).

  This terminal device is equipped with a camera and a sensor, and when the customer stands in front, customer information is automatically acquired by the camera and sensor, and the customer type is determined from the acquired information, and guidance data suitable for the customer segment is obtained. Provide to its customers. The guidance data is centrally held in the server device corresponding to the customer demographic data and provided to a plurality of terminal devices. Here, the customer residence time is also measured.

JP 2002-32553 A (FIG. 1) JP-A-11-328266 (FIGS. 1 and 2)

  However, the conventional customer information collection and management system described above has the following problems.

  The first problem is the lack of accuracy in determining customer attributes such as age, gender, and clothing.

  The reason is that shooting data for acquiring customer attribute information is not shot continuously for a plurality of frames, and is determined only by a single frame. In the above-mentioned patent document 1, image information only when purchasing a product at a register, and in the above-mentioned patent document 2 image information of a camera when facing a terminal device, a customer is determined from only the image from the front. The amount of data for is small, and it is dangerous to accurately determine the attributes of a person such as age, gender, and clothes.

  The second problem is that it is difficult to respond flexibly to each store when promoting product sales based on the personal information of customers visiting the store.

  The reason is that in Patent Documents 1 and 2, the information analysis center or server device manages customer data and sales promotion product data in a concentrated manner at a plurality of stores, and it is difficult to respond quickly to changes at each store. is there.

  The third problem is that it is difficult to reflect the purchase information of the customers who visit the store in improving the product display in the store.

  The reason is that in the above-mentioned Patent Document 1, what corresponds to the customer information is purchased product data, and there is no data other than the purchased product. This is because the above-described Patent Document 2 proposes to provide a terminal device for each product for information collection.

  The object of the present invention is to solve such problems, to obtain more accurate personal information from image information at each product provision location, and to quickly lose time to changes in the situation at each provision location. It is an object of the present invention to provide a customer information collection management method and system capable of providing corresponding product data and collecting information for improving product display position.

  The customer information collection and management system according to the present invention acquires customer image information with a camera for use in marketing, discriminates the customer from the image information with an image processing unit, and collects and manages the customer list. A plurality of first cameras deployed at least near the doorway and taking a picture of the front of the customer and outputting the person image; and images taken at least toward the passage and including the customer's face orientation to output the person image A plurality of second cameras and a plurality of images continuously acquired by the first and second cameras are received, and a personal attribute including the age and sex of the customer tracked from the image information is determined to determine the personal information of the customer The image processing unit that generates and outputs the data, the customer list in the store that receives and records the personal information generated by the image processing unit, and the information recorded in the customer list in the store is repeatedly scanned Characterized in that it comprises a customer management unit for generating general conditions including store status of a specific time period as a feature of the information obtained separately from the information take-out the personal information Te.

  For this purpose, images of all target customers are periodically acquired by the camera, the individual attributes of the customers are determined from the images of a plurality of frames acquired continuously, and the individual is determined. Each product group of interest is determined from the camera position, face orientation, and staying time, and utilized for marketing.

  With this configuration, a plurality of pieces of continuous image information can be acquired for one customer, so that personal information as customer attributes can be collected more accurately. Further, it is possible to identify the product or group of products of interest from the camera position and the face orientation analyzed from the image information, and to analyze the stay time and the duration of attention.

  In addition, when the customer's personal attributes and attention product group are acquired, sales promotion information of a product prepared in advance corresponding to the attention product group is audibly / visible and immediately notified in the store. Therefore, highly efficient marketing automation is possible.

  A notification information database that stores in advance an audible / visible notification device such as a speaker, an electronic bulletin board, and / or an information terminal, and a guidance notification and a notification condition including a sales promotion product corresponding to the notification device in the product display place. Are further provided. This image processing unit selects the product of interest from data acquired by the camera in the passage such as the person position and face direction of the consecutive human images and the residence time based on the same personal information, and the information is stored in the store Record in the customer list corresponding to the personal information. Further, when the information recorded in the in-store customer list matches the notification condition, the customer management unit acquires the guidance notification corresponding to the notification condition from the notification information database and corresponds to the notification device. Provide audible / visible information.

  Since this notification information can be provided from a database in the same store, provision conditions and guidance for provided products can be appropriately created and provided promptly without losing time. In other words, the willingness to purchase can be increased.

  In addition, by determining a customer's attention product group, its attention time, and a movement route from continuously acquired images, it can be used for marketing improvement such as improvement of the product display position in the store.

  That is, there are provided a register for registering purchased products of customers visiting the store and a customer database for recording information on the customers visited so far. This image processing unit compares the purchased customer's camera image with the registered customer's camera image when the registered product is registered in the register, and registers it in the in-store customer list. Record in the customer list during the visit. Further, the customer management unit additionally records in the customer database predetermined information of the customer correspondence information in which the store is recorded in the in-store customer list.

  Next, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a plan view showing an embodiment according to the arrangement of in-store cameras and the face orientation of a person in the present invention. The difference from the conventional example shown in FIG. 7 is that a plurality of cameras 11 to 18 are provided for acquiring imaging screens of customers 31 to 35 on the aisle in the store.

  In the plan view shown in FIG. 1, an entrance camera 11, a cash register camera 12, an exit camera 14, and in-store cameras 15 to 18 are provided, for example, on a ceiling, and each camera regularly and continuously captures images. . The store is, for example, a so-called convenience store. In the store, there is a POS (sales point information management) terminal 20 that serves as a register, there are customers 31 to 36 who are visiting the store, a speaker 41, an electronic bulletin board 42, and an information terminal 43 other than the product display shelf. Yes.

  The entrance camera 11 faces the entrance from the inside of the store and photographs the front of the customer entering the store. The cash register camera 12 photographs almost the front of the customer who presents the purchased product at the installation location of the POS terminal 20 serving as a register. The exit camera 14 faces the inside of the store from the entrance and exit side, and captures almost the front of the customer leaving the store.

  The in-store cameras 15 to 18 photograph customers on the aisle. The in-store camera 15 in FIG. 1 is intended for customers between a bookshelf and a daily goods display shelf. The in-store camera 16 is intended for customers between food and confectionery display shelves. The in-store camera 17 targets breads, lunch boxes, desserts, and customers facing the information equipment 43. The in-store camera 18 targets drinks and customers in front of the electronic bulletin board.

  In FIG. 1, a customer 31 faces his book shelf. The customer 32 faces the electronic bulletin board 42. For example, when the face direction is fixed in the direction of the electronic bulletin board 42, it can be determined that the electronic bulletin board 42 is attracting attention. Customers 33 and 34 are paying attention to food and lunch boxes, respectively. The customer 35 accesses the information terminal 43, and the customer 36 pays out the purchased goods at the register.

  Next, a customer information collection management system according to the present invention will be described with reference to FIG.

  This customer information collection and management system includes an input unit 10, a POS terminal 20, an output unit 40, a control unit 50, a customer DB (database) 60, and a notification information DB 70 in the store. The customer DB 60 may be managed in common or centrally at a plurality of stores.

  The information input unit 10 includes an entrance camera 11, a cash register camera 12, an exit camera 14, and in-store cameras 15-18. The illustrated POS terminal 20 includes a keyboard 21, a screen display 22, and a purchased product information acquisition unit 23. In the information output unit 40, a speaker 41, an electronic bulletin board 42, and an information terminal 43 are shown. The control unit 50 includes an image processing unit 51, an in-store customer list 52, and a customer management unit 53. The notification information DB 70 includes voice guidance product information 71 and electronic criminal product information 72.

  Although the cameras 11 to 18 have already been described with reference to FIG. 1, for example, image information captured continuously several frames per second is sequentially sent to the image processing unit 51 of the control unit 50. In the POS terminal 20, for information input, there is a bar code reader (not shown), for example, in addition to the keyboard 21. The information output includes, for example, a receipt output (not shown) other than the screen display 22 that displays various information on the screen. Accordingly, a purchase product information acquisition unit 23 that records and holds customer purchase product information is provided.

  The speaker 41 receives the product information stored in the voice guidance product information 71 under the control of the customer management unit 53 and broadcasts it into the store. The electronic bulletin board 42 receives product information stored in the electronic bulletin product information 72 under the control of the customer management unit 53 and displays the screen. Although the illustrated electronic bulletin board 42 faces the entrance / exit direction, a plurality of electronic bulletin boards 42 may be provided and may face the outside of the store depending on the posted information. The information terminal 43 can receive the operation of the customer and provide not only the products stored in the notification information DB 70 but also other information to the customer in a visible / audible manner.

  The image processing unit 51 performs image processing for each frame of image information obtained from the input unit 10 and extracts personal features such as sex, age, and clothes for customer tracking. By image processing, it is possible to identify a person and identify the product information of interest obtained from the camera position, face orientation, and staying time. Furthermore, the movement of one customer in the store in association with the passage of time The route can be traced. Therefore, the personal information extracted and given the customer ID (identifier) is sent to the customer list 52 during the visit and is arranged by customer ID. Further, when it is determined from the image information that the customer has left the store, the customer's exit is sent to the in-store customer list 52.

  The customer management unit 53 repeatedly scans customer information recorded in the in-store customer list 52. When the customer information matches a preset condition, the voice guidance product information 71 stored in the notification information DB 70 and / or Appropriate notice information is extracted from the electronic bulletin product information 72 and sent to the output unit 40. When a customer leaves the store, the corresponding customer data is collated with the customer DB 60 and new customer information is transferred to the customer DB 60 for addition, while the same information is discarded. In particular, when a change in information in the customer DB 60 is significant, this can be detected and reflected in sales promotion.

  In addition to the characteristics of each customer, the customer DB 60 also records product information of interest such as purchased products, travel routes, and staying time for the customer segment. Therefore, the customer DB 60 is useful for selecting sales promotion products corresponding to gender, age group, occupation, etc., and improving product display in the store.

  The notification information DB 70 is information on recommended products corresponding to the customer group created for sales promotion based on the information so far, voice information is voice guidance product information 71, video information is electronic bulletin product information 72, Each of them is stored in advance together with the customer group conditions for sales promotion, for example, sex, age group, occupation and the like. This information is periodically revised based on information stored in the customer DB 60.

  Next, the function of the image processing unit 51 will be described in detail with reference to FIGS.

  In the example shown in the figure, the image processing unit 51 has functions corresponding to camera-accepted image receptions 511, 512,. For example, the image reception 511 sequentially receives image data captured by the entrance camera 11, adds the camera number and time to these image data, and sends them to the ID editing 510.

  The ID editing 510 performs image processing on the received images of a plurality of frames for each frame, and extracts personal characteristics such as gender, age, and clothes as personal information. Specifically, for each image frame, a determination probability such as 90% probability of male and 30% probability of being in 30's is calculated, and the person feature is determined from the high determination probability. On the other hand, in each of the human images, since the place of interest can be determined from the camera position and the face orientation at the same time, a product group or a product that is interested or is a purchase target is specified.

  Next, similar image information is gathered in time series, and as shown in FIG. 4, more accurate feature extraction is realized by comprehensively judging information of multiple image frames 81 to 86 for one customer. Then, the personal information is comprehensively determined and the customer ID is given. As described above, for the human feature, a method of adopting a more reliable determination result from the determination results of a large number of image frames by tracking a plurality of cameras is employed.

  Specifically, in the comprehensive determination, only an image frame having a determination probability equal to or higher than a certain threshold is adopted. For example, when the threshold value of the determination probability is 80%, two image frames, male: 93% image frame 81 and male: 81% image frame 85, are adopted and determined as male. Thus, accurate human feature extraction can be performed by the method of eliminating ambiguous determination.

  As a result, even when a small number of low-reliability determination results are mixed in with a large number of high-reliability determination results, more accurate person feature extraction can be performed by removing the low-reliability determination results. In addition, even if there are a small number of highly reliable judgment results in the majority of judgment results with low reliability, it is possible to adopt a small number of highly reliable judgment results as well, so that accurate human feature extraction is still possible. Is possible.

  The customer tracking information and feature information obtained by the image processing unit 51 are sent to the in-store customer list 52 and additionally recorded. That is, for example, for each ID, personal information indicating customer characteristics, store entry time, stay location and stay time, etc. are recorded, and when exit from the exit is detected, store exit time can be recorded. As described above, the customer information in which the store closing time is recorded is retrieved by the customer management unit 53 from the visiting customer list 52.

  Next, with reference to FIGS. 5A and 5B, the details of the in-store customer list 52 will be described.

  The contents of the in-store customer list 52A shown in FIG. 5A include attributes such as an ID for identifying a customer, sex, age, clothes, status obtained from the face direction, and staying time. Updated in real time by the processing unit 51. Furthermore, in FIG. 5B, a gait that sequentially records places where the staying time exists is acquired as data of the moving course.

  Next, an operation procedure of the control unit 50 will be described with reference to FIGS. 1 to 5 in FIG. FIG. 6 is a flowchart showing an embodiment of a procedure centering on the main operation of the customer management unit 53.

  In the control unit 50, the image processing unit 51 knows that there is a customer coming to the store (YES in step S1) from the image data received from the entrance camera 11, and acquires the customer's personal data (step S2) and assigns an ID code. And recorded in the in-store customer list 52. The customers who are in the store are photographed by any of the cameras 11 to 18 in the store, the images are processed, and the processing results are sequentially recorded in the in-store customer list 52.

  The customer management unit 53 repeatedly scans the in-store customer list 52 and investigates the characteristics of the acquired data (step S3). As a feature, not only individuals but also general conditions such as many male customers wearing suits at lunch time purchasing lunch boxes and drinks, or many young female customers visiting the store, etc. is there.

  If step S3 is “YES” and the acquired data from the in-store customer list 52 is characteristic, the customer management unit 53 compares it with the registered information in the notification information DB 70, for example, recommending it to male customers at lunch time. The presence or absence of corresponding PR (advertisement, promotion) products such as today's products or cosmetics recommended for many young female customers is confirmed (step S4).

  When the procedure S4 is “YES” and there is a corresponding PR product, the customer management unit 53 retrieves the corresponding data from the notification information DB 70 and passes it to the output unit 40, and the PR product is announced by voice or / and electronic posting. An image is notified (step S5). For example, in the case of the voice guidance product information 71, voice guidance by announcement is played from the speaker 41. Further, in the case of the electronic bulletin product information 72, a notice is given to the visiting customer by a screen display on the electronic bulletin board 42.

  In addition, for example, if there are many young student customers in the late 10's to early 20's browsing at a magazine corner at midnight, a specific PR service is recommended, such as recommending a specific evening meal Since the service for the customer group can be performed automatically and in real time, it becomes possible to increase the customer's willingness to purchase.

  On the other hand, when the image processing unit 51 identifies a customer who has left the store from the image data received from the exit camera 14 (YES in step S6), the store is recorded at the corresponding location in the on-store customer list 52, for example, according to time. Is done. The customer management unit 53 compares the personal characteristic data of customers who have a record of closing in the in-store customer list 52 with those stored in the customer DB 60. When it is determined that the exit customer is a new customer (YES in step S7), the customer management unit 53 assigns a new ID number and registers the customer's personal data in the customer DB 60 (step S8). Predetermined data such as the product that interests the user and the staying time at the place is additionally registered as customer data, and the corresponding data is deleted from the in-store customer list 52 (step S9).

  If the procedure S7 is “NO” and an individual customer has already been registered, the customer management unit 53 skips the procedure S8 and proceeds to the procedure S9, and the number of visits to the store, purchased product information, products of interest that are different from conventional ones, etc. The predetermined customer data is additionally registered in the corresponding location, and the corresponding data is deleted from the in-store customer list 52.

  Next, the customer management unit 53 confirms the presence of the in-store customer in the in-store customer list 52 (step S10). Since there is no customer record in the visiting customer list 52, the procedure ends when this step S10 is “NO”.

  If the procedure S10 is “YES” and there are still customers in the store, the procedure S3 is “NO”, and there is no acquired data characteristic of the customer list 52, or the procedure S4 is “NO”. Similarly, if there is no corresponding PR product, the process returns to step S2, and the above-described procedure is continuously repeated from the operation of collecting customer data of customers who are visiting the store.

  In the above description, the ID of the customer data during the store visit and the ID of the customer DB are assigned independently. However, when the image processing unit assigns the ID to the customer who visited the store, the ID corresponding to the personal characteristic data is assigned to the customer DB. It is good also as taking out from and giving.

  In the above description, the imaging data of all customers in the store is obtained using a plurality of cameras. However, a method in which an omnidirectional camera is installed on the central ceiling in the store and the entire store is tracked with one camera. But you can.

  In addition, regarding the processing of a plurality of image frames with reference to FIG. 4 described above, only those having a threshold value or more are adopted, but a method of taking an average of all the image frames 81 to 85 is also conceivable. Specifically, it is a method of determining a male by calculating the probability of being 72 (= (93 + 67 + 45 + 74 + 81 + 74) / 6) percent and a probability of being 28 (= (7 + 33 + 55 + 26 + 19 + 26) / 6) percent. This method is more reliable than the determination obtained from only one image because all information taken by the customer can be captured. Further, if the facial expression can be recognized by the image processing technique, it can be added to the information on the degree of interest in the product.

  Further, although the image processing technique has been described for acquiring customer data, it is possible to further improve the recognition performance by combining voice. In this case, for example, it is possible to add a method of installing a microphone in a store and analyzing a man and woman from utterances or analyzing the reputation of a product using speech recognition technology from conversation contents.

  Although the above description is intended for stores, tracking users and extracting features with vending machines or ATMs (automatic cash-in / out devices), executing specific services for users, or using minors It can also be used to limit

  In other words, with the above-described configuration, it is possible to automate highly efficient marketing, increase customer's willingness to purchase, and improve the product display arrangement in the store.

  As described above, the customer information management in the store has been described with reference to the function blocks and procedures shown in the figure. However, changes such as distribution by function separation / consolidation or replacement before and after the procedure are free as long as the above functions are satisfied. The target is not limited to the merchandise display shelf. That is, the above description is not limited to the present invention, and this function can be applied to general customer information management using equipment that can be photographed by a camera.

As described above, according to the present invention, the image processing unit identifies a person from the person image by acquiring the person images continuously photographed by arranging cameras facing in the other directions at a plurality of locations. Therefore, it is possible to acquire accompanying situation information from the video around the person, so that it can be widely applied to the purpose of acquiring the person entering the shooting screen and the background thereof from the camera and determining the situation.

It is a top view which shows one Embodiment which concerns on arrangement | positioning of the camera in a store in this invention, and a person's face direction. It is a functional block diagram which shows one Embodiment in this invention. It is a functional block diagram which shows one Embodiment in the image process part of FIG. It is explanatory drawing which shows one Embodiment of the multiple image frame collected for the personal information comprehensive judgment in ID edit of FIG. It is a functional block diagram which shows one Embodiment in the customer list in the store of FIG. It is a functional block diagram which shows one Embodiment in the customer list in a store visit different from FIG. 5A. It is a flowchart which shows one Embodiment of the main operation | movement procedure in the control part of FIG. It is a functional block diagram which shows an example of the past.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Input part 11 Entrance camera 12 Cash register camera 14 Exit camera 15-18 In-store camera 20 POS terminal 31-36 Customer 40 Output part 41 Speaker 42 Electronic bulletin board 43 Information terminal 50 Control part 51 Image processing part 52, 52A, 52B Visiting customer List 53 Customer Management Department 60 Customer DB
70 Notification Information DB
71 Voice guidance product information 72 Electronic bulletin product information 81-86 Image frame 510 ID edit 511, 512 Image reception

Claims (3)

In a customer information collection and management system that acquires customer image information with a camera and uses the image processing unit to identify customers from the image information and collect and manage them in a customer list for use in marketing.
A plurality of first cameras arranged at least at the doorway to capture the front of the customer and output the person image;
A plurality of second cameras that are arranged at least toward the aisle and that capture an image including the customer's face orientation and output the person image;
An image processing unit that receives a plurality of images continuously acquired by the first and second cameras, determines personal attributes including the age and sex of the customer tracked from the image information, and generates and outputs the personal information of the customer When,
In-store customer list that receives and records the personal information generated by the image processing unit, and repeatedly scans and extracts the information recorded in the in-store customer list, and obtains information obtained from the information separately from the personal information. A customer information collection and management system comprising: a customer management unit that generates a general condition including the in-store situation of a specific time zone as a feature. 2. The customer information collection and management system according to claim 1, wherein at least one notification device of a speaker, an electronic bulletin board, and an information terminal, and guidance notification information including sales promotion products and notification conditions are provided at the product provision location. Further provided with a notification information database stored in advance corresponding to the device,
The image processing unit selects a product of interest from at least face orientation data acquired by the second camera among consecutive person images based on the same personal information, and stores the information in the in-store customer list Record corresponding information, and
When the information recorded in the in-store customer list matches the notification condition, the customer management unit obtains the guidance notification information corresponding to the notification condition from the notification information database and provides it to the notification device Customer information collection management system characterized by that. The customer information collection and management system according to claim 1, further comprising: a register for registering purchased products of customers visiting the store; and a customer database for recording information on the customers so far.
When the purchased product is registered in the register, the image processing unit compares the purchased customer's camera image with the registered customer list and registers it in the visiting customer list, and when the exit camera image captures the customer, the store exits. To the customer list during the visit, and
The customer management unit additionally records in the customer database predetermined information of customer correspondence information recorded as having been closed in the in-store customer list.

Images