JP7242367B2 - Packaging material data calculation device and packaging material data calculation method - Google Patents

Description

The present invention relates to a packaging material data calculation device and a packaging material data calculation method.

When packaging an article, there are cases where a packaging material on which an object such as a logo or message is described is used. In this case, it is desired that the object is arranged in a desired position, for example, in the center of the surface, in the wrapped state. Conventional methods for creating packaging materials that include objects include determining the packaging material by selecting the shape of the item to be packaged or reading the size using a scanner, then printing the folding method on the back side and the object on the front side. has been proposed (Patent Document 1).

JP 2013-65196 A

However, Patent Literature 1 has a problem that the shape of the object to be packaged must be a known shape.

In view of the above problems, the present invention provides a packaging material data calculation apparatus and packaging material data that can generate packaging material data in which objects on the packaging material are arranged at desired positions even if the package has an undefined shape. The purpose is to provide a calculation method.

In order to achieve the above object, an information processing apparatus of the present invention comprises: first acquisition means for acquiring shape data of an object to be packaged; packaging material for packaging the object to be packaged based on the shape data; second acquiring means for acquiring packaging material data; object determining means for determining an object to be placed on the packaging material data; shape data of the package, packaging material data, and object data determining means for determining the position of the object on the packaging material data based on the distance information included in the shape data; characterized by

According to the present invention, it is possible to arrange an object on a packaging material at a desired position even for an article to be packaged that does not have a known shape.

Schematic diagram showing a packaging material data calculation device according to the first embodiment Schematic diagram showing an imaging unit according to the first embodiment A diagram showing a packaging material data calculation flow according to the first embodiment A diagram showing a packaged item and shape data according to the first embodiment. A diagram showing shape data of an object to be packaged and packaging material data according to the first embodiment. A diagram showing packaging material data and an object arrangement surface according to the first embodiment. A diagram showing packaging material data according to the second embodiment A diagram showing a UI example of packaging assistance according to the second embodiment. A diagram showing an example of a configuration that realizes everything from imaging an object to be packaged to printing with a printer.

The present invention will be described in detail with reference to embodiments and drawings. The same number is used when referring to the same thing. The present invention is not limited to the contents described in each example. Further, each embodiment may be combined as appropriate.

(First embodiment)
(Device configuration)
FIG. 1(a) shows an overview of the packaging material data calculation device 100. As shown in FIG. The object to be packaged 102 is photographed by the camera device 101, which is shape data acquisition means. The camera 101 includes an imaging optical system 103 , an image sensor 104 and an arithmetic processing unit 105 .

The imaging optical system 103 has a function of forming an image of a subject on an imaging device 104, which is an imaging surface. The imaging optical system 103 is composed of a plurality of lens groups and diaphragms, and has an exit pupil 130 at a position a predetermined distance away from the imaging device 104 . Reference numeral 140 in FIG. 1A indicates the optical axis of the imaging optical system 103, and the optical axis 140 is parallel to the z-axis in this specification. Furthermore, the x-axis and the y-axis are perpendicular to each other and perpendicular to the optical axis.

A plurality of pixels are arranged in the imaging device 104, and each pixel is composed of a microlens 201, a color filter 202, and photoelectric conversion units 203A and 203B as shown in the cross-sectional view of FIG. 2(a). The imaging element 105 is given RGB (Red, Green, Blue) spectral characteristics according to the wavelength band detected by the color filter 202 for each pixel, and is arranged on the xy plane according to a known coloration pattern (not shown). The substrate 204 is formed with a photoelectric conversion section that is sensitive to the wavelength band to be detected. In addition, each pixel is provided with wiring (not shown).

FIG. 2B is a view of the exit pupil 130 of the imaging optical system 103 viewed from the intersection (center image height) of the optical axis 140 and the image sensor 104 . A first luminous flux that has passed through a first pupil region 210 and a second luminous flux that have passed through a second pupil region 220, which are different regions of the exit pupil 130, are incident on the photoelectric conversion units 203A and 203B, respectively. do. By photoelectrically converting the light beams incident on the photoelectric conversion units 203A and 203B, an A image and a B image (parallax image set) having parallax with each other are generated. The acquired A image and B image are transmitted to the arithmetic processing unit 105 which is arithmetic means, and distance values are calculated by distance measurement arithmetic processing described later and stored in the ROM 111 . Also, an image obtained by adding the A image and the B image can be used as image information. 211 indicates the barycentric position (first barycentric position) of the first pupil region 210 and 221 indicates the barycentric position (second barycentric position) of the second pupil region 220 . A base line length 222 is a center-to-center distance between the center-of-gravity position 211 and the center-of-gravity position 221 . The positions of the A image and the B image are shifted in the baseline length direction due to defocus. A relative positional shift amount (image shift amount) between the images, that is, a parallax amount between the A image and the B image is calculated from a correlation value obtained by correlation calculation. A common calculation method such as SAD (Sum of Absolute Difference) or SSD (Sum of Squared Difference) can be used to calculate the correlation value. The amount of parallax can be calculated by estimating, with sub-pixel precision, the amount of movement that maximizes the correlation between the calculated correlation values using a known interpolation method. The calculated amount of parallax can be converted into a defocus amount or an object distance by a known method. Conversion from the amount of parallax to the amount of defocus can be calculated from the geometric relationship using the baseline length. Also, the conversion from the defocus amount to the subject distance may be performed using the imaging relationship of the imaging optical system 103 . Alternatively, the amount of parallax may be converted to the amount of defocus or the subject distance by multiplying the amount of parallax by a predetermined conversion coefficient. By such a method, distance information (including parallax amount, defocus amount, and subject distance) can be calculated. It is possible to obtain a distance image shown in a format such as . A distance image indicates a data array in which values of distance information corresponding to each area of image data are arranged, and does not necessarily need to be visualized. In this embodiment, a component that constitutes at least part of a distance image is called distance information.

Although two photoelectric conversion units 203A and 203B are arranged in the x direction in FIG. 2A, they may be arranged in the y direction, and may be arranged in any way, such as arranging them in both the x direction and the y direction. . Also, it may have three or more photoelectric conversion units.

Based on the distance image acquired in this way, the length in the in-plane direction perpendicular to the optical axis 140 can also be calculated based on the geometric relationship using the optical parameters such as the focal length of the imaging optical system 103. can be done. In this way, the value of the distance (z direction) corresponding to each pixel constituting the image of the viewpoint from one direction is associated with the value of the in-plane direction (x, y direction) perpendicular to the optical axis 140. This image is called distance information.

The camera device may be composed of a twin-lens stereo camera composed of two or more optical systems and corresponding imaging devices. This is preferable from the viewpoint of improving the degree of freedom in designing the base line length and improving the ranging resolution.

FIG. 1B illustrates details of the arithmetic processing unit 105 . Arithmetic processing unit 105 has system control unit 110 and is connected to image processing unit 107 , user operation unit 108 , display unit 109 , ROM 111 , RAM 112 and recording medium 114 via bus 106 .

In this embodiment, the system control unit 110 controls not only the arithmetic processing unit 105 but also the entire packaging material data calculation apparatus 100, and is, for example, a CPU. The system control unit 110 reads the operation program of each block provided in the arithmetic processing unit 105 from the ROM 111 , develops it in the RAM 112 and executes it, thereby controlling the operation of each block provided in the arithmetic processing unit 105 .

The ROM 111 is a rewritable non-volatile memory, such as a flash ROM. The ROM 111 stores an operation program for each block included in the arithmetic processing unit 105, as well as parameters required for the operation of each block.

A RAM 112 is a rewritable volatile memory, and is used as a temporary storage area for data output during the operation of each block provided in the arithmetic processing unit 105 . The system control unit 110 and the image processing unit 107 use the RAM 112 as work memory.

The image processing unit 107 performs image processing on image data stored in the RAM 112 . Specifically, the image processing unit 107 performs, for example, processes such as white balance adjustment, color interpolation, reduction/enlargement, bandpass filter processing, generation of a defocus map, extraction and correction of a subject area of interest, Perform various image processing such as blurring. The image processing unit 107 records the processed image on the recording medium 108 .

The external I/F 113 is an interface for exchanging signals with devices outside the packaging material data calculation device 100 . The system control unit 110 can output packaging material data to an external device such as a PC or a printer via the external I/F 113, and can upload data such as packaging material data to an Internet server. .

The recording medium 114 is, for example, a memory card or the like that is removable from the arithmetic processing unit 105 . An image processed by the image processing unit 107 stored in the RAM 112, an A/D converted image input from the imaging unit 104, and the like are recorded on the recording medium 114 as recorded images.

A user operation unit 108 is an operation unit that receives instructions from the user 8 . A display unit 110 displays an image captured by the imaging element 102, various operation instructions, a menu screen for displaying a state, icons, and objects on a display such as an LCD. The display unit 109 also has a touch panel function for receiving user operations. For example, by touching the display unit, a subject of interest can be designated from among the subjects displayed on the display unit.

A bus 108 connects each block of the arithmetic processing unit 105 . Each block exchanges signals via the bus 108 .

(calculation flow)
A calculation flow relating to packaging material data calculation performed by the calculation processing unit 105 will be described with reference to FIG.

In step S<b>301 “obtain shape data”, the camera 101 is used to image the object 102 to be packaged, and the shape data of the object 102 is acquired. FIG. 4A is an example of a photographed image of the package 102. FIG. When acquiring such a photographed image, a distance image is acquired by the method described above. Next, the camera 101 is moved to a different point of view to similarly obtain a photographed image of the item to be packaged 102 to obtain a distance image. A plurality of distance images acquired in this manner are integrated to acquire shape information including three-dimensional xyz dimension values. Specifically, it is recorded as a point cloud data format in which the numerical values of the x, y, and z columns are stored side by side for each row. At this time, the coordinate origin of the point cloud data x, y, and z can be arbitrarily set, and for example, the camera coordinates when the first range image was obtained can be set as the origin. Integration of distance information is performed using a known method such as a feature point base. Based on the obtained three-dimensional shape information, shape data shown in FIG. 4B is calculated. The shape data is calculated by fitting a geometric shape based on the dimensional values of the acquired shape information, and it is also preferable to smooth the original shape information by a technique such as smoothing. The shape data is expressed in a general geometric shape data format, and is recorded in a format in which vertex coordinates vx, vy, vz and vertex normal information vnx, vny, vnz are stored for the number of vertices.

As another shape, by fitting a shape that surrounds the outer circumference of the shape information of the packaged object, shape data containing a curved surface as shown in FIG. 4(c) can be obtained. In the case of a circular shape, the above-described data format in which the circumference is divided into vertices by an arbitrary number of divisions, or a format in which the surface type, radius, and normal information of a circle are stored may be used. In the case of a curved surface, the coefficients of polynomial fitting, or the coordinates of each vertex divided into facets and normal line information may be recorded as data.

When acquiring distance images from multiple viewpoints, the photographer may be presented with the next candidate shooting point to which the camera 101 is to be moved. The point of view at that time is selected so that the feature points at the time of integration exist in the photographed image, and more distance information unacquired regions are photographed.

Shape data can be calculated by estimating the shape of the invisible area based on the range image from one viewpoint. Shape data is calculated by inverting and integrating the acquired distance information with respect to a plane perpendicular to the optical axis 140 or using machine learning. Further, shape data is calculated by a UI (User Interface) in which the user draws a shape so as to enclose the package on the acquired distance image.

In addition, the packaging material data calculation apparatus 100 acquires distance information in synchronization with the camera 101 via the external I/F 113 using another distance measurement information acquisition device such as LiDAR. A corresponding range image can also be generated.

In this manner, the camera 101 photographs the object 102 to be packaged and acquires the corresponding shape data, so that the shape data of the object to be packaged can be acquired even if the object to be packaged does not have a known shape. can be done.

In step S302 "packaging material data calculation", the size and shape data of the packaging material are calculated based on the shape data acquired in step S301 "shape data acquisition".

FIG. 5(a) shows the acquired shape data. This is developed as shown in FIG. 5(b) to create a marginal area 501 shown in FIG. 5(c). Packaging material data is calculated in this way. The packaging material data can be represented by two-dimensional data, and is recorded in a data format that stores the coordinates and identification numbers of each vertex, the surface number of each surface, and the identification numbers of the vertices forming the surface. In addition, by selecting wrapping methods such as furoshiki wrapping, caramel wrapping, etc., and applying the dimension values of the shape data acquired in step S301, the size and shape data of the wrapping material corresponding to the wrapping method can be obtained. can be calculated. A plurality of wrapping methods are stored as data in the main body memory 106, and the selection can be made at any time by the user through the UI. In this way, calculation is performed by the calculation processing unit 105 as packaging material data calculation means, and the result is stored in the main body memory 106 .

The step S303 "object determining means" selects an object to be arranged on the packaging material. Objects refer to texts such as messages to be placed on packaging materials, logos such as store names, and data such as illustrations and marks. The user selects the object list data stored in the main body memory 106 and determines it as an object. Also, text data or image data input by the user can be determined as an object. It is also possible to use machine learning to perform scene recognition from the photographed image of the object to be packaged, automatically select an object suitable for the object to be packaged, propose it to the user, and allow the user to select it. For example, when a scene is recognized as a toy, the text "Happy Birthday" can be presented as a candidate. In addition, various decoration processing such as text font and illustration color tone can be performed. In this way, calculation is performed by the calculation processing unit 105 and the result is stored in the ROM 111 .

In step S304 "determination of object position", the position and size of the object selected in step S303 "object determination means" are determined so that the wrapped object is placed on the packaging material at a desired position.

For the surface on which the object is arranged on the packaging material data, for example, the surface of the packaging material data corresponding to the surface having the maximum area in the shape data obtained from the package is selected. FIG. 6A shows the acquired shape data. For a plane 601 having the largest area, a corresponding plane 602 on the packaging material data created from the developed view is selected as a plane for arranging the object.

The center-of-gravity position of the selected surface 602 is calculated, and the object is placed on the surface 602 so that the determined center-of-gravity position of the object matches. Then, the object is scaled to a size that does not protrude from the sides forming the surface 602 . In this way, calculation is performed by the calculation processing unit 105 as a determining means, and the result is stored in the main body memory 106 .

By determining and arranging the position of the object in this way, the object will come to the center of gravity of the surface having the largest area in the outer shape surrounding the packaged item in the packaged state. Therefore, it is possible to arrange the object in a desired position by reducing the visibility of the object being poor due to the boundary between the surfaces in the wrapped state, or the object being hidden inside although it was desired to be printed on the surface. will be able to

Also, the surface on which the object is arranged can be the surface that exists at the closest distance in the first photographed image among the photographed images of the item to be packaged obtained in step S301. Since it is assumed that the user starts the operation while imagining the wrapped state, it is preferable in that the layout surface of the object desired by the user can be automatically proposed.

Also, the surface on which the object is placed can be arbitrarily selected by the user through the UI from the photographed image of the object to be packaged acquired in step S301, and the surface of the packaging material data corresponding to the selected surface can be determined. The size of the object can also be determined by the user by enlarging or reducing it to an arbitrary size. This is advantageous in that the user can select a desired object placement surface and object size.

Also, using the obtained distance image of the package 102, the surface on which the object is arranged can be selected. Specifically, if the amount of change in the distance information (distance value) within the plane, that is, the area in which the differential value in the pixel direction is approximately constant, is determined to be flat and the corresponding plane is selected as the object placement plane. . Similarly, if the area where the second order differential value of the distance value is a constant is greater than or equal to a predetermined range, the corresponding surface is determined to be a curved surface with little unevenness and is selected as the object placement surface. By selecting in this way, the position where the object exists after the packaging is completed can be a surface with less unevenness, so the phenomenon that the object is distorted due to the unevenness of the surface can be reduced, and the appearance of the object in the wrapped state can be improved. It is preferable in that it can be improved.

In step S305 "packaging material data calculation", the object-arranged packaging material data calculated in steps S301 to S304 are converted into various data formats and calculated as packaging material data. The arithmetic processing unit 105 performs arithmetic operations, and the results are stored in the ROM 111 .

By calculating the packaging material data in this way, it becomes possible to place the object on the packaging material at a desired position after the packaging is completed, even for an item to be packaged that does not have a known shape.

(Second embodiment)
This embodiment presents a method of presenting a cutout range so that the packaging material on which the object is originally printed can be wrapped in a desired arrangement, and a method of supporting the packaging procedure.

FIG. 7A shows a packaging material 701 on which objects are originally printed. By photographing this with the camera device 101, it is captured as shape data. An object area is selected from the captured shape data of the packaging material 701, and determined as an object to be determined in step S303 object determination in the flow described above. After that, the packaging material data created by a similar method is superimposed on the shape data of the previous packaging material 701 to calculate the packaging material data shown in FIG. 7(b).

It can also be applied to packaging materials on which objects have already been printed, and is suitable in terms of increasing the options of packaging materials that can be used.

Further, by using the calculated packaging material data and the shape data of the object to be packaged, it is possible to present the packaging procedure to the user and provide packaging assistance. As shown in FIG. 8, the camera 101 photographs the object to be packaged and the packaging material, and compares and analyzes the packaging material data with the captured image. Specifically, the above-described distance information is calculated from the captured image, and the position, orientation, and scale of the shape data are set so that the calculated distance information and the calculated shape data have a high matching degree. The matching degree at this time is calculated by a general method, for example, by calculating a correlation value. After that, as described above, the packaging material data associated with the shape data is superimposed on the photographed image and displayed to the user. In addition, based on the analysis results, the next folding position and direction are calculated, and indicated by an icon such as an arrow 801 on the rear liquid crystal screen of the display unit 109, thereby assisting the user in wrapping. The object to be packaged 102 and the packaging material 701 may be captured by the camera device 101 before or during packaging. It is also possible to support packaging by obtaining and displaying captured images in the moving image mode, and by updating the display position and contents of the arrow as needed as the packaging procedure progresses. This is preferable in that the user can place the wrapped object on the packaging material at a desired position while performing the actual work.

(output to printer)
FIG. 9 shows a packaging material production apparatus 910 that prints the packaging material data calculated by the packaging material data calculation apparatus 100 with a printer 901, which is common to the two embodiments described above. The packaging material data calculation apparatus 100 and the printer 901 are connected by wire or wirelessly via the external I/F 113 . The packaging material to be printed on is appropriately selected from among paper, plastic, and cloth, and the printer 901 is connected accordingly. This is preferable in that a desired packaging material can be realized using the calculated packaging material data as an actual product.

(program)
The present invention also includes a computer program in addition to the packaging material data calculation device. A computer program according to the present embodiment causes a computer to execute predetermined steps in order to calculate shape information. The program of this embodiment is installed in the computer of the shape information acquisition device or an imaging device such as a digital camera equipped with it. By executing the installed program by the computer, the above functions are realized, and desired packaging material data can be calculated.

REFERENCE SIGNS LIST 100 packaging material data calculation device 101 camera 102 package object 103 imaging optical system 104 imaging device

Claims (12)

a first acquisition means for acquiring shape data of an object to be packaged;
a second acquisition means for acquiring packaging material data related to a packaging material for packaging the object to be packaged based on the shape data;
an object determining means for determining an object to be arranged on the packaging material data;
determining means for determining the position of the object on the packaging material data based on the shape data of the object to be packaged, the packaging material data, and the data of the object ;
The information processing apparatus, wherein the object determination means determines a position where the object is arranged based on distance information included in the shape data . 2. The information processing apparatus according to claim 1 , wherein said first acquiring means acquires a plurality of sets of parallax images captured by said imaging means at different positions. 3. An information processing apparatus according to claim 2, wherein said imaging means is means for acquiring, for each position, a set of parallax images made up of light beams that have passed through different pupil regions of the imaging optical system. 3. An information processing apparatus according to claim 2, wherein said imaging means is imaging means comprising a combination of two different optical systems and imaging elements. 5. The information processing according to any one of claims 1 to 4, wherein said object determining means determines a position where an object is to be arranged on a plane having the largest area among planes included in said shape data. Device. 6. The apparatus according to any one of claims 1 to 5, wherein said object determination means determines a position where said object is arranged based on an amount of change in an in-plane direction of distance information included in said shape data. The information processing device described. a first acquiring means for acquiring shape data of an object to be packaged generated based on a plurality of sets of parallax images captured at different positions by imaging means for receiving light beams that have passed through different pupil regions of an imaging optical system;
a second acquisition means for acquiring packaging material data related to a packaging material for packaging the object to be packaged based on the shape data;
an object determining means for determining an object to be arranged on the packaging material data;
and determining means for determining a position of the object on the packaging material data based on the shape data of the object to be packaged, the packaging material data, and the data of the object. A presenting means is provided for presenting a packaging procedure to a user by using the packaging material data calculated by the information processing apparatus according to any one of claims 1 to 7 and the photographed image of the object to be packaged. Packaging support equipment. A packaging material production apparatus for producing packaging materials using the packaging material data generated by the information processing apparatus according to any one of claims 1 to 7. a first acquisition step of acquiring shape data of the object to be packaged;
a second acquiring step of acquiring packaging material data related to a packaging material for packaging the object to be packaged based on the shape data;
an object determination step of determining an object to be placed on the packaging material data; and determining the position of the object on the packaging material data based on the shape data of the package to be packaged, the packaging material data, and the data of the object. a decision step;
with
The information processing method , wherein in the object determination step, a position where the object is arranged is determined based on distance information included in the shape data. a first acquiring step of acquiring shape data of the object to be packaged generated based on a plurality of sets of parallax images captured at different positions by imaging means for receiving light beams that have passed through different pupil regions of the imaging optical system;
a second acquiring step of acquiring packaging material data related to a packaging material for packaging the object to be packaged based on the shape data;
an object determination step of determining an object to be placed on the packaging material data; and determining the position of the object on the packaging material data based on the shape data of the package to be packaged, the packaging material data, and the data of the object. a decision step;
An information processing method comprising: A program that causes a computer to execute each step of the information processing method according to claim 10 or 11 .

Images