FR2910674A1 - Building's frontage structured image analyzing method, involves merging vertical or horizontal basic histogram with vertical or horizontal contour histogram, in vertical window histogram or horizontal window histogram - Google Patents

Abstract

The method involves calculating a histogram representing a contour image such as vertical or horizontal contour histogram, along a vertical or horizontal direction. An appurtenance image is calculated at a bottom from a structured image, and a histogram i.e. vertical basic histogram or horizontal basic histogram, representing the image is calculated along a vertical or horizontal direction. The vertical or horizontal basic histogram is merged respectively with vertical or horizontal contour histogram, in a histogram such as vertical window histogram or horizontal window histogram. Independent claims are also included for the following: (1) a method for establishing a structured image (2) a structured image establishing system (3) a signal representing a visible data transports a structured image model obtained by analysis of the structured image (4) a computer program comprising instructions for implementing a method for establishing a structured image.

Description

1 Method for analyzing a structured image, method of modeling a

  The present invention generally relates to the fields of image analysis and rendering, and more specifically to the visualization of two or three dimensional urban scenes made up of a large number of images. buildings.  One of the applications of the invention is in fact the analysis of real images of building facades in order to model them and then allow an approximate but rapid reconstruction of these images during a navigation in a virtual environment.  Currently, existing geographic information systems are able to model buildings over a large area and offer fluid navigation interfaces that make it easy to understand an urban environment in two or three dimensions.  In order to make the urban environment realistic, for example to virtually visit Paris on a terminal, it is necessary that the textures displayed on each building are similar to real facades and correct resolutions.  However, the data of such environments are generally stored on a remote server, so it is not possible to send the actual images of the facades of each building to the terminal as and when the visit, because the amount of information to to transmit would be too important.  To overcome this problem, current systems propose to display generic texture images by repeating them several times within the virtual urban environment.  This allows a small amount of information to be sent, but results in a significant decrease in overall navigation quality because the textures displayed are not visually close to the actual buildings.  In order to more granularly reconstruct building facades, modeling languages for defining facade elements, such as windows or balconies, regardless of background texture, have been proposed in various works.  Thus the article "Modeling urban scenes for LBMS" of A.  F.  Coelho, A.  AT.  of Sousa and F.  NOT.  Ferreira, published in 2005 on the occasion of the tenth international conference "3D Web Technology", proposes an XML grammar, according to the English "eXtensible 5 Markup Language", which allows to describe completely large urban environments in three dimensions , and from these descriptions to rebuild buildings.  The cities are thus modeled with many details.  Similarly, to provide both a realistic urban environment navigation, good resolution and an acceptable amount of transferable data, it has been proposed within the "Motion Picture Expert Group 4" (MPEG4) standard, in its part "Animation Framework eXtension", to characterize the facade of a building, not by a single texture image, but by "procedural models of facades".  These contain facade reconstruction information from a number of facade element models, such as windows, doors, balconies, and background textures, which are stored in a window. a catalog.  These models make it possible to recreate facade images visually close to the original facades, of any size, by transmitting a small amount of information to the terminal.  These visualization techniques of scenes are however limited because they require the intervention of an operator to manually model the buildings and the facades, which is very expensive when the environment to be modeled is of urban dimension in particular.  As a result, facade image analysis to automatically fill procedural models of facades has become an important issue in the field of image analysis.  However, there are few methods to automatically analyze such images.  Given the diversity of facade architectures, types of exterior cladding and types of facade elements, one of the main difficulties to overcome is in particular to isolate the facade elements in a façade image.  K. Schindler and J. Bauer, in his article "A model-based method for building reconstruction", published in 2003 at a "Higher-Level 5 Knowledge in 3D Modeling and Motion" workshop of a conference "International Conference on Computer Vision ( ICCV) ", presents a facade analysis method, in which the windows of a facade to be analyzed are detected by a scanning algorithm based on the non-incident point density of the façade plane.  This method thus detects the windows as being objects behind the plane of the facade, but requires several starting images of the facade, and their combination according to the parameters of the taking of images, which is complex and impractical to implement. artwork.  Similarly the article by H.  Mayer and S.  Reznik, entitled "Building Facade Interpretation from Image Sequences", and published in the archives of the International Society for Photogrammetry and Remote Sensing (ISPRS), uses a method based on that of K. Schindler and J. Bauer to insulate the windows of a facade before positioning them and to detect them more finely on the plane of the facade.  This method therefore has the same disadvantages as the previous method, in terms of complexity and implementation.  A different method of facade image analysis is developed in the article "Extraction and Integration of Window in a 3D Building Model from Ground View Image", by S.  C.  Lee and R.  Nevatia, published at the 2004 ICCV conference.  This method makes it possible to isolate facade elements of a facade from a single photographic image taken from the street.  This method analyzes histograms of vertical and horizontal contours of the facade image, to cut the image into cells each containing a facade element.  However the analysis of these histograms does not allow to cut without error and precisely the facade images in 30 cells.  Indeed, some irregularities of the facades, such as decorative edges or decorative lines, give highly chiseled histograms and very difficult to analyze.  The histograms used are therefore not very robust to the variations existing in the images of facades.  It should be noted that these image analysis techniques are applied to facade images, but they could be more generally applied to the detection of unit elements in structured images, i.e. images having two directions in which are arranged elements distinguished from a background or background.  Indeed these techniques exploit the fact that in a facade image, the facade elements are aligned vertically and horizontally with respect to the axes of the facade on a background image.  Similarly, if one of the applications of the invention is the analysis of images of building facades, to extract elements such as windows, doors and balconies to build the procedural models of associated facades, the invention is also applicable to any structured image containing elements aligned along two axes.  It is an object of the present invention to overcome the disadvantages of the prior art by providing a structured image analysis method and apparatus, as well as a facade image modeling method and system.  To this end, the invention proposes a method for analyzing a structured image containing unitary elements to be detected, aligned in two horizontal and vertical directions, comprising the steps of: Detecting contours and obtaining an outline image , Calculation of a histogram in a vertical or horizontal direction, representative of said contour image, called vertical contour histogram or histogram of horizontal contours, said method being characterized in that it comprises the further steps of: Computation of a background image from said structured image, 2910674 5 Calculation of a histogram in a vertical or horizontal direction, representative of said background image, respectively called vertical background histogram or histogram of horizontal background, 5 Fusion of said vertical or horizontal background histogram with said contour histogram respectively vertical or histogram of horizontal contours, in a histogram respectively called vertical window histogram or horizontal window histogram.  Thanks to the invention, one automatically analyzes a structured image in a robust and efficient manner.  Indeed, the invention merges several types of highly correlated histograms, those obtained from a contour image, and those obtained from a background image, to obtain so-called "window" histograms. are easy to analyze.  In other words, the joint use of the background color information of the image and the contour information makes the image analysis more robust.  These histograms then make it possible to cut a structured image into cells, each of which contains an element to be detected, in a fast, precise and efficient manner with respect to the prior art.  This cell division is a preliminary step to the finer detection of the isolated elements in each cell, then to the "procedural" modeling of the structured image when it is a facade image.  In addition to performing the analysis of a facade, the invention requires a single photographic image thereof.  According to a preferred characteristic, in the melting step the vertical or horizontal background histogram is respectively multiplied by the histogram of vertical or horizontal contours.  The fusion of a contour histogram with a background histogram is feasible in several ways, for example by linear combination or by multiplication.  The best results in terms of efficiency and robustness were obtained by multiplying these two types of histograms.  According to a preferred feature, the method of analyzing a structured image according to the invention further comprises a step of detecting a vertical or horizontal color change, making it possible respectively to separate the vertical window histogram or the histogram of horizontal window in two sub-histograms each representative of a color of said structured image.  This additional step, performed prior to the analysis of the window histograms for cell division, makes it possible to split a window histogram at the locations where there are disturbances due to color changes in the image.  Sub-histograms are obtained in which these disturbances are not troublesome when these sub-histograms are analyzed separately.  Since the ground floors of the buildings often have a different façade color than the upper floors, this step is useful for separating a façade image to be analyzed into two sub-images to be analyzed separately, one containing the floor floor and the other the upper floors.  According to a preferred feature, the vertical or horizontal contour histograms are obtained after respectively horizontal or vertical filtering of said contour image.  This additional filtering step makes the histograms of vertical or horizontal contours more characteristic of the unitary elements to be detected, when these unitary elements are composed of vertical or horizontal lines, for example windows.  According to a preferred feature, the vertical color change detection step comprises the steps of: - Hough transforming of an image of the luminances of said structured image, 2910674 7 Calculation of a histogram in a vertical direction, representative of the resulting image of said transformation, called vertical Hough histogram, Calculation of a histogram called ground floor histogram by linear combination of said vertical background histogram, said vertical Hough histogram, said vertical contour histogram, and a histogram in a vertical direction representative of said vertically filtered contour image.  The ground floor histogram obtained by these substeps has a peak corresponding to the location of the color change in the structured image, which makes it possible to detect this change.  The use of such a histogram is more robust than a conventional study of color variations on the image, because the use of colorimetric information and information on the presence or absence of horizontal lines is associated. on the image.  According to a preferred characteristic, the step of calculating said ground floor histogram uses the following formula: HRDC (Y) = 3. HFY (Y) 2. Hvy (Y) + HTH (y) ûHHY (y) where HRDC (y) is the value of said ground floor histogram in line variable y of said image, HFY (y) is the value of said histogram of vertical background in said variable y, Hvy (y) is the value of said histogram representative of said vertically filtered contour image, in a vertical direction, in said variable y, HTH (y) is the value of said vertical Hough histogram in said variable y, 2910674 8 - and HHY (y) is the value of said histogram of vertical contours in said variable y.  This linear combination of different histograms obtained from the structured image maximizes the peak value of the window histogram corresponding to the color change to be detected, relative to the values of the other peaks of the histogram.  According to another feature, both a horizontal window histogram and a vertical window histogram are computed, and the analysis method according to the invention includes the additional steps of: segmenting said structured image into cells each having one of said unitary elements of said structured image, by analyzing said horizontal and vertical window histograms, detecting said unitary elements by using a rectangular region growth process using said background image and said contour image.  The joint use of the image belonging to the background and the contour image during the detection of the unitary elements improves the effectiveness of this detection, prior to for example a procedural facade modeling.  According to a preferred feature, the cell segmentation step uses frequency filtering of at least one of said horizontal and vertical window histograms.  This frequency filtering facilitates the division into cells because it smooths the histograms, which makes it possible to use segmentation thresholds without requiring complex analysis.  The invention also relates to a method for modeling a structured image comprising the steps of: inverse perspective transformation of said structured image, analysis according to the invention of said structured image thus transformed, classification of the elements of said structured image , detected at the end of said analysis, and background textures of said structured image, from a catalog of unitary elements and background textures, modeling said structured image from said catalog.  In the case of the analysis of a facade image, the unitary elements to be detected are facade elements such as doors, balconies or windows.  The façade image is decomposed, in this modeling, into facade elements superimposed on a background texture image that characterizes its coating.  This modeling method makes it possible to obtain procedural facade models, which serve to reconstruct a facade texture image visually close to the actual facade.  Thanks to the invention this obtaining of models is automatic, effective, and requires only one photograph per building facade.  Only the facade elements most characteristic of the façades to be modeled are retained in the catalog of facade elements and background textures.  The visualization on a remote browser of models in two or three dimensions of buildings faithful to reality, with textured facades of high resolution, is possible.  The invention also relates to a device embodying the method of analyzing a structured image according to the invention, or the method of modeling a structured image according to the invention.  The invention also relates to a structured image modeling system comprising: structured image analysis means implementing the method for analyzing a structured image according to the invention, and means for classifying the image; unitary elements and background textures.  The invention also relates to a signal representative of visibility data, carrying a structured image model obtained by analysis of a structured image containing unit elements aligned in two horizontal and vertical directions, said analysis having given rise to obtaining An image of contours and histograms of horizontal and vertical contours, representative of said contour image in a respectively horizontal and vertical direction, said signal being characterized in that a background image having been calculated from said structured image, and so-called vertical and horizontal background histograms representative of said background image in a respectively vertical and horizontal direction having been calculated, said analysis uses a merge of said vertical background histogram with said histogram vertical contours and said horizontal background histogram with said histogram of horizontal contours.  The invention finally relates to a computer program comprising instructions for implementing one of the methods according to the invention, when it is executed on a computer.  The device, the modeling system, the signal and the computer program have advantages similar to those of the methods.  Other features and advantages will appear on reading a preferred embodiment described with reference to the figures in which: FIG. 1 represents various steps of the facade image modeling method according to the invention, as described in this embodiment, FIG. 2 shows processes performed on a "perspective" facade image, that is to say taken from the street in a non-orthogonal manner to the plane of the image, FIG. 3 represents different steps of the image analysis method according to the invention; - FIG. 4 represents the obtaining of contour histograms from a contour image; FIG. 5 represents steps of a step of the FIG. 6 shows the frequency filtering of a window histogram obtained by the image analysis method according to the invention, FIG. analysis of the peri In Fig. 8, an embodiment of the facade image modeling system according to the invention shows the odicity of a horizontal window histogram obtained by the image analysis method according to the invention.  According to a preferred embodiment of the invention, the structured image analysis method according to the invention is used for analyzing façade images, and is integrated in the facade image modeling method according to the invention. invention.  However, this embodiment is easily adaptable to the analysis of structured images other than facade images, the principles of the invention remaining unchanged.  The facade image modeling method is shown in FIG. 1 in the form of a flowchart comprising seven steps E1 to E7, the steps E2 to E5 being part of the image analysis method according to the invention.  The first step E1 of the modeling method is the inverse perspective transformation of an IP facade image to be analyzed, represented in FIG. 2.  This step is necessary because the IP facade image is conventionally acquired by a photographic system from the street.  The facade represented on the IP image is then deformed according to perspective transformations, and contains elements foreign to the facade, such as sidewalk ends.  It is therefore necessary to crop the IP image on the edges of the facade and straighten it, in order to obtain a frontal orthographic image 10.  The initially color image 10 is transformed into a grayscale image.  For a good performance of the modeling method according to the invention, this image must be of good resolution but not too great in order to limit the computation times necessary for the analysis of the image.  In this embodiment it is for example 700 * 500 pixels resolution.  2910674 12 This El step is supervised and requires the manual selection of the four corners of the facade.  The position of the four points within the image and the assumption that the facade is plane, make it possible to calculate the perspective transformation parameters undergone by the facade and to apply the inverse transformation to recover the rectified image 10.  As a variant, this step is performed automatically, since the exact position of the shooting corresponding to the IP image is known, the parameters of the camera and the dimensions of the facade represented on the camera. IP image.  The second step E2 is the creation of so-called "window" histograms, from the image 10 obtained in step E1.  These histograms are characteristic of the positions of the windows or other elements aligned with the facade image 10.  Their analysis, described later in step E4, thus makes it possible to segment the image 10 into cells each containing only one facade element.  This step E2 of the modeling method according to the invention is broken down into steps a1 to a6 represented in FIG.  The first step a1 is a contour detection on the facade image 10, making it possible to obtain an outline image IC, represented in FIG. 4.  Examples of methods for extracting the outlines of an image are given in the following articles: - "Image Field Categorization and Edge / Corner Detection from Gradient Covariance", of S.  Ando, published in 2000 in the magazine "Institute of Electrical and Electronics Engineers (IEEE) Transactions on Pattern Analysis Machine Intelligence", - and "Edge Detection with Embedded Confidence", P.  Meer and B.  Georgescu, published in 2001 in the same magazine.  The second step a2 is a directional filtering of this contour image IC: 2910674 13 - the vertically filtered image IC gives a vertically filtered contour image IFV, represented in FIG. 4, freed from vertical lines, and the image IC Filtered horizontally gives a horizontally filtered image of 5 contours IFH, cleared of horizontal lines.  The third step a3 is the calculation of contour histograms from the IFV and IFH filtered contour images.  The pixel values of these images are summed line by line to give histograms in a vertical direction, and column by column to give histograms in a horizontal direction.  Four histograms are thus obtained: a histogram of HVX horizontal contours, obtained by column-by-column summation of the pixels of the IFV filtered image, which characterizes the facade by having high values in the 15 zones of the image containing facade elements. , and hollows in the inter-element areas, an HHX contour histogram, obtained by column-by-column summation of the pixels of the IFH filtered image, which has peaks in the vertical edge areas of the facade elements, or in the areas of vertical lines on the facade, a histogram of vertical contours HHY, obtained by row-by-line summation of the pixels of the horizontally filtered image IFH, which characterizes the facade by having strong values in the zones of the image containing facade elements, and hollows in the inter-element zones, and an HVY contour histogram, representative of the vertically filtered contour image IFV, obtained by line summation per pixel line of the IFV filtered image, and which has peaks 30 in the horizontal edge areas of the facade elements, or in the horizontal line areas on the facade.  Only histograms of HVX horizontal contours and vertical HHY are used for calculating the window histograms in this step E2, the other histograms being used for the detection of a color change on the facade in step E3.  The fourth step a4 is the calculation of a membership image of the background IAF, shown in FIG.  Each pixel of this image represents the distance, coded in gray scale, between the average color of the background of the facade image 10, and the corresponding pixel of the facade image 10.  This distance is defined as a difference in gray level values.  Thus, the more the zones of the facade image 10 are close to the average background color, the more black they appear on the IAF background image, since the distance between the corresponding pixels and the average background color is small.  To characterize the average background color, the "Largest 15 Empty Rectangle" algorithm, described in the article "Automatic Techniques for Texture Mapping in Virtual Urban Environments", by R.  G.  Laycock and A.  Mr.  Day, published in 2004 at an international conference "Computer Graphics International", or in the article "Computing the largest empty rectangle Source" by B.  Chazelle, R.  L.  Drysdale and D.  T.  Lee, published in 1986 in the 20 "Society for Industrial and Applied Mathematics (SIAM) Journal on Computing," is used.  For a set of randomly distributed points on the image, this algorithm looks for rectangles of uniform colors.  These regions make it possible to analyze the uniformly distributed colors in the image and to detect the average background color.  It should be noted that the "Largest Empty Rectange" algorithm is applied to the initial color facade image 10, transposed into the L * a * b * coding system created by the International Commission on Illumination (CIE). ).  The distances calculated between the average background color of the facade image 10, 30 and the corresponding pixel of the facade image 10 are distances expressed in the coding system L * a * b *, because this coding system The color allows to code a distance between colors consistent with the distances perceived by the human eye.  The fifth step a5 is the calculation of so-called "background" histograms representative of the IAF background image in the vertical and horizontal directions: the vertical background histogram HFY, shown in FIG. 2, is obtained by row-by-row summation of the pixel values of the background image IAF, and the horizontal background histogram HFX is obtained by column-by-column summation of the pixel values of the IAF background image. .  These background histograms characterize the adequacy of a row or a column of pixels with the background color of the facade: they contain strong values on the areas of the image containing facade elements, and 15 low values. on the inter-element zones.  The sixth step a6 is the fusion of histograms of vertical and horizontal contours previously calculated, with the vertical and horizontal background histograms respectively.  These four histograms are relatively correlated, which makes it possible to merge them to sum up the information and increase the robustness of the analysis of the facade image.  The great diversity of façade images requires the provision of a robust cell segmentation method to ensure a high rate of correct results.  We then obtain two so-called "window" histograms, one vertical and the other horizontal, defined by the formulas: HWindow (y) ù HFY (Y).  HHY (Y) H Window (x) ù HFX (x) HVX (x) y is a variable designating a pixel line of the facade image 10, where x is a variable designating a column of pixels of the facade image 10, - HWindow (y) is the value of the vertical window histogram in the row of pixels y, 5 - HWindow (x) is the value of the horizontal window histogram in the column of pixels x, - HFY (y) is the value of the vertical background histogram HFY in the line of pixels y, - HFX (x) is the value of the horizontal background histogram HFX in the column of pixels x, - HHY (y) is the value of the histogram of vertical contours HHY in the row of pixels y, - and HvX (x) is the value of the histogram of horizontal contours HVX in the column of pixels x.  It should be noted that in this embodiment the values of the histograms are multiplied with one another, but other modes of fusion can be used, for example using weighted sums.  These window histograms will then be used in cell segment image segmentation step E4 because they contain peaks on areas representing facade elements and valleys in inter-element areas.  The third step E3 of the modeling method according to the invention is the detection of a color change on the facade image 10.  Indeed, the window histograms obtained in step E2 are disturbed at the levels of color changes on the facade.  In this embodiment of the invention, only a color change is detected in a vertical direction, because the buildings very often have ground floors coated with a different color from that of the upper floors.  However, it is easy to adapt this embodiment to the detection of a color change in a horizontal direction, or to several color changes.  This step E3 is broken down into four steps b1 to b4 shown in FIG.  Step b1 is a Hough transformation of an image of the luminances IL obtained from the initial color facade image, and shown in FIG.  The image of the luminances IL is calculated by changing the color space of the image 10.  The pixels of the image 10 are initially represented in the Red / Green / Blue RGB color coding system according to the English "Red Green Blue", by triplets representing the 10 values of these three components.  These colors encoded in the RGB system are converted into coded colors in the CIE coding system L * a * b *.  The components a * and b * represent the chrominance plane while the L * axis represents the luminance variations.  The image of the luminances IL is obtained by keeping only the L * component of the colors L * a * b * thus calculated from the initial color image.  It should be noted that the CIE L * a * b * color space is used in the invention to calculate distances between colors and in particular during the "Largest Empty Rectange" algorithms during the detection of the background color and when constructing Background histograms.  The Hough transform of the IL luminance image is performed according to a known method described in US Pat. No. 3,069,654. V. C.  Hough, entitled "Method and Means for Recognizing Complex Patterns".  This transform makes it possible to detect the presence of lines within a two-dimensional signal.  The resultant ITH image of this transform contains much more marked lines than those present on the IC contour image.  Step b2 is the calculation of a HTH histogram called "Hough", in a vertical direction of the resulting image ITH.  This histogram is obtained by summing the pixel values of the ITH image line by line.  It has very sharp peaks at horizontal lines on the ITH image and smaller peaks at the horizontal edges of facade elements.  It should be noted that the Hough HTH histogram calculation does not require the complete construction of the Hough image, but only the portion of the Hough image corresponding to the vertical and horizontal lines.  Step b3 is the calculation of a so-called "ground floor" histogram, obtained by linear combination of the HHY vertical contour histogram, the HFY vertical background histogram, the Hough vertical histogram HTH and the contour histogram HVY, according to the formula: HRDc (y) = 3 • HFY (y) + 2 • H, (y) + HTH (y) -HHY (y) where y is a variable designating a line of pixels of the facade image 10, HRDC (y) is the value of the ground floor histogram in the variable y, HFY (y) is the value of the vertical background histogram HFY in the variable y, HvY (y) is the value of the contour histogram HVY in the variable y, HTH (y) is the value of the histogram of vertical Hough HTH in the variable y, and HHY (y) is the value of the histogram of vertical contours HHY in the variable y.  This ground floor histogram was constructed to obtain a peak on the boundary between two areas of different colors.  This is because, on the one hand, histograms that have high values on such transition areas, i.e., the HVY contour histogram and the Hough HTH histogram, are summed because they detect horizontal lines, and the background histogram HFY because it has high values 30 on the ground floor area, 2910674 19 and on the other hand subtract the histogram from vertical contours HHY because it must be zero on the transitions zones, which has the effect of attenuating the other peaks of the window histogram.  Similarly, if it is desired to detect a color change in a horizontal direction, it is sufficient to perform the symmetrical linear combination thereof, with the histogram of HVX horizontal contours, the HFX horizontal background histogram. , a histogram representative of the resulting ITH image of the Hough transform in a horizontal direction, and the HHX contour histogram.  Step b4 is the detection of the peak marking the boundary between the ground floor and the floors of the facade of the image 10, on the previously calculated ground floor histogram.  We are looking for a maximum in the lower half of the histogram.  Indeed, being unable to make any assumption on the number of floors in the facade, it is simply assumed that there are at least two levels in the building.  The boundary between the ground floor and the floor or floors is therefore in the lower half of the facade image 10.  So we go through the histogram of ground floor and, we look for, on all the maxima of its lower half, the peak of greater amplitude satisfying the following criteria: 20 HRDC (i)> 2/3, N Window U ) j H H Fen Fen)))) Fen Fen Fen Fen Fen Fen Fen Fen Fen Fen Fen Fen Fen Fen Fen Fen Fen Fen. 8 H Window Ci) i = ii = i where i is a pixel line variable of the facade image corresponding to a peak on the lower half of the ground floor histogram, j = 0 1 H window (J) j = 0 1> ù, 2 2910674 20 j is an index traversing the N values of the vertical window histogram from top to bottom with respect to the facade image 10, N is the size of the window. vertical window histogram, HRDC (i) is the value of the histogram of the ground floor histogram in the variable i, and HWindow (j) is the value of the vertical window histogram in the variable j.  This determines the line of the facade image 10 corresponding to the peak of ground floor.  This additional criterion of selecting the maxima of the ground floor histogram makes it possible to suppress the peaks not being on areas of variations of the average background color.  The fourth step E4 of the modeling method according to the invention is the cell segmentation of the facade image 10.  For this, the window histograms obtained in step E2 are analyzed.  The vertical window histogram is separated into two sub-histograms corresponding to two sub-images of the facade image 10, one containing the ground floor of the building whose facade is being analyzed, and the other containing the upper floors.  This separation uses the previous E3 detection step, which provides the vertical window histogram separation line.  It makes it possible to overcome disturbances on the vertical window histogram, due to the change of color on the facade image 10.  As noted above, window histograms have "valleys" in the inter-element areas.  The analysis of the minima of the vertical window sub-histogram representative of the stages thus makes it possible to determine the boundaries between stages.  Similarly, the analysis of the minima of the horizontal window histogram makes it possible to determine the boundaries between facade elements.  2910674 21 To facilitate the search for minima, the high frequencies of these histograms are filtered on a wavelet basis.  More precisely, the signals of the window histograms are broken down into frequencies on a Daubechies wavelet basis, and are filtered by suppressing the high frequencies.  This makes it possible to smooth the histograms, as shown, for example, in the diagram of FIG. 6, having on the abscissa the axis AX1 of lines of pixels and on the ordinate the axis AY1 of values of the vertical window sub-histogram representative of the stages: The unfiltered signal SO of this sub-histogram has many peaks due to irregularities of the facade elements, while the corresponding SF1 filtered signal has only a few minima and maxima remaining to be analyzed.  As the horizontal window histogram is generally more sensitive to image noise and disturbance than the vertical window histogram, the horizontal window histogram undergoes in this step E4, in addition to this frequency filtering, a process of periodic delay.  This process analyzes the periodicity of the horizontal window histogram by a study of the covariance and, if appropriate, applies a set of Gaussian centered on the minimums of the curves meeting the periodicity criterion.  The diagram of FIG. 7, having on the abscissa the axis AX2 of lines of pixels and the ordinate axis AY2 of values of the horizontal window histogram shows the signal filtered in SF2 frequencies of this histogram, a Gaussian SG applied to this filtered signal SF2, and the histogram resulting from this application SR.  The filtered signal SF1 of the vertical window sub-histogram and the resulting signal SR of the horizontal window histogram are then analyzed.  For this, the distribution spaces of the values of these signals are partitioned, and thresholds are dynamically fixed on these signals from the partitions created.  More precisely, these partitions are classified into four groups by a K-Mean algorithm, which dynamically sets three thresholds.  Thus, in the example of FIG. 6, the values of the filtered signal SF1 are distributed in four intervals as a function of their positions with respect to three thresholds S1, S2 and S3.  These thresholds make it possible to classify the minima of the signals so as to keep only the global minima: only the minima lower than the lowest threshold are kept, and two minima are fused if they are not separated by a maximum greater than the median threshold. .  it

  allows to merge two minima located in the same inter-element zone. The global minima obtained by analysis of the SF1 filtered signal delimit the stages between them and make it possible to segment the facade image floor by floor. The overall minima obtained by analysis of the resulting signal SR 10 delimit the facade elements between them and make it possible to segment the floors into cells, each comprising only one facade element. Alternatively, instead of calculating a single horizontal window histogram in step E2, step E4 computes a window histogram per floor, after step segmentation of the vertically filtered contour image, and segmentation by stages of the image belonging to the background, from the global minima obtained by analysis of the SF1 filtered signal. This makes it possible to refine the results, by inter-stage comparison of the limits of cells found by analysis of the different horizontal window histograms. This allows the cells to be aligned, which increases the visual quality of the segmentation, and to compare the segmentation of each stage to add or remove cells based on neighboring stages. The fifth step E5 of the modeling method according to the invention is the detection of the facade elements in the cells obtained in the previous step 25. This step consists in precisely extracting the facade elements from the bottom of the facade. Indeed a cell is comparable to a facade element surrounded by background areas. To accurately extract the area of each cell representing the facade element, a process of growing rectangular regions is used. The principle of this process is to enlarge a region having certain shape properties, here a rectangular shape, according to a distance map computed from the image. An example of an algorithm for growing rectangular regions to detect facade elements is described in the article by K. Schindler and J. Bauer cited above. More precisely, the distance map is created in this step E5 by adding the values of the pixels of the image belonging to the background IAF and the values of the pixels of the contour image IC. It should be noted that FIG. 4 represents for practical reasons an inverted IC contour image, the contour image IC being actually composed of white outlines on a black background. This distance map is then filtered by a Gaussian calculated according to the distribution of this map, that is to say according to the average and the variance of the map, to attenuate the noise and to minimize the values close to it. of zero, that is to say the values of the pixels of the background areas, or the values close to one, that is to say the values of the pixels on facade elements. Finally, the average value of the filtered card is subtracted from all the values of this map to obtain a positive distance map on the regions representing facade elements and negative elsewhere. The map thus obtained is then supplied to the region growth process, which from a small rectangular starting area, positioned in the center of the cells as delimited in step E4, converges towards the rectangular zone maximizing the average value. pixels of the corresponding region. The sixth step E6 of the modeling method according to the invention is the classification of the facade elements and background textures of the facade image 10, from a catalog of facade elements and background textures. This catalog is for example contained in a database BD2, represented in FIG. 8, the modeling method according to the invention being implemented in a software manner in a servicing data server SERV, connected to a database BD1 of FIG. facade images of a city. In a variant, the modeling method according to the invention is implemented on several machines operating in parallel.

2910674 24 The facade element catalog is updated as the SERV server parses the front images of the BD1 database. It contains images for reconstructing synthetic facade images as closely as possible to actual facade images. These images are of two types: images of background textures of the facades, of small dimensions, which are repeated on the background of the computer images to create high resolution images, and images corresponding to the elements of facades extracted from the cells when from step E5. The background texture images characterizing the stages of the facade image are created using the distance map created in step E5 and from which the unitary elements of facades were extracted. This map is scanned to search for the largest rectangular area for which the sum of the pixels composing it is minimum. A small uniform rectangular region corresponding to this zone is then extracted into the initial color IO facade image, which is repeated to recreate a background image of the same size as the stages of the facade image 10. Another background texture image is similarly created to characterize the ground floor. In order to allow a fast reconstruction of facade images from a remote terminal connected to a visualization data server, the catalog keeps only the images of facade elements and the most characteristic background texture images of the images analyzed in and according to the invention. The constitution of the catalog is carried out at the same time as the analysis of the images of facades, in order to optimize the construction of the procedural facade models. For this purpose, the images corresponding to the elements of façades extracted during step E5 are analyzed by color, texture and shape descriptors. Similarly, background texture images extracted from the facade images of the database BD1 are analyzed by descriptors of colors and textures. According to the result of the color change detection step E3, several background texture images per facade image are possibly described. The description of each element makes it possible to represent them in a large description space and to measure in this space the similarity between elements. The two types of images stored in the catalog require two description spaces, one for background texture images and the other for images corresponding to facade elements. Indeed, although the descriptors used to describe the background texture images and the descriptors used to describe the images corresponding to the extracted façade elements are partly identical, in particular for the description of the colors and textures, these last images are analyzed separately. They are also described by shape descriptors using a region approach, for example using the Angular Radial Transform transformation (ART). The color descriptors are created by calculating color histograms of the images to be described. This approach is proposed by M. J. Swain and D. H. Ballard in their article "Color indexing" published in 1991 in the magazine "International Journal of Computer Vision". It consists in representing the pixels of the images in the form of histograms of the values of their colors. In this embodiment of the invention, three histograms are created, one per color component of the pixels described in the CIE color space L * a * b *. These histograms are quantized to reduce the description space and the value of each quantization is stored as a descriptor. Texture descriptors use an image texture analysis. This analysis is based on the work of Yong Man Ro, Kim Munchurl, Kyung Kang Ho, BS Manjunath, and Jinwoong Kim, described in the article "MPEG-7 Homogeneous Texture Descriptor" published in 2001 in the magazine "Electronics and Telecommunications". Research Institute (ETRI) journal ".

Once all the background elements and textures of the facade images of the database BD1 indexed using these descriptors in the catalog of the database BD2, the two description spaces are segmented into classes. similar elements by the dynamic cloud 5 algorithm. This algorithm is an automatic classification method that aims to partition space into a certain number of classes. It is described in particular in the following articles: "Least squares quantization in pulse-code modulation (PCM)", by SP Lloyd, published in 1982 in the magazine "IEEE Transactions on 10 Information Theory", and "Some Methods for Classification and Analysis" "From Multivariate Observations", by JB MacQueen, published in 1967 for the fifth "Berkeley Symposium on Mathematical Statistics and Probability".

The choice of the number of classes to be kept, and therefore the final number of background textures and facade elements in the catalog, is defined according to the accuracy of the desired realism and the size of the catalog that is desired. . At the end of this segmentation, each image of background texture or facade element is associated with its class. This association is maintained for each facade image in the database BD1, which also stores the position of the elements or textures on the corresponding facades. Then we delete from the catalog all the images that are not defined by the dynamic clouds algorithm as a center of a class. This makes it possible to keep only a few facade elements, while ensuring that each element will have a visually close class representative that will replace it in the computer-generated images. The facade image 10 is associated in this embodiment with two background texture images, one for the ground floor and the other for the upper floors. Each of these texture images are classified, at the end of step E6, in a very close class of background texture images represented by the center image of this class. Likewise, the facade elements extracted in step E5 are classified into classes each represented by a very similar façade element. Depending on the desired realism or the type of facade analyzed, for example if several color changes of the facade are detected in step E3, the front image 10 is associated with more images of background textures: for example each floor of the facade image 10 is associated with a background texture image. Finally, the seventh step E7 of the modeling method according to the invention is the modeling of the facade image 10. This modeling is for example the procedural modeling proposed in the MPEG4 standard, in the "Animation Framework eXtension" part. It is carried out by filling modeling parameters defining in particular: the position and the size of the ground floor and the floors, the number of floors, 15 for each floor, the number and the position of the cells a number of class associated with the texture image characterizing the stages of the facade image 10, a class number associated with the texture image characterizing the ground floor of the facade image 10, 20 for each cell, their size and a class number associated with a facade element possibly present in this cell, as well as its position in the cell, its size, etc. The server SERV of visualization data of a city is connected to 25 client terminals by a communication network RES, shown in FIG. 8. When a user navigates virtually in the city using a browser on a remote terminal T connected to the server SERV by the network RES, the terminal T asks the server SERV, by a query REQ visualization data, the transmission of data to be displayed corresponding to the place where virtually the user is in the city. The server SERV then sends the terminal T a REP response containing facade data 2910674 to display. These data include the procedural models of the facades to be viewed, and all or part of the catalog of facade elements and background textures. The information signal conveying the data of the REP request is therefore much more compact than if it were necessary to transmit the real images of the facades, and allows a reconstruction of good resolution and similar to the original facades. 10

Claims (10)

  A method of analyzing a structured image (10) containing unit elements to be detected, aligned in two horizontal and vertical directions, comprising the steps of: Detecting (a1) contours and obtaining an outline image (IC ), Calculating (a3) a histogram in a vertical or horizontal direction, representative of said contour image (IC), respectively called vertical contour histogram (HHY) or horizontal contour histogram (HVX), said method being characterized in it comprises the additional steps of: computation (a4) of a background image (IAF) from said structured image (10), calculation (a5) of a histogram in a vertical or horizontal direction , representative of said background membership image (IAF), respectively called vertical background histogram (HFY) or horizontal background histogram (HFX), Fusion (a6) of said vertical (HFY) or horizontal (HFX) background histogram with respec said vertical contour histogram (HHY) or horizontal contour histogram (HVX) into a histogram called vertical window histogram or horizontal window histogram, respectively.   A method of analyzing a structured image (10) according to claim 1, characterized in that in the melting step (a6) the vertical (HFY) or horizontal (HFX) background histogram is multiplied respectively by the histogram of vertical (HHY) or horizontal (HVX) contours.   A method of analyzing a structured image (10) according to claim 1 or 2, further comprising a step of detecting (E3) vertical or horizontal color change, for respectively separating the vertical window histogram. or the horizontal window histogram in two sub-histograms each representative of a color of said structured image (10). 10   A method of analyzing a structured image (10) according to any one of claims 1 to 3, wherein the histograms of vertical (HHY) or horizontal (HVX) contours are obtained after horizontal or vertical filtering respectively. said contour image (IC).   A method of analyzing a structured image (10) according to claims 3 and 4, wherein the step of detecting (E3) vertical color change comprises the steps of: Hough processing (b1) of a image of the luminances (IL) of said structured image (10), Calculation (b2) of a histogram in a vertical direction, representative of the resulting image (ITH) of said transformation, called vertical Hough histogram (HTH), Calculation (b3) of a histogram called a ground floor histogram by linear combination of said vertical bottom histogram (HFY), said vertical Hough histogram (HTH), said histogram of vertical outlines (HHY), and a histogram (HVY) in a vertical direction representative of said vertically filtered contour image (IFV). 15 20 25 30 2910674 31   A method of analyzing a structured image (10) according to claim 5, wherein the step of calculating (b3) said ground floor histogram uses the following formula: HRDC (Y) = 3.HFY (Y) + 2.Hvy (Y) + HTH (y) ûHHY (y) where HRDC (y) is the value of said ground floor histogram in a line variable y of said image (10), HFY (y) is the value of said vertical background histogram (HFY) in said variable y, Hvy (y) is the value of said histogram (HVY) representative of said vertically filtered contour image (IFV), in a vertical direction, in said variable y, HTH (y) is the value of said vertical Hough histogram (HTH) in said variable y, and HHY (y) is the value of said vertical contour histogram (HHY) in said variable y.   A method of analyzing a structured image (10) according to any of claims 1 to 6, wherein both a horizontal window histogram and a vertical window histogram are computed, characterized in that it comprises the additional steps of: segmenting (E4) said structured image (10) into cells each having one of said unit elements of said structured image (10), by analyzing said horizontal and vertical window histograms, detecting (E5) said unitary elements by using a rectangular region growth process using said background membership image (IAF) and said contour image (IC). 2910674 32   The method of analyzing a structured image (IO) according to claim 7, wherein the cell segmentation step (E4) uses frequency filtering of at least one of said horizontal and vertical window histograms. 5   9. A method of modeling a structured image (IP), comprising the steps of: - transformation (E1) inverse perspective of said structured image (IP), - analysis (E2, E3, E4, E5) of said structured image and Transform 10 (10) according to any one of claims 1 to 8, - classification (E6) of the elements of said structured image, detected at the end of said analysis (E2, E3, E4, E5), and textures of background of said structured image, from a catalog of unitary elements and background textures, modeling (E7) of said structured image (IP) from said catalog. 13. Device (SERV) comprising means adapted to implement one of the methods according to any one of claims 1 to 9. 14. A structured image modeling system (IP) comprising: structured image analysis implementing the analysis method according to any one of claims 1 to 8, and means for classifying unitary elements and background textures. A representative visibility data signal, carrying a structured image pattern obtained by analysis of a structured image (10) containing unitary elements aligned in two horizontal and vertical directions, said analysis having resulted in obtaining an edge image (IC) and horizontal contour (HVX) and vertical (HHY) histograms representative of said contour image in a respectively horizontal and vertical direction, said signal being characterized in that an image of background membership (IAF) having been calculated from said structured image, and so-called vertical background (HFY) and horizontal (HFX) histograms representative of said background membership image (IAF) in one direction respectively calculated vertically and horizontally, said analysis uses a merge of said vertical bottom histogram (HFY) with said histogram of vertical contours (HHY) and said histogram horizontal basemap (HFX) with said histogram of horizontal contours (HVX). 13. A computer program comprising instructions for carrying out one of the methods of any one of claims 1 to 9 when executed on a computer.