WO2015193521A1 - Method and system for the automatic classification of kidney stones, computer program, and computer program product - Google Patents

Abstract

The method comprises: a) acquiring an image of a kidney stone; b) analysing, using computer-aided vision technology, information contained in the acquired image, referring to the characteristics of the image associated with at least the texture of the kidney stone; and c) classifying the kidney stone based on the result of said analysis. The system is suitable for carrying out the method according to the invention. The computer program carries out the analysis and classification steps of the method according to the invention, and the computer program product incorporates the computer program.

Description

 METHOD AND SYSTEM FOR AUTOMATIC CLASSIFICATION OF RENAL CALCULATIONS. COMPUTER PROGRAM AND PRODUCT PROGRAM

 COMPUTER Technical sector

 The present invention concerns in general a method and a system for the automatic classification of kidney stones, based on image analysis, and more particularly a method and a system that use computer vision techniques to carry out the analysis. of images of kidney stones and their subsequent classification as a result of such analysis.

 Other aspects of the invention concern a computer program that implements the analysis and classification stages of the method of the invention, and a product that incorporates such a computer program. Prior art

 Kidney stones are usually classified according to their chemical composition as calcium oxalate monohydrate (COM), calcium oxalate dihydrate (COD), calcium oxalate dihydrate transformed into monohydrate (TRA), brushite (BRU), apatite carbonate (CAP) also known as hydroxyapatite (PAH), struvite (STR), anhydrous uric acid (AUA), uric acid dihydrate (AUD), mixed calcium oxalate and apatite carbonate (MXD) calculations, with remarkable variability within each class.

 This chemical classification of kidney stones leads to a description of the metabolic alterations that a patient has gone through, and thus, the selection of a useful treatment to prevent recurrence of colic (recurrence). The specific treatment that each patient can receive should be based on recommendations and restrictions in the diet and suggestions on dietary supplements (which can modify some urinary parameters, such as inhibitors and promoters of stone formation), along with monitoring the levels of some components of the urine. It is important to note that the formation of kidney stones is a disease with an especially high recurrence rate. Therefore, by proper treatment of the patient, further stone formation is drastically reduced. This provides a better quality of life for the patient, along with considerable savings for health organizations. This is well known to urologists, but their access to this information is limited due to the partially incomplete results normally offered by clinical laboratories.

Existing analysis techniques: Aware of the problem, some techniques have been applied to the analysis of kidney stones. These types of samples are generally classified, using optical and spectroscopic methodologies, according to their chemical composition.

Infrared (IR) spectroscopy is the most widespread technique, since it is simple and allows a classification of stones based on the chemical composition and the percentage of the main components of the sample. The strength of this classification lies in the recognition of spectral bands (in the infrared range, defined by wave numbers ranging from 400 to 4000 cm ^"1 ), which are directly related to the chemical composition. The three main drawbacks of this Technique are:

 - The stone must be ground for analysis, so any spatial distribution of the components is inevitably lost.

 - The infrared spectra obtained need to be studied by an expert in the area of spectroscopy and with knowledge of lithiasis.

 - Only then can the results on composition and quantification be obtained, and translated into the most appropriate treatment suggestions.

 Although infrared light is sensitive to chemical components that appear naturally in kidney stones, it is not sensitive enough to detect minor components.

 On the other hand, the nature and distribution of the components found in kidney stones induce a characteristic visual appearance, which can be recognized by optical techniques. On the basis of this, a morpho-constitutional analysis based on the optical and physical characteristics of the samples can be carried out, since it is performed using a stereo microscope. Features such as hardness, color and distribution of the components are used to give a classification of the calculations. The sample is cut to observe the inner part of the calculation, if necessary. The classes resulting from this type of analysis are not the same as those obtained with infrared spectroscopy, but an extension / specialization of them because this technique is sensitive to the presence of minor components thus offering a second order classification. Thus, the classes not only depend on the chemical composition, but also on their spatial distribution. The main drawbacks of this technique are:

- It is a technique that consumes a lot of time and that requires a specialized technician duly trained in the recognition of the constitution of calculations. - Because the recognition is done visually, it is subject to the aptitude and experience of the technician.

 In the International application WO2012136874A1, a method for the characterization and classification of kidney stones is proposed, by analyzing different spectra of pre-cut kidney stones, applying a hyperspectral imaging technique. The information analyzed is relative to the intensity of the radiation reflected in the calculation at different wavelengths. The method proposed in said application comprises analyzing each pixel of each image, so that the image of the renal calculus is divided into a matrix of pixels and each of them is analyzed independently of the neighboring pixels. Hyperspectral analysis is carried out in the near-infrared spectrum (in a spectral range covering 1000-1700 nm), taking into account all the variables, relative to reflectance measured for all wavelengths, individually in an analysis multiparameter That is, in the proposal made in WO2012136874A1, a sample is considered as composed of a given number of pixels, each consisting of variables related to reflectance of energies in the near-infrared spectrum. This spectrum carries information on the chemical composition of the stone.

 In US8280496B2 a method is described to determine the type of renal calculus, in particular its composition, by lighting the renal calculus with different wavelengths and comparing the reflectances generated in the renal calculus for each wavelength, classifying the renal calculus based on the differences found in these reflectances for different wavelengths. The method is implemented by an endoscope that carries both the lighting means of different wavelengths and an image sensor sensitive to said wavelengths and that acquires images that include information related to said reflectances.

 In Japanese patent document JP2008197081A it is also proposed to perform a classification of calculations based on a multispectral analysis thereof, in particular quantifying the components in the calculation based on absorbance values of some spectral bands in the infrared region.

In none of the cited documents is an image analysis performed, understood as relative to the texture and general characteristics of the kidney stone, but simply of its spectral characteristics. Explanation of the invention. It seems necessary, therefore, to offer an alternative to the state of the art that overcomes the disadvantages of those suffering from traditional kidney stone analysis techniques.

 To this end, the present invention concerns, in a first aspect, a method for the automatic classification of kidney stones, comprising:

 a) acquire an image of a kidney stone;

 b) analyze information contained in said acquired image; Y

 c) classify said renal calculus according to the result of said analysis.

 Unlike the known automatic classification methods, in which a spectral analysis of the different parts of the acquired image is performed, in the method proposed by the first aspect of the present invention, characteristically, the analysis of step b ) is carried out using computer vision techniques, preferably together with computational learning techniques, where said information to be analyzed refers to the image characteristics associated with at least the texture of the kidney stone.

 According to a preferred embodiment, the acquired image is a digital image, and the analysis of step b) is carried out at once on the complete digital image acquired in stage a), without performing an individual classification of each pixel.

 For an exemplary embodiment, the information to be analyzed refers to associated image characteristics, in addition to texture, to other visual characteristics of renal calculus, including size and / or shape and / or color.

 According to an exemplary embodiment, step a) comprises acquiring at least two images of a fragment of the kidney stone, one corresponding to a view of an outer surface and another to a view of an exposed inner surface, and analyzing them in the stage b). When the kidney stone fragment has no exposed inner surface, the method comprises cutting it to expose said inner surface whose image is acquired in step a).

 Preferably, the method comprises acquiring in stage a) and analyzing in stage b), a plurality of images of corresponding views of each of the outer and inner surfaces of the kidney stone fragment, each under conditions of different lighting and / or exposure time.

According to an embodiment, the method comprises carrying out said image acquisitions with the same image sensor sensitive to the wavelengths associated with all the illuminations included in said different lighting conditions, which are within the range from the visible light at the initial near-infrared wavelengths. Alternatively, the method comprises carrying out said acquisitions with several image sensors sensitive, together, to all said different wavelengths.

 The method comprises, according to an embodiment, making said acquisition, in step a), and analysis, in step b), of images of the views of the outer and inner surfaces for a plurality of fragments thereof. sample.

 The classification of stage c) explained so far, that is, based on the analysis of the images of the fragment or fragments of renal calculus, is a classification of a first level, or classification of sight, which includes, for each view, a estimate of the probability of belonging to a class of renal calculus associated with chemical composition, from the calculation of a vector of probabilities for each view that includes information on the spatial distribution of said probabilities.

 The result of said classification of a first level is considered as a final result for an embodiment example, but for another more elaborate embodiment example for which the precision required for the classification result is higher, in order to obtain such more precise classification, the method comprises performing, in step c), a second level classification, or fragment classification, which comprises combining the results obtained in the classification of a first level for several views of the same kidney stone fragment , to determine a unique class for each fragment based also on chemical composition also on the location and distribution of the chemical components associated with these probabilities and to what view they correspond.

 In the case of obtaining divergent results for different views, the system contemplates the application of a previously defined cost matrix. The costs related to each class are included in a classification vector of each fragment, and are fixed, contrary to the information obtained as a result of the classification of the first level, which depends on the images and measurements taken. The cost values that are applied make it possible to correct disparate results to give a single class value to a second level.

Optionally, the method comprises correcting the result of the classification of a second level if the result for a given fragment differs from those obtained for each of the views of the same, in the classification of a first level, above a cost value determined. In general, such a determined cost value is defined prior to the analysis, and included in a classification vector, which includes an associated cost value that marks the dependence of such classification. One way of carrying out said correction comprises reclassifying said fragment taking less into account, or not taking into account at all, the location and distribution of the chemical components associated with said probabilities and to what view they correspond, that is, based on everything, or only (in the most extreme case), in the chemical composition.

 The result of said classification of a second level is considered as a final result for an embodiment example, but for another more elaborate embodiment example for which the precision required for the classification result is higher, in order to obtain such Even more precise classification, the method comprises performing steps a), b) and c) for two or more fragments of the same sample, the method comprising performing, in step c), a third level classification, or sample classification , which comprises, if the classification of a second level of said fragments is not coincident, assign a unique class for the sample.

 As indicated above, depending on the accuracy required for the classification result, it may be terminated after the classification of a first level, after the classification of a second level or after the classification of a third level.

 According to an embodiment example, the method comprises:

 - generate, prior to stage a), a set of training, or learning, for a plurality of samples of kidney stones classified manually by an expert, including, in a correlated way, information on chemical composition, spatial distribution and of visual appearance for internal and external views of different fragments of each sample represented in images obtained with different types of lighting and exposure times, and

 - make the classifications of a first, a second and a third level by consulting the images acquired in said training set and extracting the class information correlated with the images more similar to those consulted.

 The method comprises training said training set and / or automatic classifiers used to perform at least said first, second and third level classifications using the results of the classifications.

According to an embodiment, the method of the first aspect of the invention comprises carrying out the classification of step c) complementing the analysis of the stage b) with additional information regarding the patient from whom the kidney stone comes and / or obtained from the kidney stone with non-camera based sensors.

 With respect to said patient information, it includes at least one of the following information, or a combination thereof:

 - simple data related to sex and / or age and / or race and / or complexion and / or body mass index and / or health disorders associated with renal lithiasis, and / or

 - data linked to collateral analytics, including at least one of the following data collected in the patient's urine analysis: pH, calcium, oxalate, magnesium, ammonium and phosphate.

 As regards said information obtained from renal calculus with non-camera based sensors, this includes, according to an example of embodiment, at least reflectivity information in other areas of the electromagnetic spectrum not included in the image acquired in a), such as the spectrum corresponding to infrared.

 A second aspect of the invention concerns a system for the automatic classification of kidney stones, comprising:

 - means of acquiring images to acquire at least one image of a kidney stone; Y

 - an electronic system in connection with said image acquisition means and including processing means to process information contained in said acquired image and to classify said renal calculus according to the result of said analysis.

 The system proposed by the second aspect of the invention complements the method according to any one of the preceding claims, implementing the means of processing of the electronic system one or more algorithms based on computer vision techniques, and preferably also computational learning, for perform steps b) and c) of the first aspect method.

 Preferably, said image acquisition means comprise a system that has an image focusing mechanism of kidney stones, manually or automatically controllable by the electronic system.

 According to an exemplary embodiment, the system comprises a housing that defines an interior space that is lightly insulated from the exterior that houses, supported and / or fixed in an internal support structure:

- to a support for kidney stone samples, which is preferably removable with respect to said support structure and said housing; - to lighting means arranged to illuminate, with light of one or more wavelengths, the kidney stone sample (s) arranged on said support;

 - to said image acquisition means, which include an image sensor sensitive to said or said wavelengths; Y

 - at least part of said electronic system, which also includes control means for controlling at least the lighting means.

 The system also optionally comprises one or more sensors sensitive to a range of the electromagnetic spectrum (such as the corresponding infrared spectrum) different from that associated with the image acquisition means, arranged or arranged facing the kidney stone sample (s) arranged on said support, and in connection with the electronic system, to capture the global reflectivity (without performing a pixel analysis) of the sample or samples in a spectral range suitable for characterization.

 According to a preferred embodiment, the entire electronic system is local and is housed inside the housing.

 Alternatively, said part of said electronic system housed within the housing is a local part and the electronic system comprises a remote part, such as a computer, communicated bi-directionally with said local part and with the image acquisition means.

 A third aspect of the invention concerns a computer program that includes code instructions which, when executed on a computer, implements steps b) and c) of the method of the first aspect.

 A fourth aspect of the invention concerns a computer program product comprising the computer program of the third aspect.

 According to an exemplary embodiment, the computer program product comprises or is stored or implemented in a medium that may contain, store, communicate, propagate or transport the computer program for use by a system, apparatus or device of execution of instructions, or in connection with it. Said means is or comprises, according to some examples of embodiment, a physical and / or logical support readable by a computer and / or an electromagnetic, optical or acoustic signal that transports the computer program.

The present invention allows, therefore, in its different aspects, to perform an automatic classification of kidney stones in a way that is useful for doctors, that is, the results are similar to those they are accustomed to manage and comply with their needs in the allocation of treatments for patients Although the definition of the classes used for the ciasmcacion is Dasa in the chemical composition of the calculations, the way in which the samples are analyzed according to the present invention is not based on chemical parameters but on the visible characteristics (mainly the texture of the sample ). The advantages that the present invention offers over known techniques, according to its different embodiments, are:

 - The classification of the samples is automatic, because the chemical analysis is replaced by a visual analysis carried out by computer at once, avoiding the need for a qualified technician and, therefore, without depending on their aptitudes. - The sample is not destroyed, so it can be re-analyzed if necessary.

 - The spatial distribution of the components can be performed, which allows the history of renal calculus to be traced and used to prevent recurrence.

 - As far as the system is concerned, it is very robust.

 - The analysis is carried out in a few minutes during the same visit to the urologist.

 - The doctor receives the diagnosis and treatment proposed by the device and according to the type of kidney stone generated, thus facilitating the work of the urologist, which can be transmitted directly to the patient.

 - Low cost of the system, which can be amortized in a short time, since it can be implemented based on a camera, microprocessor and standard lighting components.

 - Low cost of the analysis, even in comparison with a chemical analysis, since it can be performed by the urologist himself and not by a technician trained in an infrared spectroscopy service.

 Therefore, the present invention is capable of providing the information required by the urologist, but using unconventional techniques that achieve an equal or improved result. In addition, recommendations on diet and treatment to be followed along with the classification are provided. These recommendations are useful for the urologist and their relationship with the specific kind of kidney stone not known to most of them.

The present invention has been developed, in its different aspects, by a team composed of experts in image analysis and experts in kidney stone analysis who are continually in contact with urologists who were constantly asked for advice on, for example, what Type of information they need and would expect. Brief description of the drawings

 The foregoing and other advantages and features will be more fully understood from the following detailed description of some examples of embodiment, some of which with reference to the attached drawing, which should be taken by way of illustration and not limitation, in which:

 Fig. 1 shows, schematically, the system proposed by the second aspect of the present invention, which is apt to implement the method of the first aspect.

Detailed description of some embodiments

 In this section a practical implementation of the present invention will be described, both as regards the method and the system, a prototype of which it has been manufactured and will be described later.

 First, the implementation of the method proposed by the first aspect of the invention is described, which includes the selection of the samples and the subsequent classification procedure.

 Terminology:

 - Sample: calculation or calculation fragments generated by a patient during an episode.

 - Fragment: Part of a kidney stone that has been obtained directly from the patient (after treatment with extracorporeal shock wave lithotripsy or ESWL) or after cutting an entire stone, in order to leave the discovered its internal part.

 - View: Part of the stone or fragment exposed to the chamber. There are two different types of sight, superficial and cutting; that is, the outer or inner part of a fragment or a stone.

 - Image: For each view the camera acquires different images, each one under a particular lighting source and exposure time.

 The samples were obtained from the Urology Service of the Hospital Universitari de

Bellvitge, Barcelona (Spain). The stones were expelled either naturally (so that the whole stone was received) or collected after breaking it by applying a treatment with extracorporeal lithotripsy by shock waves (fragments of the stone being received). For stones or non-fragmented stones (whole), these were cut with a surgical knife in order to reach the nucleus. When the sample contains fragments, both the inner and outer part of the stone is in general visible without the need for any manipulation (unless the fragment does not show the core of the stone, then it must be cut). After collection, the stone or fragments were rinsed with water and ethanol and then stored in clean, individual vials. Samples can thus be stored for years without visible signs of decomposition or damage to the structure.

 In order to create a database suitable for training the designed system, that is, a training set, the samples must be chosen carefully, that is, the samples cannot be chosen at random. For this, the samples were selected from a bank of 1300 samples by an expert (a qualified person) in the analysis and classification of kidney stones. Due to the remarkable variability within each class, the selection criterion was chosen in order to reflect this variation in the group of samples selected for each type of stone. The data set includes the stones that comprised all the possibilities for each class in the inner and outer part of the stones. These possibilities include the chemical composition (defined by infrared), the distribution of the components and the visual appearance of the sample (defined by morpho-constitutional analysis), both performed by a qualified specialist. The creation of this library or sample database is based on the experience of the present inventors in the study of the causes of stone formation and the classification of kidney stones. The results obtained with the method proposed by the present invention cannot be achieved if the stone data set is chosen at random, or if the selected samples do not cover the full range of possibilities for each kind of stone. This can only be developed by an expert in this field, as a result of experimental work on the classification of samples.

As stated earlier, the samples were chosen in order to cover all the different classes (which could be done from an analysis of the results obtained by infrared spectroscopy) and also the different, second-order classification (the which can only be done by a well trained person). The image acquisition procedure has been designed as follows. One or two fragments were selected from each selected sample so that both the inner and outer surfaces can be observed. Then, for each of the two surfaces or views, a series of 6 images were recorded varying the type of lighting source and the exposure time. The choice of the light source depends on the knowledge and experience gained after analyzing several spectra of kidney stones. The total number of samples selected was 346, from which 606 fragments were selected resulting in 1212 surface views indoor and outdoor (and the acquisition and registration of 6 images of each view). Therefore, a total of 7272 images were selected, all of them used for training and validation of a new hierarchical classification scheme, specifically designed for that purpose on three levels by two experts in image processing, which has already been explained in an earlier section for an embodiment of the method proposed by the present invention, but which will be described in more detail below and with reference to the experiment set forth herein. Classifier Scheme:

 First level: Classification of views.

 Using the set of 6 images of a view of a fragment, a class is determined for that view as well as the probability of each class for that view is estimated, almost always coinciding with the class determined with the most probable. This class is based entirely on the visual characteristics of size, shape, color and texture (a feature never used before). At this level it is also possible to take into account the pH level of the urine as another characteristic, if this is known. Although the measurement of chemical characteristics is not used in the present invention for classification, the output classes are comparable to those obtained by chemical analysis (COD, COM, STR ...) in order to facilitate a classification to the urologist known. The output of this classification is an estimate of the probability or belief that a fragment with such a view (external or internal) belongs to each of the previous classes. Therefore, the method and system developed by the present invention calculates a probability vector for each view of a fragment. From them you can easily infer also the most probable class for a given view, as a first approximation of the class of the fragment.

For the creation and training of this classifier, all possible species that appear naturally as main components in the stones were included. The types of stones contained in this analysis were defined according to the chemical composition (so they are comparable with IR spectroscopy), but were recognized according to visual characteristics. Calcium oxalate monohydrate (COM), calcium oxalate dihydrate (COD), calcium oxalate dihydrate monohydrate (TRA), brushite (BRU), apatite carbonate (CAP), struvite (STR), anhydrous uric acid (AUA) classes were selected , uric acid dihydrate (AUD), mixed calcium oxalate and apatite carbonate (MXD) stones. This chemical composition is the classification scheme shared with the infrared spectroscopy technique. Second level: Classification of fragments.

 Once for the two views of a fragment, one of these classes and also a probability distribution has been computed, a second classifier produces a class fragment from the results of outputs of the classifier of a first level. This is a second-order classification, a very useful information that can only be achieved by an expert in morpho-constitutional analysis and not by one in infrared spectroscopy.

 The classification of the fragments is done after both views (inner and outer parts) have been assigned to a class. The system is trained in the definition of a single specific class for the fragment based on the combination of the results for the individual views. Therefore, the classification of each fragment is not only limited to the chemical components present in the stone, but also to its location and distribution. For example, the class assigned to a fragment will be different if compound A is inside the stone and B on the surface, or if the situation is the opposite. The definition of these classes by fragments is based on the differences between the possible treatments administered to the patient. The relationship between the possible combinations for internal and external views is given below in Table 1, referring to the classification of fragments according to their internal (Interior) and external (Surface) views.

IDE interior

 COM COD TRA MXD HAP STR BRU AUA AUD CYS

COM 2 3t 3t 5b

 CYS 10

Table 1. Second level of classification

However, the system may be wrong, that is, an incorrect class is assigned to a fragment, and depending on the difference between the actual and the assigned class, the associated cost will be different (in the second classification level). In other words, not all errors that the system can make have the same influence on the treatment. To do so, the classes described in Table 1, on which the classification is based, can be reordered or assigned as described in Table 2. If such reordering is carried out, always according to the potential cost, which is also shown in Table 2 in the range of 0 to 10, 10 being associated with the highest error, the class system can be simplify, getting closer and closer to the first level of classification, based only on chemical composition, not on the distribution of components. That is, the stone classification algorithm can be very strict, but it has also been designed with an important flexibility component. This feature allows the method and system of the present invention to adapt its performance to the specific conditions that the user needs.

 Therefore, the values shown in Table 2 can be used as a percentage of probability of one class being assigned, instead, during the training process. For example, if a sample is known to be of class 2b, the cost if the assigned class (and learned) is 2ct is low, so the model will be freer to assign this other class to the fragment. However, if the same sample 2b is initially recognized as class 6, this decision will be affected and modified by the risk of assigning that class.

 As stated above, the criteria used for this transposition are shown in the cost matrix of Table 2, which is set out below. If classes are combined with a low risk of confusion, the classification is simplified to the first level of classification. Logically, the percentage accuracy of the classification increases as the subtypes of stones decrease (when different groups are combined), since errors usually occur between similar classes, which give a low value in the cost matrix.

Table 2. Cost matrix Third level: classification of the samples.

 Taking advantage of the fact that two fragments were selected for each stone, a third classifier has been designed in order to offer a final classification. If the most probable class of each of the two fragments both have a probability greater than a certain threshold, if they are the same class, this is the one assigned. If they are not but the two exceed this threshold, the method and system of the present invention has been designed to assign a single class to the sample, using the combination table as seen in Table 3, designed based on the experience and knowledge in the analysis of kidney stones. When either of the two classes does not exceed this probability threshold, it is classified based on the output results of the second level classifier.

 Ta la 3. It is the third level and classification. ragm.1 or 2 corresponds to the individual fragments. The resulting class is the final result assigned. The output or final result offered by the method and system of the present invention consists not only in the classification of kidney stones, but also in treatment recommendations for the physician. These recommendations depend directly on the type of kidney stone detected, related to the direct origin of the stone. The recommendations are linked to a strict classification system that has been defined specifically for this case. In the literature, some classification systems for kidney stones have been defined, but based on the experience of the present inventors these systems have been reconstructed and adapted to this specific problem and combined with the requirements of image analysis. In accordance with the present invention, the stones are classified according to a combination of classes defined in a table designed specifically for this purpose, which considers the outer part and the inner part of the stones. This form of kidney stone classification has never been used previously.

 The classification of images based purely on optical parameters is optionally complemented, as already indicated in a previous section, with additional information from the patient's medical history. Therefore, the rate of correctly classified samples is improved by considering some parameters such as urine pH, since this is directly related to the type of kidney stone formed. The inclusion of urine pH in the stone classification is another exclusive feature of the present invention, since no other methodology using said parameter has been described previously.

The present inventors have designed a prototype of the system proposed by the second aspect of the invention, specially configured to complement the method of the invention All components were chosen independently to meet both high performance and low cost criteria. The lighting configuration was chosen based on the experience of the present inventors in stone classification (thus the most useful energy ranges for this application were chosen) and image treatment (the physical configuration of the hardware was based on the experience in the image analysis and laboratory work, so a high number of lighting configurations were tested before selecting the most appropriate one).

 In Figure 2 the system proposed by the present invention is schematically illustrated, for an exemplary embodiment, for which the system comprises:

 - A lighting plate 3, with L LEDs of different wavelengths in the visible and near-infrared spectrum,

 - A control unit 5 (or local part of the electronic system, according to the terminology used in a previous section) to control the intensity and type of light emitted by the LEDs L of the lighting board 3, connected to an external computer 7 (or remote part of the electronic system, according to the terminology used in a previous section),

 - A digital camera 4, also connected to the external computer 7, in addition to a suitable optic 8, which includes an extension tube and a lens,

 - A housing 1 that defines an interior space insulated from the outside to insulate the sample S from ambient light,

 - An infrared sensor 6,

 - A stand or sample tray 2,

 - An internal support structure (not illustrated).

The images of the samples, which are used for the description of the characteristics of the stone necessary for classification (texture, shape and color), are taken using a conventional camera, equipped with a Silicon sensor. The lighting energies used are in the Visible range - Near infrared (400-1000 nm) within the sensitivity range of this sensor. The image information can be complemented with the use of other LEDs, which emit at specific wavelengths, allowing measurements of particular reflectance intensities. The illumination plate 3 has been specially designed so that the lenses allow to observe the renal calculus S, and the location of the LEDs L has been chosen to avoid shadows in the sample S. In addition, an infrared sensor 6 arranged in this plate 3 provides a signal of the reflected light to the computer 7. The operating parameters of the different types of lighting are controlled by the software. As regards the sample tray 2, in the prototype built this is a mobile sample tray 2 with a homogeneous bottom, designed for the placement of the different samples S in the field of view of the camera 4, and for optically distinguish the background stone fragment (segmentation). The sample tray 2 has been placed on a mobile platform, which serves for the best placement of the S stone for a simpler approach and lighting adaptation. For another embodiment, the sample tray is not mobile.

 A suitable support structure (not shown) has also been designed to support the system elements inside the housing 1. It keeps all the components in the defined position, while allowing the sample tray 2 to move as necessary, both to place the sample in the field of view of the camera and to adjust the focus of the image.

 The system operates from a computer 7 using special software designed for this purpose. This software controls the captured image, as well as the lighting conditions. In addition, an appropriate graphical user interface has been designed for stone image acquisition, classification and visualization of the result. This interface allows naming the sample and the collection and storage of several images, as well as the associated near-infrared data for each stone fragment, and also for more than one stone fragment for each sample (stones with ESWL origin generally consisted of several fragments). Once all the images and data of the stones of a given sample were recorded, they were classified using a set of supervised algorithms designed exclusively for the present invention. In addition to these images and IR information, other patient data, relevant for classification, such as age, sex and urine pH level can be entered. The results of the classification are given as the probability distribution for each class based on the degree of belief in terms of perception, illustrated, for example, by a diagram.

 A person skilled in the art could introduce changes and modifications in the described embodiments without departing from the scope of the invention as defined in the appended claims.

Claims

1.- Method for the automatic classification of kidney stones, which includes: a) acquiring an image of a kidney stone;  b) analyze information contained in said acquired image; Y  c) classify said renal calculus according to the result of said analysis; the method being characterized in that said analysis of said stage b) is carried out using computer vision techniques, where said information to be analyzed refers to the image characteristics associated with at least the texture of the kidney stone.  2. Method according to claim 1, characterized in that for the analysis of step b) computational learning techniques are also used.  3. - Method according to claim 1, characterized in that said acquired image is a digital image, and that the analysis of step b) is carried out once on the complete digital image acquired in stage a), without performing a individual classification of each pixel.  4. - Method according to claim 1, 2 or 3, characterized in that said information to be analyzed refers to associated image characteristics, in addition to texture, to other visual characteristics of renal calculus, including size and / or shape and / or color .  5. Method according to any one of the preceding claims, characterized in that said step a) comprises acquiring at least two images of a fragment of said renal calculus, one corresponding to a view of an outer surface and another to a view of an inner surface exposed, and analyze them in stage b).  6. Method according to claim 5, characterized in that if said kidney stone fragment has no exposed inner surface, the method comprises cutting it to expose said inner surface whose image is acquired in step a).  7. - Method according to claim 5 or 6, characterized in that it comprises acquiring in stage a) and analyzing in stage b), a plurality of images of corresponding views of each of said exterior and interior surfaces of said calculation fragment renal, each under different lighting conditions and / or exposure time. 8. - Method according to claim 7, characterized in that it comprises carrying out said acquisitions with the same image sensor sensitive to the wavelengths associated with all the illuminations included in said different lighting conditions, which are within the range from visible light to Near infrared, or with several image sensors sensitive, together, to all such different wavelengths.  9. - Method according to any one of claims 5 to 8, characterized in that it comprises carrying out said acquisition, in step a), and analysis, in step b), of images of said views of said exterior and interior surfaces for a plurality of fragments of the same sample.  10. - Method according to any one of claims 5 to 9, characterized in that said classification of said step c) based on the analysis of said images of said fragment or said fragments of renal calculus, is a classification of a first level, or classification of view, which includes, for each view, an estimate of the probability of belonging to a class of renal calculus associated with chemical composition, from the calculation of a vector of probabilities for each view that includes information on the spatial distribution of said odds  1. Method according to claim 10, characterized in that it comprises performing, in step c), a second level classification, or fragment classification, which comprises combining the results obtained in the classification of a first level for several views of the same renal calculus fragment, to determine a unique class for each fragment also based on chemical composition also on the location and distribution of the chemical components associated with said probabilities and to what view they correspond.  12. - Method according to claim 1, characterized in that it comprises correcting the result of said classification of a second level if the result for a given fragment differs from those obtained for each of the views thereof, in the classification of a first level , above a certain cost value.  13. Method according to claim 12, characterized in that said correction comprises reclassifying said fragment taking less into account, or not taking into account at all, the location and distribution of the chemical components associated with said probabilities and to what extent they correspond.  14. Method according to claim 11, 12 or 13, characterized in that it comprises performing said steps a), b) and c) for at least two fragments of the same sample, the method comprising performing, in step c), a classification of a third level, or sample classification, comprising, if the classification of a second level of said fragments, which are at least two, is not coincidental, assign a unique class for the sample. 15. Method according to claim 10, 1 or 14, characterized in that it comprises: - generate, prior to said stage a), a set of training, or learning, for a plurality of samples of kidney stones classified manually by an expert, including, in a correlated manner, information on chemical composition, spatial distribution and of visual appearance for internal and external views of different fragments of each sample represented in images obtained with different types of lighting and exposure times, and  - make said classifications of a first, a second and a third level by consulting the images acquired in said training set and extracting the class information correlated with the images more similar to those consulted.  16. Method according to claim 15, characterized in that it comprises training said training set and / or automatic classifiers used to perform at least said classifications of a first, a second and a third level using the results of the classifications.  17. Method according to any one of the preceding claims, characterized in that it comprises carrying out said classification of stage c) complementing the analysis of stage b) with additional information relative to the patient from whom the kidney stone and / or obtained from the renal calculus with non-camera based sensors.  18. Method according to claim 17, characterized in that said patient information includes at least one of the following information, or a combination thereof:  - simple data related to sex and / or age and / or race and / or complexion and / or body mass index and / or health disorders associated with renal lithiasis, and / or  - data linked to collateral analytics, including at least one of the following data collected in the patient's urine analysis: pH, calcium, oxalate, magnesium, ammonium and phosphate.  19. - Method according to claim 17 or 18, characterized in that said information obtained from the renal calculation with non-camera based sensors includes at least reflectivity information in other areas of the electromagnetic spectrum not included in said image acquired in a).  20. - System for the automatic classification of kidney stones, comprising: - means of acquiring images to acquire at least one image of a kidney stone; - an electronic system in connection with said image acquisition means and including processing means for processing information contained in said acquired image and for classifying said renal calculus according to the result of said analysis; the system being characterized in that it implements the method according to any one of the preceding claims, said process means complementing at least one algorithm based on computer vision techniques to perform steps b) and c) of the method.  21. - System according to claim 20, characterized in that said algorithm implementing in said processing means is also based on computational learning, to perform steps b) and c) of the method according to claim 2.  22. - System according to claim 21, characterized in that it comprises a housing (1) defining an interior space that is luminously insulated from the exterior that houses, supported and / or fixed in an internal support structure:  - to a support (2) for samples (S) of kidney stones;  - to lighting means arranged to illuminate, with light of one or more wavelengths, the sample (S) of kidney stones arranged on said support (2);  - to said image acquisition means, which include at least one image sensor (4) sensitive to said or said wavelengths; Y  - to at least part of said electronic system which also includes control means (5) for controlling at least the lighting means.  23. - System according to claim 22, characterized in that it also comprises one or more sensors (6) sensitive to a range of the electromagnetic spectrum different from that associated with said image acquisition means, arranged or arranged facing the sample (s) (S) of kidney stones arranged on said support (2), and in connection with the electronic system, to capture the overall reflectivity of the sample or samples (S) in a spectral range suitable for characterization. 24. - System according to claim 22 or 23, characterized in that said part of said electronic system housed within the housing (1) is a local part (5) and that the electronic system comprises a remote part (7) communicated bi-directionally with said local part (5) and with the means of image acquisition. 25. - Computer program that includes code instructions that, when executed on a computer, implements steps b) and c) of the method according to any one of claims 1 to 19.  26. - Computer program product comprising the computer program of claim 25.