WO2025214818A1

WO2025214818A1 - Methods and devices for evaluating an image

Info

Publication number: WO2025214818A1
Application number: PCT/EP2025/058831
Authority: WO
Inventors: Max Berg; Thomas KUENSTING; Magdalena ZIMON
Original assignee: Roche Diabetes Care GmbH
Current assignee: Roche Diabetes Care GmbH
Priority date: 2024-04-09
Filing date: 2025-04-01
Publication date: 2025-10-16
Anticipated expiration: 2026-10-09

Abstract

The present invention relates to a computer-implemented method of obtaining an item of information on a skin condition of a user, the method comprising: i. receiving at least one image (162) of a portion of a skin of a user, wherein the image (162) has been captured by at least one camera (144) of a mobile device (132), the image (162) comprising at least one portion of the skin of the user showing the at least one skin condition; ii. obtaining at least one item of information on a skin morphology of at least one skin feature (166) comprised by the at least one portion of the skin of the user showing the at least one skin condition by evaluating the received at least one image (162) by using at least one image (162) processing algorithm, preferably wherein obtaining the at least one item of information on the skin morphology of the at least one skin feature (166) comprises comparing the at least one item of information on a skin morphology of the at least one skin feature (166) to at least one reference color (149); iii. obtaining the at least one item of information on the skin condition of the user by evaluating the at least one item of information on the skin morphology of the at least one skin feature (166) by using at least one classification algorithm.

Description

Methods and devices for evaluating an image

Technical Field

The present invention relates to a computer-implemented method of obtaining an item of information on a skin condition of a user, a mobile device having at least one camera, a computer program, a computer-readable storage medium and a system.

Background art

The evaluation of skin conditions usually starts with a visual inspection. Many characteristics have been defined, such as primary lesions like macules, vesicles, etc., and secondary lesions like scales, erosions, etc. The evaluation of these skin conditions together with changes in color, local grouping and distribution across the body are, typically, used in the diagnosis of skin reactions.

Since optical images are a classic field of computer-based classification and segregation as well as machine learning, dermatological images have been of high interest in the past and still are. Due to the high relevance and complex task, many researchers focus on distinguishing cancer from non-malignant changes. Image acquisition using mobile devices and (partial) processing via cloud-based services has been described. Sometimes, non-RGB color spaces are used to improve the definition of color, but most work focuses on proper classification or learning algorithms. For the evaluation of non-malignant skin changes, the diagnostic approaches seem less heavily computerized. E.g. for the evaluation of skin lesions following the use of medical devices, a simple questionnaire has been proposed as an improvement to the current methodology.

Freckmann G, Buck S, Waldenmaier D, et al., Skin Reaction Report Form: Development and Design of a Standardized Report Form for Skin Reactions Due to Medical Devices for When not using dedicated dermatoscopic equipment, the varying relative ambient light settings and the relative skin color can influence the determination of skin conditions. Also, for larger lesions like or user skin conditions, hand-held dermatoscopes are only of limited use.

Utilizing digital information on skin topography has been described but is rather unusual and much less accessible than image information. Ali A-R, Li J, O’Shea SJ describe in Towards the automatic detection of skin lesion shape asymmetry, color variegation and diameter in dermoscopic images, 2020, PLoS ONE 15(6): e0234352, https://doi.org/10.1371/journal.pone.0234352 that asymmetry, color variegation and diameter are considered strong indicators of malignant melanoma. The subjectivity inherent in the first two features and the fact that 10% of melanomas tend to be missed in the early diagnosis due to having a diameter less than 6 mm, deem it necessary to develop an objective computer vision system to evaluate these criteria and aid in the early detection of melanoma which could eventually lead to a higher 5-year survival rate.

C. Doukas, P. Stagkopoulos, C. T. Kiranoudis and I. Maglogiannis describe in Automated skin lesion assessment using mobile technologies and cloud platforms, 2012, Annual International Conference of the IEEE Engineering in Medicine and Biology Society, San Diego, CA, USA, pp. 2444-2447, doi: 10.1109/EMBC.2012.6346458 a smart phone based system for storing digital images of skin areas depicting regions of interest (lesions) and performing self-assessment of these skin lesions within these areas. The system consists of a mobile application that can acquire and identify moles in skin images and classify them according their severity into melanoma, nevus and benign lesions. The system includes also a cloud infrastructure exploiting computational and storage resources. This cloud-based architecture provides interoperability and support of various mobile environments as well as flexibility in enhancing the classification model.

Freckmann G, Buck S, Waldenmaier D, et al. describe in Skin Reaction Report Form: Development and Design of a Standardized Report Form for Skin Reactions Due to Medical Devices for Diabetes Management, 2021, Journal of Diabetes Science and Technology, 15(4):801-806, doi: 10.1177/1932296820911105 that skin reactions due to medical devices for diabetes management have become a common problem in diabetes technology. There is a varying degree in how detailed skin reactions are described in scientific literature and diabetes practice, and no uniform structured documentation is given.

Mahbod A, Tschandl P, Langs G, Ecker R, Ellinger I. describe in The effects of skin lesion segmentation on the performance of dermatoscopic image classification, 2020, Comput Methods Programs Biomed, 197: 105725, doi: 10.1016/j.cmpb.2020.105725 that malignant melanoma (MM) is one of the deadliest types of skin cancer. Analysing dermatoscopic images plays an important role in the early detection of MM and other pigmented skin lesions. Among different computer-based methods, deep learning-based approaches and in particular convolutional neural networks have shown excellent classification and segmentation performances for dermatoscopic skin lesion images. These models can be trained end-to-end without requiring any hand-crafted features.

US11605243B2 discloses a method of determining a cosmetic skin attribute of a person. The method includes obtaining a color channel image of a person's skin, analyzing the color channel image with a computer using entropy statistics to obtain an entropy value, and then determining a cosmetic skin attribute for the person based on the entropy value.

US11298072B2 discloses a system and method for the diagnosis of skin cancer, wherein visual data is acquired from a skin lesion by a passive or active optical, electronical, thermal or mechanical method, processing the acquired visual field image into a classifier, applying a dermoscopic classification analysis to the image and converting, by sonification techniques, the data to raw audio signals, wherein the raw audio signals are analyzed with a second machine learning algorithm to increase interpretability and the precision of diagnosis.

US11176669B2 discloses a system for remote medical imaging which uses conventional mobile devices (such as two smart phones) and/or augmented reality to calibrate attributes such as the size, color, and shape of a wound, injury, skin lesion, and/or tissue abnormality on a person's body. One of the mobile devices is placed near the wound, injury, skin lesion, and/or tissue abnormality and acts as a digital fiducial marker or color key to help calibrate the size, color, and/or shape of the wound, injury, skin lesion, and/or tissue abnormality.

US11003048B1 discloses an apparatus for obtaining polarized imagery using a portable device, comprising a light source, a first polarizing filter configured to cover the light source, a second polarizing filter configured to cover the lens of a mobile device's built-in camera, one or more housing to contain the light source and filters, and couplers attached to the housings for coupling the housings to the mobile device. When the housing is coupled to the mobile device and the camera is operated, light from the light source passes through the first polarizing filter, then passes as polarized light through an illuminating path to illuminate an object being imaged. Then light from the illuminated object passes through an optical path, through the second polarizing filter to the camera lens. In embodiments, the axis of polarization of at least one of the polarizing filters can be modified to allow for cross, parallel and variable polarized imaging. US10943366B2 discloses a digital image that is captured. The captured digital image includes a calibration pattern. The calibration pattern includes displayed information about the calibration pattern. The displayed information is read to obtain calibration information about the captured digital image.

US10905370B2 discloses an allergy skin test device that can include a support strip having a top and bottom face. An allergen can be applied to an area of the bottom face forming an allergen portion. A skin viewing portion can be located proximate or at the allergen portion. An adhesive can be applied to an area of the bottom face forming an adhesion portion. When applied to the skin of a subject, the allergy skin test device can facilitate viewing of a portion of the skin beneath the allergy skin test device through the skin viewing portion. A method of testing a subject for an allergy can include applying an allergy skin test device to the skin of the subject, recording two or more images of the skin of the subject, and electronically comparing the images.

US10438356B2 discloses systems and methods for determining bacterial load in targets and tracking changes in bacterial load of targets over time. An autofluorescence detection and collection device includes a light source configured to directly illuminate at least a portion of a target and an area around the target with excitation light causing at least one biomarker in the illuminated target to fluoresce. Bacterial autofluorescence data regarding the illuminated portion of the target and the area around the target is collected and analyzed to determine bacterial load of the illuminated portion of the target and area around the target. The autofluorescence data may be analyzed using pixel intensity. Changes in bacterial load of the target over time may be tracked. The target may be a wound in tissue.

US10219736B2 discloses reference imagery of dermatological conditions that is compiled in a crowd-sourced database (contributed by clinicians and/or the lay public), together with associated diagnosis information. A user later submits a query image to the system (e.g., captured with a smartphone). Image-based derivatives for the query image are determined (e.g., color histograms, FFT-based metrics, etc.), and are compared against similar derivatives computed from the reference imagery. This comparison identifies diseases that are not consistent with the query image, and such information is reported to the user. Depending on the size of the database, and the specificity of the data, 90% or more of candidate conditions may be effectively ruled-out, possibly sparing the user from expensive and painful biopsy procedures, and granting some peace of mind (e.g., knowledge that an emerging pattern of small lesions on a forearm is probably not caused by shingles, bedbugs, malaria or AIDS). A great number of other features and arrangements are also detailed.

US2022215545A1 discloses a non-transitory computer readable medium storing data and computer implementable instructions that, when executed by at least one processor, cause the at least one processor to perform operations for generating cross section views of a wound, the operations including receiving 3D information of a wound based on information captured using an image sensor associated with an image plane substantially parallel to the wound; generating a cross section view of the wound by analyzing the 3D information; and providing data configured to cause a presentation of the generated cross section view of the wound.

US 2020327670A1 discloses a system for remote medical imaging which uses conventional mobile devices (such as two smart phones) and/or augmented reality to calibrate attributes such as the size, color, and shape of a wound, injury, skin lesion, and/or tissue abnormality on a person's body. One of the mobile devices is placed near the wound, injury, skin lesion, and/or tissue abnormality and acts as a digital fiducial marker or color key to help calibrate the size, color, and/or shape of the wound, injury, skin lesion, and/or tissue abnormality.

US20130322711A1 discloses a method, performed by a mobile communication device, including obtaining one or more images, extracting one or more features from the one or more images, and determining a dermatological classification for the obtained one or more images based on the extracted one or more images and based on a dermatological analysis model. The method may further include determining a recommendation based on the determined dermatological classification and providing information about the determined dermatological classification along with the recommendation to a user of the mobile communication device.

US2022005195A1 discloses digital imaging systems and methods for determining a userspecific skin irritation value of a user's skin after removing hair. An example method may be performed by one or more processors and may include aggregating training images comprising pixel data of skin of individuals after removing hair. A skin irritation model may be trained using the training images to output skin irritation values associated with a degree of skin irritation from least to most irritation. The method may include receiving an image of a user including pixel data of the user's skin after hair is removed from the skin, analyzing the image using the skin irritation model to determine a user-specific skin irritation value, generating a user-specific recommendation designed to address a feature identifiable within the pixel data of the user's skin, and rendering the recommendation on a display screen of a user computing device.

US2020302608A1 discloses systems and methods for clinical trial assessment of skin disease treatment. The disclosure includes obtaining a series of digital images over a period of time, wherein each digital image includes an affected area of the subject; identifying characteristic morphologies and lesions in the affected area of the subject in each of the digital images; classifying each of the detected and segmented morphologies and lesions into one or more identified categories for each of the digital images; assigning a global score to each of the digital images based on a count of the detected and segmented characteristic morphologies and lesions in each of the one or more identified categories; analyzing the global scores of each of the digital images; and making an assessment of the clinical trial based on the analysis of the global scores of each of the digital images.

US20210118133A1 discloses a system, method, apparatus and computer program product for the detection and classification of different types of skin lesions that leverages artificial intelligence (Al). SkinScreen® uses a novel approach that we have labeled as ‘serial chain classifiers’. This approach uses a binary classifier, to determine whether a skin lesion is present in the image, then if a lesion is present uses a multi-class classifier to classify the type of skin lesion. This approach removes manual human intervention in the process that is employed by current solutions while improving the accuracy and precision of the results. Using novel techniques of image transformation, the datasets used to train the Al models were expanded by a factor of 8. The larger the dataset, the more accurate and precise the results. These novel approaches have resulted in a better screening detection tool.

US20190392953A1 discloses a computerized system and method for skin lesion diagnosis. The user enters one or more images of the lesion and answers an online questionnaire. Using image analysis techniques, information from the questionnaire, information from the user profile and optionally additional external information, the system makes a diagnosis of the lesion.

WO2022144865A1 discloses systems, methods, devices, and computer readable media for capturing and analyzing images for medical examination, for generating cross section views of wounds, for analyzing wounds using standard equipment, for generating visual time series views of wounds, for rearranging and selecting frames of a medical video, for providing wound capturing guidance, for selective reaction to a failure to successfully complete a medical action using a medical image capturing application, and for displaying an overlay on wounds.

Problem to be solved

It is therefore desirable to provide devices and methods, which at least partially address the above-mentioned challenges.

It may be desirable to provide devices and methods, which are more robust against the effects of ambient light conditions on the captured images of skin conditions.

Further, it may be desirable to provide devices and methods, which may set the basis for an improved and more accurate diagnostics of skin conditions.

Further, it may be desirable to provide devices and methods, which allow a determination of skin conditions that is supported by data exchange with a cloud server or a health care professional, wherein the exchanged data requires only a small amount of data.

Further, it may be desirable to provide devices and methods, which improve the quality of the derived data.

Summary

This problem is addressed by the computer-implemented method of obtaining an item of information on the skin condition of the user, the mobile device having the at least one camera, the computer program, the computer-readable storage medium and the system with the features of the independent claims. Advantageous embodiments which might be realized in an isolated fashion or in any arbitrary combinations are listed in the dependent claims as well as throughout the specification.

In a first aspect, a computer-implemented method of obtaining an item of information on a skin condition of a user is disclosed. For this aspect, reference may be made to any definition, Embodiment and/or further aspect as disclosed elsewhere herein. The computer-implemented method of obtaining an item of information on a skin condition of a user comprises the following steps, which may be performed in the given order. A different order, however, may also be feasible. Further, two or more of the method steps may be performed simultaneously. Thereby the method steps may at least partly overlap in time. Further, the method steps may be performed once or repeatedly. Thus, one or more or even all of the method steps may be performed once or repeatedly. The method may comprise additional method steps, which are not listed herein.

The term "computer-implemented" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a method which is performed by using computer programming, and/or by using at least one computer and/or at least one computer network. Thus, as an example, one or more or even all of the method steps may be performed by appropriate software, e.g. by using computer- readable instructions which, when executed on a computer or a computer network, cause the computer or computer network to perform the method steps. The term “software” as used herein may, specifically, refer to a computer program. The computer and/or computer network may comprise at least one processor, which is configured for performing at least one, more than one or all of the method steps of the method according to the present invention. Specifically, each of the method steps is performed by the computer and/or computer network. The method may be performed completely automatically, specifically without user interaction.

The method comprises, particularly the following steps: i. receiving at least one image of a portion of a skin of a user, wherein the image has been captured by at least one camera of a mobile device, the image comprising at least one portion of the skin of the user showing the at least one skin condition; ii. obtaining at least one item of information on a skin morphology of at least one skin feature comprised by the at least one portion of the skin of the user showing the at least one skin condition by evaluating the received at least one image by using at least one image processing algorithm, preferably wherein obtaining the at least one item of information on the skin morphology of the at least one skin feature comprises comparing the at least one item of information on a skin morphology of the at least one skin feature to at least one reference color; iii. obtaining the at least one item of information on the skin condition of the user by evaluating the at least one item of information on the skin morphology of the at least one skin feature by using at least one classification algorithm.

As already disclosed, the method comprises receiving at least one image of a portion of a skin of a user, wherein the image has been captured by at least one camera of a mobile device, the image comprising at least one portion of the skin of the user showing the at least one skin condition.

The term “receiving” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to the process of getting access to and/or possession of data, such as an image, specifically a receiving device may get access to and/or possession of the element. The data may be, specifically, the image. The data may be provided by a providing device, such as the camera, configured for allowing access to the data. For allowing access to the data, the providing device may exchange the data with the receiving device, such as by sending or transferring the data directly or via a further device. The respective data may be exchanged via network, such as the internet.

For receiving the data, the data may be requested by the receiving device, such as by sending a query to the providing device. Receiving the data may comprise requesting the data. For receiving the data, the data may be generated in a process, such as a measurement process in the physical world. Receiving may comprise the process of obtaining and/or generating the data. Exemplarily, receiving the analytical measurement data may comprise the process of obtaining and/or generating the analytical measurement data, such as by performing the analytical measurement. Performing the analytical measurement may be part of the step of receiving the analytical measurement data in a manner that the measurement is actively performed by the computer-implemented analytical method.

The term “camera” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a device having at least one imaging element configured for recording or capturing spatially resolved one- dimensional, two-dimensional or even three-dimensional optical data or information. As an example, the camera may comprise at least one camera chip, such as at least one CCD chip and/or at least one CMOS chip configured for recording images.

The camera, besides the at least one camera chip or imaging chip, may comprise further elements, such as one or more optical elements, e.g. one or more lenses. As an example, the camera may be a fix-focus camera, having at least one lens which is fixedly adjusted with respect to the camera. Alternatively, however, the camera may also comprise one or more variable lenses which may be adjusted, automatically or manually. The invention specifically shall be applicable to cameras as usually used in mobile applications, such as notebook computers, tablets or, specifically, cell phones such as smartphones. Thus, specifically, the camera may be part of the mobile device which, besides the at least one camera, comprises one or more data processing devices such as one or more data processors. Other cameras, however, are feasible.

As used herein, without limitation, the term “image” specifically may relate to data recorded by using the camera, such as a plurality of electronic readings from the imaging device, such as the pixels of the camera chip. Consequently, the term “image” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a collection of information representing visual content in a digital format. Typically, image may comprise one or more pixels. The one or more pixels may be arranged in a predetermined manner, particularly a predetermined grid. Any one of the one or more pixels may comprise at least one numerical value defining a color and/or an intensity of the respective pixel. The numerical value may be binary data and/or data of a known color model, such as RGB (Red, Green, Blue) or CMYK (Cyan, Magenta, Yellow, Black). Alternatively or in addition, the image data may be vector image data. The vector image data may be at least one of: one or more points, one or more lines, one or more curves, and one or more further geometric elements.

The term “capturing at least one image” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to one or more of imaging, image recording, image acquisition, image capturing. The term “capturing at least one image” may comprise recording a single image and/or a plurality of images such as a sequence of images. For example, the recording of the image may comprise recording continuously a sequence of images such as a video or a movie. The recording of the at least one image may be initiated by a user action or may automatically be initiated, e.g. once the presence of the at least one object within a field of view and/or within a predetermined sector of the field of view of the camera is automatically detected. These automatic image acquisition techniques are known e.g. in the field of automatic bar-code readers, such as from automatic barcode reading apps. The recording of the images may take place, as an example, by acquiring a stream or “live stream” of images with the camera, wherein one or more of the images, automatically or by user interaction such as pushing a button, are stored and used as the at least one first image or the at least one second image, respectively. The image acquisition may be supported by the processor of the mobile device.

The term "skin" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an organ of a human or an animal forming the outer tissue of a body of the human or the animal.

The term "skin condition" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to at least one characterization or description of at least one observable deviation from at least one physiological norm of the skin, particularly referring to an appearance and/or a texture of the skin. The at least one observable deviation may be or may comprise the at least one observable deviation on the surface of the skin. The observable deviation of the at least one portion of the skin may be a change of the skin in respect to a healthy skin. The skin condition may be a characterization of the skin morphology, particularly in terms of the appearance and/or the texture of the at least one portion of the skin.

By obtaining the at least one item of information on the skin condition of the user, the at least one skin feature may be assigned to one or more descriptive classes. The term "skin condition" may, consequently, refer to a characteristic of the skin that is related to the at least one skin feature that is categorized into one or more descriptive classes. This categorization process may involve evaluating at least one attribute of the at least one skin feature; such as a color characteristic; a spatial characteristic, a color intensity characteristic; a topographical characteristic. The observable deviation may be observable at a specific and localized skin portion of the user. The deviation may be determined in respect to at least a portion of the surrounding skin, which may be or may comprise the reference portion of the skin of the user. The skin condition may not specify the underlying cause of the observable deviation, particularly in contrast to a medical diagnosis of the skin. The term "skin condition" does consequently not refer to or include a medical diagnosis.

Consequently, the “item of information on the skin condition” may be or may comprise information on the observable deviation of at least one portion of the skin. The item of information on the skin condition may be or may comprise a characterization of the appearance and/or the texture of the skin. The item of information on the skin condition may be or may comprise a non-diagnostic characterization of, particularly the appearance and/or the texture, of the skin. Consequently, the item of information on the skin condition may be free of information on a medical diagnosis.

The at least one item of information on the skin condition of the user may comprise or may be at least one item of information on a skin lesion of the user, particularly wherein the at least one item of information on the skin lesion of the user comprises at least one item of information on at least one of a primary lesion and a secondary lesion. The at least one item of information on the skin condition of the user may comprise or may be at least one item of information on a skin appendage, such as a hair or a nail.

The term "skin lesion" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to at least one specific characterization of the observable condition of the at least one portion of the skin of the user known from the medicine branch dermatology. The characterization may comprise classifying at least one property of the observable condition in a regular term used in the medicine branch dermatology.

A skin lesion may be characterized by at least one spatial property, specifically at least one topographical property, e.g. a skin lesion may be flat, elevated above the plane of the skin or depressed below the plane of the skin. Alternatively, or in addition, a skin lesion may be characterized by at least one color property, e.g. a skin lesion may comprise at least one area being at least one of skin-colored, red, pink, violet, brown, black, grey, blue, orange, yellow or otherwise pigmented. The term "primary skin lesion" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a skin lesion originating on a previously healthy skin being directly caused by a disease and/or reaction on the skin. The primary skin lesion may not have been altered, particularly by at least one of: a natural progression; scratching; an infection; a treatment; a healing. The at least one item of information on a primary lesion may refer to least one of: a bulla; a burrow; a cyst; a macula; a nodule; a papule; a patch; a plaque; a pustule; a telangiectasia; a tumor; a vesicle; a welt; a wheal.

The term "secondary skin lesion" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a skin lesion originating from a progression of a primary skin lesion. The progression may be caused by at least one of: a natural progression; scratching; an infection; a treatment; an healing. The at least one item of information on a secondary lesion may refer to least one of: an atrophy; a callus; a crust; a dander; an erosion; an excoriation; a fissure; an induration; a lichenification; a maceration; a phyma; an umbilication; an ulcer; a scale; a scar.

As already disclosed, the method comprises obtaining at least one item of information on a skin morphology of at least one skin feature comprised by the at least one portion of the skin of the user showing the at least one skin condition by evaluating the received at least one image by using at least one image processing algorithm, preferably wherein obtaining the at least one item of information on the skin morphology of the at least one skin feature comprises comparing the at least one item of information on a skin morphology of the at least one skin feature to at least one reference color.

The term "skin feature" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an arbitrary distinct and observable localized attribute of the skin, specifically of the surface of the skin. The skin feature may be determined by evaluating the image and/or the topographical image.

The skin feature may also be, may comprise, may be related to or may be associated with a skin appendage. The term "skin appendage" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an arbitrary skin-associated structure such as hair or nails.

The term "skin morphology" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a description of the appearance and/or structure of the at least one skin feature. The skin morphology may be determined by evaluating the image and/or the topographical image.

The at least one item of information on the skin morphology of the at least one skin feature may refer to least one of: at least one size; at least one shape; at least one texture; at least one topography; at least one color; at least one relative position; particularly wherein the relative position is between at least one of: a first skin feature of the at least one skin feature and a second skin feature of the at least one skin feature; the at least one skin feature and at least one landmark.

The term "landmark" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an arbitrary distinct and observable feature or attribute related to the user and/or on the skin of the user and comprised by the image and/or the topographical image. The landmark may be evaluated in order to derive a relative position of the at least one skin feature or body structure, particularly in respect to the landmark. The absolute position of the landmark may maintain over time and, thereby, be consistent. The at least one landmark may be selected from at least one of: an eye; a nose; a mouth; an ear; a birthmark; a scar; a belly button; one or more chloasma; a body landmark marking a bony structure, such as a tibia, a kneecap, an elbow; a body landmark marking a soft tissue, such as a palmar crease; an anatomical line, such as an axillary line, a parasternal line.

Alternatively or in addition, the at least one item of information on the skin morphology of the at least one skin feature may refer to least one of: at least one distribution of a plurality of skin features of the at least one skin feature; at least one type of at least a portion of an edge of the skin feature; at least one content of the at least one skin feature comprising at least one bodily fluid, such as serum; or blood. The term "type of at least a portion of an edge of the skin feature" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to at least one a description of at least one attribute of at least a portion of an outer boundary or an outer border of the at least one skin feature. The attribute may be selected from at least one of: a color; a color gradient; a topography or the like. The type of at least a portion of an edge of the skin feature may be selected from at least one of: a sharp edge.

The term "image processing algorithm" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to at least one computer-implemented procedure configured for determining at least one item of information, such as the item of information on a skin morphology of the at least one skin feature. The image processing algorithm may be configured for determining the item of information on the skin morphology by evaluating the image and/or the topographical image. The image processing algorithm may be configured for determining an intensity gradient. The image processing algorithm may be a similarity classifier of sub-regions-of- interest using at least one of: a deterministic algorithm; a machine learning algorithm. Alternatively or in addition, the image processing algorithm may be a canny edge detection algorithm using at least one of: a deterministic algorithm; a machine learning algorithm; a sobel filter.

Obtaining the at least one item of information on the skin morphology of the at least one skin feature by using the at least one image processing algorithm may comprise searching for at least one predetermined skin feature characteristic in the received at least one image. The at least one predetermined skin feature characteristic may be selected in accordance with at least one characteristic required to obtain the item of information on the skin morphology of the at least one skin feature. The at least one predetermined skin feature characteristic may be selected in accordance with a predetermined characteristic of a known skin lesion. Particularly thereby, the image processing algorithm may directly search for known skin lesions.

The term "predetermined skin feature characteristic" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to at least one predetermined criteria for which the image processing algorithm searches within the image and/or the topographical image in order to determine the at least one item of information on the skin morphology. The predetermined skin feature characteristic may be at least one of an area showing a color within a specific color range. The predetermined skin feature characteristic may be preselected in a manner that the predetermined skin feature characteristic is known before the method is being performed. The predetermined skin feature characteristic may be adjusted to the at least one reference color, particularly by evaluating the at least one reference color.

The at least one item of information on the skin morphology of the at least one skin feature may be obtained by the image processing algorithm by at least one of performing edge detection; performing a color space analysis; performing a color histogram analysis; performing image thresholding; performing template matching; performing image segmentation.

The term "reference color" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to at least one specific color configured for providing information on at least one true value of a further color comprised the image. The reference color may be at least one predetermined color or at least one predetermined color range. The reference color may comprise at least one of at least one predetermined color value; at least one predetermined gray level. In the context of the preceding disclosure, the term “reference color” may include white and black or gradations thereof. The at least one reference color may be evaluated in order to compensate an influence on at least one color of the image particularly related to or of the skin feature. The at least one color that is shown in the image may be influenced by an ambient light condition prevailing when the at least one image of the skin is captured. Alternatively or in addition, the at least one reference color may be evaluated in order to adjust the at least one predetermined skin feature characteristic searched for in the received at least one image.

The at least one reference color may be obtained by evaluating at least one item of information on at least one reference portion of the skin of the user showing the at least one reference color. The term "reference portion of the skin" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a specific portion of the skin of the user that is evaluated in order to determine that at least one reference color. The reference portion may further be evaluated in order to determine the observable deviation of the portion of the skin of the user. The reference portion of the skin may be a healthy or intact portion of the skin of the user. The reference portion of the skin may be free of the at least one skin feature related to the at least one skin condition.

The at least one item of information on the at least one reference portion of the skin of the user may be obtained by evaluating at least one image of a portion of the skin of the user showing the at least one reference portion, particularly wherein the at least one image of the portion of the skin of the user received in step i. further shows the at least one reference portion and is used as the at least one image of the portion of the skin of the user showing the at least one reference portion.

Alternatively or in addition, the at least one reference color may be obtained by evaluating at least one item of information on at least one portion of a color reference card showing the at least one reference color. The term “color reference card” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an arbitrary object comprising one or more color reference fields having the at least one reference color. The color reference card may have an arbitrary shape, such as a flat and cylindrical shape, e.g. a coin shape.

The term “color reference field” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a locally distributed area of the color reference card having the reference color. The color reference card may further comprise at least two color reference fields having different reference colors. Specifically, the color reference card may comprise a plurality of color reference fields having different reference colors, wherein the reference colors of the color reference fields may be selected from a predetermined color subspace. Information of the color reference field and/or on the reference color may be derived from a server, such as a cloud server, via a network.

The at least one item of information on the at least one portion of the color reference card may be obtained by evaluating at least one image of at least a portion of the color reference card showing the at least one reference color, particularly wherein the at least one image of the portion of the skin of the user received in step i. further shows the at least one portion of the color reference card and is used as the at least one image of the at least one portion of the color reference card.

Alternatively or in addition, the at least one reference color may be obtained by evaluating at least one item of information on the at least one skin condition in a historic state showing the at least one reference color. The term “historic state” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a status that lies in the past. The historic state may lie further back in the past than the point in time in which the image that is received in step i. has been captured. A time range between the historic state and the point in time in which the image that is received in step i. may be larger then one hour; one day; one week; one month; one year.

The at least one item of information on the at least one skin condition in a historic state showing the at least one reference color may be obtained, by capturing at least one historic image comprising the at least one reference color. The historic image may comprise the at least one skin condition of the skin of the user, wherein the historic reference portion of the skin of the user comprised by the historic image may be evaluated in order to determine the at least one item of information on the at least one skin condition in a historic state.

Alternatively or in addition, the at least one reference color may be obtained by evaluating at least one item of information on a predetermined reference color, particularly wherein the predetermined reference color comprises information on a portion of a skin of a further person, specifically a further user, showing the at least one reference color. The "predetermined reference color" may be at least one characteristic color value of a healthy portion of the skin of the user and/or of the skin of a further person, specifically a further user. Alternatively or in addition, the "predetermined reference color" may be at least one characteristic color value of a portion of a skin having the skin condition, wherein the skin may be a skin of the user and/or the skin of the further person, specifically the further user. When the predetermined reference color may be at least one characteristic color value of the portion of the skin having the skin condition, the at least one skin condition may be known, such as known from a previous evaluation of the skin condition. The term "comparing the at least one item of information on a skin morphology of the at least one skin feature to at least one reference color" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an assessment of at least one similarity and of at least one differences between two or more items. Specifically, at least one color shown in the at least one image of the portion of the skin of the user may be compared to at least one reference color.

Comparing the at least one item of information on the skin morphology of the at least one skin feature to the at least one reference color may comprise classifying at least one color shown in the at least one image of the portion of the skin of the user in accordance with the at least one reference color. The term "classifying" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to assigning at least one true color value to at least one color shown in the at least one image. The true color value may be an accurate and/or correct color value, particularly in reference to the at least one reference color. The true color value may be a color value obtained by the camera under ideal and known conditions.

Classifying the at least one color shown in the at least one image of the portion of the skin of the user in accordance with the at least one reference color may comprise at least one of:

- setting at least one absolute color value of the at least one color shown in the at least one image of the portion of a skin of the user;

- setting at least one color difference value between the at least one color shown in the at least one image of the portion of a skin of the user and the at least one reference color.

Classifying may comprise compensating a deviation between the true color value and the at least one color shown in the at least one image. Classifying may be performed in order to set a color value of the at least one color shown in the at least one image of the portion of a skin of the user. Alternatively or in addition, classifying may be performed in order to set the predetermined skin feature characteristic to be searched for in the image. By classifying the at least one color shown in the at least one image of the portion of the skin of the user, an influence of at least one of: an ambient light condition prevailing when the image is captured; on the color shown in the image may be compensated. Alternatively or in addition, by classifying the at least one color shown in the at least one image of the portion of the skin of the user, an influence of a skin tone on the predetermined skin feature characteristic to be searched for in the image may be set.

The at least one image processing algorithm may be selected from or comprises at least one of: at least one morphological transformation algorithm; at least one image gradient algorithm; at least one Canny edge detection algorithm; at least one Hough transformation algorithm; at least one deep convolutional network.

As already disclosed, the method comprises obtaining the at least one item of information on the skin condition of the user by evaluating the at least one item of information on the skin morphology of the at least one skin feature by using at least one classification algorithm.

The term "classification algorithm" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to at least one computer-implemented procedure configured for determining at least one item of information, such as the at least one item of information on the skin condition, on by evaluating the at least one item of information on the skin morphology.

The at least one classification algorithm may be selected from or may comprise at least one of: a look-up-table; a logistic regression; a decision tree; a support vector machine, SVM; a ^-nearest neighbors algorithm; a random forest; a gradient boosting.

Obtaining the at least one item of information on the skin morphology of the at least one skin feature may comprise identifying at least one suspicious area, wherein the at least one suspicious area is suspicious of comprising the at least one skin feature.

The term "suspicious area" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a region on the skin that is identified as potentially containing at least one skin feature. The at least one suspicious area may have a shape that differs from the shape of the at least one feature. The at least one suspicious area may have a shape that differs from the shape of the at least one feature. The at least one suspicious area may have a rectangular shape. The shape of the at least one suspicious area may be larger than the shape of the potential at least one skin fea- ture. The at least one suspicious area may be flagged for further analysis due to characteristics that may indicate the presence of the at least one skin feature. The at least one suspicious area may not comprise the reference portion of the skin of the user.

The at least one suspicious area may be determined by searching in the received at least one image for a pattern in at least one of: at least one color or at least one color range, preferably at least one color channel, such as at least one RGB-channel; at least one color intensity, whereby the at least one reference color may be used in the search to calibrate at least one of: the color or the at least one color range; the at least one color intensity.

Typically, the at least one suspicious area may be determined by analyzing the received at least one image. In the analysis, it may be searched for at least one pattern in the received at least one image. Such a pattern may be a color intensity of a specific at least one color or at least one color range being above a threshold. For example, a red area on the skin of the user may be a suspicious area. The pattern used for searching for the at least one suspicious area may differ from a pattern used to search for the at least one item of information on the skin morphology in the at least one suspicious area.

The at least one reference color may be used to adjust the color settings of the received least one image. This may ensure that the colors in the least one image are interpreted consistently, regardless of variations in lighting and/or camera settings.

By comparing the at least one color or at least one color range to the at least one reference color, at least one threshold may be set for identifying the at least one suspicious areas. For example, if the at least one reference color is a known healthy skin tone, any deviation from the at least one reference color beyond a certain threshold might be flagged as suspicious.

The at least one reference color may also be used to calibrate the intensity of colors in the image. This may help in identifying areas where the color intensity of a specific color or range exceeds a predefined threshold, such as a red area indicating potential inflammation or irritation.

For searching in the received at least one image for the pattern, at least one of:

- at least one unsupervised machine learning method;

- at least one outlier detection method;

- at least one machine learning model. The at least one machine learning model may be a pre-trained machine learning model. The at least one machine learning model may be fine-tuned by using data that is collected in the field. The data that is collected in the field may be related to a specific user that is repeatedly performing the method. Thereby, computational load of the method may be reduced.

In case no at least one suspicious area may be determined, further performing the method may be stopped. In case a suspicious area may be determined, the at least one item of information on the skin morphology of the at least one skin feature may be obtained.

Obtaining the at least one item of information on the skin morphology of the at least one skin feature may comprises generating the at least one item of information on the skin morphology by evaluating exclusively the at least one suspicious area by using the at least one image processing algorithm.

The at least one image processing algorithm may use a further pattern for searching for the at least one item of information on the skin morphology. The further pattern may be in at least one of at least one color or at least one color range, preferably at least one color channel, such as at least one RGB-channel; at least one color intensity, whereby the at least one reference color may be used in the search to calibrate at least one of the color or the at least one color range; the at least one color intensity. The further pattern may differ from the pattern used to search for the at least one suspicious area. In addition, the results obtained when searching for the at least one suspicious area may be further evaluated to obtain the at least one item of information on the skin morphology.

The at least one item of information on the skin morphology may be or may comprise: at least one color; at least one size; at least one shape of the at least one skin feature.

The at least one item of information on the skin morphology further may be or may comprise: at least one topography; at least one color; at least one relative position; particularly wherein the relative position is between at least one of: a first skin feature of the at least one skin feature and a second skin feature of the at least one skin feature; the at least one skin feature and at least one landmark.

The at least one item of information on the skin morphology may be subject to a threshold. In case the at least one item of information on the skin morphology may be not exceed the threshold, further performing the method may be stopped. In case the at least one item of information on the skin morphology may be equal to or exceed the threshold the at least one item of information on the skin condition may be obtained.

Step i. may further comprise receiving at least one topographical image, wherein the topographical image comprises the at least one portion of the skin of the user showing the at least one skin condition. The term "topographical image" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to at least one image comprising information of the topography of at least one object shown in the image. The at least one object shown in the image may be the portion of the skin of the user and/or the reference portion of the skin of the user.

The topographical image may have been captured by at least one of:

- at least one light imaging, detection and ranging, LIDAR, sensor, of the mobile device;

- at least one multi-camera technique;

- at least one shutter-technique.

The term "multi-camera technique" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a method of operating a plurality of cameras in a manner that a topographical image of an object may be derive by evaluating the different viewing angles of the cameras onto the object. The plurality of cameras may be comprised by the mobile device.

The term "shutter-technique" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a method of operating a camera, a modulator and a light source in a manner that light emitted by the light source onto a measurement object is repeatedly turned off and on so that a topographical image may of the measurement object may be derive by evaluating at least one variation in the light from the measurement object. The camera, the modulator and the light source may be comprised by the mobile device.

Obtaining the at least one item of information on the skin morphology of the at least one skin feature in step ii. further may comprise adding information on the topography of the at least one skin condition to the at least one item of information on the skin morphology of the at least one skin feature by evaluating the topographical image comprising the at least one portion of the skin of the user showing the at least one skin condition. For obtaining the at least one item of information on the skin condition of the user, the at least one item of information on the skin morphology of the at least one skin feature may be evaluated. Evaluating the at least one item of information on the skin morphology of the at least one skin feature may comprise comparing the at least one item of information on the skin morphology of the at least one skin feature to at least one item of information on the skin morphology of the at least one healthy skin.

The term "topography" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a structure or elevation of an area, particularly of the surface of the skin having the skin condition. The term "information on the topography of the at least one skin condition" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to information on the surface elevation of the skin condition. The information on the topography of the at least one skin condition may be selected from at least one of: an even surface; a rough surface; a size; a structure.

Obtaining the at least one item of information on the skin morphology of the at least one skin feature in step ii. further may comprise adding information on a skin tone of the user by evaluating the at least one item of information on the at least one reference portion of the skin of the user. For obtaining the at least one item of information on the skin condition of the user, the information on a skin tone of the user may be evaluated.

The term "skin tone" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to at least one natural color of the human skin, particularly of the healthy human skin. The skin tone may be influenced by at least one of: genetics; exposure to sunlight, particularly an ultraviolet light; a skin contact with a chemical; uptake of at least one of: a food, a chemical and a pharmaceutical; a metabolic or an endocrine disorder; an external source of energy, such as a mechanical stress, a heat and/or a body temperature; an ambient temperature; an emotional state; an activity such as an exercise. The information on a skin tone of the user may be evaluated in order to compensate an influence of the skin tone on the at least one color of the skin feature. Alternatively or in addition, the information on a skin tone of the user may be evaluated in order to set the at least one predetermined skin feature characteristic searched for in the received at least one image.

The method may further comprise: iv. obtaining at least one item of information on an evolution of the skin condition of the user by comparing the item of information on the skin condition of the user to the at least one information on the at least one portion of the skin of the user showing the at least one skin condition in the historic state.

The term "item of information on an evolution of the skin condition" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to information on a progression of the at least one skin condition in time. The progression may be influenced by at least one of: natural progression; scratching; an infection; a treatment; an healing.

The method may further comprising:

- guiding the user in a manner that the at least one image fulfils at least one image capture condition by evaluating at least one preliminary image received from the at least one camera of a mobile device, wherein guiding the user comprises indicating at least one guidance signal to the user by using at least one signal device of the mobile device.

The term "guiding" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a process of leading or directing a person, such as the user, towards at least one specific goal. Guiding may comprise giving at least one instruction to the person. The instruction may comprise information on how to act in a manner that the specific goal will be realized. Guiding may comprise monitoring and evaluating at least one condition relevant for realizing the at least one specific goal.

The at least one image capture condition may be or may comprise at least one of: - the at least one image comprising the at least one portion of the skin of the user showing the at least one skin condition, particularly wherein the user is guided to arrange the at least one camera of a mobile device in a manner that the at least one image comprises the at least one portion of the skin of the user showing the at least one skin condition;

- the at least one image comprising the at least one further portion of the reference skin, particularly wherein the user is guided to arrange the at least one camera in a manner that the at least one image comprises the at least one further portion of the reference skin;

- the ambient light condition being set in a manner that the at least one portion of the skin of the user showing the at least one skin condition is visible in the at least one image, particularly wherein the user is guided to set up the ambient light condition in a manner that the ambient light condition being set in a manner that the at least one portion of the skin of the user showing the at least one skin condition is visible in the at least one image.

The at least one signal may be or may comprise at least one of:

- a visual signal, particularly wherein a display of the mobile device is used as the signal device;

- an audio signal, particularly wherein a speaker of the mobile device is used as the signal device;

- a haptic signal, particularly wherein a haptic signal device of the mobile device is used as the signal device.

The term “visual signal” may refer to a visual stimulus that is displayed to the user. The term “audio signal” may a stimulus that is perceptible by a sense of hearing of the user. The term “tactile signal” may refer to a stimulus that is perceived by a haptic sensation, for example haptic sensations such as, but not limited to, a vibration of the measurement device.

The at least one skin condition may be caused by at least one external exposure, particularly selected from at least one of: wearing a tape, particularly wherein the tape is configured for attaching a device, preferably for attaching a medical device, more preferably for attaching a medical device used in the field of diabetes treatment, to a body part of a user; wearing a plaster, particularly wherein the plaster is configured for attaching a device, preferably for attaching a medical device, more preferably for attaching a medical device used in the field of diabetes treatment, to a body part of a user; wearing an insulin pump; wearing a, optionally body -worn, continuous analyte monitoring device, specifically continuous glucose monitoring device; being exposed to an adhesive component, particularly comprised by at least one of: the tape; the plaster; being exposed to a leachable substance, preferably of a medical device, more preferably of a medical device used in the field of diabetes treatment. The at least one skin condition may be or may comprise at least one of a non-malignant skin change; a diagnostically non-relevant skin characteristic.

In a further aspect, a mobile device having at least one camera, and, optionally, at least one processor, is disclosed. The mobile device may be configured, specifically by software configuration, for performing the computer-implemented method of obtaining an item of information on a skin condition of a user as elsewhere disclosed herein. For details, options and definitions, reference may be made to any further aspect discussed elsewhere herein.

The term “mobile device” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a mobile electronics device, specifically a personal mobile device (PDA), more specifically to a mobile communication device such as a cell phone and/or a smartphone. Additionally or alternatively, the mobile device may also refer to a notebook, a tablet computer or another type of portable computer, such as a wearable, specifically smart glasses, having at least one camera. Alternatively or in addition, a mobile device, specifically a smartphone, having an external camera may be used. The external camera may be comprised by spectacles. The mobile device may have a direct internet access, particularly in a manner that the mobile device is free of being required to connect to a network, such as a wireless lan network, to the internet. Thus, generally, the mobile device may be selected from the group consisting of a cell phone having at least one camera, specifically a smart phone; a portable computer having at least one camera, specifically at least one of a notebook and a tablet computer.

The mobile device may comprise a processor. The term “processor” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an arbitrary logic circuitry configured for performing basic operations of a computer or system, and/or, generally, to a device which is configured for performing calculations or logic operations. In particular, the processor may be configured for processing basic instructions that drive the computer or system. As an example, the processor may comprise at least one arithmetic logic unit (ALU), at least one floating-point unit (FPU), such as a math co-processor or a numeric co-processor, a plurality of registers, specifically registers configured for supplying operands to the ALU and storing results of operations, and a memory, such as an LI and L2 cache memory. In particular, the processor may be a multi-core processor. Specifically, the processor may be or may comprise a central processing unit (CPU). Additionally or alternatively, the processor may be or may comprise a microprocessor, thus specifically the processor’s elements may be contained in one single integrated circuitry (IC) chip. Additionally or alternatively, the processor may be or may comprise one or more application-specific integrated circuits (ASICs) and/or one or more field-programmable gate arrays (FPGAs) and/or one or more tensor processing unit (TPU) and/or one or more chip, such as a dedicated machine learning optimized chip, or the like. The processor specifically may be configured, such as by software programming.

In a further aspect, a computer program comprising instructions is disclosed, which, when the program is executed by the mobile device as elsewhere disclosed herein, cause the mobile device to perform the computer-implemented method of obtaining an item of information on a skin condition of a user as elsewhere disclosed herein. For details, options and definitions, reference may be made to any further aspect discussed elsewhere herein.

In a further aspect, a computer-readable storage medium, specifically a non-transient computer-readable storage medium, comprising instructions is disclosed, which, when the instructions are executed by the mobile device as elsewhere disclosed herein, cause the mobile device to perform the computer-implemented method of obtaining an item of information on a skin condition of a user as elsewhere disclosed herein. For details, options and definitions, reference may be made to any further aspect discussed elsewhere herein.

As used herein, the term “computer-readable storage medium” specifically may refer to non- transitory data storage means, such as a hardware storage medium having stored there-on computer-executable instructions. The computer-readable storage medium specifically may be or may comprise a storage medium such as a random-access memory (RAM) and/or a read-only memory (ROM). For details, options and definitions, reference may be made to any further aspect discussed elsewhere herein.

In a further aspect, a system comprising at least one mobile device having at least one camera as elsewhere disclosed herein and at least one color reference card comprising at least one reference color is disclosed. For details, options and definitions, reference may be made to any further aspect discussed elsewhere herein.

As used in the following, the terms “have”, “comprise” or “include” or any arbitrary grammatical variations thereof are used in a non-exclusive way. Thus, these terms may both refer to a situation in which, besides the feature introduced by these terms, no further features are present in the entity described in this context and to a situation in which one or more further features are present. As an example, the expressions “A has B”, “A comprises B” and “A includes B” may both refer to a situation in which, besides B, no other element is present in A (i.e. a situation in which A solely and exclusively consists of B) and to a situation in which, besides B, one or more further elements are present in entity A, such as element C, elements C and D or even further elements.

Further, it shall be noted that the terms “at least one”, “one or more” or similar expressions indicating that a feature or element may be present once or more than once typically will be used only once when introducing the respective feature or element. In the following, in most cases, when referring to the respective feature or element, the expressions “at least one” or “one or more” will not be repeated, non-withstanding the fact that the respective feature or element may be present once or more than once.

Further, as used in the following, the terms "preferably", "more preferably", "particularly", "more particularly", "specifically", "more specifically" or similar terms are used in conjunction with optional features, without restricting alternative possibilities. Thus, features introduced by these terms are optional features and are not intended to restrict the scope of the claims in any way. The invention may, as the skilled person will recognize, be performed by using alternative features. Similarly, features introduced by "in an embodiment of the invention" or similar expressions are intended to be optional features, without any restriction regarding alternative embodiments of the invention, without any restrictions regarding the scope of the invention and without any restriction regarding the possibility of combining the features introduced in such way with other optional or non-optional features of the invention.

The methods and devices according to the present invention may provide a large number of advantages compared with known methods and devices. Thus, by using the invention, the provided devices and methods are more robust against the effects of ambient light conditions on the captured images. The devices and methods according to the present invention provide an inherent digital solution, such as a standalone app, a software module or an Application Programming Interface, API, which can be incorporated into further applications that enable early manufacturer feedback for generating scientific data and/or development data.

User guidance and/or technical improvements in respect to devices and methods known from the prior art allow for layman usage of the devices and methods so that medial and technical issues can be easily recorded anytime, when they occur and the reporting is not lost or inaccurate until the next intervention of a health care professional HCP.

An improved classification of the skin condition, compared to the prior art, may be achieved by applying a physical-principles-guided image acquisition and processing approach, which compensates for a present ambient light condition prior to the classification instead of a ‘simple’ straight-forward machine learning classification approach. Additionally, a combination of optical images with topographical information from light imaging, detection and ranging, LIDAR, sensor also available on mobile devices further enriches the information content available for downward processing. An improved classification may further be achieved by making use of the large field of view of a smartphone camera, which allows analyzing intact and suspicious skin areas within the same recorded image.

By the use of devices and methods that may control and/or account for the prevailing ambient light conditions and by, additionally, referring to areas of the healthy and intact skin from the same user, higher quality data for the evaluation of skin lesions may be provided.

By using cameras of mobile devices, a determination of a skin condition of an unexperienced layman under ambulant conditions may be enabled.

A combination of the optical images with the topographical information from the LIDAR sensor may also be available when using a mobile device, such as a smartphone, which further enriches the information content available for downward processing.

A computer-program on a smartphone may be used by a user or patient for capturing images of skin reactions which arise from some external impact, such as e.g. wearing a plaster for a prolonged period of time, in particular a plaster of a continuous blood glucose monitoring, CGM, system. An image may be captured by the camera of at least one mobile device and comprises both unaffected and affected areas of the patient’s skin. When said image may be uploaded to a server, e.g. of a manufacturer of the CGM system, they may be analyzed, by comparing the unaffected and affected areas of the skin to each other, so that they can be reliably evaluated, particularly according to an internal categorization scheme.

As already disclosed, the devices and methods according to the present invention provide several advantages of the invention over the prior art. An advantage may be that a higher quality data for the evaluation of skin lesions may be obtained. A further advantage may be that a reliable classification of non-malignant skin changes, specifically as opposed to, typically, rather inaccurate descriptions provided by patients, e.g. via telephone when calling a hotline or a doctor. An image of irritated and intact skin areas, captured with a photo app by a patient, may be provided to a HCP and/or to manufacturer and/or to a developer for reliable feedback.

A further advantage may be that the description of a skin condition may be independent of the user and the environment, in particular of the ambient light of the environment. The data related to the derived skin condition may be used for further processing steps, such as comparing, monitoring, evaluating and diagnosing. This may improve the quality of these processing steps.

By separating the method into distinct steps — first obtaining morphological data, then evaluating the skin condition — each step may be optimized independently. This separation may allows for the use of at least one specialized tool and/or at least one algorithm configured for morphological analysis and/or condition evaluation, potentially reducing error rates and improving overall accuracy.

By first obtaining detailed morphological data, the system may apply a variety of classification algorithms to evaluate the skin condition. This flexibility may allow for the customization of the evaluation process based on different criteria and/or priorities, such as focusing on specific types of skin features and/or conditions.

Obtaining an item of information on the skin condition of the user by evaluating the information on skin morphology using a classification algorithm, may have the advantage of enabling a non-diagnostic characterization of the skin's appearance and/or texture. This may allow for a detailed and/or objective assessment of observable deviations from the physiological norm of the skin, such as changes in appearance and/or texture, without necessarily providing a medical diagnosis. Thereby, a valuable insight into the condition of the skin may be provided in a way that is independent of medical interpretation, making it useful for applications where a detailed description of the skin's characteristics is needed without entering the realm of medical diagnosis. This may be particularly advantageous in non-clinical settings or for applications where users seek to monitor or track changes in their skin over time without needing professional medical evaluation.

The present invention may be described in the following words:

A computer-implemented method of supporting an evaluation of local skin reactions, such as skin irritations from a tape and/or a plaster or from an adhesive component of a tape and/or a plaster, by using a mobile device having a camera, the mobile device containing “photo app” program instructions which, when executed, cause the mobile device to: receive an image of a part of a user’ s skin, the image being captured by the camera of the mobile device, the image comprising an area of intact skin of the user, and an area of non-intact skin of the user, wherein the area of non-intact skin has been exposed to an external impact, such as e.g. to a tape and/or a plaster, for a minimum period of time before the image has been captured; wherein the “photo app” program instructions take into account ambient light information during the capturing of the image and/or during processing of the image captured; and transferring the image to a receiving device, such as server, of an HCP and/or of a manufac- turer/developer of the “photo app” program instructions for further evaluation of the area of non-intact skin of the user, wherein the evaluation comprises comparing the area of intact skin and the area of non-intact skin of the user in the image.

The skin condition may be at least one of: a non-malignant skin change; a diagnostically non-relevant skin characteristic. The skin condition may be caused by an external impact, such as wearing a tape and/or a plaster, particularly for a prolonged period of time, specifically a tape and/or a plaster of a CGM system/sensor. The skin condition may be a skin irritation from a tape and/or a plaster or from an adhesive component of a tape and/or a plaster.

The algorithms used to assess the at least one skin condition may be split into the following two parts: 1. Identification of at least one suspicious area and assessment of the at least one skin feature within the at least one suspicious area. a. Identifying at least one suspicious area, e.g. by using methods such as unsupervised methods and outlier detection methods in a variety of dimensions regarding e.g. color channels and intensity. Thereby, a comparison to at least one reference color is performed. b. Evaluating skin features based on lesion/morphology specific methods, e.g. by evaluating specific spatial, intensity and/or color features by using e.g. specific algorithms or artificial neural networks. All types of features, i.e. color, intensity, and spatial based features are calculated. Based on the most substantial differentiation, the prominent skin features are selected. Thereby, a comparison with a reference color is only required if color features are evaluated.

2. Evaluation of the skin condition based on the skin features evaluated in step 1.

Further disclosed and proposed herein is a computer program including computer-executable instructions for performing the method according to the present invention in one or more of the embodiments enclosed herein when the instructions are executed on a computer or computer network. Specifically, the computer program may be stored on a computer-readable data carrier and/or on a computer-readable storage medium.

As used herein, the terms “computer-readable data carrier” and “computer-readable storage medium” specifically may refer to non-transitory data storage means, such as a hardware storage medium having stored thereon computer-executable instructions. The computer- readable data carrier or storage medium specifically may be or may comprise a storage medium such as a random-access memory (RAM) and/or a read-only memory (ROM).

Thus, specifically, one, more than one or even all of method steps a) to d) as indicated above may be performed by using a computer or a computer network, preferably by using a computer program.

Further disclosed and proposed herein is a computer program product having program code means, in order to perform the method according to the present invention in one or more of the embodiments enclosed herein when the program is executed on a computer or computer network. Specifically, the program code means may be stored on a computer-readable data carrier and/or on a computer-readable storage medium.

Further disclosed and proposed herein is a data carrier having a data structure stored thereon, which, after loading into a computer or computer network, such as into a working memory or main memory of the computer or computer network, may execute the method according to one or more of the embodiments disclosed herein.

Further disclosed and proposed herein is a non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform the method according to one or more of the embodiments disclosed herein.

Further disclosed and proposed herein is a computer program product with program code means stored on a machine-readable carrier, in order to perform the method according to one or more of the embodiments disclosed herein, when the program is executed on a computer or computer network. As used herein, a computer program product refers to the program as a tradable product. The product may generally exist in an arbitrary format, such as in a paper format, or on a computer-readable data carrier and/or on a computer-readable storage medium. Specifically, the computer program product may be distributed over a data network.

Finally, disclosed and proposed herein is a modulated data signal which contains instructions readable by a computer system or computer network, for performing the method according to one or more of the embodiments disclosed herein.

Referring to the computer-implemented aspects of the invention, one or more of the method steps or even all of the method steps of the method according to one or more of the embodiments disclosed herein may be performed by using a computer or computer network. Thus, generally, any of the method steps including provision and/or manipulation of data may be performed by using a computer or computer network. Generally, these method steps may include any of the method steps, typically except for method steps requiring manual work, such as providing the samples and/or certain aspects of performing the actual measurements.

Specifi cally, further disclosed herein are: - a computer or computer network comprising at least one processor, wherein the processor is adapted to perform the method according to one of the embodiments described in this description,

- a computer loadable data structure that is adapted to perform the method according to one of the embodiments described in this description while the data structure is being executed on a computer,

- a computer program, wherein the computer program is adapted to perform the method according to one of the embodiments described in this description while the program is being executed on a computer,

- a computer program comprising program means for performing the method according to one of the embodiments described in this description while the computer program is being executed on a computer or on a computer network,

- a computer program comprising program means according to the preceding embodiment, wherein the program means are stored on a storage medium readable to a computer,

- a storage medium, wherein a data structure is stored on the storage medium and wherein the data structure is adapted to perform the method according to one of the embodiments described in this description after having been loaded into a main and/or working storage of a computer or of a computer network, and

- a computer program product having program code means, wherein the program code means can be stored or are stored on a storage medium, for performing the method according to one of the embodiments described in this description, if the program code means are executed on a computer or on a computer network.

Summarizing and without excluding further possible embodiments, the following embodiments may be envisaged:

Embodiment 1 : A computer-implemented method of obtaining an item of information on a skin condition of a user, the method comprising: i. receiving at least one image of a portion of a skin of a user, wherein the image has been captured by at least one camera of a mobile device, the image comprising at least one portion of the skin of the user showing the at least one skin condition; ii. obtaining at least one item of information on a skin morphology of at least one skin feature comprised by the at least one portion of the skin of the user showing the at least one skin condition by evaluating the received at least one image by using at least one image processing algorithm, preferably wherein obtaining the at least one item of information on the skin morphology of the at least one skin feature comprises comparing the at least one item of information on a skin morphology of the at least one skin feature to at least one reference color; iii. obtaining the at least one item of information on the skin condition of the user by evaluating the at least one item of information on the skin morphology of the at least one skin feature by using at least one classification algorithm.

Embodiment 2: The method according to the preceding Embodiment, wherein obtaining the at least one item of information on the skin morphology of the at least one skin feature comprises identifying at least one suspicious area, wherein the at least one suspicious area is suspicious of comprising the at least one skin feature.

Embodiment 3 : The method according to the preceding Embodiment, wherein the at least one suspicious area is determined by searching in the received at least one image for a pattern in at least one of: at least one color or at least one color range, preferably at least one color channel, such as at least one RGB-channel; at least one color intensity, whereby the at least one reference color is used in the search to calibrate at least one of: the color or the at least one color range; the at least one color intensity.

Embodiment 4: The method according to the preceding Embodiment, wherein, for searching in the received at least one image for the pattern, at least one of:

- at least one unsupervised machine learning method;

- at least one outlier detection method;

- at least one machine learning model.

Embodiment 5: The method according to any one of the three preceding Embodiments, wherein obtaining the at least one item of information on the skin morphology of the at least one skin feature comprises generating the at least one item of information on the skin morphology by evaluating exclusively the at least one suspicious area by using the at least one image processing algorithm. Embodiment 6: The method according to the preceding Embodiment, wherein the at least one item of information on the skin morphology is or comprises: at least one color; at least one size; at least one shape of the at least one skin feature.

Embodiment 7: The method according to the preceding Embodiment, wherein the at least one item of information on the skin morphology further is or comprises: at least one topography; at least one color; at least one relative position; particularly wherein the relative position is between at least one of: a first skin feature of the at least one skin feature and a second skin feature of the at least one skin feature; the at least one skin feature and at least one landmark.

Embodiment 8: The method according to any one of the preceding Embodiments, wherein the at least one item of information on the skin condition of the user comprises or is at least one item of information on a skin lesion of the user, particularly wherein the at least one item of information on the skin lesion of the user comprises at least one item of information on at least one of a primary lesion and a secondary lesion.

Embodiment 9: The method according to the preceding Embodiment, wherein the at least one item of information on a primary lesion refers to least one of: a bulla; a burrow; a cyst; a macula; a nodule; a papule; a patch; a plaque; a pustule; a telangiectasia; a tumor; a vesicle; a welts; a wheal.

Embodiment 10: The method according to any one of the two preceding Embodiments, wherein the at least one item of information on a secondary lesion refers to least one of: an atrophy; a crust; an erosion; a excoriation; a fissure; an induration; a lichenification; a maceration; a phyma; an umbilication; an ulcer; a scale.

Embodiment 11 : The method according to any one of the preceding Embodiments, wherein obtaining the at least one item of information on the skin morphology of the at least one skin feature by using the at least one image processing algorithm comprises searching for at least one predetermined skin feature characteristic in the received at least one image.

Embodiment 12: The method according to the preceding Embodiment, wherein the at least one predetermined skin feature characteristic is selected in accordance with at least one characteristic required to obtain the item of information on the skin morphology of the at least one skin feature. Embodiment 13: The method according to any one of the preceding Embodiments, wherein the at least one item of information on the skin morphology of the at least one skin feature is obtained by the image processing algorithm by at least one of: performing edge detection; performing a color space analysis; performing a color histogram analysis; performing image thresholding; performing template matching; performing image segmentation.

Embodiment 14: The method according to any one of the preceding Embodiments, wherein the at least one image processing algorithm is selected from or comprises at least one of:

- at least one morphological transformation algorithm;

- at least one image gradient algorithm;

- at least one Canny edge detection algorithm;

- at least one Hough transformation algorithm;

- at least one deep convolutional network.

Embodiment 15: The method according to any one of the preceding Embodiments, wherein step i. further comprises receiving at least one topographical image, wherein the topographical image comprises the at least one portion of the skin of the user showing the at least one skin condition.

Embodiment 161 : The method according to the preceding Embodiment, wherein the topographical image has been captured by at least one of:

- at least one multi-camera technique;

- at least one shutter-technique.

Embodiment 17: The method according to any one of the two preceding Embodiments, wherein obtaining the at least one item of information on the skin morphology of the at least one skin feature in step ii. further comprises adding information on the topography of the at least one skin condition to the at least one item of information on the skin morphology of the at least one skin feature by evaluating the topographical image comprising the at least one portion of the skin of the user showing the at least one skin condition.

Embodiment 18: The method according to any one of the preceding Embodiments, wherein comparing the at least one item of information on the skin morphology of the at least one skin feature to the at least one reference color comprises classifying at least one color shown in the at least one image of the portion of the skin of the user in accordance with the at least one reference color.

Embodiment 19: The method according to the preceding Embodiment, wherein classifying the at least one color shown in the at least one image of the portion of the skin of the user in accordance with the at least one reference color comprises at least one of:

Embodiment 20: The method according to any one of the preceding Embodiments, wherein the at least one reference color is obtained by evaluating at least one of:

- at least one item of information on at least one reference portion of the skin of the user showing the at least one reference color;

- at least one item of information on at least one portion of a color reference card showing the at least one reference color;

- at least one item of information on the at least one skin condition in a historic state showing the at least one reference color;

- at least one item of information on a predetermined reference color, particularly wherein the predetermined reference color comprises information on a portion of a skin of a further person, specifically a further user, showing the at least one reference color.

Embodiment 21 : The method according to the preceding Embodiment, wherein the at least one item of information on the at least one reference portion of the skin of the user is obtained by evaluating at least one image of a portion of the skin of the user showing the at least one reference portion, particularly wherein the at least one image of the portion of the skin of the user received in step i. further shows the at least one reference portion and is used as the at least one image of the portion of the skin of the user showing the at least one reference portion.

Embodiment 22: The method according to any one of the two preceding Embodiments, wherein obtaining the at least one item of information on the skin morphology of the at least one skin feature in step ii. further comprises adding information on a skin tone of the user by evaluating the at least one item of information on the at least one reference portion of the skin of the user.

Embodiment 23 : The method according to any one of the three preceding Embodiments, wherein the at least one item of information on the at least one portion of the color reference card is obtained by evaluating at least one image of at least a portion of the color reference card showing the at least one reference color, particularly wherein the at least one image of the portion of the skin of the user received in step i. further shows the at least one portion of the color reference card and is used as the at least one image of the at least one portion of the color reference card.

Embodiment 24: The method according to any one of the preceding Embodiments, wherein the at least one classification algorithm is selected from or comprises at least one of:

- a look-up-table;

- a logistic regression;

- a decision tree;

- a support vector machine, SVM;

- a ^-nearest neighbors algorithm;

- a random forest;

- a gradient boosting.

Embodiment 25: The method according to any one of the preceding Embodiments, the method further comprising: iv. obtaining at least one item of information on an evolution of the skin condition of the user by comparing the item of information on the skin condition of the user to the at least one information on the at least one portion of the skin of the user showing the at least one skin condition in the historic state.

Embodiment 26: The method according to any one of the preceding Embodiments, the method further comprising:

- guiding the user in a manner that the at least one image fulfils at least one image capture condition by evaluating at least one preliminary image received from the at least one camera of a mobile device, wherein guiding the user comprises indicating at least one guidance signal to the user by using at least one signal device of the mobile device. Embodiment 27: The method according to the preceding Embodiment, wherein the at least one image capture condition is or comprises at least one of:

- the at least one image comprising the at least one portion of the skin of the user showing the at least one skin condition, particularly wherein the user is guided to arrange the at least one camera of a mobile device in a manner that the at least one image comprises the at least one portion of the skin of the user showing the at least one skin condition;

Embodiment 28: The method according to the preceding Embodiment, wherein the at least one signal is or comprises at least one of:

Embodiment 29: The method according to any one of the preceding Embodiments, wherein the at least one skin condition is caused by at least one external exposure, particularly selected from at least one of:

- wearing a tape, particularly wherein the tape is configured for attaching a device, preferably for attaching a medical device, more preferably for attaching a medical device used in the field of diabetes treatment, to a body part of a user; - wearing a plaster, particularly wherein the plaster is configured for attaching a device, preferably for attaching a medical device, more preferably for attaching a medical device used in the field of diabetes treatment, to a body part of a user;

- wearing an insulin pump;

- wearing a, optionally body -worn, continuous analyte monitoring device, specifically continuous glucose monitoring device;

- being exposed to an adhesive component, particularly comprised by at least one of the tape; the plaster;

- being exposed to a leachable substance, preferably of a medical device, more preferably of a medical device used in the field of diabetes treatment.

Embodiment 30: The method according to any one of the preceding Embodiments, wherein the at least one skin condition is or comprises at least one of a non-malignant skin change; a diagnostically non-relevant skin characteristic.

Embodiment 31 : The method according to any one of the preceding Embodiments, wherein the at least one item of information on the skin morphology of the at least one skin feature refers to least one of

- at least one size;

- at least one shape;

- at least one texture;

- at least one topography;

- at least one color;

- at least one relative position; particularly wherein the relative position is between at least one of o a first skin feature of the at least one skin feature and a second skin feature of the at least one skin feature; o the at least one skin feature and at least one landmark;

- at least one distribution of a plurality of skin features of the at least one skin feature;

- at least one type of at least a portion of an edge of the skin feature;

- at least one content of the at least one skin feature comprising at least one bodily fluid, such as serum; or blood. Embodiment 32: A mobile device having at least one camera, and, optionally, at least one processor, the mobile device being configured, specifically by software configuration, for performing the computer-implemented method of obtaining an item of information on a skin condition of a user according to any one of the preceding Embodiments referring to a computer-implemented method of obtaining an item of information on a skin condition of a user.

Embodiment 33: A computer program comprising instructions which, when the program is executed by the mobile device according to the preceding Embodiment, cause the mobile device to perform the computer-implemented method of obtaining an item of information on a skin condition of a user according to any one of the preceding Embodiments referring to a computer-implemented method of obtaining an item of information on a skin condition of a user.

Embodiment 34: A computer-readable storage medium, specifically a non-transient computer-readable storage medium, comprising instructions which, when the instructions are executed by the mobile device according to the Embodiment 32, cause the mobile device to perform the computer-implemented method of obtaining an item of information on a skin condition of a user according to any one of the preceding Embodiments referring to a computer-implemented method of obtaining an item of information on a skin condition of a user.

Embodiment 35: A system comprising at least one mobile device having at least one camera according to the preceding Embodiment referring to a mobile device and at least one color reference card comprising at least one reference color.

Short description of the Figures

Further optional features and embodiments will be disclosed in more detail in the subsequent description of embodiments, preferably in conjunction with the dependent claims. Therein, the respective optional features may be realized in an isolated fashion as well as in any arbitrary feasible combination, as the skilled person will realize. The scope of the invention is not restricted by the preferred embodiments. The embodiments are schematically depicted in the Figures. Therein, identical reference numbers in these Figures refer to identical or functionally comparable elements. In the Figures:

Figure 1 shows an exemplary computer-implemented method of obtaining an item of information on a skin condition of a user;

Figure 2 shows a portion of the exemplary computer-implemented method of obtaining an item of information on the skin condition of the user; and

Figure 3 shows further steps of the exemplary computer-implemented of obtaining an item of information on the skin condition of the user.

Detailed description of the embodiments

In Figure 1, an exemplary computer-implemented method 110 of obtaining an item of information on a skin condition of a user according is shown. The method comprises: i. receiving at least one image 162 of a portion of a skin of a user (denoted by reference number 112), wherein the image 162 has been captured by at least one camera of a mobile device, the image 162 comprising at least one portion of the skin of the user showing the at least one skin condition; ii. obtaining at least one item of information on a skin morphology of at least one skin feature 166 comprised by the at least one portion of the skin of the user showing the at least one skin condition (denoted by reference number 114) by evaluating the received at least one image 162 by using at least one image processing algorithm, preferably wherein obtaining the at least one item of information on the skin morphology of the at least one skin feature 166 comprises comparing the at least one item of information on a skin morphology of the at least one skin feature 166 to at least one reference color; iii. obtaining the at least one item of information on the skin condition of the user (denoted by reference number 116) by evaluating the at least one item of information on the skin morphology of the at least one skin feature 166 by using at least one classification algorithm.

Obtaining the at least one item of information on the skin morphology of the at least one skin feature 166 by using the at least one image processing algorithm may comprise searching for at least one predetermined skin feature 166 characteristic in the received at least one image 162. The at least one predetermined skin feature 166 characteristic may be selected in accordance with at least one characteristic required to obtain the item of information on the skin morphology of the at least one skin feature 166.

Obtaining the at least one item of information on the skin morphology of the at least one skin feature 166 (denoted by reference number 114) may comprise identifying at least one suspicious area (denoted by reference number 113), wherein the at least one suspicious area is suspicious of comprising the at least one skin feature 166.

Obtaining the at least one item of information on the skin morphology of the at least one skin feature 166 (denoted by reference number 114) may comprises generating the at least one item of information on the skin morphology (denoted by reference number 115) by evaluating exclusively the at least one suspicious area by using the at least one image processing algorithm.

The at least one item of information on the skin morphology of the at least one skin feature 166 may be obtained by the image processing algorithm by at least one of: performing edge detection; performing a color space analysis; performing a color histogram analysis; performing image thresholding; performing template matching; performing image segmentation. The at least one image processing algorithm may be selected from or comprises at least one of: at least one morphological transformation algorithm; at least one image gradient algorithm; at least one Canny edge detection algorithm; at least one Hough transformation algorithm; at least one deep convolutional network.

Step i. (denoted by reference number 112) may further comprise receiving at least one topographical image, wherein the topographical image comprises the at least one portion of the skin of the user showing the at least one skin condition. The topographical image may have been captured by at least one of: at least one light imaging, detection and ranging, LIDAR, sensor, of the mobile device; at least one multi-camera technique; at least one shutter-technique.

Obtaining the at least one item of information on the skin morphology of the at least one skin feature 166 in step ii. (denoted by reference number 114) further may comprise adding information on the topography of the at least one skin condition to the at least one item of information on the skin morphology of the at least one skin feature 166 by evaluating the topographical image comprising the at least one portion of the skin of the user showing the at least one skin condition.

Comparing the at least one item of information on the skin morphology of the at least one skin feature 166 to the at least one reference color may comprise classifying at least one color shown in the at least one image 162 of the portion of the skin of the user in accordance with the at least one reference color.

Classifying the at least one color shown in the at least one image 162 of the portion of the skin of the user in accordance with the at least one reference color may comprise at least one of:

- setting at least one absolute color value of the at least one color shown in the at least one image 162 of the portion of a skin of the user;

- setting at least one color difference value between the at least one color shown in the at least one image 162 of the portion of a skin of the user and the at least one reference color.

The at least one reference color may be obtained by evaluating at least one of:

The at least one item of information on the at least one reference portion of the skin of the user may be obtained by evaluating at least one image 162 of a portion of the skin of the user showing the at least one reference portion, particularly wherein the at least one image 162 of the portion of the skin of the user received in step i. (denoted by reference number 112) further shows the at least one reference portion and is used as the at least one image 162 of the portion of the skin of the user showing the at least one reference portion. Obtaining the at least one item of information on the skin morphology of the at least one skin feature 166 in step ii. (denoted by reference number 114) further may comprise adding information on a skin tone of the user by evaluating the at least one item of information on the at least one reference portion of the skin of the user.

The at least one item of information on the at least one portion of the color reference card may be obtained by evaluating at least one image of at least a portion of the color reference card showing the at least one reference color, particularly wherein the at least one image 162 of the portion of the skin of the user received in step i. (denoted by reference number 112) further shows the at least one portion of the color reference card and is used as the at least one image of the at least one portion of the color reference card.

The method may further comprise: iv. obtaining at least one item of information on an evolution of the skin condition of the user (denoted by reference number 118) by comparing the item of information on the skin condition of the user to the at least one information on the at least one portion of the skin of the user showing the at least one skin condition in the historic state.

Figure 2 shows steps ii. (denoted by reference number 114) and iii. (denoted by reference number 116) of the exemplary computer-implemented method 110.

As already disclosed, step ii. (denoted by reference number 114) refers to obtaining the at least one item of information on a skin morphology of at least one skin feature 166 comprised by the at least one portion of the skin of the user showing the at least one skin condition.

The at least one item of information on the skin morphology of the at least one skin feature 166 may refers to least one of: at least one size; at least one shape; at least one texture; at least one topography; at least one color; at least one relative position; particularly wherein the relative position is between at least one of: a first skin feature 166 of the at least one skin feature 166 and a second skin feature 166 of the at least one skin feature 166; the at least one skin feature 166 and at least one landmark; at least one distribution of a plurality of skin features 166 of the at least one skin feature 166; at least one type of at least a portion of an edge of the skin feature 166; at least one content of the at least one skin feature 166 comprising at least one bodily fluid, such as serum; or blood.

As already disclosed, step iii. (denoted by reference number 116) refers to obtaining the at least one item of information on the skin condition of the user.

The at least one skin condition may be caused by at least one external exposure, particularly selected from at least one of: wearing a tape, particularly wherein the tape is configured for attaching a device, preferably for attaching a medical device, more preferably for attaching a medical device used in the field of diabetes treatment, to a body part of a user; wearing a plaster, particularly wherein the plaster is configured for attaching a device, preferably for attaching a medical device, more preferably for attaching a medical device used in the field of diabetes treatment, to a body part of a user; wearing an insulin pump; wearing a, optionally body -worn, continuous analyte monitoring device, specifically continuous glucose monitoring device; being exposed to an adhesive component, particularly comprised by at least one of: the tape; the plaster; being exposed to a leachable substance, preferably of a medical device, more preferably of a medical device used in the field of diabetes treatment. The at least one skin condition may be or may comprise at least one of: a non-malignant skin change; a diagnostically non-relevant skin characteristic.

The at least one item of information on the skin condition of the user may comprise or may be at least one item of information on a skin lesion of the user, particularly wherein the at least one item of information on the skin lesion of the user comprises at least one item of information on at least one of a primary lesion and a secondary lesion.

The at least one item of information on a primary lesion may refer to least one of: a bulla; a burrow; a cyst; a macula; a nodule; a papule; a patch; a plaque; a pustule; a telangiectasia; a tumor; a vesicle; a welts; a wheal. The at least one item of information on a secondary lesion may refer to least one of: an atrophy; a crust; an erosion; a excoriation; a fissure; an induration; a lichenification; a maceration; a phyma; an umbilication; an ulcer; a scale.

A step of obtaining at least one item of information on a diagnosis of the portion of the skin of the user showing the at least one skin condition by evaluating the at least one item of information on the skin condition of the user (denoted by reference number 120) may be excluded from the computer-implemented method 110 of obtaining an item of information on the skin condition. A typical diagnosis may refer to the user having an acne; a rash; a nevus and/or a melanoma.

Figure 3 shows further optional steps of the exemplary computer-implemented method 110. The black circle denoted be reference number 122 denotes a possible start of the method 110.

The method 110 may further comprise:

- guiding the user in a manner that the at least one image fulfils at least one image capture condition by evaluating at least one preliminary image received from the at least one camera of a mobile device (denoted by reference number 124), wherein guiding the user comprises indicating at least one guidance signal to the user by using at least one signal device of the mobile device.

The at least one image capture condition may be or may comprise at least one of:

The at least one signal may be or may comprise at least one of:

- a visual signal, particularly wherein a display 142 of the mobile device is used as the signal device; - an audio signal, particularly wherein a speaker 152 of the mobile device is used as the signal device;

- a haptic signal, particularly wherein a haptic signal device 154 of the mobile device is used as the signal device.

The method 110 may further comprise:

- synchronizing a database of the mobile device 132 to a database of a cloud server 134 comprising a plurality of known items of information on the skin condition and at least one of: a feedback; a guidance; an advice, paired with a respective known item of information on the skin condition of the user.

The databases 132, 134 may be stored on a storage device 156, 158.

As already disclosed, the method 110 comprises: i. receiving the at least one image of the portion of the skin of the user (denoted by reference number 112), wherein the image has been captured by at least one camera of a mobile device, the image comprising at least one portion of the skin of the user showing the at least one skin condition.

Particularly before receiving the at least one image of the portion of the skin of the user (denoted by reference number 112), the method 110 may further comprise: capturing the at least one image by using the at least one camera of a mobile device (denoted by reference number 126).

As already disclosed, step i. (denoted by reference number 112) may further comprise receiving at least one topographical image, wherein the topographical image comprises the at least one portion of the skin of the user showing the at least one skin condition. Particularly before further receiving the at least one topographical image of the portion of the skin of the user (denoted by reference number 112), the method 110 may further comprise: capturing the at least one topographical image (denoted by reference number 141) by using at least one of: the at least one light imaging, detection and ranging, LIDAR, sensor 150, of the mobile device; the at least one multi-camera technique; the at least one shutter-technique.

As already disclosed, the method 110 comprises: ii. obtaining at least one item of information on a skin morphology of at least one skin feature 166 comprised by the at least one portion of the skin of the user showing the at least one skin condition (denoted by reference number 114) by evaluating the received at least one image 162 by using at least one image processing algorithm, wherein obtaining the at least one item of information on the skin morphology of the at least one skin feature 166 comprises comparing the at least one item of information on a skin morphology of the at least one skin feature 166 to at least one reference color, particularly in order to classify at least one color shown in the at least one image 162 of the portion of the skin of the user in accordance with the at least one reference color.

Using at least one image processing algorithm may further comprise:

- performing at least one image preprocessing step on the received at least one image of the portion of the skin of the user (denoted by reference number 122).

The image preprocessing step might comprise verifying if at least one of: the at least one portion of the skin of the user showing the at least one skin condition; the at least one reference portion of the skin of the user is shown in the at least one image. Alternatively or in addition, the image preprocessing step might comprise excluding specular reflections. At least one further arbitrary artifact in the at least one image may be identified and/or excluded.

Using at least one image processing algorithm may further comprise obtaining at least one item of information on an ambient light condition (denoted by reference number 128). The item of information on an ambient light condition may be obtained by evaluating the at least one item of information on at least one portion of a color reference card showing the at least one reference color. Alternatively or in addition, The method 110 may further comprise:

- receiving the at least one item of information on an ambient light condition (denoted by reference number 144) generated by at least one ambient light sensor comprised by the mobile device.

The item of information on an ambient light condition may be received by at least one ambient light sensor 160 comprised by the mobile device (denoted by reference number 146).

As already disclosed, the method 110 comprises:

- defining the spatial location of at least one of: the portion of the skin of the user showing the at least one skin condition; the at least one reference portion of the skin of the user (denoted by reference number 128).

As already disclosed, the method 110 comprises: iii. obtaining the at least one item of information on the skin condition of the user (denoted by reference number 116) by evaluating the at least one item of information on the skin morphology of the at least one skin feature 166.

The method 110 may further comprise:

- storing the at least one item of information on the skin condition of the user (denoted by reference number 130) in at least one of: the database of the mobile device 132, the database of the cloud server 134.

The method 110 may further comprise:

- verifying if information on the at least one item of information on the skin condition of the user is available (denoted by reference number 136) by comparing the at least one item of information on the skin condition of the user to at least one known item of information on the skin condition of the user comprised by at least one of: the database of the mobile device 132, the database of a cloud server 134.

The method 110 may further comprise:

- transmitting the information on the at least one item of information on the skin condition of the user to a health care professional (denoted by reference number 138) if no information on the at least one item of information on the skin condition is available.

The method 110 may further comprise:

- providing at least one of: a feedback; a guidance; an advice to the user (denoted by reference number 140), particularly by using a display of the mobile device in accordance with at least one item of information on the skin condition of the user, if information on the at least one item of information on the skin condition is available.

In Figure 4, an exemplary mobile device 132 having at least one camera 144, and, optionally, at least one processor, is disclosed. The mobile device may be configured, specifically by software configuration, for performing the computer-implemented method of obtaining an item of information on a skin condition of a user as elsewhere disclosed herein.

The mobile device 132 may further comprise at least one of: - at least one display 142;

- at least one light imaging, detection and ranging, LIDAR, sensor 150;

- at least one speaker 152;

- at least one haptic signal device 154;

- at least one storage device 156;

- at least one ambient light sensor 160.

A computer program comprising instructions is disclosed, which, when the program is executed by the mobile device 132, cause the mobile device 132 to perform the computer-implemented method 110 of obtaining an item of information on a skin condition of a user.

A computer-readable storage medium, specifically a non-transient computer-readable storage medium, comprising instructions is disclosed, which, when the instructions are executed by the mobile device 132, cause the mobile device to perform the computer-implemented method 110 of obtaining an item of information on a skin condition of a user.

In Figure 4, further, a system 146 comprising at least one mobile device 132 having at least one camera 144 and at least one color reference card 148 comprising at least one reference color 149 is disclosed.

Figure 5 shows a typical at least one image 162 of a portion of a skin of a user. In the at least one image 162 of a portion of a skin of a user a suspicious area 164 that is suspicious of comprising the at least one skin feature 166 is depicted. In Figure 5, further a legend 168 is shown. The legend indicates the intensity of different colors in the image 162 outside of the suspicious area 164 (denoted by reference number 170). The legend further indicates the intensity of different colors in the image 162 in the suspicious area 164 (denoted by reference number 172).

List of reference numbers

110 computer-implemented method of obtaining an item of information on a skin condition of a user

112 receiving at least one image of a portion of a skin of a user

114 obtaining at least one item of information on a skin morphology obtaining the at least one item of information on the skin condition of the user obtaining at least one item of information on an evolution of the skin condition of the user obtaining at least one item of information on a diagnosis of the portion of the skin of the user possible start of the method guiding the user in a manner that the at least one image fulfils at least one image capture condition capturing the at least one image by using the at least one camera of a mobile device obtaining at least one item of information on an ambient light condition storing the at least one item of information on the skin condition of the user mobile device cloud server verifying if information on the at least one item of information on the skin condition of the user is available transmitting the information on the at least one item of information on the skin condition of the user to a health care professional providing at least one of: a feedback, a guidance, an advice to the user capturing the at least one topographical image display camera system color reference card reference color light imaging, detection and ranging, LIDAR, sensor speaker haptic signal device storage device of the mobile device storage device of the cloud server ambient light sensor at least one image of a portion of a skin of a user suspicious area skin feature legend 170 intensity of different colors in the image outside of the suspicious area

172 intensity of different colors in the image in the suspicious area

Claims

1. A computer-implemented method of obtaining an item of information on a skin condition of a user, the method comprising: i. receiving at least one image (162) of a portion of a skin of a user, wherein the image (162) has been captured by at least one camera (144) of a mobile device (132), the image (162) comprising at least one portion of the skin of the user showing the at least one skin condition; ii. obtaining at least one item of information on a skin morphology of at least one skin feature (166) comprised by the at least one portion of the skin of the user showing the at least one skin condition by evaluating the received at least one image (162) by using at least one image (162) processing algorithm, preferably wherein obtaining the at least one item of information on the skin morphology of the at least one skin feature (166) comprises comparing the at least one item of information on a skin morphology of the at least one skin feature (166) to at least one reference color (149); iii. obtaining the at least one item of information on the skin condition of the user by evaluating the at least one item of information on the skin morphology of the at least one skin feature (166) by using at least one classification algorithm, wherein the term skin condition does not include a medical diagnosis.

2. The method according to the preceding claim, wherein obtaining the at least one item of information on the skin morphology of the at least one skin feature (166) comprises identifying at least one suspicious area (164), wherein the at least one suspicious area (164) is suspicious of comprising the at least one skin feature (166), wherein the at least one suspicious area (164) is determined by searching in the received at least one image (162) for a pattern in at least one of: at least one color or at least one color range; at least one color intensity, whereby the at least one reference color is used in the search to calibrate at least one of: the color or the at least one color range; the at least one color intensity.

3. The method according to the preceding claim, wherein obtaining the at least one item of information on the skin morphology of the at least one skin feature (166) comprises generating the at least one item of information on the skin morphology by evaluating exclusively the at least one suspicious area (164) by using the at least one image processing algorithm.

4. The method according to any one of the preceding claims, wherein the at least one item of information on the skin condition of the user comprises or is at least one item of information on a skin lesion of the user, wherein the at least one item of information on the skin lesion of the user comprises at least one item of information on a primary lesion and a secondary lesion.

5. The method according to the preceding claim, wherein the at least one item of information on a primary lesion refers to least one of: a bulla; a burrow; a cyst; a macula; a nodule; a papule; a patch; a plaque; a pustule; a telangiectasia; a tumor; a vesicle; a welts; a wheal, wherein the at least one item of information on a secondary lesion refers to least one of: an atrophy; a crust; an erosion; a excoriation; a fissure; an induration; a lichenification; a maceration; a phyma; an umbilication; an ulcer; a scale.

6. The method according to any one of the preceding claims, wherein step i. further comprises receiving at least one topographical image, wherein the topographical image comprises the at least one portion of the skin of the user showing the at least one skin condition, wherein obtaining the at least one item of information on the skin morphology of the at least one skin feature (166) in step ii. further comprises adding information on the topography of the at least one skin condition to the at least one item of information on the skin morphology of the at least one skin feature (166) by evaluating the topographical image comprising the at least one portion of the skin of the user showing the at least one skin condition.

7. The method according to any one of the preceding claims, wherein comparing the at least one item of information on the skin morphology of the at least one skin feature (166) to the at least one reference color (149) comprises classifying at least one color shown in the at least one image (162) of the portion of the skin of the user in accordance with the at least one reference color (149).

8. The method according to the preceding claim, wherein classifying the at least one color shown in the at least one image (162) of the portion of the skin of the user in accordance with the at least one reference color (149) comprises at least one of: - setting at least one absolute color value of the at least one color shown in the at least one image (162) of the portion of a skin of the user;

- setting at least one color difference value between the at least one color shown in the at least one image (162) of the portion of a skin of the user and the at least one reference color (149).

9. The method according to any one of the preceding claims, wherein the at least one reference color (149) is obtained by evaluating at least one of:

- at least one item of information on at least one reference portion of the skin of the user showing the at least one reference color (149);

- at least one item of information on at least one portion of a color reference card (148) showing the at least one reference color (149);

- at least one item of information on the at least one skin condition in a historic state showing the at least one reference color (149);

- at least one item of information on a predetermined reference color (149), wherein the predetermined reference color (149) comprises information on a portion of a skin of a further person showing the at least one reference color (149).

10. The method according to any one of the preceding claims, the method further comprising: iv. obtaining at least one item of information on an evolution of the skin condition of the user by comparing the item of information on the skin condition of the user to the at least one information on the at least one portion of the skin of the user showing the at least one skin condition in the historic state.

11. The method according to any one of the preceding claims, wherein the at least one skin condition is caused by at least one external exposure, particularly selected from at least one of:

- wearing a tape;

- wearing a plaster;

- wearing an insulin pump;

- wearing a continuous analyte monitoring device;

- being exposed to an adhesive component, particularly comprised by at least one of: the tape; the plaster;

- being exposed to a leachable substance.

12. The method according to any one of the preceding claims, wherein the at least one skin condition is or comprises at least one of: a non-malignant skin change; a diagnostically non-relevant skin characteristic.

13. The method according to any one of the preceding claims, wherein the at least one item of information on the skin morphology of the at least one skin feature (166) refers to least one of:

- at least one size;

- at least one shape;

- at least one texture;

- at least one topography;

- at least one color;

- at least one relative position; particularly wherein the relative position is between at least one of: o a first skin feature (166) of the at least one skin feature (166) and a second skin feature (166) of the at least one skin feature (166); o the at least one skin feature (166) and at least one landmark;

- at least one distribution of a plurality of skin features (166) of the at least one skin feature (166);

- at least one type of at least a portion of an edge of the skin feature (166);

- at least one content of the at least one skin feature (166) comprising at least one bodily fluid, such as serum; or blood.

14. A mobile device (132) having at least one camera (144), and, optionally, at least one processor, the mobile device (132) being configured, specifically by software configuration, for performing the computer-implemented method of obtaining an item of information on a skin condition of a user according to any one of the preceding claims referring to a computer-implemented method of obtaining an item of information on a skin condition of a user.

15. A computer program comprising instructions which, when the program is executed by the mobile device (132) according to the preceding claim, cause the mobile device (132) to perform the computer-implemented method of obtaining an item of information on a skin condition of a user according to any one of the preceding claims referring to a computer-implemented method of obtaining an item of information on a skin condition of a user.

16. A computer-readable storage medium comprising instructions which, when the in- structions are executed by the mobile device (132) according to the claim 14, cause the mobile device (132) to perform the computer-implemented method of obtaining an item of information on a skin condition of a user according to any one of the preceding claims referring to a computer-implemented method of obtaining an item of information on a skin condition of a user.

17. A system (146) comprising at least one mobile device (132) having at least one camera (144) according to the preceding claim referring to a mobile device (132) and at least one color reference card (148) comprising at least one reference color (149).