CN111860760A - Training of generators for domain transformation - Google Patents

Abstract

Method of training a generator, the generator being configured for transforming a data set with measurement data from a first domain a to a second domain B according to a first set of learning data sets belonging to domain a and a second set of learning data sets belonging to domain B, the behavior of the generator being characterized by adaptable parameters and the parameters being adapted gradually to optimize the value of a pre-given cost function, the cost function containing a confidence contribution for a metric of to what extent: at least one characteristic embodied in at least one learning dataset belonging to domain a remains unchanged upon conversion of the learning dataset to domain B; at least one characteristic embodied in at least one learning data set belonging to domain a is similar to a corresponding characteristic of at least one learning data set belonging to domain B after conversion to domain B. A generator obtained with this method. A method for training a module capable of training using a generator. A method of utilizing an effect chain comprised of a generator and a module capable of training.

Description

Training of generators for domain transformation

technical field

The invention relates to a training method with which a generator can be trained to convert a dataset with measurement data between two domains A and B. The use of such a generator in turn makes it possible to alleviate the lack of learning data sets, for example, in the case of training trainable modules for at least partially automated driving.

Background technique

The driving of vehicles by human drivers in road traffic is usually trained by subjecting the driving learner repeatedly to certain criteria (Kanon) for a number of situations within the scope of his own training. The driving learner must react to these situations individually and get feedback through comments or even the intervention of the driving instructor: whether their own responses were correct or incorrect. Training with a limited number of situations should enable the driving learner to also become competent in unknown situations in independent driving of the vehicle.

In order to enable vehicles to participate in road traffic fully or partially automatically: these vehicles are controlled with modules that can be trained in a very similar way. These modules acquire, for example, sensor data from the vehicle environment as input variables and as output variables to provide actuation signals with which the operation of the vehicle is intervened and/or semi-finished products from which such actuations are formed Signal. For example, the classification of objects in the environment of the vehicle may be such a semi-finished product.

The time- and cost-intensive advance in such training is the necessity to create a sufficient number of learning datasets. The learned values for the input parameters and the learned values for the output parameters of the trainable module, which the trainable module should thereby generate in a correctly trained state, are respectively required.

SUMMARY OF THE INVENTION

Within the scope of the present invention, a method for training a generator is developed which is designed to transform a data set with measurement data from a first domain A to a second domain B.

The concept of measurement data here includes very generally data which have been observed by means of physical measurement processes and/or by partly or fully simulating such measurement processes and/or by partly or fully simulating with such measurement processes obtained from the technical system. Measurement data are therefore always data which characterize a process and/or a physical observation in a technical system. For better readability and in view of the fact that in many applications simulations are an almost equally valuable alternative to physical experiments, these data are hereinafter referred to only as "measurement data" without regard to whether the data are Obtained through physical measurements or through simulations.

A data set with measurement data may, for example, include images that have been obtained by observing the environment of the vehicle with one or more cameras. However, these datasets may also include, for example, radar-reflected point clouds, which have been obtained by observing the environment of the vehicle using one or more radar sensors.

These fields A and B can, for example, represent different types or categories of situations that affect the physical or simulated recording of the measurement data. If the measurement data comprise images, for example, the fields A and B can represent, for example, different seasons, times of day, lighting situations and/or weather situations. The images of each domain then have the following style (Stil), which is collectively applied (aufprägen) to the images through the corresponding situation. E.g:

• Images recorded during the day, at dusk or at night,

• Images recorded in sunny conditions or in rainy weather conditions, and

• Images recorded in spring, summer, autumn and winter

Each has its own style.

These large number of possible styles increase the cost in the case of training of trainable modules that should draw inferences from the images for controlling or steering the vehicle. It must therefore be ensured, for example, that certain traffic situations, including traffic signs and other traffic participants present in the images, are always correctly recognized in the images, irrespective of which style is precisely applied to these images. This means that image styles that are not related to the semantic content of the traffic situation are not allowed to influence the recognition of the semantic content. To ensure this, trainable modules are typically trained on images whose variability includes a very large number of possible image styles.

Trainable modules are in particular considered modules that are embodied (verkörpern) with great power for generalization: a function (Funktion) that is parameterized with adaptable parameters. These parameters can be adapted, in particular, during the training of the trainable module in such a way that the values of the associated learned output variables are reproduced as well as possible when the learned input variables are input into the module. The trainable module may comprise, inter alia, an artificial neural network, KNN; and/or may be a KNN.

Inadequacies with regard to the training data can arise, in particular, in that, when collecting real images for training, certain traffic situations do not often occur in combination with certain image styles. For example, snowfall may occur only very infrequently at a certain location, so that only a small number of images are available, which show the location in snow-covered conditions. In actual driving operation, certain hazardous situations (“corner cases”), for example, may in principle also occur infrequently, in which the risk of an accident is significantly increased. Thus, it is more and more difficult to obtain images in which these situations are combined (gepaart) with many different image styles.

In these cases, generators for domain transformations (Domänenübersetzung) are used. For example, images recorded in summer can be made to belong to domain A, and images recorded in winter can be made to belong to domain B. The generator can then transform any image recorded in summer such that the image appears to be recorded in winter. In the ideal case of a certain traffic situation, therefore, only one image with any style is required and this image can then be converted into all other styles required for the training of the trainable modules.

The final practical application of the generator is therefore to improve the training of trainable modules, which in turn are designed to control and/or monitor technical systems. The control and/or monitoring is thus also improved indirectly.

The generator is trained in its aspects within the scope of the method. For this purpose, a first set of learning datasets belonging to domain A and a second set of learning datasets belonging to domain B are used. Here, it is not necessary that the learning datasets in domains A and B respectively have the same semantic content. This means that the training can also be performed with unpaired (ungepaart) learning datasets.

The behavior of the generator is characterized by adaptable parameters. These parameters are gradually adapted within the scope of the training in order to optimize the values of the predetermined cost function. This cost function contains the credibility contribution (Plausibilitäts-Beitrag). To what extent this confidence contribution is used for the following measures:

• At least one characteristic embodied in at least one learning dataset belonging to domain A remains unchanged when transforming that learning dataset to domain B; and/or

• At least one characteristic embodied in at least one learning dataset belonging to domain A is similar to the corresponding characteristic of at least one learning dataset belonging to domain B after transformation to domain B.

In this way, additional information about the domains A and B, about the physical or simulated process, and about the semantic content of the measurement data to which the measurement data in the learning dataset belong can be taken into account when training the generator. B, Measured data were obtained using the physical or simulated process. It has been recognized that the training of the generator can be made more stable in such a way that the generator has a higher probability of being able to convert a dataset with measurement data from domain A to domain B after training has ended available (brauchbar) tools.

In particular, it is possible, for example, to reduce the probability of unintentionally changing the semantic content of these measurement data, which is relevant for the respective application, in the event of a transition from domain A to domain B. If, for example, a visually well-achieved image transition from style "summer" to style "winter" introduces a small artefact into the image, which leads to misinterpretation of a stop sign as a speed 70 sign, This can then be manifested in the confidence contribution used for the cost function (niederschlagen). The training of the generator can, for example, react to it in such a way that the transformed image looks less visually "pretty", but for this the semantic content is correctly conveyed.

The cost function used for the training of the generator thus has at least two contributions in total. There are contributions that involve the domain transformation itself and depend on its specific modality (Macart) (eg Generative Adversarial Network, GAN). These contributions evaluate: to what extent, for example, the rewriting (Übertragung) of an image from one style to another is "achieved (gelingen)". This is accompanied by a credibility contribution (hierzu gesellt sich) which evaluates whether the transformed image can also be used for its own intended other applications. This is especially important when the dataset that has been transformed from domain A to domain B should then be used for training of trainable modules, eg artificial neural networks. The performance and reliability of such trainable modules depend decisively on the quality of the data used in training.

In a particularly advantageous configuration, the cost function additionally contains at least one circular argument contribution. The circular argument contributes to this by a measure of the extent to which at least one learning dataset belonging to domain A is consistently reproduced after transformation from domain A to domain B and subsequent transformation from domain B back to domain A using the generator . The circular argument contributions therefore belong to the following contributions, which involve the domain transformation itself.

For example, the generator can be trained in the category of CycleGAN (Cyclic GAN) method. In the case of such a method, a total of four trainable modules, eg neural networks, can be trained, for example. The first generator is trained in view of transforming the datasets from domain A to domain B in such a way that these datasets can be distinguished from datasets recorded directly in domain B, either physically or analogously. The second generator is trained to transform the datasets in the opposite direction from domain B to domain A in such a way that these datasets can be distinguished from those recorded directly in domain A, either physically or analogously. Furthermore, the first discriminator (Diskriminator) is trained to distinguish the datasets recorded directly in domain B, physically or analogously, from the datasets transformed into domain B by the first generator. The second discriminator is trained to distinguish between datasets recorded directly in domain A, physically or analogically, from datasets transformed into domain A by the second generator.

In the case of such a CycleGAN approach, the cost function may, for example, contain two circular argument contributions. This first circular argument contribution is a measure for at least one learning dataset belonging to domain A after being transformed from domain A to domain B with the first generator and then transformed back from domain B with the second generator To what extent domain A is subsequently reproduced consistently. This second circular argument contribution is a measure for at least one learning dataset belonging to domain B after being transformed to domain A with the second generator and after being subsequently transformed from domain A back to domain B with the first generator To what extent are they reproduced consistently.

It has been recognized that, when trained according to the method, the two generators can learn (erlernen) distortions (Verfälschung) of these datasets; these distortions cancel each other out in a full (komplett) run of the circular argument and thus make the use of These circular contributions to the cost function do not change. For example, the first generator can learn to exchange (Vertauschung) in the assignment of colors to different object types, and the second generator can learn the opposite exchange. This complete circular argument can then, for example, reproduce the input image exactly again. If, for example, the colors assigned to the road on the one hand and the sky on the other hand are exchanged during the transition between domains, or even if the colors of the image are completely inverted during the transition, this is therefore The circular argument contribution for the cost function is completely ignored (entgehen). Images distorted in this way are, for example, no longer usable for training trainable modules.

This plausibility contribution can now be monitored, for example, in addition: in the case of a transition from domain A to domain B, the base color tone remains unchanged and/or only changes in a plausible manner. Therefore, the color tone of the road is, for example, a gray tone in most cases. The hue can be changed by covering with white snow when switching to the field "Winter" or by covering with brown-toned fallen leaves when switching to the field "Autumn". However, it is difficult to imagine the real motives (Anlass) for which the hue of the road is changed, for example, to blue, green or red. Likewise, it is possible, for example, to change the color of the sky from blue to gray or white depending on the weather, or to change the color of the sky to a red hue at sunrise or sunset. However, it is hard to imagine for what real motive the color of the sky should be changed to green, for example.

Thus, for example, the color swap or color reversal no longer remains unnoticed in the training of the generator, but is "bestrafened" in the cost function, so that the generator eventually recovers from this misbehavior again to move away (abrücken).

In this way, the training of the generator becomes more reproducible overall. Typically, the parameters characterizing the behavior of the generator are randomly initialized at the beginning of training. In experiments so far with cost functions with no credibility contribution, it therefore depends on chance that these generators learn the described distortions, which cancel each other out in a circular argument. In the unfavorable case, the training must be restarted more than ten times in order to obtain a usable generator. With this confidence contribution, the described distortions are "aberziehen" by the generator during training, irrespective of the initial configuration of the generator's parameters. Associated with this is a significant saving in time and money for training, as a full training can require multiple days of computing time given the strength of the graphics processing unit (GPU), and is either provided by own hardware or by in For leases in the cloud, this computing time must be paid in either form.

The cost function may, for example, comprise an optionally also weighted sum consisting of the plausibility contribution and the circular argument contribution.

In a further particularly advantageous configuration, the training of the generator is continued with the cost function in response to the course of the curve and/or the value of the plausibility contribution of the cost function satisfying predetermined criteria: In this cost function, the confidence contribution is weighted or removed less. It has been recognized that the learning of the described distortions, which cancel each other out in a complete circular argument, preferably occurs at the beginning of the training starting from a random initialization of the parameters. The further the training progresses, the less likely the generator will also develop a trend for the described distortion.

As explained above, in a particularly advantageous configuration, domains A and B are differentiated with respect to at least one physical condition and/or a simulation boundary condition, wherein the associated learning data sets for domains A and B are each already in the Detected under the physical conditions and/or the simulated boundary conditions. These conditions may include, for example: time of day, season, weather conditions or lighting conditions.

In particular, the credibility contribution can be used, for example, to pre-train the generator before finally considering the circular argument contribution for the cost function. But it is also possible, for example, to alternate between training optimized only for plausibility contributions and training optimized only for circular argument contributions.

In another particularly advantageous configuration, the following physical properties are chosen as properties that should remain unchanged when the learning dataset is transferred from domain A to domain B: the simulation boundary between domains A and B Differences in conditions and/or physical conditions do not affect this physical property or only up to a predetermined maximum measure (Höchstmaß). As explained above, the color tone of certain objects visible in the image can be changed, for example only within predetermined limits, when the weather and/or lighting conditions change. Simple changes such as lighting conditions may also not radically change the texture of eg tree canopies or roads.

In a further advantageous configuration, the learning data sets each comprise the detected distribution of at least one physically observable measurement variable in a two-dimensional or three-dimensional spatial region. In this case, the measured variable can have any dimensions. This distribution assigns values of any dimension of the measurement variable to positions which are in turn characterized by two-dimensional or three-dimensional coordinates within the spatial region. Examples for such distributions are two-dimensional or three-dimensional images. Thus, the two-dimensional image recorded with the image sensor represents, for example, the spatially resolved distribution of the light intensity and/or color of the region via the image sensor. Another example for a distribution is a point cloud of radar data that assigns, for example, a distance and one or more corners to intensities incident through a location characterized by the distance and the one or more corners.

The term "physically observable measured variables" should not be interpreted restrictively in the sense that the values of these measured variables must be checked compulsorily by physical measurements. Similar to the term "measurement data", it simply means that such physically detected measurement variables are available. However, it is also possible to use simulations as an alternative to physical detection without slightly changing the subsequent application of the measured variable.

In a further particularly advantageous configuration, at least one location at which the physically observable measurement variable has been detected, at least one attribute of the at least one physical object is selected as a function of the transfer of the learning data set from the domain A to the domain Case B should maintain the same characteristics. For this purpose, the measurement data in the learning dataset can be semantically segmented, for example, using any method. The accuracy with which this segmentation is made is only secondary; what is decisive is whether the result of this segmentation changes after the transformation of the learning dataset to domain B. The semantic segmentation also does not have to be performed fully automatically, for example, but the image regions which belong to the two domains A and B can, for example, also remain manually selected from the image.

As previously stated, styles applied to an image that depend, for example, on season, time of day, weather conditions and/or lighting conditions are substantially irrelevant to the semantic content of the image. If a mere transition to another style is thus made, the semantic content of the image should not change.

The results of this semantic segmentation can also react sensitively to small changes in the image. An example of this (Paradebeispiel) is the so-called "Adversarial Example". This is a change in an image that is intentionally introduced, is visually inconspicuous, or is not discernible to humans at all with the naked eye, and that drastically alters the results of semantic segmentation of the image or other classification of the image result. In the same way, the results of this semantic segmentation may also be affected by artifacts that are inadvertently generated in the domain transformation. Therefore, it makes sense to monitor to what extent the assignment from locations to objects remains constant.

If the result of this semantic segmentation changes in the case of switching from domain A to domain B, there are different possibilities: how this can be taken into account in the confidence contribution.

For example, the confidence contribution of the cost function may depend on the number and/or dimension (Ausdehnung) of locations whose assignment to at least one physical object changes when the learning dataset is transformed from domain A to domain B. The confidence contribution may then be, for example, a measure for which part of the image the change in semantic segmentation relates to.

Alternatively or also in combination with this, the physical objects can be divided into classes, to which positions are respectively assigned to these physical objects. The confidence contribution of the cost function can then depend on which class of objects the at least one position in domain A is assigned to and to which class the assignment is transformed in the case of a transition to domain B. In this way, for example, changes which have a particularly disadvantageous effect on the further processing of the transformed data set, for example in the training of the trainable module, can be given a higher weight in the plausibility contribution. In the context of at least partially automated driving, for example, it is particularly critical that other traffic participants or other objects are classified as freely traversable zones after the transition to domain B.

As long as there is a skalar or vectorial measure for a change in a property that should remain unchanged in the transition from domain A to domain B, the confidence contribution of the cost function can be made to include, for example, via the The numerical sum or the sum of squares of the measure of the two- or three-dimensional spatial region over which the physically observable measurement variable is detected. The plausibility contribution then takes into account not only the magnitude of the change but also the corresponding spatial extent (Ausmaß) of the change.

In a further particularly advantageous configuration, the aggregated value of the physically observable measurement variable over a predetermined subregion of a two- or three-dimensional spatial region is selected as a property which is The case where the learning dataset is transformed from domain A to domain B should remain unchanged.

If the learning data set contains images, for example, the mean value of the colors in at least a subregion of the image can be selected, for example, as a characteristic which is to be preserved when the learning data set is converted from domain A to domain B The invariant feature or the feature should be similar to its corresponding feature of at least one learning dataset belonging to domain B after the transformation.

For example, the three lamps and the housing of a traffic signal each have a specified color that does not change or only changes insignificantly when transitioning to other daylight hours, seasons, weather conditions and/or lighting conditions. Likewise, saturated colors on traffic signs, such as black, red or blue, do not change or only change insignificantly with such a domain change. Instead, the shading (Farbgebung) of the sky, for example, changes in a defined way in the transition from noon time to sunrise or sunset. A nominally white area on a traffic sign can also change its own color visible in the image, eg depending on the position of the sun.

The adaptation of the color mean value to at least one learning dataset belonging to the domain B can be measured, for example, separately according to the object types, which in turn can be derived from the corresponding images in the two domains A and B by means of semantic segmentation was determined in. For this purpose, for example, the color mean value of all pixels of the determined object type, which occurs after the conversion from domain A to domain B, can be compared with the color mean value of all pixels of the learning data set belonging to domain B, wherein these pixels are assigned to the same object type. For example, the color mean of all pixels of an image converted to domain B that is assigned to the object type "road" can thus be assigned to the object type the same as the images in the learning dataset belonging to domain B from the beginning (vornherein) Color average comparison of all pixels of "road". The comparison can take place, for example, with a suitable reference (Norm) in the RGB color space (Farbraum). The reference is close to zero (Null) if the regions assigned to the same object type are very similar in color in the image converted to domain B as well as in the image that belonged to domain B from the beginning. Conversely, the greater the difference in hue and/or in brightness between the image areas being compared, the greater the value the reference may have. However, the metric used for this similarity included into the confidence contribution can also be interpreted as RGB in other color spaces, for example, depending only on the selective color channel (Farbkanal) and/or using other fiducial definitions ( Normdefinition). The confidence contribution for the cost function may eg be proportional to this similarity measure. But the confidence contribution for the cost function can also depend on the similarity measure in any other function. This is especially interesting if the confidence contribution measures one or more other criteria in addition to the similarity measure for the color mean.

In another particularly advantageous configuration, learning data sets are selected which contain time series of measurement data. In this way, the principle of this domain transformation can be exploited, for example, in order to diversify the learning data set of the trainable modules for predicting the behavior of the engine or other components of the vehicle.

It is therefore desirable, for example, in engine development to predict the exhaust gas behavior of the engine under real driving conditions (Real Driving Emissions (RDE)). Therefore, the engine can also be adapted during development if necessary, so that the final product passes the RDE exhaust gas test with a high probability. For prediction, it is possible to use trainable modules (eg artificial neural networks), which take as input the driving cycles that exist as time programs (Zeitprogramm) and which combine one or more harmful substances The total emitted amount is output as an output. Here, the execution of the test drive for obtaining the learning data set is relatively time-consuming and there is a need to convert, for example, the learning data set for the test drive performed in summer to the data for the test drive performed in the winter set. In this way, the number of test runs actually to be performed can be reduced.

Although time series and 2D or 3D images are the most prominent examples of measurement data used to transform from domain A to domain B using generators, this does not therefore include limiting the dimensions of measurement data to those of time series and images. Higher dimensional learning datasets can be produced, for example, by combining different data types into one and the same learning dataset. The driving cycle mentioned above as an example can be characterized, for example, by a combination of a time sequence on the one hand and an image on the other hand, wherein the image is used, for example, in order to characterize the time of day, weather conditions and/or seasons, where The time program has departed (abgefahren) at said daytime, weather conditions and/or season.

Very generally speaking, it is preferred to select a learning dataset whose measurement data has been obtained by observing and/or simulating the environment of the vehicle and/or by observing and/or simulating the crew and/or structural assemblies of the vehicle at least one state of .

The entity that trains the generator (Entität) need not necessarily be the entity that uses the generator. Instead, the trained generator is a standalone product. For example, the generator may be trained with images from a large number of situations, which are not limited to road traffic, generically in view of transitions between seasons, time of day and weather conditions and/or lighting conditions. The generator can then be used, for example, by the first enterprise for the domain transformation associated with the observation of the vehicle environment and by the second enterprise for the domain transformation in conjunction with monitoring the operating area.

Accordingly, the invention generally also relates to a generator for transforming a data set from domain A to domain B and/or to a data set with adaptable parameters which characterize the generator. The generator or the dataset is obtained using the methods previously described.

As explained before, an important application of the generator trained according to the previously described method is to transform the learning data set for the trainable module and thus in particular to the domain in which the learning data is The set is insufficient for alleviating the deficiency. Accordingly, the invention also relates to a method for training a trainable module, which method converts one or more input parameters into one or more output parameters. The behavior of the trainable module is characterized by other adaptable parameters. The other adaptable parameters are gradually adapted so that the learned values for the input variables are mapped on average to the associated learned values for the output variables.

For example, these input parameters may be pixel values of the image and the output parameters may then represent, for example, the classification of objects visible in the image.

In this method, at least one learning dataset with learned values for input parameters is transformed from domain A to domain B using the generator described previously. Within the scope of the training of the trainable module, the learned values for the input variables, which are converted to the domain B, are supplied to the trainable module.

In this way, the learned values belonging to domain B for the input parameters can be provided without having these learned values all recorded directly in domain B, either physically or analogously. In particular, situations for which measurement data were recorded only in domain A (eg "summer") can be presented to a trainable module as if these situations occurred in domain B (eg "winter").

In a further particularly advantageous configuration, a learned value for the input variable is selected which, within the range A, is based on at least one characteristic embodied in the learned value with at least one The learned values for the output parameters are used for labeling, wherein the generator leaves the properties unchanged and/or adapts the properties to the corresponding properties in the domain B in the case of transformation. According to a common language idiom in the field of machine learning, "Gelabelt" in this context means that the learned value for the output parameter has been assigned as additional information to the learning dataset. Thus, images showing different objects, for example, can be tagged with information: which objects are involved in each case.

By making the features that label the learning dataset based on itself invariant in case of domain transition and/or adapting to domain B, ensuring that the assignment of labels to the learning dataset is still in content after transition to domain B above applies (zutreffend). If, for example, the label indicates that certain objects are contained in the image in the field A, these objects are also well recognized in the image converted to the field B. The trainable module can thus learn from the applicable information which features in the image respectively indicate the presence of an object according to the label. Conversely, if these objects are no longer recognized in the image converted to domain B, the trainable module may have some features of the image that are not at all relevant to the objects present in terms of labels with properties that just indicate that these objects. Thus, the object recognition may be degraded rather than improved by the trainable module.

As mentioned at the outset, an important application of trainable modules is the control or manipulation of vehicles, for which training of these modules in turn uses the described generator. Therefore, the present invention also relates to other methods.

In this method, the generator is trained using the method described previously for training the generator. In the case of using the generator, the trainable module is trained using the method for training of the trainable module. The trainable module is operated in such a way that measurement data are supplied to the module as input variables, which measurement data have been obtained by observing the environment of the vehicle and/or by observing at least parts of the components and/or structural components of the vehicle. obtained by a state. The vehicle and/or the components or structural components of the vehicle are actuated by means of actuating signals according to the output variables provided by the trainable module.

The described method may be implemented entirely or partly by computer. Thus, these methods can be embodied, for example, in software. Accordingly, the invention also relates to a computer program having machine-readable instructions which, when executed on a computer, cause the computer to perform one of the described methods.

Likewise, the invention also relates to a machine-readable data carrier or download product having a computer program. A download product is a digital product that can be transmitted over a data network, ie can be downloaded by a user of the data network, which can be sold, for example, in an online store for immediate download.

Furthermore, a computer can be equipped with the computer program, the machine-readable data carrier or the download product.

Description of drawings

Further measures for improving the invention are further illustrated in the following in conjunction with the description of preferred embodiments of the invention on the basis of the drawings. in:

Figure 1 shows an embodiment of a method 100 for training a generator 1;

Figure 2 illustrates an embodiment of a method 200 for training a trainable module 2;

FIG. 3 shows an embodiment of a method 300 for training the generator 1 for the training of the trainable module 2 and for the subsequent operation of the trainable module 2;

FIG. 4 shows an exemplary domain transformation ( FIG. 4 a ) after training generator 1 with method 100 as images of learning dataset 11 a and after training generator 1 without method 100 as images of learning dataset 11 a Exemplary domain switching (Fig. 4b).

Detailed ways

According to FIG. 1 , in step 110 of the method 100 , the learning dataset 11 a is transformed from domain A to domain B in generator 1 to be trained. In addition to the converted result 11a', in this case, the value of the cost function 13 is provided, with which the result 11a' is evaluated.

The cost function 13 contains a confidence contribution 13a. The confidence contribution 13a may measure to what extent the characteristic 14a embodied in the learning dataset 11a remains invariant, ie survives in the domain B, when the learning dataset 11a is transferred to the domain B to what extent the (leben) result 11a' exists as before. Alternatively or in combination therewith, the confidence contribution 13a also measures: to what extent the characteristic 14b embodied in the learning dataset 11a after the learning dataset 11a has been transformed to domain B is similar to at least one in domain B The corresponding feature 14b* of the learning dataset 11b that survives in .

This cost function 13 also contains a circular argument contribution 13b. The circular argument contribution 13b measures: the extent to which the original learning data set 11a in domain A is consistently reproduced when the transformed surviving results 11a' in domain B are transformed back into domain A . This conversion back can in particular take place with a second generator, which is trained similarly to generator 1 with a method corresponding to method 100 .

In step 120 , new values for the adaptable parameters 12 are determined based on the values of the cost function 13 , which new values characterize the behavior of the generator 1 . This can be done, for example, using gradient descent.

In step 130 it is checked whether the course of the curve and/or the value of the plausibility contribution 13a of the cost function 13 satisfies a predetermined criterion 13c. If this is the case (true value 1), the training is continued in step 140 with a modified cost function 13' in which the confidence contribution 13a is less weighted or removed. Conversely, if the criterion 13c is not fulfilled (true value 0), the cost function 13 therefore remains unchanged for further training. The removal or lower weighting (Untergewichten) of the confidence contribution 13a can especially be considered in the case where the training of generator 1 is prone to such transition errors at the very beginning, which in the transition from domain B back to domain A is compensated again.

In particular, the training can be continued, for example, until a predetermined interruption condition is fulfilled for the value of the cost function 13 and/or the course of the curve. This corresponding check is not depicted in FIG. 1 for reasons of clarity.

According to block 102, in particular the learning data sets 11a, 11b comprising images can be selected. Alternatively or in combination with this, a learning data set 11 a , 11 b containing a time series (Zeitreihe) of measurement data can be selected according to block 103 . In general, according to block 104 , in particular, learning data sets 11 a , 11 b can be selected whose measurement data have been combined by observing the environment 51 of the vehicle 50 and/or by observing the crew 52 and/or structural combination of the vehicle 50 . obtained from at least one status of the item.

According to block 105, the following physical properties may be selected as properties 14a that should remain unchanged if the learning dataset 11a is converted to domain B: where the simulated boundary conditions and/or at least one physical property between domains A and B The difference in conditions does not affect the physical properties or only affects them up to a predetermined maximum measure. This may eg be the semantic meaning of the learning dataset 11a in the context of the corresponding application. Therefore, if, for example, a stop sign has switched from the domain "summer" to the domain "winter" and is covered with snow, the stop sign itself must also be recognizable.

Thus, for example, a property 14a that should remain unchanged in the case of a domain transformation can be selected according to block 106 for at least one location to at least one object at which at least one location has been detected in the learning data set 11a A physically observable measurement parameter recorded in .

According to block 107 , the value of the physically observable measurement variable aggregated over a predetermined subregion of the spatial region can be selected as the characteristic 14 a that should remain unchanged in the case of the field conversion, in which the spatial region The physically observable measurement parameters recorded in the learning data set 11a have been detected. The following expression (Ausdruck) can arise in this respect: In the case of the domain switching, the conditions under which the physically observable measurement variables are to be mainly detected do not change radically.

For example, according to block 108 , the color mean value in at least a partial area of the image can be selected as the characteristic 14a, 14b, which should remain unchanged in the case of domain conversion or be adapted to domain B. For example, it can be provided that on a traffic sign signaling a prohibition (Verbot), such as no overtaking or speed limit, the ring around the more detailed marking (Bezeichnung) surrounding the prohibition is kept outside the red tone after the domain switch The red tint should also be maintained. For example, it can be provided that, on a traffic sign signaling a command (Gebot), the background on which the command is marked in more detail, which remains blue, should also remain blue after the field switch.

FIG. 2 shows an embodiment of a method 200 for the training of a trainable module 2 that converts one or more input parameters 21 into one or more output parameters 23 . The behavior of trainable modules is characterized by adaptable parameters 22 .

In step 210, at least one learning dataset with learned values 21a for the input parameters 21 is converted from domain A to domain B using the previously described, trained generator 1 . Within the scope of the training 220 of this module 2, the learned values 21a' converted to domain B are fed to module 2. The learned value 21a originally present in domain A can thus be utilized in order to alleviate the deficiency of the learned value in domain B. The trainable module 2 can additionally contain learned values 21 a that survive in the domain A. Thus, for example, a trainable module 2 that classifies the content of the images can use not only the learning images recorded in the domain A "summer" but also the learning images thus generated with the generator 1 to survive in the domain B "winter" learning images.

In the context of the training 220 it is checked to what extent the learned values 21 a for the input variables 21 are mapped by the trainable module 2 to the associated learned values 23 a for the output variables 23 . This consistency is evaluated using the cost function 24 . In step 230 , the adaptable parameters 22 of the trainable module 22 are changed with the goal of improving the value of the cost function 24 .

In particular, the training can be continued, for example, until predetermined interruption conditions are met for the value of the cost function 24 and/or the course of the curve. The corresponding check is not depicted in FIG. 2 for reasons of clarity.

According to block 205, for example such a learned value 21a for the input parameter may be selected, wherein the learned value within the domain A is based on at least one of the following characteristics embodied in the learned value with at least one The labeling is performed on the learned value 23a of the output variable 23, wherein the generator 1 keeps the characteristic unchanged in the case of the transformation and/or adapts the characteristic to the corresponding characteristic in the domain B. In this way, these tags can also continue to be utilized after transition to domain B. This means that from a tagged learned value 21a surviving in domain A to a tagged learned value 21a' surviving in domain B, without re-labeling.

The method 300 shown by way of example in FIG. 3 combines the effect chain (Wirkkette) formed with the generator 1 and the trainable module 2 . In step 310 , generator 1 is trained using method 100 . Using this generator, in step 200, trainable module 2 is trained. In step 330 , the trainable module 2 is operated by supplying the module with measurement data as input variables 21 , which measurement data have been obtained by observing the environment 51 of the vehicle ( 50 ) and/or by observing the vehicle ( 50 ). At least one state of the unit 51 and/or structural assembly of 50 is obtained. In step 340 , the vehicle 50 and/or the units 52 or structural assemblies of the vehicle 50 are actuated using the actuation signal 3 as a function of the output variables 23 provided by the trainable module 2 .

Figure 4a schematically shows an exemplary domain transformation of an image 11a belonging to domain A to a result 11a' belonging to domain B. This image 11 a contains sky 61 , trees 62 , grass 63 and road 64 . Since it is pre-specified when training the generator 1 used in the case of domain transformation: the assignment of positions to semantic objects 61, 62, 63, 64 should be invariant in the case of domain transformation 14a, in When each of these objects 61 , 62 , 63 , 64 is shown, the image style changes respectively, but each object 61 , 62 , 63 , 64 should be identified again in the correct position in the result 11 a ′.

For comparison, FIG. 4b shows an exemplary domain transformation on the same image 11a with generator 1 not trained according to method 100 . Here, in result 11a, the image styles in which the objects 61, 62, 63, 64 are shown have not only been transformed to domain B, respectively, but have also been in these objects 61, 62, 63 and 64 were swapped between. The graphic styles of sky61 and tree62 have been interchanged with each other. Likewise, the image styles of the grass 63 and the road 64 have been interchanged with each other. The exchange is reflected in the reference numbers with which the objects in result 11a' are marked in Fig. 4b. By exchange, this result 11a' differs from the real situation in domain B to the extent that it is no longer available as training material for trainable module 2.

Claims (22)

1. A method (100) for training a generator (1) which is designed to convert a data set (11) with measurement data from a first domain a to a second domain B as a function of a first set of learning data sets (11 a) belonging to the domain a and a second set of learning data sets (11B) belonging to the domain B, wherein the behavior of the generator (1) is characterized by adaptable parameters (12) and wherein the parameters (12) are gradually adapted (110, 120) in order to optimize the value of a predefined cost function (13), wherein the cost function (13) contains a reliability contribution (13 a) which is a measure for the degree to which:

At least one characteristic (14 a) embodied in at least one learning data set (11 a) belonging to the domain a remains unchanged in the case of a conversion of the learning data set (11 a) into the domain B; and/or

At least one characteristic (14B) embodied in at least one learning data set (11 a) belonging to said domain a is similar to a corresponding characteristic (14B) of at least one learning data set (11B) belonging to said domain B after conversion to said domain B.

2. The method (100) according to claim 1, wherein the cost function (13) additionally contains at least one cyclic demonstrative contribution (13 b), which is thus a measure for: to what extent at least one learning data set (11 a) belonging to the domain a is consistently reproduced after conversion from the domain a to the domain B with the generator (1) and after subsequent conversion back from the domain B to the domain a.

3. Method (100) according to one of claims 1 to 2, wherein the training of the generator (1) is continued (140) with a cost function (13 ') in which the plausibility contribution (13 a) is weighted or removed less heavily (13') in response to the curve course and/or the value of the plausibility contribution (13 a) of the cost function (13) satisfying (130) a predefined criterion (13 c).

4. Method (100) according to one of claims 1 to 3, wherein the domain A and the domain B are distinguished with respect to at least one physical condition and/or simulated boundary condition under which the belonging learning data sets (11 a, 11B) for the domain A and the domain B, respectively, have been detected.

5. The method (100) according to claim 4, wherein the following physical characteristics are selected (105) as the characteristics (14 a) that should remain unchanged in case of converting a learning data set (11 a) from the domain A to the domain B: the differences between the domains a and B in terms of simulated boundary conditions and/or physical conditions do not affect the physical properties or only affect the physical properties with up to a predefined maximum measure.

6. Method (100) according to one of claims 1 to 5, wherein the learning data sets (11 a, 11 b) each comprise a detected distribution of at least one physically observable measurement variable in a two-dimensional or three-dimensional spatial region.

7. The method (100) according to claim 6, wherein the property (106) of at least one location at which a physically observable measurement variable has been detected to at least one physical object is selected as a property (14 a) that should remain unchanged in the case of a conversion of a learning data set (11 a) from the domain A to the domain B.

8. The method (100) according to claim 7, wherein the trustworthiness contribution (13 a) of the cost function (13) depends on the number and/or dimensions of locations whose assignment to at least one physical object changes if the learning data set (11 a) transitions from the domain A to the domain B.

9. Method (100) according to one of claims 7 to 8, wherein the following physical objects are divided into a plurality of classes, wherein positions are assigned to these physical objects, and wherein the plausibility contribution (13 a) of the cost function (13) depends on which class of objects at least one position in the domain A is assigned to and to which class the assignment transforms with a transition to domain B.

10. The method (100) according to one of claims 6 to 9, wherein the confidence contribution (13 a) of the cost function (13) comprises: a sum of values or a sum of squares over a two-or three-dimensional spatial region of a measure for a change of a property (14 a) that should remain unchanged if the learning data set (11 a) is converted from the domain a to the domain B.

11. Method (100) according to one of claims 6 to 10, wherein a predefined partial-region aggregated value of a two-dimensional or three-dimensional spatial region of the physically observable measured variable is selected (107) as a property (14 a), which property (14 a) is to remain unchanged if the learning data set (11 a) is converted from the domain a to the domain B.

12. The method (100) according to one of claims 1 to 11, wherein a learning data set (11 a, 11 b) comprising images is selected (102).

13. Method (100) according to claim 12, selecting (108) the color mean value within at least one partial region of the image as a characteristic (14 a, 14B) which should remain unchanged in the case of a conversion of a learning data set (11B) from the domain a to the domain B or which should resemble a corresponding characteristic (14B) of at least one learning data set (11B) belonging to the domain B after the conversion.

14. The method (100) according to one of claims 1 to 13, wherein a learning data set (11 a, 11 b) is selected (103) comprising a time series of measurement data.

15. Method (100) according to one of claims 1 to 14, wherein a learning data set (11 a, 11 b) is selected (104), the measurement data of which have been obtained by observing the environment (51) of a vehicle (50) and/or by observing at least one state of a crew (52) and/or a structural assembly of the vehicle (50).

16. A generator (1) for converting a data set (11) from domain a to domain B, and/or a data set having adaptable parameters (12), wherein the adaptable parameters (12) characterize the generator (1), wherein the generator and/or the data set having adaptable parameters are obtained with a method (100) according to one of claims 1 to 15.

17. A method (200) for training a trainable module (2), wherein the trainable module converts one or more input variables (21) into one or more output variables (23), wherein the behavior of the trainable module (2) is characterized by further adaptable parameters (22) and wherein the further adaptable parameters (22) are gradually adapted (220, 230) such that learned values (21 a) for the input variables (21) are averagely mapped to associated learned values (23 a) for the output variables (23), wherein the method has the following steps:

converting (210) at least one learning data set with learning values (21 a) for the input quantities (21) from domain a to domain B with the generator (1) according to claim 16;

-supplying (221) the learning value (21 a') converted into the domain B for the input variable (21) to the training-capable module (2) in the context of a training (220) of the training-capable module.

18. The method (200) according to claim 17, wherein a learning value (21 a) for the input variable (21) is selected (205), which is labeled within a domain a with at least one learning value (23 a) for the output variable (23) on the basis of at least one characteristic embodied in the learning value, wherein the generator (1) in the case of the conversion leaves the characteristic unchanged and/or adapts the characteristic to a corresponding characteristic in the domain B.

19. A method (300) having the steps of:

training (310) the generator (1) with the method (100) according to one of claims 1 to 15;

training (320) a training-capable module (2) with the method (200) according to one of claims 17 to 18, with the generator (1);

the trained module (2) is operated (330) in such a way that measurement data are supplied to the trained module as input variables (21), wherein the measurement data have been obtained by observing the environment (51) of a vehicle (50) and/or by observing at least one state of a component (52) and/or a structural assembly of the vehicle (50);

-operating (340) the vehicle (50) and/or the aggregate (52) or the structural assembly of the vehicle (50) with an operating signal (3) as a function of the output variable (23) provided by the training module (2).

20. A computer program comprising machine-readable instructions which, when executed on a computer, cause the computer to perform the method (100, 200; 300) according to one of claims 1 to 19.

21. A machine-readable data carrier and/or download product having a computer program according to claim 20.

22. A computer provided with a computer program according to claim 20 and/or with a machine-readable data carrier according to claim 21 and/or with a download product.

Images