ES3023268T3 - Method for controlling rail traffic of a plurality of railway vehicles, computing unit and computer program - Google Patents

Abstract

The invention relates to a method (100) for controlling railway traffic of a plurality of railway vehicles (215), comprising: - receiving (101) railway traffic status data; - receiving (103) a control objective (KPI1, KPI2, KPIN) for controlling railway traffic of the plurality of railway vehicles (215); - selecting (105) at least one action selection rule (Π1, Π2, ΠN) from a plurality of action selection rules (Π1, Π2, ΠN) based on the control objective (KPI1, KPI2, KPIN), where the action selection rules (Π1, Π2, ΠN) are configured to determine rail traffic control actions based on the rail traffic status data, where by executing the control actions by the railway vehicles (215), the railway traffic is brought to an optimized state with respect to a control objective (KPI1, KPI2, KPIN). can be transferred; - executing (107) the at least one selected action selection rule (Π1, Π2, ΠN) on the received status data and determining control actions; and - providing (109) the control actions for controlling the plurality of railway vehicles (215). (Automatic translation with Google Translate, no legal value)

Description

DESCRIPTION

Procedure for controlling railway traffic of a plurality of railway vehicles, computer unit and computer program

The present invention relates to a method for controlling railway traffic of a plurality of railway vehicles.

Rail traffic control involving a plurality of rail vehicles, such as in a metro or tram network, can be optimized with regard to various control objectives, for example, in terms of advantageous energy consumption or the shortest possible delay times. To this end, optimized timetables can be generated accordingly, according to which rail traffic, and in particular the plurality of rail vehicles, can be controlled. These optimized timetables are typically determined or calculated offline, i.e., not while rail transport is in operation. However, if situations arise during rail traffic operation in which the predefined and optimized timetable must be deviated from, the problem arises of how to achieve the most optimal rail traffic control possible while online, i.e., during operation.

Energy optimization for operating a fleet of rail vehicles is known from application WO 2020/043397 A1. Furthermore, application US 6,459,964 B1 describes a method for improving a train timetable, and application EP 2619 067 B1 describes a method for automatically controlling a plurality of guided vehicles.

The object of the present invention is to provide an improved method for controlling railway traffic of a plurality of railway vehicles.

This object is achieved by a method for controlling rail traffic of a plurality of rail vehicles according to independent claim 1. Advantageous configurations are indicated in the dependent claims.

According to one aspect of the invention there is provided a method for controlling rail traffic of a plurality of rail vehicles, the method comprising:

- receiving status data from the plurality of railway vehicles of railway traffic, where the status data describes control states of the plurality of railway vehicles and comprises at least position data of the railway vehicles within a railway traffic network;

- receiving a control target for controlling railway traffic of a plurality of railway vehicles;

- selecting at least one action selection rule from a plurality of action selection rules based on the control objective, where the action selection rules are configured to, based on rail traffic state data, determine rail traffic control actions, where by executing the control actions by the rail vehicles, the rail traffic can be brought into a state optimized with respect to a control objective, where each action selection rule is configured for optimization with respect to an individual control objective, and where at least one selected action selection rule is configured for optimization with respect to the received control objective;

- execution of at least one action selection rule based on the received status data and determination of control actions; and

- provision of control actions to monitor the plurality of railway vehicles.

This provides the technical advantage that an improved method for controlling rail traffic of a plurality of rail vehicles can be provided, in which a control objective can be modified during the operation of rail traffic and the movement of rail vehicles within a rail traffic network, according to which the rail traffic can be controlled or optimized. For this purpose, a plurality of different action selection rules are available, by means of which, based on rail traffic status data, control actions can be determined which, when executed by the plurality of rail vehicles in the rail traffic, enable optimization of rail traffic with respect to a predetermined control objective. Upon receipt of a control objective to be modified accordingly, an action selection rule can be selected that is optimized or configured for the respective control objective to determine corresponding control actions which, when executed by the respective rail vehicles, are designed to achieve the respective control objective. This makes it possible to modify the control objective as desired during rail traffic operation, and easily determine control actions to optimize rail traffic with respect to the selected control objective by executing the appropriate action selection rules. This eliminates the need for complex adaptation of rail traffic control.

An action selection rule, in the sense of the request, is a policy known in the field of reinforcement learning, which is configured to, in any state of rail traffic, determine corresponding control actions that are suitable for bringing rail traffic into an optimized state with respect to a predetermined control objective.

A control objective, within the meaning of the application, is a target specification according to which rail traffic for a plurality of rail vehicles is to be optimized or improved. A control objective or corresponding optimization criterion for rail traffic, for example, may comprise energy consumption, total delay of the rail vehicles, or similar factors relevant to the operation of a plurality of rail vehicles.

A railway traffic, within the meaning of the application, is a totality of a plurality of railway vehicles within a railway traffic network, and is characterized by states, those control states of the individual railway vehicles comprising at least position data of the railway vehicles within the railway traffic network, as well as time data, for example delay times of the plurality of railway vehicles relative to a predetermined time, energy consumption or other control parameters of the totality of the plurality of railway vehicles. A change in railway traffic during the control of the railway vehicles within the railway traffic network is described by state changes in railway traffic, in which the control states of the plurality of railway vehicles, as well as control parameters, such as energy consumption or total delay, undergo a time change.

Control actions, within the meaning of the application, are control actions or criteria according to which the plurality of railway vehicles are to be controlled. Control actions, within the meaning of the application, together with explicit control orders for individual railway vehicles, comprise time data that, for example, describe arrival times, departure times, or stopping times of individual railway vehicles at stops within the railway traffic network. Thus, the railway vehicles are to be controlled in such a way that they comply with or adhere to the corresponding time data of the specifications.

An optimization of rail traffic with respect to a selected or predetermined control objective, within the meaning of the application, comprises an improvement of the actual condition of the rail vehicle with respect to the respective control objective. An optimization of rail traffic, therefore, within the meaning of the application, does not necessarily have to lead to an optimal state of rail traffic with respect to the respective control objective, but can only be limited by an improvement relative to the control objective and in comparison to previous rail traffic conditions.

The method according to the invention for controlling rail traffic can thus be carried out during rail traffic operations, as individual rail vehicles move within the rail traffic network. Rail traffic optimization can be carried out in this case with respect to a predetermined schedule, according to which the plurality of rail vehicles within the rail traffic network are to be controlled.

In one embodiment, the received control objective is a combined control objective and comprises a plurality of control objectives, where the control objectives with individual weights are included in the combined control objective; comprising:

- selecting a plurality of action selection rules, where each action selection rule is respectively configured for optimization with respect to one of the plurality of control objectives of the combined control objective; and

- executing the plurality of selected action selection rules according to an execution relationship, where the execution relationship, for each selected selection rule, determines a time portion in a total execution duration of the plurality of selected action selection rules, and where an execution of the plurality of action selection rules, according to the execution relationship, meets the combined control objective.

This provides the technical advantage that a further improvement in rail traffic control can be achieved by enabling combined control objectives and corresponding optimization of rail traffic with respect to control objectives of that type. A combined control objective comprises a plurality of predetermined control objectives, which are each weighted accordingly in the combined control objective. This enables rail traffic control with respect to a plurality of control objectives.

A combined control objective, within the meaning of the application, is defined by a plurality of at least two predetermined control objectives, which are included in the combined control objective with a corresponding weighting. Rail traffic can thus be controlled simultaneously with respect to the control objectives of energy consumption and delay reduction, for example in a ratio of 60 to 40. To this end, a plurality of corresponding action selection rules, each optimized with respect to individual predetermined control objectives, are selected from the combined control objective and executed according to an execution ratio. In this case, the execution ratio is configured such that the execution of the selected action selection rules, according to the execution ratio, leads to the fulfillment of the combined control objective.

An execution of a plurality of action selection rules, according to an execution relationship, in the sense of the request, corresponds to an execution of the control actions respectively determined by the individual action selection rules, according to the respective execution relationship. An execution relationship, here in the sense of the request, for each action selection rule to be considered accordingly, describes a time portion of the respective execution of the action selection rule within a predetermined execution period. The execution of the individual control actions of the individual action selection rules, according to the execution relationship, in the sense of the request means that, according to the respective execution relationship, rail traffic is controlled according to the respective action selection rules.

Depending on the execution method, the execution relationship determines an execution duration and/or a sequence in time of the plurality of action selection rules for each action selection rule.

This allows for the technical advantage that a precise and easily determinable execution relationship is possible, in which an execution duration is determined for each action selection rule within the respective execution period and/or an execution order is determined for the plurality of action selection rules involved.

According to one form of execution, by means of the execution relation, for each selected action selection rule, an execution probability is determined, according to which the action selection rule must be executed within an execution period.

This provides the technical advantage of providing a precise and easily determined execution ratio. To this end, a probability of execution of the respective action selection rule within a corresponding execution period is determined for each action selection rule involved. The execution probability describes the probability that rail traffic will be controlled according to the control actions determined by the respective action selection rule within a predetermined period.

According to one embodiment, the selected action selection rules are executed cyclically, whereby rail traffic is controlled cyclically according to the control actions determined by the action selection rules.

This provides the technical advantage of enabling precise control of rail traffic.

In one embodiment, the selection and/or execution of the action selection rules is performed using a time-slicing function, where the time-slicing function is trained using machine learning to determine corresponding execution relationships for combined control objectives.

This allows for the technical advantage of enabling precise selection of the respective control actions that can be executed and precise determination of the respective execution ratios. Using the time-division function, which is trained using machine learning techniques to determine corresponding execution ratios for desired combined control objectives, highly complex control of rail traffic comprising a plurality of rail vehicles and optimization thereof with respect to combined control objectives can be achieved precisely and reliably. The use of machine learning techniques in this case enables more precise determination of the corresponding execution ratios and, consequently, more effective and efficient control of rail traffic.

According to the invention, the action selection rules are trained using reinforcement learning. In one embodiment, the time-slicing function is trained using reinforcement learning.

This provides the technical advantage of providing accurate and efficient training of the time-slicing function as well as action selection rules.

In one embodiment, the training of the action selection rules and/or the time-slicing function is performed based on simulation data, where the simulation data is based on a simulation of rail traffic of a plurality of rail vehicles.

This provides the technical advantage of enabling effective and efficient training of action selection rules and the time-slicing function. Furthermore, generating training data based on actual journeys of the multiple rail vehicles can be eliminated.

According to the invention, the training of the action selection rules comprises a maximization of reward functions and the reward functions define control objectives and/or combined control objectives.

In one embodiment, training the time-slicing function comprises maximizing reward functions, where the reward functions define control objectives and/or combined control objectives.

This provides the technical advantage that effective and efficient training of action selection rules and the time-slicing function can be achieved.

In one embodiment, the action selection rules and the time-slicing functions are trained simultaneously. This allows for the technical advantage of enabling improved rail traffic control. By simultaneously training the time-slicing function and the plurality of action selection rules with respect to combined control objectives, the individual control actions of the individual action selection rules can be adapted to their interaction with the other control selection rules. Through adaptation, the action selection rules can achieve better performance than would be possible with individual training of the individual action selection rules with respect to individual control objectives, and only a combination of the individually trained action selection rules, using the time-slicing function, to achieve a combined control objective. In one embodiment, the action selection rules and/or the time-slicing function are designed as neural networks.

This provides the technical advantage that efficient and robust configuration of action selection rules or time-slicing functions can be provided.

According to one embodiment, the control objectives comprise: minimizing an energy consumption of the plurality of railway vehicles and/or minimizing energy consumption of individual railway vehicles, and/or minimizing a total delay of the plurality of railway vehicles and/or minimizing a maximum delay of individual railway vehicles, and/or minimizing a sum of squared or absolute deviation of the delays with respect to a mean or median delay value of the plurality of railway vehicles.

This provides the technical advantage of enabling precise rail traffic control, which allows rail traffic to be optimized with respect to relevant control objectives.

According to one embodiment, the control actions comprise arrival times and/or departure times and/or stopping times of the plurality of railway vehicles at stops within the railway traffic network.

This allows for the technical advantage of enabling efficient rail traffic control by considering combined control objectives. By considering departure times and/or arrival times and/or stopping times of rail vehicles at stops within the rail traffic network in the respective control actions, precise rail traffic control can be achieved. The corresponding arrival times and/or departure times and/or stopping times in this case can be linked to a predetermined optimized timetable. In this case, individual rail traffic vehicle control can be configured so that the respective rail vehicles observe the departure times and/or arrival times and/or stopping times defined in the control actions.

According to a second aspect, a computing unit is provided, the computing unit being designed to execute the method for controlling a railway traffic of a plurality of railway vehicles according to one of the preceding embodiments.

According to a third aspect of the invention, there is provided a computer program product comprising commands which, when the program is executed by a data processing unit, arrange the program to execute the method for controlling rail traffic of a plurality of rail vehicles, according to one of the preceding embodiments.

The features and advantages of this invention, described above, as well as the manner of achieving them, are clarified and made more understandable by the explanation of the following highly simplified representations of preferred embodiments. They show respectively:

Figure 1 a schematic representation of a system for controlling railway traffic, according to one embodiment.

Figure 2 a flowchart of a procedure for controlling railway traffic, according to one embodiment;

Figure 3 a diagram of a railway traffic optimization with respect to two control objectives;

Figure 4 a schematic representation of an execution of two action selection rules according to an execution relationship;

Figure 5 a schematic representation of an architecture for a time division function, according to one embodiment;

Figure 6 another schematic representation of an architecture for a time division function, according to another form of implementation;

Figure 7 shows a hypervolume diagram of a railway traffic optimization state; and Figure 8 shows a schematic representation of a software product.

Figure 1 shows a schematic representation of a system 200 for optimizing railway traffic, according to one embodiment.

A system 200 for optimizing railway traffic, according to the embodiment shown, comprises a plurality of modules that can be executed on a computer unit 201. The system 200 can furthermore be divided into an offline subsystem 202 and an online subsystem 204, where the offline subsystem 202 is executed offline, i.e. independently of an operation of the railway vehicles 215, while the online subsystem 204 is executed during the operation of the railway vehicles 215.

The central component of the offline subsystem 202 is a schedule optimization module 211. The schedule optimization module 211 is used to create an optimized schedule for rail traffic of a plurality of rail vehicles in a rail traffic network. The schedule optimization module 211, via a first interface, can transmit the optimized schedule created in the offline subsystem 202 to the online subsystem 204, so that the schedule can be executed to control the rail traffic.

In the actual operation of railway traffic, the timetable, now referred to as the online timetable, is managed by a timetable management module 205. For this purpose, position data of individual railway vehicles in the railway traffic can be transmitted to the timetable management module 205 via an automatic railway vehicle tracking module 207, via a second interface S2, so that it can perform a comparison between target movements in the timetable and actual movements actually performed by the railway vehicles in the railway traffic.

The timetable management module 205 can transmit commands for selecting railway routes that can be used accordingly via a third interface S3 to an automatic track selection module 209. This enables appropriate railway routes to be reserved in good time to ensure that railway vehicles operate with minimal delays.

The timetable management module 205 further has a bidirectional interface S2, S4 with an automatic rail vehicle control module 203. If the online timetable is to be changed due to a new target time of the timetable optimization module 211 or due to changes in a switchman, the automatic rail vehicle control module 203 can be informed accordingly in order to trigger a change in the online timetable. For this purpose, the automatic rail vehicle control module 203 additionally requires actual position data of the rail vehicles in order to determine actual rail vehicle delays and, in the case of larger deviations, to be able to initiate appropriate controls.

These regulations may include control actions to be executed by railway vehicles, such as departure times and/or arrival times and stopping times of railway vehicles at stops on railway routes in the railway traffic network, which are desired or which must be achieved or observed by railway vehicles in order to avoid delays and to modify the timetable. The corresponding regulations and/or control actions may be transmitted to the timetable management module 205 via a fifth S5 interface.

Finally, for timetable adaptation, the corresponding regulations and control actions and the corresponding optimized arrival times/departure times/stop times are transmitted from the automatic rail vehicle management module 205 via a sixth interface S6 to automatic rail vehicle control modules 213 of the individual rail vehicles 315 of the rail traffic, so that they can execute the desired modification or control actions to achieve or observe the optimized arrival times/departure times/stop times.

In the case of railway vehicles operating automatically, the automatic railway vehicle control module 213 can determine energy-optimized travel paths for the railway vehicle 203 that best fit the actual line schedule. In the case of manual operation, the vehicle driver is informed, but is free to implement the regulations.

In the embodiment shown, the automatic rail vehicle control module 203 is configured, based on status data of a rail traffic to be optimized, of a plurality of rail vehicles, to execute the method according to the invention for controlling a rail traffic of a plurality of rail vehicles 215.

Figure 2 shows a flowchart of a procedure 100 for controlling railway traffic, according to one embodiment.

The method 100 according to the invention for controlling a railway traffic of a plurality of railway vehicles 215 can be carried out by a system 200 according to the embodiment in Figure 1 and in particular by an automatic railway vehicle regulation module 203.

In order to control rail traffic according to the inventive method 100, first, in a method step 101, rail traffic status data are received from the plurality of rail vehicles 215. The status data describe rail traffic statuses and comprise at least position data of the plurality of rail vehicles, according to which positioning of the rail vehicles within the rail traffic network is possible. The status data may further comprise rail traffic control criteria, for example, energy consumption of the rail traffic or of individual rail vehicles, or delays of the rail vehicles, both as a whole and individually, and other criteria relevant to controlling rail traffic.

The method can be executed during a rail traffic operation, in which rail vehicles travel within a rail network according to a schedule. The status data here describes rail traffic states and describes at least the positions within the rail network where vehicles are located at a given time. The status data can also comprise time data, for example, data on the delay of individual rail vehicles relative to an optimized schedule.

In a further step of the procedure 103, a control objective KPI<1>, KPI<2>, KPIn is received. The control objective can be given, for example, by an energy consumption of all railway vehicles or a total delay of all railway vehicles, by individual delays of individual railway vehicles.

The control objective KPI<1>, KPI<2>, KPI<n> can be obtained during rail traffic operation and thus during the movement of rail vehicles within the rail network. This enables the control objective KPI<1>, KPI<2>, KPIn to be varied during online rail traffic operation. The control objective here indicates a criterion according to which rail traffic control can be carried out. For example, rail traffic can be controlled so that the total energy consumption of rail vehicles is minimal. Alternatively, rail traffic can be controlled to prevent or minimize delays of rail vehicles with respect to the schedule.

In a method step 105, at least one action selection rule ni, n<2>, nN is then selected from a plurality of previously stored action selection rules m, n<2>, nN. The action selection rules m, n<2>, nN are configured to determine control actions based on rail traffic status data, which, in the case of executions by rail traffic vehicles, lead to an optimization of rail traffic with respect to a predetermined control objective KPI<1>, KPI<2>, KPI<n>. The individual action selection rules m, n<2>, nN are optimized with respect to an individual control objective KPI<1>, KPI<2>, KPI<n>. For example, an action selection rule rn, n<2>, nN is optimized with respect to an optimization with respect to a total energy consumption of rail traffic, while another action selection rule is optimized with respect to a minimization of a total delay of the rail vehicles of the rail traffic. An optimization of an action selection rule m, n<2>, nN with respect to a control objective KPI<1>, KPI<2>, KPI<n> can be understood such that the respective action selection rule m, n<2>, nN is configured to determine corresponding control actions that, in the case of execution by the rail vehicles, lead to an optimization of the rail traffic with respect to the respective control objective KPI<1>, KPI<2>, KPIn. Accordingly, in method step 105, from the plurality of previously stored action selection rules m, n<2>, nN, the action selection rule rn, n<2>, nN that is optimized with respect to the received control objective KPI<1>, KPI<2>, KPIn is selected.

The control actions determined by the action selection rules m, n<2>, nN on the basis of railway traffic status data, together with, or alternatively to, actual control orders for the railway vehicles, may comprise time specifications in the form of arrival, departure and/or stopping times of the railway vehicles at stops on the railway network. An action selection rule m, n<2>, nN optimized for example with respect to an energy consumption of the plurality of railway vehicles is thus configured to, based on railway traffic status data, in which at least the positions of the railway vehicles at a given time are described, determine arrival, departure and/or stopping times of the railway vehicles for stops that are to be controlled in the future, where the arrival, departure and/or stopping times are configured such that when controlling the railway vehicles, it is achieved that the railway vehicles comply with the arrival, departure and/or stopping times, reducing the energy consumption of the vehicles to a minimum.

The correspondingly determined arrival, departure, and/or stopping times may vary here for different control objectives. For example, to reduce energy consumption, it may be advantageous for a vehicle to stay longer at a stop in order to synchronize the vehicle's start with the acceleration and/or braking of other rail vehicles. In contrast, to reduce a delay for rail vehicles, longer stopping times may not be required, as this would result in significantly greater delays. Furthermore, the correspondingly determined arrival, departure, and/or stopping times depend on rail traffic data and, in particular, on the positions of the individual vehicles.

In a method step 107, at least one action selection rule rn, n<2>, nN selected with respect to the received rail traffic status data is executed, and corresponding control actions are determined. The determined control actions, for example, the corresponding arrival, departure, and/or stopping times, are configured in this case such that by controlling the rail vehicles accordingly, the rail traffic can be brought to a state that is optimized with respect to the respective control objective or at least improved compared to previous states in time. For example, in a state generated in this way, a delay relative to a predetermined schedule can be reduced or an energy consumption can be reduced.

In a method step 109, control actions according to the embodiment shown in FIG. 1 are provided to the individual modules of the system 200, thereby enabling rail traffic control. Control in this case comprises controlling the railway vehicles such that the railway vehicles observe the arrival, departure, and/or stopping times determined in the method step. The corresponding arrival, departure, and/or stopping times, in this case, in addition to a stop directly ahead of a respective vehicle on a track of the transport network, may include a plurality of stops that are reached by the railway vehicle on the track within a predetermined time period in the case of a corresponding control. The corresponding control actions for a railway vehicle may thus comprise a set of arrival, departure, and/or stopping times for a plurality of stops.

According to one embodiment, a combined KPI control objective is received in method step 103. The combined KPI control objective comprises a plurality of the predetermined control objectives KPI<1>, KPI<2>, KPI<n>, which are respectively included with individual weights W<1>, W<2>, W<n> in the combined KPI control objective. A combined KPI control objective can thus comprise, for example, simultaneous optimization with respect to energy consumption and with respect to minimization of a total delay of rail traffic, where the optimization with respect to energy consumption and the optimization with respect to total delay can be included in the combined KPI control objective in any ratio, for example from 60 to 40. In this way, control can be carried out such that, for example, a 60% reduction in energy consumption and a 40% reduction in delay are achieved. The control objectives described and the ratios and percentages indicated are illustrative only and do not represent any limitation of the invention.

In method step 105, a plurality of action selection rules n, n<2>, nN are then selected from the previously stored action selection rules ni, n<2>, nN. The action selection rules n, n<2>, nN selected in this case are optimized with respect to corresponding control objectives KPI<1>, KPI<2>, KPIn of the combined control objective KPI.

In method step 107, the action selection rules n, n<2>, nN of the selected plurality are then executed according to an execution relationship. The execution relationship, in this case for each of the selected action selection rules n, n<2>, nN, describes a time portion in a total execution duration of the plurality of executed action selection rules n, n<2>, nN and makes it possible to meet the combined KPI control objective by executing the selected action selection rules rn, n<2>, nN according to the corresponding execution relationship. In order to control rail traffic, railway vehicles are controlled in the respective execution relationship according to the control actions determined by the selected action selection rules rn, n<2>, nN. In this case, the execution ratio may correspond to the weighting of the individual control objectives KPI<1>, KPI<2>, KPIn of the combined control objective KPI, so that, according to the example described above, rail traffic is controlled, for example, in 60% of a control period based on an optimization with respect to energy consumption and in 40% of the execution period based on an optimization with respect to total delay. The control of the railway vehicles thus takes place according to the control actions determined by the action selection rules rn, n<2>, m. The execution ratio, however, may also differ from the respective weighting W<1>, W<2>, W<n> of the individual control objectives KPI<1>, KPI<2>, KPIn of the combined control objective KPI.

The execution relationship, in this case, according to one embodiment, may describe an execution duration t<1>, t2, tN for each of the individual action selection rules n, n<2>, nN and/or may determine a temporal sequence of the plurality of action selection rules n, n<2>, m. Thus, in accordance with the example described above, rail traffic may be optimized with respect to energy consumption by a correspondingly determined execution duration t<1>, t2 and optimized with respect to a total delay by a corresponding duration t<1>, t2. The execution durations t<1>, t2 may in this case be divided into corresponding time windows, so that, for example, rail traffic may be optimized alternately with respect to one control objective and then with respect to the other.

Rail traffic control according to the execution ratio and the correspondingly determined execution duration t<1>, t<2>, tN for the individual action selection rules n, n<2>, m means here that the railway vehicles are controlled for one execution duration such that those arrival, departure and/or stopping times are reached or observed, which are suitable for achieving a first control objective, and are controlled for another execution duration in order to reach or observe other arrival, departure and/or stopping times, which are configured to achieve a further control objective of the combined control objective.

According to another execution method, the execution ratio, for each selected action selection rule ni, n<2>, nN, can determine an execution probability P, according to which the action selection rule rn, n<2>, nN must be executed in an execution period. Depending on the execution ratio, the probability with which each action selection rule ni, n<2>, nN must be executed in a corresponding execution period is stochastically determined. This means that rail traffic, depending on the corresponding execution probability P, can be optimized during an execution period with respect to energy consumption or total delay. An execution time, for example, can be given by a cycle time in the case of cyclic rail traffic control, such that for each cycle time of a cycle to be executed of rail traffic control, according to the corresponding execution probabilities P, rail traffic can be optimized with respect to the respective control objectives KPIi, KPI<2>, KPI<n>.

The selection in method step 105 and/or the execution in method step 107 of the plurality of action selection rules ni, n<2>, nN may be executed by a time-slicing function 300. The time-slicing function 300, in this case by machine learning techniques, may determine corresponding execution ratios taking into account combined KPI control objectives. The time-slicing function 300, for example, may be trained by reinforcement learning. In particular, the time-slicing function 300 may be designed by a correspondingly trained neural network 303.

Correspondingly, the action selection rules rn, n<2>, nN are trained using reinforcement learning to optimize the respective control objectives KPIi, KPI<2>, KPIn. The action selection rules rn, n<2>, nN can also be designed as correspondingly trained neural networks.

The training of the time-division function 300, as well as the action selection rules rn, n<2>, nN, can in this case be performed using simulation data from a railway traffic simulation. The simulation data, in this case, can be provided by a corresponding simulation, for example by the Falko simulation program.

The training of the action selection rules rn, n<2>, nN comprises a maximization of a reward function and/or the training of the time-slicing function 300 may in this case comprise a maximization of a reward function which, correspondingly, comprises the respective control objectives KPIi, KPI<2>, KPI<n> or combined KPI control objectives.

The training of the action selection rules rn, n<2>, nN can be carried out individually here, such that each action selection rule rn, n<2>, nN is trained for an individual control objective KPIi, KPI<2>, KPIn. Based on the pre-trained action selection rules rn, n<2>, nN, the time-slicing function 300 can then be trained for various combined control objectives KPI, which, taking into account different weights Wi, W<2>, W<n>, comprise the control objectives KPIi, KPI<2>, KPIn of the individual action selection rules rn, n<2>, nN, in terms of determining corresponding execution relationships.

Alternatively, a simultaneous training of the action selection rules ni, n<2>, nN and the time-slicing function 300 may be performed. In a simultaneous training of the action selection rules rn, n<2>, nN and the time-slicing function 300, thus, the action selection rules rn, n<2>, nN pre-trained and optimized with respect to the individual control objectives KPIi, KPI<2>, KPIn may be trained to meet the combined control objective KPI, while the time-slicing function 300 is trained to determine a corresponding execution ratio.

The method according to the invention can be applied to a plurality of desired control objectives KPIi, KPI<2>, KPIn and corresponding combined control objectives KPI, and can be applied with respect to a corresponding desired number of action selection rules rn, n<2>, nN trained accordingly.

Figure 3 shows a diagram of a railway traffic optimization with respect to two control objectives (KPIi, KPI<2>, KPIn).

Figure 3 shows a Pareto front diagram of rail traffic that is controlled or optimized with respect to two control objectives KPIi, KPI<2>. The diagram in Figure 3 shows in this case a percentage optimization Oi with respect to the first control objective KPIi and a percentage optimization O<2> with respect to the second control objective KPI<2>. Several optimized states Opt, Opt1, Opt2 are shown. The plotted points of the Pareto front show optimizations of rail traffic with respect to the two optimization objectives KP1<1>, KPI<2> for different weights W<2> of the second control objective KPI<2>. Point Opt1 shows here an optimized state of rail traffic for a weight W<2> equal to 0 of the second control objective KPI<2>, such that rail traffic, in the shown optimization state Opt1, is 100 % optimized with respect to the first control objective KPI<1>. The optimized state Opt2, on the other hand, shows an optimization of rail traffic for a weighting W<2> of 100 of the second control objective KPI<2>, so that in this optimized state, rail traffic is optimized exclusively with respect to the second control objective KPI<2>. The other optimized states show different relationships of the two control objectives KPI<1>, KPI<2>, which, depending on the respective weights W<1>, W<2>, are combined into a combined control objective KPI.

The diagram shown represents an optimization of railway traffic with respect to a reference state Ref, such that the percentage optimizations O<1>, O<2> are represented as improvements of railway traffic with respect to the first and second control objectives KPI<1>, KPI<2> with respect to the reference state Ref.

The numerical values shown are for illustrative purposes only and do not represent an actual optimization of rail traffic. The illustrated rail traffic optimization process can, for example, describe a situation in which, due to the operation of rail vehicles, rail traffic has deviated from a predetermined optimized timetable, and thus, by performing the method according to the invention and by correspondingly optimizing rail traffic with respect to the respective control objectives KPI<1>, KPI<2>, KPI<n>, an improvement in or redirection of rail traffic to the predetermined optimized timetable is to be achieved, taking into account the respective control objectives.

Figure 4 shows a schematic representation of an execution of two action selection rules (rn, n<2>) according to an execution relationship.

Figure 4 shows a time curve of an execution of two action selection rules ni, n<2> over an execution period which, in the execution form shown, is given by a cycle time tc. In the execution form shown, the first action selection rule rn is executed for a first execution duration t<1>, while the second action selection rule n<2> is then executed for a second execution duration t<2>. In the execution form shown, the execution period of cycle time tc is executed in individual time intervals ts. The action selection rules m, n are executed in time intervals ts and thus, depending on the respective control actions, cause the rail traffic to transition to different states St, St+<1>, ..., St+<9>. The representation in Figure 4 shows an example of an execution relationship in which each action selection rule m, n<2> is associated with a corresponding execution duration t<1>, t<2>. Alternatively, corresponding execution probabilities P can be associated with the first and second action selection rules m, n<2>, such that the action selection rules m, n<2> are executed in the individual time intervals ts according to the respectively associated execution probabilities P.

Figure 5 shows a schematic representation of an architecture for a time division function 300, according to one embodiment.

Figure 5 shows an architecture of a time-slicing function 300 for the execution and control of rail traffic. In the execution function shown, the time-slicing function 300 is applied to control rail traffic with respect to a combined control objective KPI comprising control objectives KPI<1>, KPI<2>, KPI<n> provided with corresponding weights W<1>, W<2>, W<n>. The time-slicing function 300 in this case is configured to generate an execution relation according to the execution manner shown in Figure 4, which for each action selection rule m, n<2>, nN determines a corresponding execution duration.

In the shown embodiment, the time-slicing function 300 comprises a time-slicing module 301 and a selection module 302. The time-slicing module 301 is configured to determine corresponding execution relationships for combined KPI control objectives. Furthermore, the selection module 302 is configured to, based on the determined execution relationships, select respective action selection rules rn, n<2>, m. In the shown embodiment, the time-slicing module 301 is designed as a neural network 303. The selection module 302, in the shown embodiment, further comprises an Argmax function that is configured to select respective action selection rules m, n<2>, m.

In addition to the control objectives KPIi, KPI<2>, KPIn provided with the weights Wi, W<2>, Wn, a modulation operator mod is included as an activation function in the neural network 303 of the time-slicing module 301. The modulation operator mod enables the division of the cycle time tc as an execution period into the time intervals ts shown in Figure 4. In contrast to the execution method shown, the modulation operator mod can also be arranged as an activation function in deeper layers of the neural network. Overall, the neural network structures shown in Figures 5, 6 are merely illustrative and do not describe neural network structures that can be effectively implemented.

In the architecture shown, the time-slicing function 300 is trained to determine a corresponding execution ratio for desired combined KPI control objectives, according to which the action selection rules ni, n<2>, nN optimized with respect to the individual KPIi, KPI<2>, KPI<n> control objectives should be executed.

In order to execute the time division function 300 and the action selection rules ni, n<2>, nN for controlling rail traffic according to the method 100 according to the invention, status data of the rail vehicles are received during the operation of the rail traffic and during the operation of the plurality of rail vehicles. In this case, for example, the status data may indicate that an actual status of the rail traffic differs from a predetermined timetable.

Rail traffic control can thus be designed to bring rail traffic back into line with the optimized timetable and, accordingly, reduce delays. This control can be planned with respect to various control objectives: KPIi, KPI<2>, and KPIn. For example, rail traffic control can be planned to be as energy-efficient as possible. Alternatively, delay reduction can be planned as quickly as possible.

In the embodiment shown, rail traffic control is provided taking into account a combined control objective KPI comprising a plurality of control objectives KPIi, KPI<2>, KPI<n> in a given weighting. For example, rail traffic control can be provided such that the restoration of the optimal timetable takes place taking into account minimized energy consumption and the shortest possible time until the optimized timetable is again aligned.

For rail traffic control, therefore, a corresponding combined KPI control objective is received, comprising the respective planned control objectives KPIi, KPI<2>, KPI<n> with corresponding weights, where the weights Wi, W2, W<n>, for each KPIi, KPI<2>, KPI<n>, describe a priority within rail traffic control.

For control, the time division function 300 is applied with respect to the received combined control objective KPI, and a corresponding execution relationship is determined. The execution relationship indicates a relationship according to which the action selection rules m, n<2>, nN corresponding to the respective control objectives KPIi, KPI<2>, KPI<n> and optimized with respect thereto are to be executed, and according to which the rail traffic is to be controlled with respect to the respective control actions of the action selection rules m, n<2>, nN.

According to the determined execution relationship, the corresponding action selection rules m, n<2>, nN are selected by means of the selection module 302, and the selected action selection rules m, n<2>, nN are applied with respect to the received rail traffic status data. The application of the action selection rules m, n<2>, nN with respect to the status data, in the manner described above, comprises the determination of corresponding control actions and, if applicable, corresponding arrival, departure and stopping times that are optimized with respect to the respective control objectives. Each action selection rule m, n<2>, nN and the control actions that can be determined correspondingly by means thereof are optimized with respect to one of the control objectives.

By executing an m, n<2>, nN action selection rule on the railway traffic state data, the m, n<2>, nN action selection rule is able to determine control actions that are designed to, when executed by the railway vehicles, transition the railway traffic from an actual state into a state that is optimized with respect to the respective control objective or that is at least improved compared to the preceding state in time.

In the execution form shown, the execution relationship describes a time relationship that associates an execution duration to each action selection rule m, n<2>, nN involved, indicating the time interval during which the respective action selection rule m, n<2>, nN is to be executed. The time relationship in this case is configured such that the respective combined KPI control objective is achieved.

The rail traffic control can be executed cyclically, such that according to predetermined control cycles, status data and control objectives are received, the time division function 300 is applied with respect to the control objectives, corresponding execution relationships are determined, the corresponding action selection rules m, n<2>, nN are executed according to the respective execution relationships with respect to the received status data, corresponding control actions are determined and the railway vehicles are controlled according to the control actions and, thereby, the railway traffic is brought to an optimized or at least improved state with respect to the respective control objective.

The depiction of the time-division function 300 shown is for illustrative purposes only. The depicted neural network 301 does not represent a neural network that should actually be implemented.

Figure 6 shows another schematic representation of a training architecture for a 300-time slice function, according to another implementation form.

Figure 6 shows another embodiment of an architecture for a time-slicing function 300. In the embodiment shown, the time-slicing function 300 is trained to determine an execution relation that, for each action selection rule ni, n<2>, nN, determines corresponding execution probabilities P. Unlike the embodiment in Figure 5, the selection module 300 comprises a Softmax function 305 that is configured to, based on the execution probabilities P determined by the time-slicing module 301, select corresponding action selection rules ni, n<2>, nN.

The execution of the architecture takes place analogously to the execution of the architecture in Figure 6. Instead of the execution duration described there, in the architecture represented in Figure 7, however, the individual action selection rules are executed at predetermined time intervals according to the execution probabilities P determined by means of the time division function 300.

Figure 7 shows a hypervolume HV diagram of an optimization state of a railway traffic.

Figure 7 shows a graphical representation of a hypervolume HV for different optimized states of a rail traffic that needs to be optimized with respect to two control objectives KPIi, KPI<2>. In this case, the hypervolume can be used as an objective variable to determine the quality of a training process or the quality of the optimization capabilities of individual action selection rules m, n<2>, nN, or of the time slice function 300.

Figure 8 shows a schematic representation of a computer program product 40 comprising commands which, when the program is executed by a computing unit, arrange the program to execute the method 100 according to one of the aforementioned execution forms.

The computer program product 400, in the embodiment shown, is stored on a storage medium 401. In this case, the storage medium 401 may be any storage medium known to the prior art.

Although the invention has been illustrated and described in detail by means of the preferred embodiment, the invention is not limited by the examples described, and the expert can deduce from them other variations, without abandoning the scope of protection of the invention.

Claims (15)

1. Method (100) for controlling railway traffic of a plurality of railway vehicles (215), comprising: - receiving (101) railway traffic status data, where the status data describe railway traffic control states of the plurality of railway vehicles (215) and comprise at least position data of the plurality of railway vehicles; - receiving (103) a control objective (KPI<1>, KPI<2>, KPIn) for controlling the railway traffic of the plurality of railway vehicles (215); - selecting (105) at least one action selection rule (m, n<2>,<hn>) from a plurality of action selection rules (m, n<2>, hn) based on the control objective (KPI<1>, KPI<2>, KPIn), the action selection rules (ni, n<2>, hn) being configured to, based on rail traffic status data, determine rail traffic control actions, by executing the control actions by the rail vehicles (215), the rail traffic can be brought into an optimized state with respect to a control objective (KPI<1>, KPI<2>, KPI<n>), each action selection rule (m, n<2>,<hn>) being configured for optimization with respect to an individual control objective (KPI<1>, KPI<2>, KPI<n>), and at least one action selection rule (m, n<2>,<hn>) being configured for optimization with respect to an individual control objective (KPI<1>, KPI<2>, KPI<n>). n<2>,<hn>) selected is configured for optimization with respect to the received control objective (KPI<1>, KPI<2>, KPIn); - execution (107) of at least one action selection rule (m, n<2>, hn) selected with respect to the received state data and determination of control actions; and - provision (109) of control actions for controlling the plurality of railway vehicles (215), where the action selection rules (m, n<2>,<hn>) are trained by reinforcement learning, the training of the action selection rules (m, n<2>,<hn>) comprises a maximization of reward functions and the reward functions define control objectives (KPI<1>, KPI<2>, KPIn) and/or combined control objectives (KPIs). 2. Method (100) according to claim 1, wherein the received control objective (KPI<1>, KPI<2>, KPI<n>) comprises a combined control objective (KPI) and a plurality of control objectives (KPI<1>, KPI<2>, KPIn), and wherein the control objectives (KPI<1>, KPI<2>, KPIn) with individual weights (W1, W2, WN) are included in the combined control objective (KPI); comprising: - selecting (105) a plurality of action selection rules (KPI<1>, KPI<2>, KPIn), wherein each action selection rule (m, n<2>, <hn>) is respectively configured for optimization relative to one of the plurality of control objectives (KPI<1>, KPI<2>, KPIn) of the combined control objective (KPI); and - executing (107) the plurality of selected action selection rules (m, n<2>,<hn>) according to an execution relationship, where the execution relationship, for each selected action selection rule (m, n<2>, hn>), determines a time portion in a total execution duration of the plurality of selected action selection rules (m, n<2>,<hn>), and where an execution of the plurality of action selection rules, according to the execution relationship, meets the combined control objective (KPI). 3. Method (100) according to claim 2, wherein by means of the execution relationship, for each action selection rule (m, n<2>,<hn>), an execution duration (t<1>, t2) and/or a sequence in time of the plurality of action selection rules (m, n<2>, hn>) is determined. 4. Method (100) according to claim 2, wherein by means of the execution relation, for each action selection rule (m, n<2>, nN) selected, an execution probability (P) is determined, according to which the action selection rule (m, n<2>, hn) must be executed in an execution period. 5. Method (100) according to one of the preceding claims 2 to 4, wherein the selected action selection rules (m, n<2>, hn) are executed cyclically, and wherein the railway traffic is controlled cyclically according to the control actions determined by the action selection rules (m, n<2>, hn). 6. Method (100) according to one of the preceding claims, wherein the selection and/or execution of the action selection rules (ni, n<2>,<hn>) is executed by means of a time division function (300), and where the time division function (200) is trained by means of machine learning, to determine corresponding execution relationships for combined control objectives (KPIs). 7. Method (100) according to one of the preceding claims, wherein the time division function (200) is trained by reinforcement learning. 8. Method (100) according to claim 7, wherein the training of the action selection rules (ni, n<2>, hn) and/or the time division function (300) is executed based on simulation data, and where the simulation data is based on a simulation of a railway traffic of a plurality of railway vehicles (215). 9. Method (100) according to claim 7 or 8, wherein the training of the time-slicing function (300) comprises a maximization of reward functions, and where the reward functions define control objectives (KPI<1>, KPI<2>, KPI<n>) and/or combined control objectives (KPI). 10. Method (100) according to claim 7, 8 or 9, wherein the action selection rules (m, n<2>, hn) and the time division functions (300) are trained simultaneously. 11. Method (100) according to one of the preceding claims, wherein the action selection rules (m, n<2>, hn) and/or the time division function (300) are designed as neural networks (303). 12. Method (100) according to one of the preceding claims, wherein the control objectives (KPI<1>, KPI<2>, KPI<n>) comprise: minimizing an energy consumption of the plurality of railway vehicles (215) and/or minimizing energy consumption of individual railway vehicles (215), and/or minimizing a total delay of the plurality of railway vehicles (215) and/or minimizing a maximum delay of individual railway vehicles (215), and/or minimizing a sum of squared or absolute deviation of the delays with respect to a mean or median delay value of the plurality of railway vehicles (215). 13. Method (100) according to one of the preceding claims, wherein the control actions comprise arrival times and/or departure times and/or stopping times of the plurality of railway vehicles (215) at stops within the railway traffic network. 14. Computer unit (201), wherein the computer unit (201) is designed to execute the method (100) for controlling a railway traffic of a plurality of railway vehicles according to one of the preceding claims 1 to 13. 15. Computer program product (400) comprising commands which, when the program is executed by a data processing unit, arrange the same to execute the method (100) for controlling a railway traffic of a plurality of railway vehicles according to one of the preceding claims 1 to 13.