WO2025171818A1 - Data processing method, apparatus and device, and computer-readable storage medium - Google Patents

Abstract

Disclosed in the present application are a data processing method, apparatus and device, and a computer-readable storage medium. The method comprises: using a policy network, a value function network and a tail risk network that are to be trained to interact with an environment for multiple times, so as to obtain a feedback sample set; using the feedback sample set to separately perform at least one round of training on the policy network, the value function network and the tail risk network; and after each network has undergone at least one round of training and a preset training finish condition being satisfied has been detected, obtaining a trained policy network, so as to adjust hedging portfolios of financial derivatives on the basis of the policy network.

Description

Data processing method, device, equipment and computer-readable storage medium

Related applications

This application claims priority to Chinese patent application No. 202410179678.4 filed on February 18, 2024, the entire contents of which are incorporated herein by reference.

Technical Field

The present application relates to the fields of financial risk control and artificial intelligence technology, and in particular to a data processing method, apparatus, device, and computer-readable storage medium.

Background Art

In the financial derivatives market-making business of securities firms, real-world enterprises need to purchase financial derivatives from securities firms to manage financial risk. For example, steel producers often hold iron ore inventories. To hedge against the risk of future price declines, they need to purchase iron ore put options from securities firms to protect their value. After selling financial derivatives to real-world enterprises or financial institutions, securities firms need to effectively hedge to mitigate the financial risks they assume, particularly the tail risk of hedging errors within the hedge portfolio. Specifically, they need to construct a hedge portfolio for the financial derivative, consisting of cash (i.e., a risk-free asset) and hedging assets to replicate the financial derivative. Specifically, by adjusting the number of hedging assets in the hedge portfolio, the total value of the hedging portfolio, over the life of the financial derivative contract, is equal to or not less than the value of the financial derivative—the value of one or more payments (i.e., the payment amount) stipulated in the financial derivative contract to the buyer of the financial derivative. The payment amount is determined by the price of the underlying asset. For example, if the financial derivative is a CSI 300 Index put option, the underlying asset is the CSI 300 Index. At the expiration date of the CSI 300 Index put option, the contractually agreed-upon amount payable to the option buyer is equal to the difference between the contractually agreed-upon strike price and the market price of the underlying asset (i.e., the CSI 300 Index), and the maximum of zero. The hedging error of a hedge portfolio is the difference between the value of the hedge portfolio and the value of the financial derivative. When the hedging error is less than zero—that is, when the value of the hedge portfolio falls below the value of the derivative—the securities firm incurs a loss. Therefore, securities firms need to effectively manage the financial risks associated with selling financial derivatives, particularly the tail risk of hedging error, which is the risk of the firm incurring large losses. Currently, trained models can be used to help adjust the hedged assets in a hedge portfolio to reduce this tail risk. However, while current model training methods consider the tail risk of hedging errors in a hedge portfolio, the resulting models are not applicable to the same financial derivative with different initial state information (e.g., different underlying initial prices, different strike prices, and different expiration times). Consequently, different models must be trained for each financial derivative with different initial state information, resulting in high training costs in terms of time, computing power, and hardware resources. Furthermore, existing methods that consider tail risk can only train using data generated by simulations based on parametric models, rather than directly using real market observation data. This prevents direct and effective use of real market data, prevents modeling errors, and prevents errors in parameter estimation. Consequently, securities companies are unable to effectively reduce the tail risk of hedging errors.

Summary of the Invention

The main purpose of this application is to provide a data processing method, apparatus, device and computer-readable storage medium, which aims to solve the technical problem that the current model training method cannot reduce the time cost, computing power cost and hardware resource cost of training while considering the tail risk of the hedging error of the hedge portfolio, and solve the technical problem that the existing method considering tail risk can only use data generated by simulation based on parameter model for training, but cannot directly use real market observation data for training, thereby failing to directly and effectively use the information of real market data, and failing to avoid modeling errors and parameter estimation errors.

To achieve the above objectives, the present application provides a data processing method, which includes the following steps:

A feedback sample set is obtained by performing multiple interactions with the environment using a policy network to be trained, a value function network to be trained, and a tail risk network to be trained, wherein the policy network is used to output policy information based on state information at a target moment, and the policy information is used to generate an adjustment action for the financial derivatives hedging portfolio at the target moment. The feedback sample set includes multiple samples, each of which includes sample data used to calculate a loss value, at least one of which is calculated based on reward information, the reward information is calculated based on a hedging error of the financial derivatives hedging portfolio and a tail risk of the hedging error, the tail risk is calculated based on risk information, and the risk information is predicted by the tail risk network based on initial state information corresponding to the financial derivative and/or state information corresponding to the target moment;

Using the feedback sample set to perform at least one round of training on the policy network, the value function network, and the tail risk network respectively;

After each network is trained for at least one round and it is detected that a preset training end condition is met, the trained strategy network is obtained for adjusting the financial derivatives hedging portfolio based on the strategy network.

In one embodiment, the step of using the policy network to be trained, the value function network to be trained, and the tail risk network to be trained to interact with the environment multiple times to obtain a feedback sample set includes:

Obtaining trajectory data corresponding to a target financial derivative in a preset environmental data set, the trajectory data including a price trajectory, a delivery amount trajectory, a risk-free interest rate trajectory, a price trajectory of each asset within the contract period, and a cash flow trajectory of each asset within the contract period for the target financial derivative;

Using the to-be-trained policy network, the to-be-trained value function network, and the to-be-trained tail risk network to interact at least once with an environment defined by the trajectory data corresponding to the target financial derivative, to obtain a sub-feedback sample set corresponding to the target financial derivative, wherein the sub-feedback sample set includes a plurality of samples, each of which includes sample data for calculating a loss value;

The feedback sample set is generated by using the sub-feedback sample sets corresponding to a plurality of financial derivatives with different initial state information.

In one embodiment, the value function network is configured to output a state value based on state information input at a target time, at least one item of sample data is calculated based on the state value, and the step of using the to-be-trained policy network, the to-be-trained value function network, and the to-be-trained tail risk network to interact at least once with an environment defined by the trajectory data corresponding to the target financial derivative to obtain a sub-feedback sample set corresponding to the target financial derivative includes:

determining initial state information corresponding to the target financial derivative according to the trajectory data corresponding to the target financial derivative, wherein the initial state information is state information corresponding to the start time of the contract period;

Using the starting time as the target time, inputting state information corresponding to the target time into the strategy network to obtain strategy information at the target time, and generating an adjustment action for the financial derivatives hedging portfolio at the target time based on the strategy information;

Calculating, based on the adjustment action and the trajectory data corresponding to the target financial derivative, state information at a moment immediately following the target moment and a hedging error of the hedging portfolio at a moment immediately following the target moment after the adjustment action is performed;

Inputting the state information of the target moment into the value function network to be trained to obtain the state value corresponding to the target moment;

Calculating reward information after performing the adjustment action at the target time, wherein, when the moment immediately following the target time is the end time of the contract period, the reward information is calculated based on the hedging error of the target financial derivative hedging portfolio at the moment immediately following the target time and the tail risk of the hedging error, wherein the tail risk is calculated based on risk information, which is calculated by inputting initial state information corresponding to the target financial derivative and/or state information corresponding to the target time into a tail risk network to be trained;

Taking the moment after the target moment as the new target moment, and returning to the step of inputting the state information corresponding to the target moment into the policy network to obtain the policy information for the target moment, until the moment after the target moment is the end moment of the contract period;

Generate a first sample corresponding to each moment based on the state information, adjustment action, reward information, and state value corresponding to each moment within the contract period, wherein the first sample includes sample data for calculating the policy loss value corresponding to the policy network and sample data for calculating the evaluation loss value corresponding to the value function network;

generating a second sample based on the initial state information of the target financial derivative and the hedging error corresponding to the termination time of the contract period, wherein the second sample includes sample data for calculating the risk information predicted loss value corresponding to the tail risk network;

A sub-feedback sample set corresponding to the target financial derivative is generated based on the first sample and the second sample.

In one embodiment, when the risk information is calculated by inputting the initial state information corresponding to the target financial derivative into the tail risk network to be trained, the second sample includes the initial state information of the target financial derivative and the hedging error corresponding to the end time of the contract period.

In one embodiment, when the risk information is calculated by respectively inputting the initial state information corresponding to the target financial derivative and the state information corresponding at the target time into the tail risk network to be trained, the second sample includes the initial state information of the target financial derivative, the state information corresponding to the target time, and the hedging error corresponding to the end time of the contract period.

In one embodiment, when the risk information is calculated by inputting the state information corresponding to the target financial derivative at the target time into the tail risk network to be trained, the second sample includes the state information corresponding to the target financial derivative at the target time and the hedging error corresponding to the end time of the contract period.

In one embodiment, before the step of obtaining trajectory data corresponding to a target financial derivative in the preset environmental data set, the method further includes:

For any financial derivative with a preset asset as the underlying asset and having different initial state information within a preset time period, collecting trajectory data corresponding to the financial derivative based on a preset financial information data interface;

The environmental data set is obtained according to the trajectory data corresponding to each financial derivative with different initial state information.

In one embodiment, the feedback sample set includes a plurality of first samples and second samples, wherein the first samples include sample data for calculating the policy loss value corresponding to the policy network and sample data for calculating the evaluation loss value corresponding to the value function network, and the second samples include sample data for calculating the risk information prediction loss value corresponding to the tail risk network;

The steps of using the feedback sample set to perform a round of training on the policy network, the value function network, and the tail risk network respectively include:

Calculating a policy loss value corresponding to the policy network and an evaluation loss value corresponding to the value function network using the first sample;

Calculating the gradient values corresponding to the network parameters in the policy network and the value function network according to the policy loss value and the evaluation loss value, and updating the network parameters according to the gradient values;

Calculating a risk information predicted loss value corresponding to the tail risk network using the second sample, calculating a gradient value corresponding to a network parameter in the tail risk network according to the risk information predicted loss value, and updating the network parameter according to the gradient value;

After using the feedback sample set to update the network parameters in the policy network, the value function network and the tail risk network for at least one round, a round of training process for the policy network, the value function network and the tail risk network is completed.

In one embodiment, the second sample includes initial state information corresponding to a target financial derivative and a hedging error of the hedging combination of the target financial derivative at the time of contract termination of the target financial derivative;

The steps of using the second sample to calculate the risk information predicted loss value corresponding to the tail risk network, calculating the gradient value corresponding to the network parameter in the tail risk network according to the risk information predicted loss value, and updating the network parameter according to the gradient value include:

Inputting the initial state information of the second sample into the tail risk network prediction to obtain risk information corresponding to the second sample;

Substituting the risk information corresponding to the second sample and the hedging error in the second sample into a preset risk information prediction loss function to obtain a risk information prediction loss value corresponding to the second sample;

Predicting loss values based on risk information corresponding to a plurality of second samples in the feedback sample set, calculating gradient values corresponding to network parameters in the tail risk network, and updating the network parameters based on the gradient values.

In one embodiment, the status information corresponding to the financial derivative at the target moment includes the amount of cash in the hedging portfolio of the financial derivative at the target moment, the amount of each hedging asset, the price of each hedging asset, the price of each underlying asset of the financial derivative, the remaining maturity time of the financial derivative, the parameters affecting the price of the financial derivative agreed in the contract of the financial derivative or the ratio of the parameters affecting the price of the financial derivative agreed in the contract of the financial derivative to the prices of each underlying asset of the financial derivative at the initial moment, the hedging error of the hedging portfolio and the risk-free interest rate at the target moment.

In one embodiment, the policy network includes two independent multi-layer feedforward neural networks, each of which includes an input layer, several hidden layers and an output layer.

In one embodiment, the value function network includes two independent multi-layer feedforward neural networks, each of which includes an input layer, several hidden layers and an output layer, and the number of nodes in the input layer is consistent with that in the policy network. Each hidden layer contains several nodes, and the output layer has only one node for outputting a value function variable for evaluating the value of the input state information.

To achieve the above objectives, the present application further provides a data processing device, comprising:

An interaction module, configured to employ a policy network to be trained, a value function network to be trained, and a tail risk network to be trained to interact multiple times with an environment to obtain a feedback sample set, wherein the policy network is configured to output policy information based on state information at a target moment, the policy information being configured to generate an adjustment action for the financial derivatives hedging portfolio at the target moment, the feedback sample set comprising a plurality of samples, each of which includes various sample data items used to calculate a loss value, at least one of which is calculated based on reward information, the reward information being calculated based on a hedging error of the financial derivatives hedging portfolio and a tail risk of the hedging error, the tail risk being calculated based on risk information, and the risk information being predicted by the tail risk network based on initial state information corresponding to the financial derivative and/or state information corresponding to the target moment;

A training module is used to use the feedback sample set to perform at least one round of training on the strategy network, the value function network and the tail risk network respectively; after performing at least one round of training on each network and detecting that the preset training end conditions are met, the trained strategy network is obtained for adjustment of the financial derivatives hedging portfolio based on the strategy network.

To achieve the above-mentioned objectives, the present application also provides a data processing device, which includes: a memory, a processor, and a data processing program stored in the memory and executable on the processor, wherein the data processing program implements the steps of the data processing method described above when executed by the processor.

In addition, to achieve the above-mentioned purpose, the present application also proposes a computer-readable storage medium, on which a data processing program is stored. When the data processing program is executed by a processor, the steps of the data processing method described above are implemented.

In an embodiment of the present application, a policy network to be trained, a value function network to be trained, and a tail risk network to be trained are used to interact with the environment multiple times to obtain a feedback sample set, wherein the policy network is used to output policy information based on state information at a target time, and the policy information is used to generate an adjustment action for the financial derivatives hedging portfolio at the target time. The feedback sample set includes multiple samples, each of which includes sample data for calculating a loss value, at least one sample data item is calculated based on reward information, the reward information is calculated based on a hedging error of the financial derivatives hedging portfolio and a tail risk of the hedging error, the tail risk is calculated based on risk information, and the risk information is predicted by the tail risk network based on initial state information corresponding to the financial derivative and/or state information corresponding to the target time; the policy network, the value function network, and the tail risk network are each trained for at least one round using the feedback sample set; after each network is trained for at least one round and a preset training end condition is detected to be met, the trained policy network is obtained for use in adjusting the financial derivatives hedging portfolio based on the policy network. Because during the training of the policy network, the policy loss value is calculated based on reward information, which is calculated based on the tail risk of the hedging error of the financial derivative hedging portfolio, adjustment actions made based on the trained policy network can achieve a lower tail risk of the hedging error. Furthermore, because the risk information used to calculate the tail risk is predicted by the tail risk network based on the input initial state information and/or the state information corresponding to the target time, and the tail risk network is trained based on the loss value predicted by the risk information, it is possible to calculate different tail risks of the hedging error for the same financial derivative with different initial state information, thereby allowing the feedback sample set to include multiple sub-feedback sample sets corresponding to the same financial derivative with different initial state information. Therefore, the policy network trained based on the feedback sample set is applicable to each financial derivative of the same type with different initial state information, eliminating the need to train different models for each financial derivative of the same type with different initial state information, thereby reducing the time cost, computing power cost, and hardware resource cost of training the policy network. In addition, existing methods for considering tail risk can only use data generated by parameter model simulation for training, and cannot directly use real market observation data for training. This makes it impossible to directly and effectively use the information of real market data, nor can it avoid modeling errors and parameter estimation errors. The solution provided in the embodiment of the present application can only use real market data for training, so it can avoid modeling errors and parameter estimation errors, and can also directly use the information of market data, thereby effectively reducing the tail risk of hedging errors, which helps securities companies to carry out risk management.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flow chart of an embodiment of a data processing method of the present application;

FIG2 is a schematic diagram of the structure of the hardware operating environment involved in the embodiment of the present application;

FIG3 is a flow chart of another embodiment of the data processing method of the present application;

FIG4 is a flow chart of another embodiment of the data processing method of the present application;

FIG5 is a flow chart of another embodiment of the data processing method of the present application;

FIG6 is a flow chart of another embodiment of the data processing method of the present application;

FIG7 is a flow chart of another embodiment of the data processing method of the present application.

The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings.

DETAILED DESCRIPTION

It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

Refer to FIG1 , which is a flow chart of an embodiment of a data processing method of the present application.

The embodiments of the present application provide embodiments of the data processing method. It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in an order different from that shown here. In this embodiment, the execution subject of the data processing method can be a smart phone, a personal computer, a server and other devices, which are not limited in this embodiment. In this embodiment, for ease of description, the execution subject is omitted for elaboration. In this embodiment, the data processing method includes the following steps:

Step S10, using the policy network to be trained, the value function network to be trained, and the tail risk network to be trained to interact with the environment multiple times to obtain a feedback sample set, wherein the policy network is used to output policy information based on the state information at the target moment, and the policy information is used to generate an adjustment action for the financial derivatives hedging portfolio at the target moment, and the feedback sample set includes multiple samples, and the samples include various sample data for calculating the loss value, at least one sample data is calculated based on the reward information, and the reward information is calculated based on the hedging error of the financial derivatives hedging portfolio and the tail risk of the hedging error, and the tail risk is calculated according to the risk information, and the risk information is predicted by the tail risk network based on the initial state information corresponding to the financial derivative and/or the state information corresponding to the target moment.

Financial derivatives are financial products whose value depends on the value of the underlying asset. They are an important tool for businesses and financial institutions to manage financial risk. For example, steel producers often hold iron ore inventories. To hedge against the risk of future price declines, they may purchase iron ore put options from securities firms to hedge their value. The underlying asset of these put options is iron ore, and their value depends on the price of iron ore. When securities firms sell financial derivatives, they construct a hedge portfolio to replicate the derivative. The hedge portfolio consists of cash and hedge assets. Before the expiration of a financial derivative contract, securities firms adjust the amount of hedge assets in the portfolio to ensure that, during the contract period, the total value of the hedge portfolio is equal to or greater than the value of the derivative. This is the value of one or more payments (i.e., the payment amount) stipulated in the contract to the buyer of the derivative. The payment amount is determined by the price of the underlying asset (which may be negative). For example, a European iron ore put option contract stipulates that on the option's expiration date, the option seller must pay the option buyer the maximum value of the difference between the strike price agreed in the contract and the price of iron ore on the option's expiration date and zero. The hedging error of a hedge portfolio is the difference between the value of the hedge portfolio and the value of the financial derivative. When the hedging error is less than zero, that is, when the value of the hedge portfolio is lower than the value of the financial derivative, the securities company will bear the loss. Therefore, securities companies need to effectively control the financial risks associated with selling financial derivatives, especially the tail risk of the hedging error, which is the risk of the securities company suffering large losses. In this embodiment, the policy network can be a neural network that outputs policy information based on the state information of the financial derivative at the target time. The policy information is used to generate adjustments to the financial derivative's hedging portfolio at the target time. The target time refers to any time within the contract period of the financial derivative. The contract period starts at the time the financial derivative is sold, and ends at the expiration time specified in the financial derivative contract (hereinafter referred to as the contract expiration time).

The state information corresponding to the financial derivative at the target time may include the values of each state variable corresponding to the financial derivative at the target time. The each state variable corresponding to the financial derivative may include information variables that affect or are correlated with the hedging error of the financial derivative's hedging portfolio, the price of the financial derivative, the price or cash flow of the hedging asset, or the price or cash flow of the underlying asset. For example, the information variables may include the volatility of the underlying asset of the financial derivative, the number of each hedging asset in the hedging portfolio of the financial derivative, etc. The specific information variables included can be set as needed and are not limited here. For example, in one embodiment, the state information corresponding to the financial derivative at the target time may include the amount of cash in the hedging portfolio of the financial derivative at the target time (which can be represented by _Bt , where t represents the target time), the number of n hedging assets in the hedging portfolio (which can be represented by _δt = (δt _{, 1} , ..., δt _{, n} )), the prices of the n hedging assets in the hedging portfolio (which can be represented by _Gt = (Gt _{, 1} , ..., Gt _{, n} )), the The price of the underlying asset (can be (represented by) the financial derivative The volatility of the underlying asset (can be t), the remaining maturity time of the financial derivative (which can be represented by _τt ), the parameters affecting the price of the financial derivative agreed in the contract of the financial derivative or the ratio of the parameters affecting the price of the financial derivative agreed in the contract of the financial derivative to the prices of the underlying assets of the financial derivative at the initial time, the hedging error of the hedging portfolio (which can be represented by _Wt ) and the risk-free interest rate at the target time (which can be represented by Represented). Among them, for different types of financial derivatives, the types of parameters that affect the price of the financial derivatives agreed in the contract of the financial derivatives are different. For example: the parameter that affects the price of the European option agreed in the contract of the European option is the strike price of the European option (which can be represented by K); the parameters that affect the price of the knock-out call option agreed in the contract of the knock-out call option are the strike price and the knock-out barrier price of the knock-out call option; the parameters that affect the price of the snowball option agreed in the contract of the snowball option are the knock-in barrier price, the knock-out barrier price and the coupon rate of the snowball option. It should be noted that the above-mentioned various status information can be increased or decreased according to actual conditions, and they are not listed one by one here. For example, in some embodiments, the status information of the target moment corresponding to the financial derivative can also include the sensitivity information of the financial derivative to the price change of the underlying asset and other information that affects the value of the derivative and the hedging error. Among them, the sensitivity information can include the Delta of the financial derivative (which can be represented by Gamma (can be expressed as Indicates), Vega (can be used denoted) and Theta (can be expressed as It should be noted that when the hedging asset in the hedging portfolio of a financial derivative is the same as the underlying asset of the financial derivative, _St and _Gt are the same, and _Gt can be omitted in the status information.

For example, the state information s _t corresponding to the European option at time t can be expressed as:

When S _t and G _t are the same, G _t can be omitted in the status information. S ₀ represents the price of the underlying asset of the European option at the initial moment. It can also be replaced by K. For example, the state information s t at time _t corresponding to the upward knock-out call option can be expressed as:

Where J is the knock-out barrier agreed in the contract for the upward knock-out call option, and K is the strike price.

For example, the state information s t corresponding to the Snowball option at time _t can be expressed as:

Among them, J is the knock-out barrier agreed in the Snowball options contract, L is the knock-in barrier agreed in the Snowball options contract, and D is the dividend rate agreed in the Snowball options contract.

The adjustment action made to a hedge portfolio can be reflected by the number of hedge assets in the adjusted hedge portfolio. That is, the number of hedge assets in the adjusted hedge portfolio can be generated based on the policy information. For example, the adjustment action selected based on the state information s t at time _t is defined as a _t , which represents the number of hedge assets included in the hedge portfolio after the adjustment at time t, i.e., a _t ∈ ^{R n} , where n is the number of hedge assets at time t . The policy information output by the policy network can be predefined information capable of generating adjustment actions. For example, in one embodiment, the policy network can be configured to output a parameter value for the probability distribution of the adjustment action to be made at the target time, and the adjustment action can be sampled based on the probability distribution defined by the parameter value. For another example, in another embodiment, the policy network can be configured to output an action value for the adjustment action to be made at the target time, which is the number of hedge assets in the adjusted hedge portfolio.

The network parameters in the policy network, value function network, and tail risk network need to be updated through multiple rounds of training to ultimately obtain a policy network that meets the set expectations. The hedging portfolio of financial derivatives can then be adjusted based on this policy network that meets the set expectations. In this embodiment, a policy network whose network parameter values have not yet been finalized is referred to as a policy network to be trained. Similarly, there are also concepts of a value function network to be trained and a tail risk network to be trained. In a specific embodiment, before the start of the first round of training, the network parameters in the policy network to be trained, the value function network to be trained, and the tail risk network to be trained can be initialized as needed. Before the start of each subsequent round of training, the network parameters in the policy network to be trained, the value function network to be trained, and the tail risk network to be trained are the network parameters updated in the previous round of training. The training process for each round of training for the policy network to be trained, the value function network to be trained, and the tail risk network to be trained are similar. Therefore, the following description uses a single round of training as an example.

The process of using the policy network to be trained, the value function network to be trained, and the tail risk network to be trained to interact with the environment can refer to the process of interaction between an intelligent agent in reinforcement learning based on a policy function and the environment in related technologies, and is not limited here. After multiple interactions, a feedback sample set can be obtained. The feedback sample set includes multiple samples, each sample including at least one sample data item, and each sample data item is used to calculate the loss value. In specific embodiments, the specific sample data included in the sample varies depending on the selected loss function. In this embodiment, at least one sample data item is calculated based on reward information, and the reward information is calculated based on the hedging error and the tail risk of the hedging error of the hedging portfolio of financial derivatives. That is, the tail risk of the hedging error is considered in the loss function, thereby minimizing the tail risk of the hedging error at the expiration of the financial derivative contract. In this embodiment, the method of calculating the sample data based on the reward information and the method of calculating the loss value based on the sample data are not limited, and specifically vary depending on the selected loss function. The method of calculating the other sample data items is also not limited in this embodiment, and specifically varies depending on the selected loss function. Furthermore, in this embodiment, the tail risk of the hedged error of the hedging portfolio is calculated based on risk information, which is predicted by the tail risk network to be trained based on the initial state information of the financial derivative and/or the state information corresponding to the target time. The initial state information corresponding to the financial derivative refers to the state information corresponding to the starting time of the contract period of the financial derivative. By setting up a tail risk network, the tail risk network predicts risk information based on the initial state information of the financial derivative, and calculates the tail risk of the hedged error of the hedging portfolio using the risk information. The tail risk network is trained based on the loss value predicted by the risk information. This allows different tail risks of hedging errors to be calculated for the same financial derivative with different initial state information, thereby allowing the feedback sample set to include multiple sub-feedback sample sets corresponding to the same financial derivative with different initial state information. Therefore, a strategy network trained on a feedback sample set that meets the set expectations can be applied to multiple financial derivatives of the same type with different initial state information. These multiple financial derivatives with different initial state information may have different underlying initial prices, different contract parameters affecting the price of the financial derivative, different contract expiration times, etc. This eliminates the need for separate model training for each financial derivative with different initial state information. This increases the applicability of the trained strategy network for adjusting financial derivative hedging portfolios and reduces the hardware resource costs, computing power costs, and time costs required for training. Financial derivatives include, but are not limited to, European call options, European put options, knock-in barrier options, knock-out barrier options, lookback options, Asian call options, Asian put options, snowball options, interest rate options, and bond options.

In a specific embodiment, the policy network may be represented by, but is not limited to, a multi-layer feedforward neural network.

In one embodiment, the policy network can have two independent multi-layer feedforward neural networks, each feedforward neural network consists of an input layer, several hidden layers and an output layer. Its structure can be set to: the input layer dimension is the same as the dimension of the state information s _t , and a total of three layers of linear hidden layers with 64 neurons are included. After each layer of linear hidden layer, a nonlinear activation function (such as Swish) and a batch normalization layer are applied, and the output layer dimension is the same as the dimension of the action a _t . In addition, the value range of the output variable can be controlled by applying a constant multiple of the Sigmoid activation function before output according to the characteristics of the target financial derivative and the meaning of the output variable. For example, when the target financial derivative is a European call option with 1 stock as the underlying asset and the output variable is the mean variable of the adjustment action, the Sigmoid activation function can be applied before output so that the value range of the output variable is [0,1] (when the target financial derivative is a European put option, the Sigmoid activation function is applied before output and multiplied by -1).

The two multi-layer feedforward neural networks output two parameter variables of the probability distribution that the adjustment action obeys. At time t, the state information is s _t , and the agent is at time t It can be set according to the situation of the target financial derivative. For example, when the target financial derivative is a European option with one share of stock as the underlying asset, b=1 can be set. The network structure can be set as follows: the input layer dimension is the same as the dimension of the state information s _t , and it contains three linear hidden layers with 64 neurons. After each linear hidden layer, a nonlinear activation function (such as Swish) and a batch normalization layer are applied. The output layer dimension is the same as the dimension of the action a _t . Two policy networks When the target asset is a European call option, the Sigmoid activation function can be applied before output so that the action value range is [0,1]. (When the target financial derivative is a European put option, the Sigmoid activation function is applied before output and multiplied by -1.)

The risk information output by the tail risk network may be predefined information that can be used to calculate the tail risk of the hedging error of the hedging combination. For example, in one embodiment, the risk information may be the α quantile of the inverse of the hedging error of the financial derivatives hedging combination at the time of contract expiration (the end of the contract period). That is, the risk information may be the quantile of the inverse of the hedging error of the financial derivatives hedging combination at the time of contract expiration under a given level α (e.g., α=0.975). In another embodiment, the risk information may be the quantile of the inverse of the hedging error of the financial derivatives hedging combination at the time of contract expiration (the end of the contract period). Quantile, where 0<α ₁ <1 is a fixed constant.

In one embodiment, when the policy network is configured to output two parameter variables of the probability distribution obeyed by the adjustment action, the tail risk network can be represented as ω(s ₀ ; ζ), where s ₀ is the initial state information corresponding to the financial derivative, and ζ is a network parameter in the tail risk network. The tail risk network can adopt a neural network structure, with the network input being s ₀ and the output being the risk information. Taking a fully connected neural network as an example (other network types, such as residual neural networks, can also be used), its structure can be configured as follows: the input layer dimension is the same as the dimension of the state information s _t , and it contains three linear hidden layers with 64 neurons. Each linear hidden layer is followed by a nonlinear activation function (such as Swish) and a batch normalization layer, and the output layer is 1-dimensional. In specific embodiments, the tail risk network can be divided into two cases: 1. The model is trained on samples of financial derivatives with unique initial state information. In this case, s ₀ is a fixed vector, so ω(s ₀ ; ζ) is a parameter ζ that does not depend on the state information, that is, ω(s ₀ ; ζ) = ζ. 2. The case of training the model for samples of multiple financial derivatives with different initial state information: In this case, ω(s ₀ ;ζ) is a function with the initial state information s ₀ as input, and the tail risk network of the aforementioned neural network structure can be used.

In another embodiment, when the policy network is set to output action values, a tail risk network and a target tail risk network can be built, represented as ω(s ₀ ; ζ ₁ ) and ω(s ₀ ; ζ ₂ ), respectively, where s ₀ is the initial state information corresponding to the financial derivative, ζ ₁ and ζ ₂ represent the network parameters in the two tail risk networks, respectively. The structures of the two tail risk networks are exactly the same, and the initialized network parameters are also the same. The two tail risk networks can adopt a neural network structure, with the input of the network being s ₀ and the output being risk information. Taking a fully connected neural network as an example (other types of networks can be used, such as a residual neural network), its structure can be set as follows: the dimension of the input layer is the same as the dimension of the state information s _t , and it contains three linear hidden layers with 64 neurons. A nonlinear activation function (such as Swish) and a batch normalization layer are applied after each linear hidden layer, and the output layer is 1-dimensional. In a specific embodiment, the tail risk network can be divided into two cases: 1. The model is trained on samples of financial derivatives with unique initial state information. In this case, _s0 is a fixed vector, so ω( _s0 ; _ζ1 ) is a parameter _{ζ1 that} does not depend on the state information, and ω( _s0 ; _ζ2 ) is a parameter _ζ2 that does not depend on the state information, that is, ω( _s0 ; _ζ1 ) = _ζ1 and ω( _s0 ; _ζ2 ) = _ζ2 . 2. The model is trained on samples of multiple financial derivatives with different initial state information. In this case, ω( _s0 ; _ζ1 ) is a function that takes the initial state information _s0 as input. The tail risk network with the aforementioned neural network structure can be used, and the structure of ω( _s0 ; _ζ2 ) is the same as that of ω( _s0 ; _ζ1 ).

In one embodiment, when the policy network is set to output two parameter variables of the probability distribution obeyed by the adjustment action, the value function network can be used to output the state value based on the state information input at the target time. The value function network can be represented by a multi-layer feedforward neural network, consisting of an input layer, several hidden layers and an output layer, and the number of nodes in the input layer is consistent with that of the policy network, each hidden layer contains several nodes, and the output layer has only one node for outputting a value function variable for evaluating the value of the input state information (i.e., the state value). The value function network can be represented as V _φ (s), where φ is the network parameter of the value function network, the input is the state information s _t , and the output is the state value; taking a fully connected neural network as an example (other types of networks can be used, such as residual neural networks), its structure can be set as follows: the input layer dimension is the same as the dimension of the state information s _t , and it contains three linear hidden layers with 64 neurons. After each linear hidden layer, a nonlinear activation function (such as Swish) and a batch normalization layer are applied, and the output layer is 1-dimensional.

In another embodiment, when the policy network is set to output action values, the value function network can be used to output state action values based on the state information and adjustment actions input at the target time. In this case, the value function network can also be called a Q function network. A Q function network and a target Q function network can be built. The Q-function network consists of three hidden layers and one output layer, where the input layer dimension is equal to the dimension of the state information plus the dimension of the action. Each hidden layer contains several nodes, while the output layer has only one node that outputs a Q-function variable, which is used to evaluate the value of the input pair of state information and the adjustment action (i.e., the state-action value). Taking a fully connected neural network as an example (other types of networks, such as residual neural networks, can be used), the structure of the Q-function network can be set as follows: the input layer dimension is equal to the dimension of the state information s _t plus the dimension of the action a _t , and it contains three linear hidden layers with 64 neurons. Each linear hidden layer is followed by a nonlinear activation function (such as Swish) and a batch normalization layer, and the output layer is 1-dimensional.

It should be noted that the implementation of the policy network and value function network provided in the above embodiments is only an exemplary description and does not impose any limitation on this application. That is, it is not limited to the aforementioned network structure, and the network structure can change accordingly with the specification of the probability distribution that the action obeys.

Step S20: Using the feedback sample set to perform at least one round of training on the policy network, the value function network, and the tail risk network.

The policy network, value function network, and tail risk network are each trained for at least one round using the feedback sample set. During each round of training, the loss value of the loss function can be calculated based on the samples in the feedback sample set. The gradient values of the network parameters of each network are calculated based on the loss value, and the network parameters of each network are updated based on the gradient values. During each round of training, the network parameters of each network can be updated at least once. There are many specific training methods, which are not limited in this embodiment.

Step S30: After each network is trained for at least one round and a preset training end condition is detected to be satisfied, the trained strategy network is obtained for adjusting the financial derivatives hedging portfolio based on the strategy network.

During the training process, it is possible to detect whether a preset training end condition is met. Specifically, the detection can be performed after each round of training, or after the network parameters of each network are updated. The preset training end condition can be set as needed and is not limited in this embodiment. For example, it can be set to when the loss function converges, when the number of training rounds reaches a set number, or when the training duration reaches a set duration.

If the preset training end conditions are still not met after one round of training, the next round of training can be carried out. That is, the policy network, value function network and tail risk network after the latest updated network parameters interact with the environment to obtain a set of feedback samples in the new round of training, and then a new round of training is carried out on each network after the latest updated network parameters based on the feedback sample set.

After detecting that a preset training termination condition has been met, the policy network with the most recently updated network parameters can be used as the trained policy network. Based on the trained policy network, policy information can be obtained by inputting state information corresponding to a particular financial derivative at a particular moment. Based on this policy information, an adjustment action to be executed on the hedging portfolio of the financial derivative at that moment can be generated, and the hedging portfolio can be adjusted based on this adjustment action. Because during the training of the policy network, the policy loss value is calculated based on reward information, and the reward information is calculated based on the tail risk of the hedging error of the financial derivative hedging portfolio, adjustment actions made based on the trained policy network can achieve a lower tail risk of the hedging error. Furthermore, because the risk information used to calculate the tail risk is predicted by the tail risk network based on the input initial state information and/or state information corresponding to the target moment, and the tail risk network is trained based on the loss value predicted by the risk information, different tail risks of the hedging error can be calculated for the same financial derivative with different initial state information, thereby enabling the feedback sample set to include multiple sub-feedback sample sets corresponding to the same financial derivative with different initial state information. Therefore, the strategy network trained based on the feedback sample set can be applied to all the same financial derivatives with different initial state information, so there is no need to train different models for the same financial derivatives with different initial state information, which reduces the time cost, computing power cost and hardware resource cost of training the strategy network. In addition, the existing methods for considering tail risk can only use data generated by simulation based on parameter models for training, and cannot directly use real market observation data for training. This makes it impossible to directly and effectively use the information of real market data, and also cannot avoid modeling errors and parameter estimation errors. The solution provided in the embodiment of the present application can only use real market data for training, so it can avoid modeling errors and parameter estimation errors, and can also directly use market data information, thereby effectively reducing the tail risk of hedging errors, which is helpful for securities companies to carry out risk management.

Based on the above embodiment, another embodiment of the data processing method of the present application is proposed. In this embodiment, referring to FIG3 , step S10 includes:

Step S101, obtaining trajectory data corresponding to a target financial derivative in a preset environmental data set, wherein the trajectory data includes the price trajectory, delivery amount trajectory, risk-free interest rate trajectory, price trajectory of each asset in the contract period, and cash flow trajectory of each asset of the target financial derivative during the contract period.

An environmental data set can be set in advance. The environmental data set can include trajectory data corresponding to multiple financial derivatives with given initial state information. The initial state information of each financial derivative can be different. The following takes one of the financial derivatives as an example and calls it the target financial derivative for distinction.

The trajectory data corresponding to the target financial derivative may include the target financial derivative's price trajectory, delivery amount trajectory, risk-free interest rate trajectory, price trajectory of each asset during the contract period, and cash flow trajectory of each asset. The cash flow of each asset refers to the cash flow generated by each asset. For example, the cash flow generated by stock assets includes stock dividends and cash distributions, and the cash flow generated by bond assets includes bond coupons. In some embodiments, the trajectory data may also include the volatility trajectory of the target financial derivative's underlying asset during the contract period, information on the sensitivity of the target financial derivative's price to changes in the underlying asset price and other variables (which may include the financial derivative's Delta, Gamma, Vega, and Theta), and other information variables that affect or are correlated with the hedging error of the financial derivative's hedging portfolio, the price of the financial derivative, the price or cash flow of the hedged asset, or the price or cash flow of the underlying asset. The specific information variables included can be set as needed and are not limited here. The delivery amount _Ut of the target financial derivative at time t refers to the cash value (which may be a negative value) that the seller of the financial derivative is required to pay to the buyer at time t, as specified in the financial derivative contract. It should be noted that the target financial derivative may have one or more underlying assets, as specified in the financial derivative contract. The assets within the contract period include the hedging assets in the target financial derivative's hedging portfolio and the target financial derivative's underlying assets. It should be noted that there may be one or more hedging assets, selected based on the financial derivative and its underlying assets. The hedging assets can be the same as the underlying assets.

The method of obtaining the environmental data set is not limited in this embodiment. For example, it can be obtained through data model simulation, or by obtaining real market information data based on a preset financial information data interface. This is not limited in this embodiment.

In one embodiment, obtaining trajectory data corresponding to a target financial derivative through data model simulation may specifically include:

Specify the model for the hedge asset price and cash flow, the model for the underlying asset price and cash flow, and the model for the risk-free rate. For example, you can assume that the underlying asset follows a geometric Brownian motion model:

Where μ and σ are given model parameters. We can also assume that the underlying asset follows the GARCH model:

Where λ, a, b, c, and d are given model parameters, _zt is an independent and identically distributed standard normal random variable, and _σt is the random volatility. The model for the underlying asset's cash flow _Dt depends on the specific definition of the underlying asset. For example, when the underlying asset is a stock, its cash flow _Dt includes dividends and bonuses, and a model for _Dt can be established using historical data. When the underlying asset is a bond, its cash flow includes bond coupons, and the model for _Dt depends on the terms of the bond contract and market information variables such as market interest rates. Similarly, the price of the hedging asset _Gt and its cash flow _It can be assumed to follow a model. It should be noted that the hedging asset _Gt and the underlying asset _St can be identical; in the absence of cash flows, _Dt or _It is zero. The risk-free interest rate model can be assumed to be the instantaneous spot CIR model.

Assume that S ₀ ,σ ₀ ,G ₀ obey a certain initial distribution. Then, based on the above model and a given discrete time interval Δt, we can simulate and generate a preset number of underlying asset prices, underlying asset cash flows, underlying asset volatility, hedge asset prices, and hedge asset cash flow trajectories. Where T is the expiration time of the financial derivative contract. T can be specified as a fixed value or multiple different values according to the needs. If the model is a geometric Brownian motion, then at any time in all volatility trajectories According to the contract definition of financial derivatives, according to the specified underlying asset price and cash flow model, for each trajectory of the underlying asset price, underlying asset cash flow, underlying asset volatility, hedge asset price and hedge asset cash flow At each time t, based on (S _t , D _t , σ _t ), calculate the value of the delivery amount U _t at time t according to the contract definition of the financial derivative, and calculate the price Z _t of the financial derivative after excluding U _t at time t, and calculate the sensitivity information of the financial derivative price at time t For example, if the financial derivative is a European put option with an exercise price of K and an expiration time of T, then U _t = 0, t = 0, ..., T-1, U _T = max(KS _T , 0), and Z _T = 0. Save the traces of all underlying asset prices, underlying asset cash flows, underlying asset volatility, hedge asset prices, hedge asset cash flows, financial derivative prices, financial derivative delivery amounts, risk-free rates, and sensitivity information. It should be noted that when the hedge asset and the underlying asset are the same, G _t and I _t can be omitted in the trajectory.

Step S102: The policy network to be trained, the value function network to be trained, and the tail risk network to be trained are used to interact at least once with the environment defined by the trajectory data corresponding to the target financial derivative to obtain a sub-feedback sample set corresponding to the target financial derivative, wherein the sub-feedback sample set includes multiple samples, and the samples include various sample data used to calculate the loss value.

The process of using the strategy network to be trained, the value function network to be trained, and the tail risk network to be trained to interact with the environment defined by the target financial derivatives can refer to the process of interaction between the intelligent agent and the environment in the relevant technology, and is not limited in this embodiment.

Step S103 : generating the feedback sample set by combining the sub-feedback sample sets corresponding to a plurality of financial derivatives having different initial state information.

The number of samples in the collected feedback sample set can be pre-set as needed and is not limited in this embodiment.

In one embodiment, when the policy network is configured to output two parameter variables of a probability distribution obeyed by the adjustment action, and the value function network is configured to output a state value based on the state information input at the target time, at least one item of the sample data is calculated based on the state value. Referring to FIG. 4 , step S102 includes:

Step S1021 : determining initial state information corresponding to the target financial derivative according to the trajectory data corresponding to the target financial derivative, wherein the initial state information is state information corresponding to the start time of the contract period.

The initial state information corresponding to the target financial derivative can be determined based on the trajectory data corresponding to the target financial derivative. The specific implementation method of determining the initial state information based on the trajectory data varies depending on the specific information items included in the state information.

Step S1022: Taking the starting time as the target time, inputting the state information corresponding to the target time into the strategy network, obtaining the strategy information of the target time, and generating an adjustment action for the financial derivatives hedging portfolio at the target time based on the strategy information.

Step S1023, based on the adjustment action and the trajectory data corresponding to the target financial derivative, calculate the state information of the next moment after the target moment and the hedging error of the hedging combination after the adjustment action is made at the next moment after the target moment.

Step S1024: input the state information of the target moment into the value function network to be trained to obtain the state value corresponding to the target moment.

Step S1025, calculating the reward information after the adjustment action is made at the target moment, wherein, when the next moment after the target moment is the end moment of the contract period, the reward information is calculated based on the hedging error of the target financial derivative hedging combination at the next moment after the target moment and the tail risk of the hedging error, wherein the tail risk is calculated based on risk information, and the risk information is calculated by inputting the initial state information corresponding to the target financial derivative and/or the state information corresponding to the target moment into the tail risk network to be trained.

Step S1026, taking the next moment after the target moment as the new target moment, and returning to the step of inputting the state information corresponding to the target moment into the policy network to obtain the policy information of the target moment, until the next moment after the target moment is the end moment of the contract period.

Exemplarily, the starting time is t=0, and the state information at the starting time is s ₀ . s ₀ is input into the policy network to obtain the policy information π ₀ at time t=0. Based on the policy information π ₀ , the adjustment action a ₀ at time t=0 is generated. Based on the adjustment action a ₀ and the trajectory data corresponding to the target financial derivative, the state information s ₁ at time t=1 and the hedging error W ₁ of the hedging combination at time t=1 after making the adjustment action a ₀ are calculated. The reward information r ₀ at time t=0 is calculated, and the state information s 0 at time t= ₀ is input into the value function network to be trained to calculate the state value V(s ₀ ) at time t=0. Then, the state information s ₁ at time t = 1 is input into the policy network to obtain the policy information π ₁ at time t = 1. Based on the policy information π ₁ , the adjustment action a ₁ at time t = 1 is generated. Based on the trajectory data corresponding to the adjustment action a ₁ and the target financial derivative, the state information s ₂ at time t = 2 and the hedging error W ₂ of the hedge portfolio at time t = 2 after the adjustment action a ₁ are calculated. The reward information r ₁ at time t = 1 is calculated. The state information s 1 at time t = ₁ is input into the value function network to be trained to calculate the state value V(s ₁ ) at time t = 1. This process is repeated until the state information s _{T-1 at time t = T-1} , the adjustment action a _T-1 , the tail risk, the reward information r _T-1 , the state value V(s T _-1 ), and the hedging error W _T at time t = T are calculated. It should be noted that the reward information r T-1 at time T _-1 is calculated based on the tail risk of the hedging error of the target financial derivative hedging portfolio and the hedging error W _T. The tail risk is calculated by inputting s ₀ and/or the state information s _{T-1 at time T-1} into the tail risk network to be trained, and then calculating the risk information ω. Where T is the end time of the contract period of the target financial derivative. The calculation method of reward information at other times is

Among them, δ ₀ :＝0, τ ₀ ＝T, T is the expiration time of the financial derivative, σ ₀ , S ₀ , G ₀ are specified according to the trajectory data, B ₀ ＝Z ₀ is the selling price of the European option. The end time of this trajectory is T. The following are the definitions of 5 different reward information r _t . During training, you can choose one of them to calculate the reward variable r _t :

Wherein, λ ₁ >0 and λ ₂ ≥0 are preset constants. It should be noted that the definition of the reward information variable r _t is not limited to the above five types.

The calculation method of the reward variable r _t is based on the following hedging error tail risk:

Where ω(s ₀ ;ζ) is the risk information corresponding to the initial state information s ₀ , and represents the α quantile of the opposite number of the hedging error W _{T. W t+1} is the hedging error at time t+1, and its value can be calculated based on the state information s t at time _t , the adjustment action a _t , and the state information s _{t+1 at time t+1} according to the following formula: δ _t+1 = a _t , t = 0, 1, …, T-1,

Where n is the number of hedge assets, G _t+1 = (G _t+1,1 ,...,G _t+1,n ), G _t+1,i is the price of the i-th hedge asset at time t+1, Z _t+1 is the price of the financial derivative at time t+1 after excluding U _t+1 , and U _t+1 is the delivery amount at time t+1. All three can be obtained from the price trajectory. δ _t+1,i is the i-th component of δ _t+1 , that is, the amount of the i-th hedge asset before adjustment at time t+1. B _t+1 can be calculated as follows:

Where a _t,i , δ _{t,i ,} and G _t,i are the i-th components of a _t , δ _t , and G _{t ,} respectively. C _t is the transaction fee at time t. I _t+1,i is the cash flow generated by the i-th hedge asset at time t+1, obtained from the cash flow trajectory. Compared to the related art's assumption of a frictionless market with no transaction fees, this embodiment incorporates transaction fees in the calculation process, which is more consistent with actual market conditions and makes the trained policy network more suitable for real-world applications.

In other embodiments, other hedging error tail risks may be used. For example, assuming a constant 0<α ₁ <1, the following hedging error tail risk may be used:

Step S1027: Based on the state information, adjustment action, reward information and state value corresponding to each moment within the contract period, generate a first sample corresponding to each moment, wherein the first sample includes sample data for calculating the strategy loss value corresponding to the strategy network and sample data for calculating the evaluation loss value corresponding to the value function network.

In this embodiment, the policy loss function of the policy network can be selected according to specific needs. Therefore, there is no restriction on the calculation method of the policy loss value, and there is no restriction on the specific data included in the sample data used to calculate the policy loss value in the first sample. In this embodiment, the evaluation loss function of the value function network can be selected according to specific needs. Therefore, there is no restriction on the calculation method of the evaluation loss value, and there is no restriction on the specific data included in the sample data used to calculate the evaluation loss value in the first sample.

In one embodiment, each first sample generated based on a piece of trajectory data of a target financial derivative can be expressed as: That is, T first samples can be generated. is the advantage function variable, is the accumulation reward variable, where γ∈[0,1] is the discount rate, which can be set in advance according to needs; λ ^gae is the parameter in the generalized advantage function, which can be set in advance according to needs.

Step S1028: Generate a second sample based on the initial state information of the target financial derivative and the hedging error corresponding to the termination time of the contract period, wherein the second sample includes sample data for calculating the risk information predicted loss value corresponding to the tail risk network.

In this embodiment, the method for generating the second sample based on the initial state information of the target financial derivative and the hedging error corresponding to the end time of the contract period is not limited, and different generation methods can be adopted depending on the risk information prediction loss function selected for the tail risk network. In this embodiment, the risk information prediction loss function of the tail risk network can be selected according to specific needs, and therefore, the method for calculating the risk information prediction loss value is not limited.

In one embodiment, when the risk information in step S1025 is calculated by inputting the initial state information corresponding to the target financial derivative into the tail risk network to be trained, the second sample may include the initial state information of the target financial derivative and the hedging error corresponding to the end time of the contract period. When the tail risk network is trained based on the second sample, the risk information predicted loss value of the tail risk network can be calculated based on the second sample, and then the gradient value of the network parameter of the tail risk network is calculated according to the risk information predicted loss value, and the network parameters of the tail risk network are updated according to the gradient value.

In another embodiment, when the risk information in step S1025 is obtained by respectively inputting the initial state information corresponding to the target financial derivative and the state information corresponding at the target time into the tail risk network to be trained, the second sample may include the initial state information of the target financial derivative, the state information corresponding to the target time and the hedging error corresponding to the end time of the contract period. When the tail risk network is trained based on the second sample, the risk information predicted loss value of the tail risk network can be calculated based on the second sample, and then the gradient value of the network parameter of the tail risk network can be calculated according to the risk information predicted loss value, and the network parameters of the tail risk network can be updated according to the gradient value.

In another embodiment, when the risk information in step S1025 is calculated by inputting the state information corresponding to the target financial derivative at the target time into the tail risk network to be trained, the second sample may include the state information corresponding to the target financial derivative at the target time and the hedging error corresponding to the termination time of the contract period. When the tail risk network is trained based on the second sample, the risk information predicted loss value of the tail risk network can be calculated based on the second sample, and then the gradient value of the network parameter of the tail risk network is calculated according to the risk information predicted loss value, and the network parameters of the tail risk network are updated according to the gradient value.

Step S1029: Generate a sub-feedback sample set corresponding to the target financial derivative based on the first sample and the second sample.

The generated first sample and the second sample can be combined to obtain a sub-feedback sample set corresponding to the target financial derivative.

In one embodiment, when the policy network is configured to output action values and the value function network is configured to output state-action values based on the state information input at the target time and the adjustment action, the sub-feedback sample set can be generated in the following manner:

Determine the initial state information of the target financial derivative based on the trajectory data corresponding to the target financial derivative. The initial state information is the state information corresponding to the start time of the contract period. The start time is used as the target time, and the state information corresponding to the target time is input into the strategy network. Obtain the action value at the target moment, and generate the adjustment action for the financial derivatives hedging portfolio at the target moment based on the action value. Based on the trajectory data corresponding to the adjustment action and the target financial derivative, calculate the state information at the next moment of the target moment and the hedging error of the hedging portfolio at the next moment of the target moment after the adjustment action is made; calculate the reward information after the adjustment action is made at the target moment, wherein, when the next moment of the target moment is the end moment of the contract period, input the initial state information corresponding to the target financial derivative and/or the state information corresponding to the target moment into the tail risk network to obtain risk information, calculate the tail risk of the hedging error of the target financial derivatives hedging portfolio at the next moment of the target moment based on the risk information, and then calculate the reward information based on the hedging error and tail risk of the target financial derivatives hedging portfolio at the next moment of the target moment; input the state information and adjustment action at the target moment into the Q function network to be trained. Obtain the state-action value corresponding to the target moment; take the next moment after the target moment as the new target moment, and return to the step of inputting the state information corresponding to the target moment into the policy network to obtain the policy information of the target moment, until the next moment after the target moment is the end moment of the contract period; based on the state information, adjustment action, reward information and state information of the next moment corresponding to each moment in the contract period, generate a first sample corresponding to each moment, the first sample includes the information for calculating the policy network The sample data of the corresponding policy loss value and the network used to calculate the Q function Sample data of the corresponding assessed loss value; generating a second sample based on the initial state information of the target financial derivative and/or the state information corresponding to the target moment and the hedging error corresponding to the termination moment of the contract period; the second sample includes sample data for calculating the risk information predicted loss value corresponding to the tail risk network ω(s;ζ ₁ )_; generating a sub-feedback sample set corresponding to the target financial derivative based on the first sample and the second sample.

For example, the starting time is t=0, the state information at the starting time is s ₀ , and s ₀ is input into the strategy network Get the action value at time t=0 Randomly sample a noise term ε ₀ (e.g., from a normal distribution Random sampling, where β ₀ is a preset parameter), adds a noise term to a ₀ , and obtains the adjustment action Then, based on the trajectory data corresponding to the adjustment action _a0 and the target financial derivative, the state information _s1 at time t=1 and the hedging error _W1 of the hedging combination at time t=1 after making the adjustment action _a0 are calculated. The reward information _r0 at time t=0 is calculated, and the state information s0 at time t= ₀ and the adjustment action _a0 are input into the Q function network to be trained. Calculate the state action value at time t=0 Then the state information s1 at time t= ₁ is input into the strategy network Get the action value at time t=1 Randomly sample a noise term ε ₁ (e.g., from a normal distribution Random sampling), add a noise term to a ₁ , and get the adjustment action Then, based on the trajectory data corresponding to the adjustment action a ₁ and the target financial derivative, the state information s ₂ at time t = 2 and the hedging error W ₂ of the hedging combination at time t = 2 after the adjustment action a ₁ are calculated. The reward information r ₁ at time t = 1 is calculated, and the state information s 1 at time t = ₁ and the adjustment action a ₁ are input into the Q function network to be trained. Calculate the state action value at time t=1 And so on, until we get the state information s _T-1 , adjustment action a _T-1 , tail risk, reward information r _T-1 , state action value at time t=T-1 and the hedging error W _T at time t=T. It should be noted that the reward information r _{T-1 at time T-1} is calculated based on the hedging error W T of the target financial derivative's hedging portfolio at time t= _T and the tail risk of this hedging error. This tail risk is calculated by inputting s ₀ and/or the state information s _{T-1 at time T-1} into the tail risk network to be trained, ω(s; ζ ₁ ), and then calculating the risk information. Where T is the end time of the contract period of the target financial derivative. The calculation method of reward information at other times is not limited here. For example, it can be calculated based on the hedging error, set to 0, or calculated based on the tail risk and hedging error.

For example, the trajectory data corresponding to a target financial derivative can be obtained from the environmental data set. The following operations are performed. First, the initial state s ₀ is determined. s ₀ is obtained by taking t=0 from the above expression of s _t . For example, when the target financial derivative is a European option, the initial state s ₀ is:

Where δ ₀ :=0, τ ₀ =T, where T is the expiration time of the financial derivative, σ ₀ , S ₀ , and G ₀ are specified based on the trajectory data, B ₀ =Z ₀ is the net cash value obtained from selling the European option, and W ₀ = 0. When the underlying asset S _t and the hedge asset G _t are the same, G ₀ can be omitted in S ₀ .

Then, at the beginning of the kth round of training, the parameters of the policy network, value function network, and tail risk network are and Starting from time t=0, the state information s _t is input into the strategy network to obtain the action value Then, a noise term _εt is randomly sampled (e.g., from a normal distribution Random sampling), adding noise terms to get the adjustment action Then, an adjustment action a _t is made, and the environment transfers from the state variable s _t to the subsequent state variable s _t+1 . The environment also provides feedback to determine the reward variable r _t . This process continues until the end time T of this trajectory. The following are the definitions of five different reward information r _t . During training, you can choose one of them to calculate the reward variable r _t :

Wherein, λ ₁ >0 and λ ₂ ≥0 are preset constants. It should be noted that the definition of the reward information variable r _t is not limited to the above five types.

The calculation method of the reward variable r _t is based on the following hedging error tail risk:

in, is the risk information corresponding to the initial state information s ₀ , representing the quantile of the opposite number of the hedging error W _{T. W t+1} is the hedging error at time t+1, and its value can be calculated based on the state information s t at time _t , the adjustment action a _t , and the state information s _{t+1 at time t+1} according to the following formula:

δ _t+1 =a _t ,t＝0,1,…,T-1,

Where n is the number of hedge assets, G _t+1 = (G _t+1,1 ,...,G _t+1,n ), G _t+1,i is the price of the i-th hedge asset at time t+1, Z _t+1 is the price of the financial derivative at time t+1 after excluding U _t+1 , and U _t+1 is the delivery amount at time t+1. All three can be obtained from the price trajectory. δ _t+1,i is the i-th component of δ _t+1 , that is, the amount of the i-th hedge asset before adjustment at time t+1. B _t+1 can be calculated as follows:

Where a _t,i , δ _{t,i ,} and G _t,i are the i-th components of a _t , δ _t , and G _{t ,} respectively. C _t is the transaction fee at time t. I _t+1,i is the cash flow generated by the i-th hedge asset at time t+1, obtained from the cash flow trajectory. Compared to the related art's assumption of a frictionless market with no transaction fees, this embodiment incorporates transaction fees in the calculation process, which is more consistent with actual market conditions and makes the trained policy network more suitable for real-world applications.

In other embodiments, other hedging error tail risks may be used. For example, assuming a constant 0<α ₁ <1, the following hedging error tail risk may be used:

In one embodiment, each first sample generated based on a piece of trajectory data of a target financial derivative can be expressed as: That is, T first samples can be generated.

Based on the above embodiments, another embodiment of the data processing method of the present application is proposed. In this embodiment, the feedback sample set may include multiple first samples and second samples. The first samples include sample data for calculating the policy loss value corresponding to the policy network and sample data for calculating the evaluation loss value corresponding to the value function network. The second samples include sample data for calculating the risk information prediction loss value corresponding to the tail risk network. Referring to Figure 5, step S20 includes:

Step S201: Calculate the policy loss value corresponding to the policy network and the evaluation loss value corresponding to the value function network using the first sample.

Step S202: Calculate the gradient values corresponding to the network parameters in the policy network and the value function network according to the policy loss value and the evaluation loss value, and update the network parameters according to the gradient values.

Step S203: use the second sample to calculate the risk information predicted loss value corresponding to the tail risk network, calculate the gradient value corresponding to the network parameter in the tail risk network according to the risk information predicted loss value, and update the network parameter according to the gradient value.

Step S204, after using the feedback sample set to update the network parameters in the policy network, the value function network and the tail risk network for at least one round, a round of training process for the policy network, the value function network and the tail risk network is completed.

In one embodiment, a subset of first samples may be extracted from each first sample in the feedback sample set to form a subset. For each first sample in the subset, the policy loss value corresponding to the policy network and the evaluation loss value corresponding to the value function network are calculated using the first sample. An average loss value is calculated based on the policy loss values and evaluation loss values corresponding to each first sample in the subset. Gradient values corresponding to network parameters in the policy network and the value function network are calculated based on the average loss value, and the network parameters in the policy network and the value function network are updated based on the gradient values. A subset of first samples may then be extracted from each remaining first sample in the feedback sample set to form a subset. The network parameters in the policy network and the value function network are updated based on the subset, and so on, until all first samples in the feedback sample set are exhausted. For each second sample in the feedback sample set, the risk information predicted loss value corresponding to the tail risk network is calculated using the second sample. An average loss value is calculated based on the risk information predicted loss values corresponding to each second sample in the feedback sample set. The gradient values corresponding to the network parameters in the tail risk network are calculated based on the average loss value, and the network parameters in the tail risk network are updated based on the gradient values.

In one embodiment, the second sample includes initial state information corresponding to a target financial derivative and the hedging error of the hedging combination of the target financial derivative at the time of contract termination of the target financial derivative. Referring to FIG. 6 , step S203 includes:

Step S2031: input the initial state information of the second sample into the tail risk network prediction to obtain risk information corresponding to the second sample.

In a specific embodiment, when the risk information in step S1025 is calculated by inputting the initial state information corresponding to the target financial derivative into the tail risk network to be trained, the second sample includes the initial state information of the target financial derivative and the hedging error corresponding to the end time of the contract period, and this step inputs the initial state information in the second sample into the tail risk network to predict the risk information corresponding to the second sample.

When the risk information in step S1025 is calculated by respectively inputting the initial state information corresponding to the target financial derivative and the state information corresponding at the target time into the tail risk network to be trained, the second sample includes the initial state information of the target financial derivative, the state information corresponding to the target time and the hedging error corresponding to the end time of the contract period, and this step inputs the initial state information and the state information corresponding to the target time in the second sample into the tail risk network to predict the risk information corresponding to the second sample.

When the risk information in step S1025 is calculated by inputting the state information corresponding to the target financial derivative at the target time into the tail risk network to be trained, the second sample includes the state information corresponding to the target financial derivative at the target time and the hedging error corresponding to the end time of the contract period, and this step inputs the state information corresponding to the target time in the second sample into the tail risk network to predict the risk information corresponding to the second sample.

Step S2032: Substitute the risk information corresponding to the second sample and the hedging error in the second sample into a preset risk information prediction loss function to obtain a risk information prediction loss value corresponding to the second sample.

Step S2033: predicting loss values based on the risk information corresponding to the plurality of second samples in the feedback sample set, calculating gradient values corresponding to network parameters in the tail risk network, and updating the network parameters according to the gradient values.

In one embodiment, the policy network is configured to output two parameter variables of the probability distribution obeyed by the adjustment action, and the value function network is configured to output the state value based on the state information input at the target time. At the beginning of the kth round of training, the parameters of the policy network, the value function network, and the tail risk network are θ=θ ^k-1 , φ=φ ^k-1 , and ζ=ζ ^k-1 , respectively. First, a feedback sample set is obtained interactively. Then, at the beginning of each update process, the first sample in the feedback sample set is randomly sorted, and then the m first samples are sequentially used as a subsample set. The first subsample set can be recorded as First, use the samples in the first subsample set to determine the following loss function:

Where c ₁ >0 and c ₂ ≥0 are preset constants, the first term on the right side of the equation is the strategy loss value, ε>0 is a preset constant, and the probability ratio is limited to a smaller range; the second term is the evaluation loss value; the third term, Entropy(π _θ (·| _si )), is the entropy of the probability distribution π _θ (·| _si ), and its calculation formula is Entropy(π _θ (·| _si ))＝-∫π _θ (x|si)log(π _θ (x| _si ))dx. In this embodiment, π _θ (·| _si ) is a diagonal Gaussian distribution with a mean variable of The standard deviation variable is Therefore, the entropy of π _θ (·|s _i ) is It should be noted that the loss function used in the above embodiment is only an illustrative description and does not limit the present application in any way, that is, it is not limited to the use of the aforementioned loss function.

Then, according to the loss function, the gradient descent algorithm is used to update the parameters θ of the policy network and the parameters φ of the value function network: and Among them, α _θ and α _φ are the training step size or learning rate, and Represent the gradients of parameters θ and φ respectively. Then, obtain the second subsample set of the feedback sample set, which contains samples from m+1 to 2m in the feedback sample set, and repeat the previous steps to update the parameters θ of the policy network and the parameters φ of the value function network, and so on, until all subsample sets in the feedback sample set are used up. Then, based on all the second samples in the feedback sample set Calculate the predicted loss value of risk information:

In one embodiment, the following function is used for tail risk:

When used to measure, the risk information prediction loss function Defined as

In one embodiment, the following function is used for tail risk:

When used to measure (where α ₁ is a fixed constant, 0<α ₁ <1), the risk information prediction loss function Defined as

Then, based on the risk information, the loss value is predicted and the gradient descent algorithm is used to update the parameter ζ of the tail risk network: Among them, _αζ is the training step size or learning rate, Denotes the gradient corresponding to the parameter ζ. This is considered an update process.

The samples in the feedback sample set are then randomly sorted before entering the next update process. After updating the network parameters of each network at least once using the feedback sample set, the parameters of the policy network, value function network, and tail risk network are updated to ^θk = θ, ^φk = φ, and ^ζk = ζ, respectively. The next round of training then begins, where a new feedback sample set is obtained based on the updated network, and the network parameters are updated multiple times based on this new feedback sample set.

In a specific embodiment, the optimizer used by the gradient descent algorithm can be set as needed. For example, an adaptive moment estimation (Adam) optimizer can be used, and the learning rate can be 0.0005.

In one embodiment, when the policy network is configured to output action values and the value function network is configured to output state action values based on the state information input at the target time and the adjustment action, at the beginning of the kth round of training, the policy network Q-Function Network and the parameters of the tail risk network ω(s ₀ ;ζ ₁ ) are and Target Policy Network Target Q-function network The parameters of the target tail risk network ω(s ₀ ;ζ ₂ ) are and First, the feedback sample set is obtained interactively. At the beginning of each update process, the first sample in the feedback sample set is randomly sorted, and then the m first samples are taken as a subsample set in turn. The first subsample set can be recorded as For each sample point, according to the policy network and Q-function network Calculate the corresponding target value Where γ∈[0,1] is the discount rate. The following loss function is thus determined:

Wherein, c ₁ >0 is a preset constant, the first term on the right side of the equation is the strategy loss value, and the second term is the evaluation loss value. It should be noted that the loss function used in the above embodiment is only for illustrative purposes and does not limit the present application in any way, i.e., it is not limited to the use of the aforementioned loss function.

According to the loss value of the loss function, the policy network is adjusted using the gradient descent algorithm. Parameters _θ1 and Q function network The parameter φ ₁ is updated: and Among them, α _θ and α _φ are the training step size or learning rate, and Represent the gradients of parameters θ ₁ and φ ₁ respectively. In addition, a replication parameter 0≤ρ≤1 is preset. For the target policy network The parameters θ ₂ and the target Q function network The parameter φ ₂ of is updated proportionally according to the replication parameter ρ: θ ₂ = (1-ρ)θ ₂ + ρθ ₁ and φ ₂ = (1-ρ)φ ₂ + ρφ ₁ . Then, obtain the second subsample set of the feedback sample set, which contains the m+1th to 2mth samples in the feedback sample set, and repeat the previous steps to update the policy network. Parameters and Q-function network Update the parameters of the target strategy network Parameters and target Q function network The parameters of the feedback sample set are updated proportionally, and so on, until all subsample sets in the feedback sample set are used up.

Then based on all second samples in the feedback sample set Calculate the predicted loss value of risk information:

In one embodiment, the following function is used for tail risk:

When used to measure, the risk information prediction loss function Defined as

In one embodiment, the following function is used for tail risk:

When used to measure, the risk information prediction loss function Defined as

Then, the loss value is predicted based on the risk information, and the parameter ζ ₁ of the tail risk network ω(s ₀ ;ζ ₁ ) is updated using the gradient descent algorithm: Among them, _αζ is the training step size or learning rate, Denotes the gradient corresponding to the parameter. Meanwhile, the parameter ζ ₂ of the target tail risk network ω(s ₀ ;ζ ₂ ) is updated proportionally to the replication parameter ρ _ζ : ζ ₂ = (1-ρ _ζ )ζ ₂ + ρ _ζ ζ ₁ , where 0 ≤ _{ρ ζ} ≤ 1 is the preset replication parameter. This is considered a single update process.

Then, the samples in the feedback sample set are randomly sorted and the next update process is entered. After the network parameters of each network are updated at least once using the feedback sample set, the parameters of the two policy networks, the two value function networks, and the two tail risk networks are updated to Then enter the next round of training, that is, obtain a new set of feedback samples based on the updated network, and update the network parameters multiple times based on the new set of feedback samples.

In one embodiment, based on the above embodiments, another embodiment of the data processing method of the present application is proposed. In this embodiment, before step S101, referring to FIG. 7 , the method further includes:

Step S40: for any financial derivative with a preset asset as the underlying asset and having different initial state information within a preset time period, collecting trajectory data corresponding to the financial derivative based on a preset financial information data interface;

Step S50 , obtaining the environmental data set according to the trajectory data corresponding to each financial derivative having different initial state information.

This embodiment proposes a specific implementation method for acquiring an environmental dataset. A preset financial information data interface is a pre-configured, customized financial information data interface. Through this interface, historical, real-world information related to financial derivatives can be obtained, thereby obtaining trajectory data corresponding to these financial derivatives. Preset assets and time periods can be configured as needed and are not limited in this embodiment.

In one embodiment, for each day within a preset asset (e.g., the SSE 50 ETF) and a preset time period (e.g., January 1, 2015, to December 31, 2022), all preset types of financial derivatives (e.g., SSE 50 ETF put options) with the preset asset as the underlying asset in the market on that day can be obtained through a preset financial information data interface, and for each financial derivative, all prices (transaction prices or the middle price between the best ask price and the best bid price) from t=0 on that day (i.e., the start time of the contract period of the financial derivative) to the expiration time t=T of the financial derivative can be extracted as the price trajectory of the financial derivative. At the same time, the parameters that affect the price of the financial derivatives agreed in the contract of the financial derivatives (such as the strike price and expiration time, etc.) can be recorded. For each of the preset types of financial derivatives, the preset financial information data interface can be used to collect the contract period of the financial derivatives. The price of the underlying asset Cash flow of underlying assets The trajectory of the n hedge asset prices G _t = (G _t,1 ,...,G _t,n ) and the hedge asset cash flows _It = (It _,1 ,...,It _,n ) in the hedge portfolio And record the delivery amount of the financial derivative during the contract period You can specify the Shanghai Interbank Offered Rate (SHIBOR) at each moment during the contract period of the financial derivative as the risk-free interest rate trajectory In one embodiment, the implied volatility trajectory of the underlying asset corresponding to the price of the financial derivative can be obtained. In one embodiment, the volatility trajectory can also be It is defined as the historical volatility trajectory or predicted volatility trajectory of the underlying asset of the financial derivative. In one embodiment, if the financial derivative has sensitivity data of the derivative price, the sensitivity data of the financial derivative can be obtained. In one embodiment, other information variables that influence or are correlated with the hedging error of the financial derivative hedging portfolio, the price of the financial derivative, the price of the hedged asset, the cash flow of the hedged asset, and the price or cash flow of the underlying asset can be obtained through a preset financial information data interface. The specific information variables can be set as needed and are not limited here. This process continues until all financial derivatives of the preset type on all dates within a preset period are processed.

In this embodiment, by constructing an environmental data set based on real data information obtained based on a preset financial information data interface, there is no need to establish a parameter model for the underlying asset price, underlying asset cash flow, hedge asset price, hedge asset cash flow, risk-free interest rate and other information variables, which can effectively avoid model errors and model parameter estimation errors of the parameter model.

In addition, an embodiment of the present application further provides a data processing device, the device comprising:

An interaction module, configured to employ a policy network to be trained, a value function network to be trained, and a tail risk network to be trained to interact multiple times with an environment to obtain a feedback sample set, wherein the policy network is configured to output policy information based on state information at a target moment, the policy information being configured to generate an adjustment action for the financial derivatives hedging portfolio at the target moment, the feedback sample set comprising a plurality of samples, each of which includes various sample data items used to calculate a loss value, at least one of which is calculated based on reward information, the reward information being calculated based on a hedging error of the financial derivatives hedging portfolio and a tail risk of the hedging error, the tail risk being calculated based on risk information, and the risk information being predicted by the tail risk network based on initial state information corresponding to the financial derivative and/or state information corresponding to the target moment;

A training module is used to use the feedback sample set to perform at least one round of training on the strategy network, the value function network and the tail risk network respectively; after performing at least one round of training on each network and detecting that the preset training end conditions are met, the trained strategy network is obtained for adjustment of the financial derivatives hedging portfolio based on the strategy network.

In one embodiment, the interaction module is further configured to:

Obtaining trajectory data corresponding to a target financial derivative in a preset environmental data set, the trajectory data including a price trajectory, a delivery amount trajectory, a risk-free interest rate trajectory, a price trajectory of each asset within the contract period, and a cash flow trajectory of each asset within the contract period for the target financial derivative;

Using the to-be-trained policy network, the to-be-trained value function network, and the to-be-trained tail risk network to interact at least once with an environment defined by the trajectory data corresponding to the target financial derivative, to obtain a sub-feedback sample set corresponding to the target financial derivative, wherein the sub-feedback sample set includes a plurality of samples, each of which includes sample data for calculating a loss value;

The feedback sample set is generated by using the sub-feedback sample sets corresponding to a plurality of financial derivatives with different initial state information.

In one embodiment, the value function network is configured to output a state value based on state information input at a target time, at least one item of sample data is calculated based on the state value, and the interaction module is further configured to:

determining initial state information corresponding to the target financial derivative according to the trajectory data corresponding to the target financial derivative, wherein the initial state information is state information corresponding to the start time of the contract period;

Using the starting time as the target time, inputting state information corresponding to the target time into the strategy network to obtain strategy information at the target time, and generating an adjustment action for the financial derivatives hedging portfolio at the target time based on the strategy information;

Calculating, based on the adjustment action and the trajectory data corresponding to the target financial derivative, state information at a moment immediately following the target moment and a hedging error of the hedging portfolio at a moment immediately following the target moment after the adjustment action is performed;

Inputting the state information of the target moment into the value function network to be trained to obtain the state value corresponding to the target moment;

Calculating reward information after performing the adjustment action at the target time, wherein, when the moment immediately following the target time is the end time of the contract period, the reward information is calculated based on the hedging error of the target financial derivative hedging portfolio at the moment immediately following the target time and the tail risk of the hedging error, wherein the tail risk is calculated based on risk information, which is calculated by inputting initial state information corresponding to the target financial derivative and/or state information corresponding to the target time into a tail risk network to be trained;

Taking the moment after the target moment as the new target moment, and returning to the step of inputting the state information corresponding to the target moment into the policy network to obtain the policy information for the target moment, until the moment after the target moment is the end moment of the contract period;

Generate a first sample corresponding to each moment based on the state information, adjustment action, reward information, and state value corresponding to each moment within the contract period, wherein the first sample includes sample data for calculating the policy loss value corresponding to the policy network and sample data for calculating the evaluation loss value corresponding to the value function network;

generating a second sample based on the initial state information of the target financial derivative and the hedging error corresponding to the termination time of the contract period, wherein the second sample includes sample data for calculating the risk information predicted loss value corresponding to the tail risk network;

A sub-feedback sample set corresponding to the target financial derivative is generated based on the first sample and the second sample.

In one embodiment, when the risk information is calculated by inputting the initial state information corresponding to the target financial derivative into the tail risk network to be trained, the second sample includes the initial state information of the target financial derivative and the hedging error corresponding to the end time of the contract period.

In one embodiment, when the risk information is calculated by respectively inputting the initial state information corresponding to the target financial derivative and the state information corresponding at the target time into the tail risk network to be trained, the second sample includes the initial state information of the target financial derivative, the state information corresponding to the target time, and the hedging error corresponding to the end time of the contract period.

In one embodiment, when the risk information is calculated by inputting the state information corresponding to the target financial derivative at the target time into the tail risk network to be trained, the second sample includes the state information corresponding to the target financial derivative at the target time and the hedging error corresponding to the end time of the contract period.

In one embodiment, the data processing device further includes:

The collection module is configured to collect, based on a preset financial information data interface, trajectory data corresponding to any financial derivative with different initial state information and a preset asset as an underlying asset within a preset time period; and obtain the environmental dataset based on the trajectory data corresponding to each financial derivative with different initial state information.

In one embodiment, the feedback sample set includes a plurality of first samples and second samples, wherein the first samples include sample data for calculating the policy loss value corresponding to the policy network and sample data for calculating the evaluation loss value corresponding to the value function network, and the second samples include sample data for calculating the risk information prediction loss value corresponding to the tail risk network;

The training module is also used to:

Calculating a policy loss value corresponding to the policy network and an evaluation loss value corresponding to the value function network using the first sample;

Calculating the gradient values corresponding to the network parameters in the policy network and the value function network according to the policy loss value and the evaluation loss value, and updating the network parameters according to the gradient values;

Calculating a risk information predicted loss value corresponding to the tail risk network using the second sample, calculating a gradient value corresponding to a network parameter in the tail risk network according to the risk information predicted loss value, and updating the network parameter according to the gradient value;

After using the feedback sample set to update the network parameters in the policy network, the value function network and the tail risk network for at least one round, a round of training process for the policy network, the value function network and the tail risk network is completed.

In one embodiment, the second sample includes initial state information corresponding to a target financial derivative and a hedging error of the hedging combination of the target financial derivative at the time of contract termination of the target financial derivative;

The training module is also used to:

Inputting the initial state information of the second sample into the tail risk network prediction to obtain risk information corresponding to the second sample;

Substituting the risk information corresponding to the second sample and the hedging error in the second sample into a preset risk information prediction loss function to obtain a risk information prediction loss value corresponding to the second sample;

Predicting loss values based on risk information corresponding to a plurality of second samples in the feedback sample set, calculating gradient values corresponding to network parameters in the tail risk network, and updating the network parameters based on the gradient values.

In one embodiment, the status information corresponding to a financial derivative at a target time includes the cash amount in the hedging portfolio of the financial derivative at the target time, the amount of each hedging asset, the price of each hedging asset, the price of each underlying asset of the financial derivative, the remaining time to maturity of the financial derivative, the parameters affecting the price of the financial derivative stipulated in the financial derivative contract, or the ratio of the parameters affecting the price of the financial derivative stipulated in the financial derivative contract to the initial price of each underlying asset of the financial derivative, the hedging error of the hedging portfolio, and the risk-free interest rate at the target time. The status information corresponding to the financial derivative at the target time may also include the volatility of each underlying asset of the financial derivative at the target time.

In one embodiment, the policy network includes two independent multi-layer feedforward neural networks, each of which includes an input layer, several hidden layers and an output layer.

In one embodiment, the value function network includes two independent multi-layer feedforward neural networks, each of which includes an input layer, several hidden layers and an output layer, and the number of nodes in the input layer is consistent with that in the policy network, each of the hidden layers contains several nodes, and the output layer has only one node for outputting a value function variable for evaluating the value of the input state information.

In addition, the present application also provides a data processing device, as shown in Figure 2, which is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present application. It should be noted that the data processing device in the embodiment of the present application can be a smartphone, a personal computer, a server, etc., and is not specifically limited here.

As shown in Figure 2, the data processing device may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 1003 may optionally include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

Those skilled in the art will understand that the device structure shown in FIG2 does not constitute a limitation on the data processing device, and may include more or fewer components than shown, or a combination of certain components, or a different arrangement of components.

As shown in Figure 2 , memory 1005, a computer storage medium, may include an operating system, a network communication module, a user interface module, and a data processing program. The operating system is a program that manages and controls the device's hardware and software resources, supporting the execution of the data processing program and other software or programs. In the device shown in Figure 2 , user interface 1003 is primarily used for data communication with the client; network interface 1004 is primarily used for establishing a communication connection with the server; and processor 1001 is used to invoke the data processing program stored in memory 1005 and perform the following operations:

A feedback sample set is obtained by performing multiple interactions with the environment using a policy network to be trained, a value function network to be trained, and a tail risk network to be trained, wherein the policy network is used to output policy information based on state information at a target moment, and the policy information is used to generate an adjustment action for the financial derivatives hedging portfolio at the target moment. The feedback sample set includes multiple samples, each of which includes sample data used to calculate a loss value, at least one of which is calculated based on reward information, the reward information is calculated based on a hedging error of the financial derivatives hedging portfolio and a tail risk of the hedging error, the tail risk is calculated based on risk information, and the risk information is predicted by the tail risk network based on initial state information corresponding to the financial derivative and/or state information corresponding to the target moment;

Using the feedback sample set to perform at least one round of training on the policy network, the value function network, and the tail risk network respectively;

After each network is trained for at least one round and it is detected that a preset training end condition is met, the trained strategy network is obtained for adjusting the financial derivatives hedging portfolio based on the strategy network.

In one embodiment, the operation of using the to-be-trained policy network, the value function network, and the tail risk network to interact with the environment multiple times to obtain a feedback sample set includes:

Obtaining trajectory data corresponding to a target financial derivative in a preset environmental data set, the trajectory data including a price trajectory, a delivery amount trajectory, a risk-free interest rate trajectory, a price trajectory of each asset within the contract period, and a cash flow trajectory of each asset within the contract period for the target financial derivative;

Using the to-be-trained policy network, the to-be-trained value function network, and the to-be-trained tail risk network to interact at least once with an environment defined by the trajectory data corresponding to the target financial derivative, to obtain a sub-feedback sample set corresponding to the target financial derivative, wherein the sub-feedback sample set includes a plurality of samples, each of which includes sample data for calculating a loss value;

The feedback sample set is generated by using the sub-feedback sample sets corresponding to a plurality of financial derivatives with different initial state information.

In one embodiment, the value function network is configured to output a state value based on state information input at a target time, at least one item of sample data is calculated based on the state value, and the operation of using the to-be-trained policy network, the value function network, and the tail risk network to interact multiple times with an environment defined by the trajectory data corresponding to the target financial derivative to obtain a sub-feedback sample set corresponding to the target financial derivative includes:

determining initial state information corresponding to the target financial derivative according to the trajectory data corresponding to the target financial derivative, wherein the initial state information is state information corresponding to the start time of the contract period;

Using the starting time as the target time, inputting state information corresponding to the target time into the strategy network to obtain strategy information at the target time, and generating an adjustment action for the financial derivatives hedging portfolio at the target time based on the strategy information;

Calculating, based on the adjustment action and the trajectory data corresponding to the target financial derivative, state information at a moment immediately following the target moment and a hedging error of the hedging portfolio at a moment immediately following the target moment after the adjustment action is performed;

Inputting the state information of the target moment into the value function network to be trained to obtain the state value corresponding to the target moment;

Calculating reward information after performing the adjustment action at the target time, wherein, when the moment immediately following the target time is the end time of the contract period, the reward information is calculated based on the hedging error of the target financial derivative hedging portfolio at the moment immediately following the target time and the tail risk of the hedging error, wherein the tail risk is calculated based on risk information, which is calculated by inputting initial state information corresponding to the target financial derivative and/or state information corresponding to the target time into a tail risk network to be trained;

Taking the moment after the target moment as the new target moment, and returning to the step of inputting the state information corresponding to the target moment into the policy network to obtain the policy information for the target moment, until the moment after the target moment is the end moment of the contract period;

Generate a first sample corresponding to each moment based on the state information, adjustment action, reward information, and state value corresponding to each moment within the contract period, wherein the first sample includes sample data for calculating the policy loss value corresponding to the policy network and sample data for calculating the evaluation loss value corresponding to the value function network;

generating a second sample based on the initial state information of the target financial derivative and the hedging error corresponding to the termination time of the contract period, wherein the second sample includes sample data for calculating the risk information predicted loss value corresponding to the tail risk network;

A sub-feedback sample set corresponding to the target financial derivative is generated based on the first sample and the second sample.

In one embodiment, before obtaining the trajectory data corresponding to a target financial derivative in the preset environmental data set, the processor 1001 may also be configured to call a data processing program stored in the memory 1005 to perform the following operations:

For any financial derivative with a preset asset as the underlying asset and having different initial state information within a preset time period, collecting trajectory data corresponding to the financial derivative based on a preset financial information data interface;

The environmental data set is obtained according to the trajectory data corresponding to each financial derivative with different initial state information.

In one embodiment, the feedback sample set includes a plurality of first samples and second samples, wherein the first samples include sample data for calculating the policy loss value corresponding to the policy network and sample data for calculating the evaluation loss value corresponding to the value function network, and the second samples include sample data for calculating the risk information prediction loss value corresponding to the tail risk network;

The operation of using the feedback sample set to perform a round of training on the policy network, the value function network, and the tail risk network respectively includes:

Calculating a policy loss value corresponding to the policy network and an evaluation loss value corresponding to the value function network using the first sample;

Calculating the gradient values corresponding to the network parameters in the policy network and the value function network according to the policy loss value and the evaluation loss value, and updating the network parameters according to the gradient values;

Calculating a risk information predicted loss value corresponding to the tail risk network using the second sample, calculating a gradient value corresponding to a network parameter in the tail risk network according to the risk information predicted loss value, and updating the network parameter according to the gradient value;

After using the feedback sample set to update the network parameters in the policy network, the value function network and the tail risk network for at least one round, a round of training process for the policy network, the value function network and the tail risk network is completed.

In one embodiment, the second sample includes initial state information corresponding to a target financial derivative and a hedging error of the hedging combination of the target financial derivative at the time of contract termination of the target financial derivative;

The operation of calculating the risk information predicted loss value corresponding to the tail risk network using the second sample, calculating the gradient value corresponding to the network parameter in the tail risk network according to the risk information predicted loss value, and updating the network parameter according to the gradient value includes:

Inputting the initial state information of the second sample into the tail risk network prediction to obtain risk information corresponding to the second sample;

Substituting the risk information corresponding to the second sample and the hedging error in the second sample into a preset risk information prediction loss function to obtain a risk information prediction loss value corresponding to the second sample;

Predicting loss values based on risk information corresponding to a plurality of second samples in the feedback sample set, calculating gradient values corresponding to network parameters in the tail risk network, and updating the network parameters based on the gradient values.

In one embodiment, the status information corresponding to a financial derivative at a target time includes the cash amount in the hedging portfolio of the financial derivative at the target time, the amount of each hedging asset, the price of each hedging asset, the price of each underlying asset of the financial derivative, the remaining time to maturity of the financial derivative, the parameters affecting the price of the financial derivative stipulated in the financial derivative contract, or the ratio of the parameters affecting the price of the financial derivative stipulated in the financial derivative contract to the initial price of each underlying asset of the financial derivative, the hedging error of the hedging portfolio, and the risk-free interest rate at the target time. The status information corresponding to the financial derivative at the target time may also include the volatility of each underlying asset of the financial derivative at the target time.

In addition, an embodiment of the present application further provides a computer-readable storage medium, on which a data processing program is stored. When the data processing program is executed by a processor, the steps of the data processing method described above are implemented.

The various embodiments of the data processing device and computer-readable storage medium of the present application can refer to the various embodiments of the data processing method of the present application, and will not be repeated here.

It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims (15)

A data processing method, wherein the method comprises the following steps: A feedback sample set is obtained by performing multiple interactions with the environment using a policy network to be trained, a value function network to be trained, and a tail risk network to be trained, wherein the policy network is used to output policy information based on state information at a target moment, and the policy information is used to generate an adjustment action for the financial derivatives hedging portfolio at the target moment. The feedback sample set includes multiple samples, each of which includes sample data used to calculate a loss value, at least one of which is calculated based on reward information, the reward information is calculated based on a hedging error of the financial derivatives hedging portfolio and a tail risk of the hedging error, the tail risk is calculated based on risk information, and the risk information is predicted by the tail risk network based on initial state information corresponding to the financial derivative and/or state information corresponding to the target moment; Using the feedback sample set to perform at least one round of training on the policy network, the value function network, and the tail risk network respectively; After each network is trained for at least one round and it is detected that a preset training end condition is met, the trained strategy network is obtained for adjusting the financial derivatives hedging portfolio based on the strategy network. The data processing method according to claim 1, wherein the step of using the policy network to be trained, the value function network to be trained, and the tail risk network to be trained to interact with the environment multiple times to obtain a feedback sample set comprises: Obtaining trajectory data corresponding to a target financial derivative in a preset environmental data set, the trajectory data including a price trajectory, a delivery amount trajectory, a risk-free interest rate trajectory, a price trajectory of each asset within the contract period, and a cash flow trajectory of each asset within the contract period for the target financial derivative; Using the to-be-trained policy network, the to-be-trained value function network, and the to-be-trained tail risk network to interact at least once with an environment defined by the trajectory data corresponding to the target financial derivative, to obtain a sub-feedback sample set corresponding to the target financial derivative, wherein the sub-feedback sample set includes a plurality of samples, each of which includes sample data for calculating a loss value; The feedback sample set is generated by using the sub-feedback sample sets corresponding to a plurality of financial derivatives with different initial state information. The data processing method according to claim 2, wherein the value function network is used to output a state value based on state information input at a target time, at least one item of the sample data is calculated based on the state value, and the step of using the to-be-trained policy network, the to-be-trained value function network, and the to-be-trained tail risk network to interact at least once with an environment defined by the trajectory data corresponding to the target financial derivative to obtain a sub-feedback sample set corresponding to the target financial derivative comprises: determining initial state information corresponding to the target financial derivative according to the trajectory data corresponding to the target financial derivative, wherein the initial state information is state information corresponding to the start time of the contract period; Using the starting time as the target time, inputting state information corresponding to the target time into the strategy network to obtain strategy information at the target time, and generating an adjustment action for the financial derivatives hedging portfolio at the target time based on the strategy information; Calculating, based on the adjustment action and the trajectory data corresponding to the target financial derivative, state information at a moment immediately following the target moment and a hedging error of the hedging portfolio at a moment immediately following the target moment after the adjustment action is performed; Inputting the state information of the target moment into the value function network to be trained to obtain the state value corresponding to the target moment; Calculating reward information after performing the adjustment action at the target time, wherein, when the moment immediately following the target time is the end time of the contract period, the reward information is calculated based on the hedging error of the target financial derivative hedging portfolio at the moment immediately following the target time and the tail risk of the hedging error, wherein the tail risk is calculated based on risk information, which is calculated by inputting initial state information corresponding to the target financial derivative and/or state information corresponding to the target time into a tail risk network to be trained; Taking the moment after the target moment as the new target moment, and returning to the step of inputting the state information corresponding to the target moment into the policy network to obtain the policy information for the target moment, until the moment after the target moment is the end moment of the contract period; Generate a first sample corresponding to each moment based on the state information, adjustment action, reward information, and state value corresponding to each moment within the contract period, wherein the first sample includes sample data for calculating the policy loss value corresponding to the policy network and sample data for calculating the evaluation loss value corresponding to the value function network; generating a second sample based on the initial state information of the target financial derivative and the hedging error corresponding to the termination time of the contract period, wherein the second sample includes sample data for calculating the risk information predicted loss value corresponding to the tail risk network; A sub-feedback sample set corresponding to the target financial derivative is generated based on the first sample and the second sample. The data processing method according to claim 3, wherein, when the risk information is calculated by inputting initial state information corresponding to the target financial derivative into the tail risk network to be trained, the second sample includes the initial state information of the target financial derivative and the hedging error corresponding to the termination time of the contract period. The data processing method according to claim 3, wherein, when the risk information is calculated by respectively inputting initial state information corresponding to the target financial derivative and state information corresponding to the target time into the tail risk network to be trained, the second sample includes the initial state information of the target financial derivative, the state information corresponding to the target time, and the hedging error corresponding to the termination time of the contract period. The data processing method according to claim 3, wherein, when the risk information is calculated by inputting state information corresponding to the target financial derivative at the target time into the tail risk network to be trained, the second sample includes the state information corresponding to the target financial derivative at the target time and the hedging error corresponding to the termination time of the contract period. The data processing method according to claim 2, wherein, before the step of obtaining trajectory data corresponding to a target financial derivative in the preset environmental data set, the method further comprises: For any financial derivative with a preset asset as the underlying asset and having different initial state information within a preset time period, collecting trajectory data corresponding to the financial derivative based on a preset financial information data interface; The environmental data set is obtained according to the trajectory data corresponding to each financial derivative with different initial state information. The data processing method according to claim 1, wherein the feedback sample set includes a plurality of first samples and second samples, the first samples including sample data for calculating a policy loss value corresponding to the policy network and sample data for calculating an evaluation loss value corresponding to the value function network, and the second samples including sample data for calculating a risk information prediction loss value corresponding to the tail risk network; The steps of using the feedback sample set to perform a round of training on the policy network, the value function network, and the tail risk network respectively include: Calculating a policy loss value corresponding to the policy network and an evaluation loss value corresponding to the value function network using the first sample; Calculating the gradient values corresponding to the network parameters in the policy network and the value function network according to the policy loss value and the evaluation loss value, and updating the network parameters according to the gradient values; Calculating a risk information predicted loss value corresponding to the tail risk network using the second sample, calculating a gradient value corresponding to a network parameter in the tail risk network according to the risk information predicted loss value, and updating the network parameter according to the gradient value; After using the feedback sample set to update the network parameters in the policy network, the value function network and the tail risk network for at least one round, a round of training process for the policy network, the value function network and the tail risk network is completed. The data processing method according to claim 8, wherein the second sample includes initial state information corresponding to a target financial derivative and a hedging error of a hedging combination of the target financial derivative at the time of contract termination of the target financial derivative; The steps of using the second sample to calculate the risk information predicted loss value corresponding to the tail risk network, calculating the gradient value corresponding to the network parameter in the tail risk network according to the risk information predicted loss value, and updating the network parameter according to the gradient value include: Inputting the initial state information of the second sample into the tail risk network prediction to obtain risk information corresponding to the second sample; Substituting the risk information corresponding to the second sample and the hedging error in the second sample into a preset risk information prediction loss function to obtain a risk information prediction loss value corresponding to the second sample; Predicting loss values based on risk information corresponding to a plurality of second samples in the feedback sample set, calculating gradient values corresponding to network parameters in the tail risk network, and updating the network parameters based on the gradient values. The data processing method according to claim 1, wherein the status information corresponding to the financial derivative at a target time includes the amount of cash in the hedging portfolio of the financial derivative at the target time, the amount of each hedging asset, the price of each hedging asset, the price of each underlying asset of the financial derivative, the remaining time to maturity of the financial derivative, the parameter affecting the price of the financial derivative agreed in the contract of the financial derivative or the ratio of the parameter affecting the price of the financial derivative agreed in the contract of the financial derivative to the price of each underlying asset of the financial derivative at an initial time, the hedging error of the hedging portfolio, and the risk-free interest rate at the target time. The data processing method according to claim 1, wherein the policy network comprises two independent multi-layer feedforward neural networks, each of the feedforward neural networks comprising an input layer, a plurality of hidden layers and an output layer. The data processing method according to claim 1, wherein the value function network includes two independent multi-layer feedforward neural networks, each of the feedforward neural networks includes an input layer, several hidden layers and an output layer, and the number of nodes in the input layer is consistent with that in the policy network, each of the hidden layers contains several nodes, and the output layer has only one node for outputting a value function variable for evaluating the value of the input state information. A data processing device, wherein the device comprises: An interaction module, configured to employ a policy network to be trained, a value function network to be trained, and a tail risk network to be trained to interact multiple times with an environment to obtain a feedback sample set, wherein the policy network is configured to output policy information based on state information at a target moment, the policy information being configured to generate an adjustment action for the financial derivatives hedging portfolio at the target moment, the feedback sample set comprising a plurality of samples, each of which includes various sample data items used to calculate a loss value, at least one of which is calculated based on reward information, the reward information being calculated based on a hedging error of the financial derivatives hedging portfolio and a tail risk of the hedging error, the tail risk being calculated based on risk information, and the risk information being predicted by the tail risk network based on initial state information corresponding to the financial derivative and/or state information corresponding to the target moment; A training module is used to use the feedback sample set to perform at least one round of training on the strategy network, the value function network and the tail risk network respectively; after performing at least one round of training on each network and detecting that the preset training end conditions are met, the trained strategy network is obtained for adjustment of the financial derivatives hedging portfolio based on the strategy network. A data processing device, wherein the data processing device comprises: a memory, a processor, and a data processing program stored in the memory and executable on the processor, wherein when the data processing program is executed by the processor, the steps of the data processing method according to any one of claims 1 to 12 are implemented. A computer-readable storage medium, wherein a data processing program is stored on the computer-readable storage medium, and when the data processing program is executed by a processor, the steps of the data processing method according to any one of claims 1 to 12 are implemented.