CN116701696A - Picture description method based on pulse transducer model - Google Patents

Abstract

The invention discloses a picture description method based on a pulse transducer model, which comprises the following steps: firstly, designing a novel pulse neuron PLMP which is more in accordance with biodiversity and has a learnable membrane potential time constant and a voltage threshold value, wherein the neuron can optimize the problem that the training gradient of a pulse model disappears; modifying a common self-attention mechanism in a transducer into a pulse attention mechanism by using PLMP neurons; constructing a pulse transducer by using a pulse self-attention mechanism for training a picture description model; finally, a pulse transducer model which is applicable to the field of picture description, energy-saving and capable of generating high-quality picture description is obtained.

Description

A picture description method based on impulse Transformer model

technical field

The invention relates to the application of a pulse neural network in the field of picture description, in particular to a picture description method based on a pulse Transformer model.

Background technique

In April 2022, Google released the language model PaLM based on the "general AI architecture". On November 30, 2022, OpenAI announced a new conversational AI model ChatGPT fine-tuned by the GPT-3.5 series of large-scale speech models. It can not only conduct natural multi-round dialogues, efficient and precise question and answer, but also generate programming codes, emails, papers, novels and other texts. The popularity of ChatGPT has set off a passion for exploring large models at home and abroad. With the maturity of large language models, more and more technology companies and researchers are focusing on the research of multimodal large models. Google launched PaLM-E, the largest visual language multimodal model in history, on March 6, 2023, demonstrating extraordinary performance in the field of robot manipulation, visual question answering, picture description, and pure language tasks. On March 14, 2023, OpenAI launched ChatGPT-4, a multimodal large language model with a larger model scale, richer knowledge base, and stronger context understanding capabilities. The popularity of ChatGPT has set off a passion for exploring large models at home and abroad. Domestic academic circles and technology companies have also announced or will launch similar robot dialogue models, such as Baidu's Wenxinyiyan, Ali's "Tongyi Qianwen", Huawei's Pangu model, Tencent Hunyuan model, etc. Large models have become the general trend. However, the cost of training a large model is huge. With the continuous increase in the growth rate of parameter scale, there are still bottlenecks in computing power and training costs. At the Artificial Intelligence Large Model Technology Summit Forum held on April 8, 2023, Tian Qi, head of Huawei's large model, said that the development and training of a large model cost 12 million U.S. dollars, of which 7.2 million U.S. dollars were spent on electricity, and the cost of large models was reduced and increased. There are two aspects to efficiency, one is to optimize computing power, and the other is to optimize power. There is a huge space for power cost reduction and efficiency increase.

Spiking Neuron Networks (SNNs), as the third-generation artificial neural network, has the advantages of small amount of calculation, low power consumption, and fast information transmission. The traditional artificial neural network (Artificial Neural Networks, ANNs) is a computing model based on the information processing method of the biological nervous system. As a subdivision field of multimodality, picture description is a subdivision task of multimodal large models. It can provide useful information for visually impaired people, and can be used for automatic image and video labeling. It has huge research value. In the future, neuromorphic computing, as an efficient method, may realize the cost reduction and efficiency increase of multi-modal large model training through the co-evolution of neuromorphic chips and SNNs algorithms. The invention studies an energy-saving pulse picture description model, which is an exploration and application of an energy-saving multi-mode model. However, due to the binary and non-differentiable nature of peak activity, direct training of SNNs may suffer from severe gradient vanishing and network degradation problems. Moreover, the currently commonly used spiking neurons are single, and the membrane potential time constant and membrane voltage threshold need to be specified as hyperparameters based on experience or optimization methods, which violates the biological diversity of neurons.

Contents of the invention

The object of the present invention is to provide a picture description method based on the impulse Transformer model to address the deficiencies of the prior art.

The object of the present invention is achieved by the following technical solutions: a method for describing pictures based on the pulse Transformer model, comprising the following steps:

(1) Design a spiking neuron PLMP, each PLMP unit contains multiple parallel LIF units with different membrane potential time constants and voltage thresholds;

(2) Based on the PLMP unit design to realize the pulse self-attention mechanism;

(3) Build Transformer model; Described Transformer model adopts encoder-decoder frame to realize, and wherein encoder is made up of a Swin Transformer and N refinement encoder blocks, and decoder is made up of N decoder blocks;

(4) Change the Transformer model in step (3) to a pulse Transformer model based on the PLMP unit and the pulse self-attention mechanism;

(5) obtain the data set of picture description field, divide data set into training set, verification set and test set, described training set is used for training pulse Transformer model; Described verification set is used for selecting optimal pulse Transformer model; The test set is input to the optimal pulse Transformer model, and the picture description is output.

Further, in the step (1), after receiving the input, each parallel LIF unit will update the membrane potential according to its respective membrane potential time constant. If the membrane potential exceeds the voltage threshold corresponding to the parallel LIF unit, the parallel LIF unit will The unit produces a peak; the output of the PLMP is the aggregate of the peaks produced by all parallel LIF units.

Further, the forward process of the spiking neuron PLMP is as follows:

Vth _k ＝tanh(z _k )

The PLMP unit introduces two trainable parameters m and z, representing the learnable membrane potential time constant parameter and the learnable membrane voltage parameter, respectively. z _k represents the learnable membrane voltage parameter of the kth LIF unit in each PLMP unit, and Vth _k is obtained through the hyperbolic tangent function, and Vth _k is the membrane voltage threshold of the kth LIF unit. _mk represents the learnable membrane potential time constant parameter of the kth LIF unit in each PLMP unit, calculated from _τk , where _τk is the membrane potential time constant of the kth LIF unit in each PLMP unit. p(n-1) represents the number of neurons in the (n-1)th layer. is the presynaptic input to the i-th neuron in layer n at time t. /> is the synaptic weight from the jth neuron in the (n-1)th layer to the ith neuron in the nth layer, /> is biased. and /> Represent the membrane potential vector and output vector of the k-th LIF unit of the i-th neuron in the n-th layer at time t+1, respectively. /> Represents the final output of the i-th PLMP unit in the n-th layer at time t+1, and it will participate in the calculation of all neurons connected to the unit in the n+1-th layer.

Further, in the step (2), the Query, Key and Value are converted into pulses by the pulse neuron PLMP, and the formula is as follows:

Q _i ＝PLMP(BN(XW _i ^Q ))

K _i =PLMP(BN(XW _i ^K ))

V _i =PLMP(BN(XW _i ^V ))

head _i = Q _i K _i ^T V _i

S'＝Concat(head ₁ ,...,head _n )

SpikingMSA(Q,K,V)=PLMP(BN(Linear(S')))

X is the input of the self-attention mechanism, is a learnable linear matrix. i=1,2,...,h, h means that the attention mechanism has h heads. V _i represents the vector of the input feature of the i-th attention head, Q _i and K _i are the feature vectors of the i-th attention head to calculate the weight of the attention weight. BN is Batch Normalization batch normalization operation. head _i represents the output of the i-th attention head. S' is the result of putting together the output of h attention heads. SpikingMSA is a modified Spiking Self-Attention mechanism.

Further, in the step (3), the Swin Transformer is used to extract grid features from the input image, and average pooling is performed on the grid features to obtain global features; the refinement encoder is used to capture grid features and grid features, The internal relationship between grid features and global features; capture the relationship between grid features and grid features using SW/W-MSA; capture the relationship between grid features and global features using MSA, in which global features as self- Key in the attention mechanism; each refined encoder block in the encoder first sends the obtained grid features and global features to the self-attention mechanism, sums the input of each self-attention mechanism with its output, and normalizes After normalization, it is passed into the feed-forward neural network, and finally the residuals are summed and normalized to output, and the refined global features and grid features are obtained.

Further, the formulation of each refined encoder block in the encoder is as follows:

FeedForward(x)=W ₂ ReLU(W ₁ x)

and /> Denote the output grid features and global features of the lth Encoder block, respectively. /> W ₁ , W ₂ are learnable parameters. /> Represents the concate operation on grid features and global features. /> It is the result of adding the input and output of the sliding window multi-head self-attention mechanism and then performing layer normalization. /> It is the result of adding the input and output of the ordinary multi-head self-attention mechanism and then performing layer normalization.

Further, the refined global features are first fused into the input of the decoder, and the global visual context information is captured through the first multimodal interaction to obtain Then captured by the Language Masked MSA module /> The word-to-word intramodal relations in get />

Represents the input of the (l-1)th refinement encoder, the output of the (l-1)th refinement encoder will be used as the input of the lth refinement encoder at time t, W _f is a learnable linear layer parameters; /> and /> is a learnable parameter, /> Represents the embedding vector corresponding to the generated word at (t-1) time, and each word calculates the attention map of the previously generated word;

Finally, through the Cross MSA Module module pair Modeling the multimodal relationship between features and refined mesh features captures local visual context information to generate image captions.

Further, the implementation formula of the Cross MSAModule module is as follows:

in, and W _x are learnable parameters. /> is the output of the Cross MSA, where /> As the Query of Cross MSA, /> As the Key and Value of the Cross MSA. /> Is the final output of the Cross MSA Module.

Further, in the step (4), the impulse self-attention mechanism is used to replace the self-attention mechanism, and the PLMP unit is used to replace the ReLU unit in the FeedForward unit, and the implementation of the impulse FeedForward unit is as follows:

SpikingFeedForward(x) = W ₂ PLMP(W ₁ x).

Further, the datasets in the field of picture description include MSCOCO 2014, Flickr30K and Flickr8K, Flickr30K and Flickr8K, VizWiz, TextCaps, Fashion Captioning and CUB-200.

The beneficial effects of the present invention are:

1. A new pulse activation unit PLMP is proposed. Compared with the LIF unit commonly used in SNN models, it has multi-level learnable membrane potential time constants and thresholds. This method can effectively alleviate the problem of gradient disappearance, weaken the influence of parameter initial value settings, accelerate the training process, and have better activation effects.

2. A new spike-type self-attention mechanism Spiking-MSA/W-MSA/SW-MSA is implemented based on PLMP. Query, Key, and Value in the form of sparse spikes are used to avoid multiplication during the calculation process and effectively reduce the amount of calculation.

3. Innovatively combine the spiking neural network and Transformer model in the field of picture description and achieve competitive results. It is a pulse Transformer model suitable for the field of picture description, energy-saving, and capable of generating high-quality picture descriptions ; and, this is the first application of spiking neural networks in the field of image description.

Description of drawings

Fig. 1 is the general flowchart of the present invention;

Figure 2 Schematic diagram of the structure of the impulse self-attention mechanism;

Figure 3 Schematic diagram of the pulse Transformer model structure.

Detailed ways

The specific implementation manners of the present invention will be further described in detail below in conjunction with the drawings and specific implementation manners. The following drawings are used to illustrate the present invention, but not to limit the scope of the present invention.

Embodiment one

Please refer to FIG. 1 , FIG. 2 and FIG. 3 , and FIG. 1 is a flow chart of a picture description algorithm based on an impulse Transformer model provided by an example of the present invention. Including the following steps:

Step 1: Design a new type of spiking neuron PLMP (Parallel LIF with Multistage Learnable Parameters);

Each PLMP unit contains multiple parallel LIF units with different membrane potential time constants and thresholds. After receiving the input, each LIF unit will update the membrane potential according to its respective membrane potential time constant. If the membrane potential exceeds the corresponding threshold, the LIF unit will generate a peak. The output of the PLMP is the aggregate of the peaks produced by all parallel LIF units. In this process, the membrane potential time constant and threshold are automatically optimized during training, and are no longer manually set as hyperparameters before training. The forward process of a PLMP neuron is as follows:

Vth _k ＝tanh(z _k )

The PLMP unit introduces two trainable parameters m and z, representing the learnable membrane potential time constant parameter and the learnable membrane voltage threshold parameter, respectively. z _k represents the learnable membrane voltage parameter of the kth LIF unit in each PLMP unit, and Vth _k is obtained through the hyperbolic tangent function, and Vth _k is the membrane voltage threshold of the kth LIF unit. _mk represents the parameter of the learnable membrane potential time constant of the kth LIF unit in each PLMP unit, calculated from _τk , where _τk is the membrane potential time constant of the kth LIF unit in each PLMP unit. n and p(n-1) denote the number of neurons in layer n and (n-1) respectively. is the presynaptic input to the i-th neuron in layer n at time t. /> is the synaptic weight from the jth neuron in the (n-1)th layer to the ith neuron in the nth layer, /> is biased. /> and /> Represent the membrane potential vector and output vector of the k-th LIF unit of the i-th neuron in the n-th layer at time t, respectively, and K is the total number of LIF units. /> Represents the final output of the i-th PLMP unit in the n-th layer at time t, and it will participate in the calculation of all neurons connected to the unit in the n+1-th layer.

In this embodiment, it is taken that each PLMP unit includes three LIF units as an example. The initial value of the membrane voltage threshold of each LIF unit was 0.6, 1.6, 2.6; the initial membrane potential time constant was 0.25; TimeStep was 4.

Step 2: Implement the impulse self-attention mechanism based on PLMP unit design

The Query, Key and Value are converted into pulses by the pulse neuron PLMP to avoid the floating-point multiplication of the matrix, and the operations between the matrices can be completed by logical AND operations and addition. Since the attention matrix calculated by Q, K, and V in pulse form is naturally non-negative, Softmax is not required to maintain the non-negative value of the attention matrix. The specific formula is as follows:

Q _i ＝PLMP(BN(XW _i ^Q ))

K _i =PLMP(BN(XW _i ^K ))

V _i =PLMP(BN(XW _i ^V ))

head _i = Q _i K _i ^T V _i

S'＝Concat(head ₁ ,...,head _n )

SpikingMSA(Q,K,V)=PLMP(BN(Linear(S')))

X is the input of the self-attention mechanism, is a learnable linear matrix. i=1,2,...,h, h means that the attention mechanism has h heads. V _i represents the vector of the input feature of the i-th attention head, Q _i and K _i are the feature vectors of the i-th attention head to calculate the weight of the attention weight. BN is Batch Normalization batch normalization operation. head _i represents the output of the i-th attention head. S' is the result of putting together the output of h attention heads. SpikingMSA is a modified Spiking Self-Attention mechanism.

Step 3: Build the Transformer model

Implemented using the widely used encoder-decoder framework. The encoder consists of a Swin Transformer and 3 refined encoder blocks, and the decoder consists of 3 decoder blocks. The pre-trained Swin Transformer is responsible for extracting grid features from the input image, performing average pooling on the grid features to obtain global features, and the refinement encoder is responsible for capturing the relationship between grid features and grid features, grid features and global features. internal relationship. Capturing mesh features and relationships between mesh features employs SW/W-MSA. MSA is used to capture the relationship between grid features and global features, and global features are used as the Key in the self-attention mechanism. Each refined encoder block of the encoder first sends the obtained grid features and global features to the self-attention mechanism. Here, the residual structure is introduced to solve the problem of gradient disappearance in the deep network, and the input and the output of the attention mechanism The summation is passed into the feed-forward neural network after normalization, and the final residual summation and normalization is output. The decoder utilizes the refined image grid features to generate captions verbatim by capturing the correlation between text and image grid features. The formula for each refinement encoder block in the Encoder is as follows:

FeedForward(x)=W ₂ ReLU(W ₁ x)

and /> Denote the output grid features and global features of the Lth Encoder block, respectively. /> W ₁ , W ₂ are learnable parameters. /> Represents the concate operation on grid features and global features. /> It is the result of adding the input and output of the sliding window multi-head self-attention mechanism and then performing layer normalization. /> It is the result of adding the input and output of the ordinary multi-head self-attention mechanism and then performing layer normalization.

In the Decoder, the refined Global Feature is first fused into the input of the decoder, and the global visual context information is captured through the first multimodal interaction to obtain Then captured by the Language Masked MSA module The word-to-word intramodal relations in get />

Represents the input of the (l-1)th refinement encoder, the output of the (l-1)th refinement encoder will be used as the input of the lth refinement encoder at time t, W _f is a learnable linear layer parameters; /> and /> is a learnable parameter, /> Represents the embedding vector corresponding to the generated word at (t-1) time, and each word only allows the calculation of the attention map of the previously generated word;

Finally, through the Cross MSA Module module pair Model the multi-modal relationship between the grid feature and the grid feature, and capture the local visual context information to generate a picture description. Through two multimodal interactions between Global Feature and Grid Feature and sentences, the reasoning ability is enhanced. The implementation formula of the Cross MSAModule module is as follows:

in, and W _x are learnable parameters. /> is the output of the Cross MSA, where /> As the Query of Cross MSA, /> As the Key and Value of the Cross MSA. /> Is the final output of Cross MSA Module, by /> Calculated according to the above formula.

Step 4: Change the Transformer model in step (3) to a pulse Transformer model based on the PLMP unit and the pulse self-attention mechanism;

Based on the impulse self-attention mechanism, the entire Transformer model is impulsively transformed. Replace the self-attention mechanism with the pulse self-attention mechanism in step 2, replace the ReLU unit in the FeedForward unit with the PLMP unit in step 1, represent the image information and words as event-driven peaks, and then train the entire network to obtain a Suitable for the field of picture description, energy-saving, pulse Transformer model that can generate high-quality picture descriptions. The implementation of pulse self-attention refers to step 2, and the implementation of the pulse FeedForward unit is as follows:

SpikingFeedForward(x) = W ₂ PLMP(W ₁ x)

Step 5: obtain data set, divide data set into training set, verification set and test set, described training set is used for training impulse Transformer model; Described verification set is used for selecting optimal impulse Transformer model; Described test set Input the optimal pulse Transformer model and output a picture description.

Train the Impulse Transformer model. Trained on the MSCOCO 2014 dataset, which contains 123,287 images, each with 5 reference captions, following the "Karpathy" segmentation to re-partition MSCOCO, where 113,287 images are used for training; 5,000 images are used for Select hyperparameters, select the optimal pulse Transformer model; 5000 images are used for offline evaluation, and output descriptions about the images. The present invention can also adopt Flickr30K and Flickr8K, Flickr30K and Flickr8K, VizWiz, TextCaps, Fashion Captioning, CUB-200 and other commonly used data sets in the field of picture description for training.

The final effect of the present invention is that a picture can be input, and a text description of the picture can be obtained, and the effect is shown in FIG. 3 .

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims (10)

1. The picture description method based on the pulse transducer model is characterized by comprising the following steps of:

(1) Designing a pulse neuron PLMP, wherein each PLMP unit comprises a plurality of parallel LIF units with different membrane potential time constants and voltage thresholds;

(2) Realizing a pulse self-attention mechanism based on PLMP unit design;

(3) Constructing a transducer model; the transducer model is realized by adopting an encoder-decoder framework, wherein the encoder consists of one Swin transducer and N refinement encoder blocks, and the decoder consists of N decoder blocks;

(4) Changing the transducer model in step (3) to a pulse transducer model based on the PLMP unit and the pulse self-attention mechanism;

(5) Acquiring a data set in the field of picture description, and dividing the data set into a training set, a verification set and a test set, wherein the training set is used for training a pulse transducer model; the verification set is used for selecting an optimal pulse transducer model; and inputting the test set into an optimal pulse transducer model, and outputting a picture description.

2. The method of claim 1, wherein in the step (1), after receiving the input, each parallel LIF unit updates the membrane potential according to the respective membrane potential time constant, and if the membrane potential exceeds the voltage threshold corresponding to the parallel LIF unit, the parallel LIF unit generates a peak value; the output of PLMP is the set of peaks produced by all parallel LIF units.

3. The picture description method based on the pulse transducer model according to claim 1, wherein the forward process of the pulse neuron PLMP is as follows:

Vth _k ＝tanh(z _k )

the PLMP unit introduces two trainable parameters m and z, which respectively represent a learnable membrane potential time constant parameter and a learnable membrane voltage parameter; z _k Learning film voltage parameters representing the kth LIF unit in each PLMP unit, vth is obtained by hyperbolic cut-off function _k ，Vth _k Is the membrane voltage threshold of the kth LIF unit; m is m _k A learnable membrane potential time constant parameter representing the kth LIF unit in each PLMP unit, represented by τ _k Calculated τ _k Is the membrane potential time constant of the kth LIF unit in each PLMP unit; p (n-1) represents the number of neurons of the (n-1) th layer;is the presynaptic input of the nth layer of the ith neuron at time t; />Is the synaptic weight from the (n-1) th layer (j) th neuron to the (n) th layer (i) th neuron,/v>Is biased; /> and />Respectively representing a membrane potential vector and an output vector of a kth LIF unit of an nth layer of an ith neuron at the time t+1;representing the final output of the nth PLMP unit of the nth layer at time t+1, would participate in the calculation of all neurons connected to that unit of the nth layer.

4. The picture description method based on the pulse Transformer model according to claim 1, wherein in the step (2), the Query, key and Value are converted into pulses by the pulse neurons PLMP, and the formula is as follows:

Q _i ＝PLMP(BN(XW _i ^Q ))

K _i ＝PLMP(BN(XW _i ^K ))

V _i ＝PLMP(BN(XW _i ^V ))

head _i ＝Q _i K _i ^T V _i

S'＝Concat(head ₁ ,...,head _n )

SpikingMSA(Q,K,V)＝PLMP(BN(Linear(S')))

x is the input of the self-attention mechanism, W _i ^Q， ，W _i ^K ，W _i ^V Is a linear matrix that can be learned; i=1, 2,..h, h represents that the attention mechanism has h heads; v (V) _i Input features representing the ith attention headerVector, Q of (1) _i 、K _i Is the feature vector of the ith attention head for calculating the attention weight; BN was a Batch Normalization lot normalization operation; head part _i Representing the output of the ith attention head; s' is the result of the multi-head which is the combination of the outputs of the h attention heads; spikingMSA is a modified pulsed self-attention mechanism.

5. The picture description method based on the pulse Transformer model according to claim 1, wherein in the step (3), swin Transformer is used for extracting grid features from an input image, and the grid features are averaged and pooled to obtain global features; the refinement encoder is used for capturing grid features and internal relations among the grid features, the grid features and the global features; capturing the relation between the grid characteristics and the grid characteristics by adopting an SW/W-MSA; capturing the relation between grid features and global features by adopting MSA, wherein the global features are used as keys in a self-attention mechanism; each refined encoder block in the encoder firstly sends the obtained grid features and global features into a self-attention mechanism, sums the input of each self-attention mechanism with the output of each self-attention mechanism, normalizes the input of each self-attention mechanism, sends the normalized input into a feedforward neural network, and finally, sums the residual errors, normalizes the normalized output and obtains the refined global features and grid features.

6. The picture description method based on the pulse Transformer model according to claim 5, wherein the formula of each refinement encoder block in the encoder is as follows:

FeedForward(x)＝W ₂ ReLU(W ₁ x)

and />Respectively representing the output grid characteristics and the global characteristics of the first Encoder block; />W ₁ ，W ₂ Is a learnable parameter; />Performing conccate operation on the grid features and the global features; />The result is obtained by adding the input and output of the multi-head self-attention mechanism of the sliding window and then carrying out layer normalization; />Is the result of adding the input and output of the common multi-head self-attention mechanism and then carrying out layer normalization.

7. The picture description method based on pulse transducer model as claimed in claim 5, wherein the refined global features are fused into the input of the decoder, and the global visual context information is captured through the first multi-modal interaction to obtainThen capture by Language Masked MSA module->The internal mode relation from medium word to word is obtained +.>

Representing the input of the (l-1) th refinement encoder, the output of which will be the input of the (l-1) th refinement encoder at time t, W _f Is a learnable parameter of the linear layer; /> and />Is a parameter that can be learned and is,representing the embedding vector corresponding to the generated word at time (t-1), each word calculating an attention map of the word it generated before;

finally, the Cross MSA Module pair is passedAnd modeling the multi-modal relationship between the refined grid features, capturing local visual context information to generate a picture description.

8. The picture description method based on the pulse Transformer model according to claim 7, wherein the Cross msamodel module is implemented as follows:

wherein , and W_x Is a learnable parameter; />Is the output of Cross MSA, wherein +.>Query as Cross MSA, < ->Key and Value as Cross MSA; />Is the final output of the Cross MSA Module.

9. The picture description method based on the pulse Transformer model according to claim 1, wherein in the step (4), a pulse self-attention mechanism is used to replace the self-attention mechanism, a PLMP unit is used to replace a ReLU unit in a FeedForward unit, and the implementation of the pulse FeedForward unit is as follows:

SpikingFeedForward(x)＝W ₂ PLMP(W ₁ x)。

10. the picture description method based on the pulse transducer model according to claim 1, wherein the data set in the picture description field comprises MSCOCO 2014, flickr30K, flickr8K, flickr K, flickr8K, vizWiz, textCaps, fashion Captioning and CUB-200.