Transcranial magnetic stimulation (TMS) is a noninvasive brain stimulation technique in which a time-varying (shortpulse) magnetic field is used to cause electric current to flowin a small region of the brain via electromagnetic induction,thus stimulating new neural activity or modulating ongoing activity. Electroencephalography (EEG) is then used to monitor this brain activity response to TMS cortical stimulation.
The combination has become a promising tool for measuring and even intervening in cortical function, for example asessing the connection between the primary motor cortex and a muscle to diagnose damage from stroke, movement disordersand motor neuron disease, or optimizing TMS treatment fordepression. The major drawback of EEG measurement ofTMS effects is that there is a relatively long lasting artifacts generated by TMS impulse that has an amplitude that is muchlarger than brain activity measeured by the EEG.
This makes it very difficult to evaluate EEG in the critical first 20ms after the TMS pulse. Despite many years of work on artifact mitigation, existing solutions (extracting independent sources with ICA, applying an offline Kalman filter and simply appying sample-and-hold method) have significant drawbacks.
This project demonstrates a novel method to characterize and remove the long lasting TMS artifacts efficiently using different recurrent neural network (RNN) architectures, specifically Long Short-Term Memory (LSTM) network and Gated Recurrent Unit (GRU) with a sliding window of size 5.
Both LSTM and GRU networks have 2 stacks of hidden layers with each having 64 hidden states and the hidden states of the last hidden layer are connected to a fully connected layer with 64 nodes. The output of fully connected layer is the sample predicted following the sliding window.
Model is compiled seperately using Adam optimizer with a decaying learning rate initialized as 0.005 and L-BFGS with a decaying learning rate initialized as 0.5. L-BFGS is a Quasi-Newton method that estimates inverse Hessian matrix to steer its search through variable space. Although L-BFGS is memory-greedy, it is a lot more numerically stable than the variants of stochastic optimization in finding local minima and maxima. Hence it is well-suited for small datasets. In the optimization process, the objective function was mean square error, because stability was more important than robustness in the fitting process.
Since the range of the samples varies between the order of magnitudes 10-3 and 105, before model training data processing was essential. The data for each EEG trial was scaled using a linear scaling method Min-Max and non-linear scaling method Log-Scaling with a log base of 5. The results show the lowest loss in the predictions was obtained by using Min-Max scaling during data processing, GRUs for setting the RNN architecture, and L-BFGS for optimization.
Since the dataset per EEG channel is composed of only 30 TMS stimulations, in order to avoid the network’s possible proneness to overfitting, the model was trained with only 22 trials, validated with 5 trials and tested with 3 trials. No cross-validation was applied. In the training step, the hidden states were adjusted at each epoch, while during validation neither weights were adjusted nor parameters were tuned, only the loss function was tracked to validate the network does not overfit. In testing, the final model was tested on dataset that has not been seen before in order to confirm the actual predictive power of the network.
The code parses arguments from the command line, available for optimization with both quasi-Newton based L-BFGS and a variant of stochastic optimization Adam as well as LSTM and GRU for creating the network architecture and Min-Max / log scaling for data processing. The model was tested with dummy data generated from a sinusoid, Mackey-Glass time series and a multiple sine wave oscillator before training on the TMS-EEG dataset.
Author: Andac Demir
Contact: andacdemir@gmail.com