Changes in hippocampal neural response by external reward

Kimoon Kang, Kyungjin You, Minwhan Jung, Hyunchool Shin

(1. Department of Electronic Engineering, Soongsil University, Seoul156-743, Korea; 2. Institute for Medical Sciences, Ajou University School of Medicine, Suwon, Korea)

Changes in hippocampal neural response by external reward

Kimoon Kang1, Kyungjin You1, Minwhan Jung2, Hyunchool Shin1

(1. Department of Electronic Engineering, Soongsil University, Seoul156-743, Korea; 2. Institute for Medical Sciences, Ajou University School of Medicine, Suwon, Korea)

The hippocampus is thought to contribute largely to memory processing and spatial navigation. Various research projects have shown evidence regarding these two crucial roles. However, many unknown functions of hippocampus remain. A great deal of research on the hippocampus is ongoing, but much of this research deals with single neuron, and little research has been conducted on what happens between neurons in hippocampus as they play these roles. In this paper, we intend to examine what changes hippocampal neurons undergo in response to a stimulus. Using an imbalanced, index of time-dependent Gini's coefficient, the firing balance between neurons during the moment in which a stimulus is

is examined. More importantly, the different firing balances are observed in reward situation. The result demonstrates the multi-neuron in hippocampus fires with balance for a while when the rat has reward. In addition, time-dependent Gini's coefficient is a feature that can verify what is not shown by using existing features.

hippocampus； multi-neuron firing inequality； time-dependent Gini's coefficient

CLD number： Q42 Document code： A

Much research on the hippocampus has been conducted recently. The hippocampus has two important roles: short-term to long-term memory processing and spatial navigation. However, many unknown functions of the hippocampus remain.

The water maze experiment and radial arm maze test had been used in past hippocampus research. The radial arm maze test shows that the hippocampus plays a role in short-term memory processing[1]. The water maze experiment shows that the hippocampus has a role in spatial navigation[2].

There have been various research projects concerning how the hippocampus is related to short-term memory processing. The research found that a sub-millisecond synchronized spike is an efficient carrier for information processing[3]. A single neuron's activity and the relationship between neurons in the various situations were investigated using a cross-correlogram[4]. A cross-correlogram was also used to investigate the relationship between hippocampal neurons and neurons in the cortex[5].

Spatial navigation has also been the subject of various research projects recently. By using the ensemble firing pattern of a place cell, inhomogeneous Poisson process modeling and a Bayesian statistical paradigm were able to predict a rat's position[6,7]. There was also an investigation of the reactivation of place cells that were activated during the daytime during sleep[8].

A great deal of research on the hippocampus is ongoing, but much of this research is concerned with a single neuron, and the multi-neuron analyses mainly examine the relationship between neurons. There has been no research that has observed and analyzed the firing rate balance of multiple neurons with or without a stimulus. This paper examines the firing balance of hippocampal neurons according to stimulus.

1 Main research work

1.1 Experiment protocol

The experiment protocol is approved by the Ethics Review Committee for Animal Experimentation of the Ajou University School of Medicine. The experiments are performed with young male Sprague Dawley rat[9].

Animals are trained to perform a dynamic two-armed bandit task with five behavioral stages. Fig.1 shows the image of the two-armed bandit task.

There are four blocks of trials, which can be distinguished via the reward probabilities[8]. For more information about the experiment protocols, see the materials and method section in Ref.[9].

Fig.1 Image of two-armed bandit task. Arrows indicate the locations of photo beam detectors. Discs on the both arms represent where the reward is dispensed

1.2 Theory analysis

We use two methods to analyze the data: single trials and grouped trials. When we grouped the trials, we grouped them into the trials with a reward and the trials without a reward. The total number of trials in the reward trial group was 100, and the total number of trials in the no-reward trial group was 67. When using these two trial groups for analysis, time section in which a spike existes in less than 20% of the total trials is excluded from the analysis.

The firing rate is a count of the spikes generated per unit time in a neuron. The firing rate is the basis of neuron analysis, and we calculate various features by using the firing rate data. We obtain the firing rate data by using alpha kernel method as

where w(τ)=[α2τexp(-ατ)]+, and the notation [z]+is

The neuron-averaged firing rate means the average of the rate at which multiple neurons fire at the same time.

The neuron-averaged firing rate can be expressed as

where N represents the total number of neurons, and fn,tstands for the firing rate of the n-th neuron in the t-th bin. Similarly, the variance of multiple neurons can be calculated.

Entropy, or Shannon entropy, is a measure of uncertainty in a random variable. Entropy is an expectation of self-information. Self-information is defined as

h(p(xk))=-log2p(xk),

and p(xk) can be written as

Because entropy is the expectation of self-information, it can be expressed as

The unit of entropy is the bit because the base of the logarithm is 2. In this paper, entropy is normalized using the maximum possible bit.

Gini's coefficient is an economic concept that was created in order to represent the imbalance of income within a population, and it has a value between 0 and 1[10]. The closer it is to 1, the more imbalanced the distribution is.

The equation for Gini's coefficient is

The x originally means the income of the population, and r represents the total number of people in the population. However, in this paper, x is the firing rate vector of each neuron, r is the total number of neurons. and p is a permutation matrix that sorts xtin ascending order[10].

A time-dependent Gini's coefficient is a measure of the degree of inequality with respect to time. The time-dependent Gini's coefficient is

G(xt) is Gini's coefficient for the t-th bin. Similar to the above, G(xt) can be written as

All the parameters are the same as the parameters from Gini's coefficient, except xtand T. T is total number of bins, and xtis a vector of the firing rates of each neuron in the t-th bin.

2 Experimental results

Through the analysis of a single trial, it is verified that there is no association between mean firing rate and time-dependent Gini's coefficient.

Fig.2 shows the basic information from two single trials. The top images shows the firing rate of 23 neurons. The top images indicate the directions chosen (position of circle) with reward existence or nonexistence (filled: rewarded). The bottom graphs show the time-dependent Gini's coefficient and the neuron-averaged firing rate.

It can be seen from Fig.2 that the peak values of the neuron-averaged firing of these two trials during the reward stage are similar. However, the time-dependent Gini's coefficients in the reward stage are quite different. In Fig.2(a), the time-dependent Gini's coefficient becomes lower, but there is no change in Fig.2(b).

Fig.2 Basic information of two single trials

When the trials are divided into a reward group and a no-reward group, as shown in Fig.3, changes are shown in the reward group during the reward stage. There is no special change that can be used to differentiate the reward group from the no-reward group based on other features of the graph. However, in the case of the mean Gini's coefficient, there is a visible decrease for the reward group during the reward stage. Therefore, the phenomenon seen in Fig.3(a) of the single trial analysis appears in the overall reward group. According to the results of enumerating each trial in the two groups, it can be verified that this phenomenon appears in a majority of trials in the reward group, as shown in Fig.4.

Fig.3 Mean of four features of the reward and no-reward trial groups

Fig.4 Cumulative single trials with four features

In order to examine the changes in time-dependent Gini's coefficient, the probability of the time-dependent Gini's coefficient being over the threshold according to time is examined. The threshold is determined by using the mean and standard deviation of the mean Gini's coefficient for all trials.

In the reward stage of the reward trial group, the probability of the time-dependent Gini's coefficient being under the threshold increased dynamically; the maximum probability in the reward stage is 0.57. For the no-reward trial group, the probability of the time-dependent Gini's coefficient being under the threshold does not increase; the maximum probability during the reward stage is less than 0.03, as shown in Fig.5.

Fig.5 Probability of Gini's coefficient being under the threshold

3 Discussion and conclusion

The single-trial analysis shows that the firing rate does not affect the time-dependent Gini's coefficient. Fig.2 shows that similar neuron-averaged firing rates do not have similar time-dependent Gini's coefficients. If one compares the top left image of Figs.2(a) and (b), one will notice that the two trials have different firing patterns. However, in Fig.2(a), there is a section in which numerous neurons have similar firing rates, a small number of neurons show overwhelmingly high firing rates in Fig.2(b).

This shows that the firing balance of the neurons affect the time-dependent Gini's coefficient. From the mean trial analysis, only the mean Gini's coefficient shows an identical difference between the two groups. The reason that the graph is shorter in the reward stage of the no-reward trials than that of the reward trials is that the duration of the reward stage is shorter in the no-reward trials. It is unreasonable to examine the firing balance with the mean firing rate and the variance of firing rate. In the case of entropy, although it can detect somewhat inequality, it is unreasonable to sense the balance changes of the moment because it needs long enough time to calculate. The time-dependent Gini's coefficient is a feature that can verify what is not shown by using the existing features.

Changes observed through the time-dependent Gini's coefficient in the hippocampal neuron show that the hippocampus responds to the stimulus of reward. Further study is necessary to determine exactly what this response is. Research on changes in firing patterns should also be expanded to include diverse stimuli and rewards.

[1] Olton D S，Samuelson R J. Remembrance of places pa-ssed: spatial memory in rats. Journal of Experimental Psychology: Animal Behavior Processes, 1976, 2(2): 97-116.

[2] Morris R. Developments of a water-maze procedure for studying spatial learning in the rat. Journal of Neuroscience Methods, 1984, 11(1): 47-60.

[3] Takahashi S, Sakurai Y. Sub-millisecond firing synchrony of closely neighboring pyramidal neurons in hippocampal CA1 of rats during delayed non-matching to sample task. Front Neural Circuits, 2009, 3: 9.

[4] Sakurai Y. Hippocampal and neocortical cell assemblies encode memory processes for different types of stimuli in the rat. The Journal of Neuroscience, 1996, 16(8): 2809-2819.

[5] Sakurai Y. Dependence of functional synaptic connections of hippocampal and neocortical neurons on types of memory. Neuroscience Letters, 1993, 158(2): 181-184.

[6] Brown E N, Frank L M, Tang D T, et al. A statistical paradigm for neural spike train decoding applied to position prediction from ensemble firing patterns of rat hippocampal place cells. The Journal of Neuroscience, 1998, 18(18): 7411-7425.

[7] Shimazaki H, Amarim S I, Brown E. N, et al. State-space analysis on time-varying correlations in parallel spike sequences. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP'09), 2009: 3501-3504.

[8] Wilson M A, McNaughton B L. Reactivation of hippocampal ensemble memories during sleep. Science, 1994, 265(5172): 676-679.

[9] Kim H, Sul J H, Huh N, et al. Role of striatum in updating values of chosen actions. The Journal of Neuroscience, 2009, 29(47): 140701-14712.

[10] XU Kuan. How has the literature on Gini's index evolved in the past 80 years? China Economic Quarterly， 2003，2(4): 757-763.

date： 2013-08-22

The MSIP (Ministry of Science, ICT & Future Planning), Korea, under the ITRC (Information Technology Research Center) support program (NIPA-2013-H0301-13-2006) supervised by the NIPA (National IT Industry Promotion Agency); The Brain Research Program through the National Research Foundation of Korea funded by the Ministry of Science, ICT & Future Planning (2011-0019212)

Hyunchool Shin (shinhc@ssu.ac.kr)

1674-8042(2013)04-0397-05

10.3969/j.issn.1674-8042.2013.04.020