Semantic category-based decoding of human brain activity using a Gabor-based model by estimating intracranial field potential range in temporal cortex

Abstract

Predicting and decoding the recorded neural activity for visual stimuli is the topic of many studies. This prediction can be made by comparing the model's responses to different stimuli with the available recorded brain signal. The neural activities can be decoded then by finding the stimulus which has generated the nearest model's response to the recorded signal. In this study, a model is proposed which can estimate the response of human brain to images from different conceptual categories by inserting the visual stimuli as the model input after filtering by Gabor wavelets. This helped us to find each image's low level visual features. Afterward, the extracted image features were applied to the input of a curve fitting neural network. As the output, the range of intracranial field potential was estimated. This was performed separately for each pixel of the image. To evaluate the model's accuracy, two factors were used, namely the Pearson correlation and Normalized root mean square error. The results show that the proposed model can accurately estimate the brain's response to conceptual categories To decode the brain’ activity based on the observed semantic category in each test observations by using of the model, we calculated the distance between the recorded signal and the model responses to all stimuli from different categories and assigned the category of the nearest model response to brain's response in that trial. To this end, a K-nearest neighbors classifier based on Euclidean distance was used. This leaded to a classification accuracy which was significantly higher than chance level. So, the proposed model can be used to decode the activity of the brain in response to the visual stimuli.

Keywords

Brain modeling Gabor wavelets neural network K-nearest neighbors classifier intracranial field potential

Get full access to this article

View all access options for this article.

References

Baldassi

, Alemi-Neissi

, Pagan

, DiCarlo

J.J.

, Zecchina

and Zoccolan

, Shape similarity, better than semantic membership, accounts for the structure of visual object representations in a population of monkey inferotemporal neurons, PLoS Comput Biol 9 (2013), e1003167. doi:10.1371/journal.pcbi.1003167.

DiCarlo

J.J.

, Zoccolan

and Rust

N.C.

, How does the brain solve visual object recognition?, Neuron 73 (2012), 415–434. doi:10.1016/j.neuron.2012.01.010.

Fukushima

, Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position, Biological Cybernetics 36 (1980), 193–202. doi:10.1007/BF00344251.

Kay

K.N.

, Naselaris

, Prenger

R.J.

and Gallant

J.L.

, Identifying natural images from human brain activity, Nature 452 (2008), 352–355. doi:10.1038/nature06713.

Kourtzi

and Connor

C.E.

, Neural representations for object perception: Structure, category, and adaptive coding, Annual Review of Neuroscience 34 (2011), 45–67. doi:10.1146/annurev-neuro-060909-153218.

Liu

, Agam

, Madsen

J.R.

and Kreiman

, Timing, timing, timing: Fast decoding of object information from intracranial field potentials in human visual cortex, Neuron 62 (2009), 281–290. doi:10.1016/j.neuron.2009.02.025.

Lowe

D.G.

, Object recognition from local scale-invariant features, in: Computer Vision, 1999. The Proceedings of the Seventh IEEE International Conference on, Vol. 2, IEEE, 1999, pp. 1150–1157.

McKee

J.L.

, Riesenhuber

, Miller

E.K.

and Freedman

D.J.

, Task dependence of visual and category representations in prefrontal and inferior temporal cortices, Journal of Neuroscience 34 (2014), 16065–16075. doi:10.1523/JNEUROSCI.1660-14.2014.

Pinto

, Doukhan

, DiCarlo

J.J.

and Cox

D.D.

, A high-throughput screening approach to discovering good forms of biologically inspired visual representation, PLoS Comput Biol 5 (2009), e1000579. doi:10.1371/journal.pcbi.1000579.

10.

Riesenhuber

and Poggio

, Hierarchical models of object recognition in cortex, Nature Neuroscience 2 (1999), 1019–1025. doi:10.1038/14819.

11.

Seger

C.A.

and Miller

E.K.

, Category learning in the brain, Annual Review of Neuroscience 33 (2010), 203. doi:10.1146/annurev.neuro.051508.135546.

12.

Seyedhosseini

, Shushruth

, Davis

, Ichida

J.M.

, House

P.A.

, Greger

, Angelucci

and Tasdizen

, Informative features of local field potential signals in primary visual cortex during natural image stimulation, Journal of Neurophysiology 113 (2015), 1520–1532. doi:10.1152/jn.00278.2014.

13.

Ungerleider

L.G.

and Haxby

J.V.

, ‘What’ and ‘where’ in the human brain, Current Opinion in Neurobiology 4 (1994), 157–165. doi:10.1016/0959-4388(94)90066-3.

14.

Ward

A.A.

and Thomas

L.B.

, The electrical activity of single units in the cerebral cortex of man, Electroencephalography and Clinical Neurophysiology 7 (1955), 135–136. doi:10.1016/0013-4694(55)90067-5.

15.

Yamins

D.L.

, Hong

, Cadieu

C.F.

, Solomon

E.A.

, Seibert

and DiCarlo

J.J.

, Performance-optimized hierarchical models predict neural responses in higher visual cortex, Proceedings of the National Academy of Sciences (USA) 111 (2014), 8619–8624. doi:10.1073/pnas.1403112111.