In this paper, I investigate conceptual categories derived from developmental processing in a deep neural network. The similarity matrices of deep representation at each layer of neural network are computed and compared with their raw representation. While the clusters generated by raw representation stand at the basic level of abstraction, conceptual categories obtained from deep representation shows a bottom-up transition procedure. Results demonstrate a developmental course of learning from specific to general level of abstraction through learned layers of representations in a deep belief network.
DilkinaK.Lambon RalphM. A. (2012) Conceptual Structure within and between Modalities. Frontiers in Human Neuroscience6: 333.http://doi.org/10.3389/fnhum.2012.00333.
4.
FukushimaK. (1980) Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics36: 193–202.
5.
GüçlüU.van GervenM. A. J. (2015) Deep neural networks reveal a gradient in the complexity of neural representations across the ventral stream. Journal of Neuroscience35: 10005–10014.
6.
HintonG. E. (2012) A practical guide to training restricted boltzmann machines. In: Neural networks: Tricks of the trade, Berlin Heidelberg: Springer, pp. 599–619.
HintonG. E.SalakhutdinovR. R. (2006) Reducing the dimensionality of data with neural networks. Science (New York, N.Y.)313: 504–7. http://doi.org/10.1126/science.1127647.
9.
HubelD. H.WieselT. N. (1962) Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. Journal of Physiology160: 106.
10.
KrizhevskyA.SutskeverI.HintonG. E. (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems. 1097–1105.
11.
KrugerN.JanssenP.KalkanS.LappeM.LeonardisA.PiaterJ.WiskottL. (2013) Deep hierarchies in the primate visual cortex: What can we learn for computer vision?IEEE Transactions on Pattern Analysis and Machine Intelligence35: 1847–1871.
12.
LalP.KingS. (2013) Cross-lingual automatic speech recognition using tandem features. IEEE Transactions on Audio, Speech, and Language Processing21: 2506–2515.
13.
LeH.-S.OparinI.AllauzenA.GauvainJ.YvonF. (2013) Structured output layer neural network language models for speech recognition. IEEE Transactions on Audio, Speech, and Language Processing21: 197–206.
14.
LewkowiczD. J.GhazanfarA. A. (2009) The emergence of multisensory systems through perceptual narrowing. Trends in Cognitive Sciences13: 470–478.
15.
MandlerJ. M.McDonoughL. (1998) On developing a knowledge base in infancy. Developmental Psychology34: 1274.
16.
MarrD. (1982) Vision: A computational investigation into the human representation and processing of visual informationVol. 2, New York, NY: Henry Holt and Co. Inc.
17.
McRaeK.CreeG. S.SeidenbergM. S.McNorganC. (2005) Semantic feature production norms for a large set of living and nonliving things. Behavior Research Methods37: 547–559.
18.
PauenS. (2002) Evidence for knowledge – Based category discrimination in infancy. Child Development73: 1016–1033.
19.
SadeghiZ.McClellandJ. L.HoffmanP. (2014) You shall know an object by the company it keeps: An investigation of semantic representations derived from object co-occurrence in visual scenes. Neuropsychologia2012: 1–10. http://doi.org/10.1016/j.neuropsychologia.2014.08.031.
20.
Sadeghi, Z., Araabi, B. N., & Ahmadabadi, M. N. (2015). A computational approach towards visual object recognition at taxonomic levels of concepts. Computational intelligence and neuroscience, 2015, 72.
21.
SaxeA. M.McClellandJ. L.GanguliS. (2013) Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. arXiv Preprint arXiv. 1312.6120.
22.
SerreT.WolfL.BileschiS.RiesenhuberM.PoggioT. (2007) Robust object recognition with cortex-like mechanisms. IEEE Transactions on Pattern Analysis and Machine Intelligence29: 411–426.
23.
StoianovI.ZorziM. (2012) Emergence of a “visual number sense” in hierarchical generative models. Nature Neuroscience15: 194–196. http://doi.org/10.1038/nn.2996.
24.
TestolinA.StoianovI.De GraziaM. D. F.ZorziM. (2013) Deep unsupervised learning on a desktop PC: a primer for cognitive scientists. Frontiers in Psychology4.
25.
TorralbaA.FergusR.WeissY. (2008) Small codes and large image databases for recognition. In: IEEE conference on computer vision and pattern recognition, 2008. CVPR 2008. 1–8.
26.
ZorziM.TestolinA.StoianovI. (2013) Modeling language and cognition with deep unsupervised learning: A tutorial overview. Frontiers in Psychology4: 515, http://doi.org/10.3389/fpsyg.2013.00515.