Abstract
We describe a model of visual processing in which feedback connections from a higher- to a lower-order visual cortical area carry predictions of lower-level neural activities, whereas the feedforward connections carry the residual errors between the predictions and the actual lower-level activities. When exposed to natural images, a hierarchical network of model neurons implementing such a model developed simple-cell-like receptive fields. A subset of neurons responsible for carrying the residual errors showed endstopping and other extra-classical receptive-field effects. These results suggest that rather than being exclusively feedforward phenomena, nonclassical surround effects in the visual cortex may also result from cortico-cortical feedback as a consequence of the visual system using an efficient hierarchical strategy for encoding natural images.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 12 print issues and online access
$259.00 per year
only $21.58 per issue
Buy this article
- Purchase on SpringerLink
- Instant access to the full article PDF.
USD 39.95
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Top-down perceptual inference shaping the activity of early visual cortex
Efficient coding of natural images in the mouse visual cortex
The developmental emergence of reliable cortical representations
References
Hubel, D. H. & Wiesel, T. N. Receptive fields and functional architecture in two non-striate visual areas (18 and 19) of the cat. J. Neurophysiol. 28, 229–289 (1965).
Hubel, D. H. & Wiesel, T. N. Receptive fields and functional architecture of monkey striate cortex . J. Physiol. (Lond.) 195, 215–243 (1968).
Bolz, J. & Gilbert, C. D. Generation of end-inhibition in the visual cortex via interlaminar connections. Nature 320, 362–365 (1986).
Hubel, D. H. & Livingstone, M. S. Segregation of form, color, and stereopsis in primate area 18. J. Neurosci. 7, 3378–3415 (1987).
Desimone, R. & Schein, S. J. Visual properties of neurons in area V4 of the macaque: sensitivity to stimulus form. J. Neurophysiol. 57, 835–868 ( 1987).
Allman, J., Miezin, F. & McGuinness, E. Stimulus specific responses from beyond the classical receptive field: Neurophysiological mechanisms for local-global comparisons in visual neurons. Annu. Rev. Neurosci. 8, 407–429 (1985).
Dobbins, A., Zucker, S. W. & Cynader, M. S. Endstopped neurons in the visual cortex as a substrate for calculating curvature. Nature 329, 438 –441 (1987).
Bolz, J., Gilbert, C. D. & Wiesel, T. N. Pharmacological analysis of cortical circuitry. Trends Neurosci. 12, 292–296 (1989).
Peterhans, E. & von der Heydt, R. in Representations of Vision. Trends and Tacit Assumptions (eds Gorea, A., Frégnac, Y., Kapoulis, Z. & Findlay, J.) 111–124 (Cambridge Univ. Press, Cambridge, UK, 1991).
Grossberg, S., Mingolla, E. & Ross, W. D. Visual brain and visual perception: how does the cortex do perceptual grouping? Trends Neurosci. 20, 106–111 (1997).
Peterhans, E. & von der Heydt, R. Subjective contours—bridging the gap between psychophysics and physiology. Trends Neurosci. 14, 112–119 ( 1991).
Rao, R. P. N. & Ballard, D. H. Dynamic model of visual recognition predicts neural response properties in the visual cortex. Neural Comput. 9, 721–763 ( 1997).
Gallant, J. L., Connor, C. E. & Van Essen, D. C. Neural activity in areas V1, V2 and V4 during free viewing of natural scenes compared to controlled viewing. Neuroreport 9, 2153–2158 ( 1998).
Attneave, F. Some informational aspects of visual perception. Psychol. Rev. 61, 183–193 ( 1954).
MacKay, D. M. in Automata Studies (eds Shannon, C. E. & McCarthy, J.) 235– 251 (Princeton Univ. Press, Princeton, NJ, 1956).
Barlow, H. B. in Sensory Communication (ed. Rosenblith, W. A.) 217– 234 (MIT Press, Cambridge, MA, 1961).
Atick, J. J. Could information theory provide an ecological theory of sensory processing? Network Comput. Neural Sys. 3, 213– 251 (1992).
Buchsbaum, G. & Gottschalk, A. Trichromacy, opponent colours coding and optimum colour information transmission in the retina. Proc. R. Soc. Lond. B Biol. Sci. 220, 89– 113 (1983).
Srinivasan, M. V., Laughlin, S. B. & Dubs A. Predictive coding: A fresh view of inhibition in the retina. Proc. R. Soc. Lond. B Biol. Sci. 216, 427–459 (1982).
Dan, Y., Atick, J. J. & Reid, R. C. Efficient coding of natural scenes in the lateral geniculate nucleus: experimental test of a computational theory. J. Neurosci. 16, 3351–3362 (1996).
Dong, D. W. & Atick, J. J. Temporal decorrelation: a theory of lagged and nonlagged responses in the lateral geniculate nucleus. Network Comput. Neural Sys. 6, 159– 178 (1995).
Mumford, D. On the computational architecture of the neocortex. II. The role of cortico-cortical loops. Biol. Cybern. 66, 241– 251 (1992).
Pece, A. E. C. in Artificial Neural Networks 2 (eds Aleksander, I. & Taylor, J.) 865–868 (Elsevier, Amsterdam, 1992).
Softky, W. R. in Advances in Neural Information Processing Systems 8 (eds Touretzky, D., Mozer, M. & Hasselmo, M.) 809–815 (MIT Press, Cambridge, MA, 1996).
Ullman, S. in Large-Scale Neuronal Theories of the Brain (eds Koch, C. & Davis, J. L.) 257–270 (MIT Press, Cambridge, MA, 1994).
Olshausen, B. A. & Field, D. J. Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Res. 37, 3311–3325 ( 1997).
Dayan, P., Hinton, G.E., Neal, R.M. & Zemel, R.S. The Helmholtz machine. Neural Comput. 7, 889– 904, (1995).
Luettgen, M. R. & Willsky, A. S. Likelihood calculation for a class of multiscale stochastic models, with application to texture discrimination. IEEE Trans. Image Proc. 4, 194–207 (1995).
Felleman, D. J. & Van Essen, D. C. Distributed hierarchical processing in the primate cerebral cortex. Cereb. Cortex 1, 1–47 ( 1991).
Field, D. J. Relations between the statistics of natural images and the response properties of cortical cells. J. Opt. Soc. Am. A 4, 2379–2394 (1987).
Bell, A. J. & Sejnowski, T. J. The "independent components" of natural scenes are edge filters. Vision Res. 37, 3327–3338 (1997).
Olshausen, B. A. & Field, D. J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607– 609 (1996).
Zipser, K., Lamme, V. A. F. & Schiller, P. H. Contextual modulation in primary visual cortex. J. Neurosci. 16, 7376–7389 (1996).
Sandell, J. H. & Schiller, P. H. Effect of cooling area 18 on striate cortex cells in the squirrel monkey. J. Neurophysiol. 48, 38–48 (1982).
Knierim, J. & Van Essen, D. C. Neural responses to static texture patterns in area V1 of the alert macaque monkey. J. Neurophysiol. 67, 961–980 ( 1992).
Sillito, A. M., Grieve, K. L., Jones, H. E., Cudeiro, J. & Davis, J. Visual cortical mechanisms detecting focal orientation discontinuities. Nature 378, 492–496 (1995).
Hupé, J. M. et al. Cortical feedback improves discrimination between figure and background by V1, V2 and V3 neurons. Nature 394 , 784–787 (1998).
Murphy, P. C. & Sillito, A. M. Corticofugal feedback influences the generation of length tuning in the visual pathway. Nature 329, 727–729 (1987).
Gilbert, C. D. Adult cortical dynamics. Physiol. Rev. 78, 467–485 (1998).
Lee, D. D. & Seung, H. S. in Advances in Neural Information Processing Systems 9 (eds Mozer, M., Jordan, M. & Petsche, T.) 515–521 (MIT Press, Cambridge, MA, 1997).
Heeger, D. J., Simoncelli, E. P. & Movshon, J. A. Computational models of cortical visual processing. Proc. Natl. Acad. Sci. USA 93, 623– 627 (1996).
Miller, E. K., Li, L. & Desimone, R. A neural mechanism for working and recognition memory in inferior temporal cortex. Science 254, 1377–1379 (1991).
Gross, C. G. & Sergent, J. Face recognition. Curr. Opin. Neurobiol. 2, 156–161 (1992).
Logothetis, N. K. & Pauls, J. Psychophysical and physiological evidence for viewer-centered object representations in the primate. Cereb. Cortex 5, 270–288 (1995).
Poggio, T. & Edelman, S. A network that learns to recognize 3D objects. Nature 343, 263– 266 (1990).
Bell, C., Bodznick, D., Montgomery, J. & Bastian, J. The generation and subtraction of sensory expectations within cerebellum-like structures. Brain Behav. Evol. 50 Suppl. 1, 17–31 (1997).
Schultz, W. et al. in Models of Information Processing in the Basal Ganglia (eds Houk, J. C., Davis, J. L. & Beiser, D. G.) 233– 248 (MIT Press, Cambridge, MA, 1995).
Creutzfeldt, O. D. Generality of the functional structure of the neocortex. Naturwissenschaften 64, 507–517 ( 1977).
Rao, R. P. N. An optimal estimation approach to visual perception and learning. Vision Res. (in press).
Rissanen, J. Stochastic Complexity in Statistical Inquiry (World Scientific, Singapore, 1989).
Acknowledgements
We thank Christof Koch for comments on the manuscript and Mary Hayhoe, Terrence Sejnowski and members of the Computational Neurobiology Lab at the Salk Institute for discussions. This work was supported by research grants from the National Institute of Health (NIH), the National Science Foundation (NSF) and the Alfred P. Sloan Foundation.
Rights and permissions
About this article
Cite this article
Rao, R., Ballard, D. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat Neurosci 2, 79–87 (1999). https://doi.org/10.1038/4580
Received:
Accepted:
Issue date:
DOI: https://doi.org/10.1038/4580
Share this article
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative
