Project Summary
How is the representation of complex visual objects organized in inferotemporal (IT) cortex, the large brain
region responsible for object recognition? To date, areas selective for a few categories such as faces, bodies, and
scenes have been found, but the vast majority of IT cortex is “wild,” lacking any known specialization. Various
schemes have been proposed for parceling IT, but a comprehensive understanding of IT organization remains
elusive. Here, we propose to use fMRI, microstimulation, and electrophysiology to develop a unified
understanding of the organization and coding principles of macaque IT. The experiments are
motivated by a major advance in computer vision and two key preliminary results from our lab. First, the advent
of deep networks trained for object classification makes it possible to generate a parametric object
space, providing a quantitative framework to decipher the feature selectivity of single IT cells. Second, our
preliminary results suggest that a large portion of macaque IT cortex is topographically organized
according to the first two principal components of object space. This topography encompasses at least
four distinct networks, each with at least three hierarchical nodes of increasing view invariance, and includes the
previously described face and body patch networks. Furthermore, single cells within each network are projecting
incoming objects, formatted as vectors in the object space, onto specific preferred axes. Taken together, these
results suggest a new hypothesis for IT organization: IT cortex is tiled by networks (i.e., sets of
functionally connected nodes, where a node is a patch of IT cells) whose organization and coding principles are
very similar to that of the face patch network, and the layout of these networks follows a regular
topography specified by the statistical structure of object space.
We propose three Specific Aims to rigorously test this hypothesis. In Aim 1, we will systematically map all
networks within IT of individual animals. In Aim 2, we will record responses of cells in each identified network
to a large, common set of object stimuli and determine their coding scheme. In Aim 3, we will perturb activity in
each network and quantitatively assess effect on object recognition behavior. Together, these three Aims seek to
build a comprehensive understanding of IT organization that bridges fMRI, single units, and behavior.
Our lab has developed powerful experimental techniques to tackle each of these Aims and has previously applied
them to the macaque face patch system. We believe the time is ripe to apply these techniques to the
larger problem of how all objects are represented--not just faces. In the same way that Mendeleev’s
arrangement of chemical elements according to their atomic mass and chemical properties helped elucidate the
electronic structure of atoms, we believe systematic mapping and characterization of all networks in
IT will help elucidate the fundamental neural mechanism for object recognition.