Perceptual integration of luminance, texture and color cues for visual boundary segmentation - Project Summary
One of the most essential computations performed by the visual system is segmenting images
into regions corresponding to distinct surfaces. This in turn requires identifying the boundaries
separating image regions, a process known as boundary segmentation. Computational analyses
of natural images have revealed that many visual cues are available at region boundaries,
including differences in luminance, texture, and color. It is known that these cues combine for
tasks like edge localization and orientation discrimination. However, it remains unclear how these
various cues are weighted and combined for boundary segmentation.
In collaborative work with Canadian colleagues at McGill University in Montreal, we have
developed a novel machine learning framework for characterizing human performance on
boundary segmentation tasks using naturalistic micro-pattern stimuli. Our method makes use of
the Filter-Rectify-Filter (FRF) model often applied to characterizing texture boundary
segmentation. The major innovation of our approach is that we fit the FRF model directly to
thousands of psychophysical stimulus-response observations to estimate its major defining
parameters. We have recently applied this approach to investigating spatial strategies for contrast
boundary segmentation and comparing competing hypotheses of how contrast modulation is
integrated across orientation channels. In this grant, we propose to apply both classical
psychophysical techniques and our novel machine learning methodology to understanding the
computations employed to combine luminance, texture and color cues for segmentation.
In Aim 1, we focus on modeling segmentation of luminance-defined boundaries,
comparing the case where each surface has uniform luminance, giving rise to a sharp edge
(luminance step), to the more naturalistic case where the two surfaces have differing proportions
of dark and light micro-patterns on either side of the boundary with no sharp edge (luminance
texture). We will apply our machine learning methodology to test the hypothesis that different
neural mechanisms may be involved in segmenting these two different kinds of luminance
boundaries. In Aim 2, we ask how observers integrate first-order (luminance) and second-order
(texture) cues for boundary segmentation, and if there are differences in cue combination
strategies for luminance steps and luminance textures. We will also compare models embodying
competing hypotheses of the underlying neural mechanisms of cue combination. In Aim 3, we
extend the analyses in Aims 1 and 2 beyond simple luminance differences to include differences
in color. Finally, Aim 4 is a pedagogical aim of promoting undergraduate research.
