Psychophysical study on spatiotemporal information processing by human visual system

Shin'ya Nishida

It is believed that the visual system separately analyses different visual attributes such as form and motion, combining the results of each analysis to recover the coherent visual scene in the brain. Our understanding of this processing structure remains superficial, however, as long as we know little about how the brain represents space and time on which each visual attribute is located. The purpose of this project is to realise the cortical mechanisms of spatiotemporal processing that enables us to effectively extract useful information from dynamically changing retinal images, and to construct a stable visual world. Based on the psychophysical analyses of subjective perception and behavioural response to various visual stimuli, functional models of visual processing will be developed. A list of to be addressed questions includes: (A) How does the brain process form in motion? It should be clarified whether direction selective sensors contribute to the form perception [1]. (B) How does the brain integrate motion and form to generate a coherent spatiotemporal stream of visual experience? We have already shown that motion signal strongly modulates the perception of from [2]. (C) What is subjective time? We are proposing the time marker theory of subjective temporal judgment in order to account for the colour-motion asynchrony illusion (see figure) [3]. We are planning to test this theory in more general contexts, and to clarify the neural basis of time marker. We have also found (D) the gaze modulation of visual aftereffects that presumably relates to transformation of visual space from retinal to body-centred coordinates [4], and (E) an afterimage of filled-in surface that provides a profound insight into the cortical representation of space [5].

Fig.1 Stretch-induced remodeling

(A) Perceptual asynchrony of motion and colour [6]. When a green pattern moving upwards and a red pattern moving downwards alternates with the indicated time course, it is hard to judge which colour and direction are shown together. For motion and colour to be seen in phase, the direction change has to lead the colour change by about 100 ms for an inter-change interval of 250 ms. (B) Conventional account of the apparent motion delay in terms of perceptual processing lag. Motion takes longer to be consciously perceived than colour. (C) New account in terms of time markers [3]. Marker flags indicate transitions (first-order temporal changes) in the two sequences. The observers cannot perform the required task to match direction reversals with colour transitions due to difficulty in detecting the time of direction reversals (turning points / second-order temporal changes) at rapid alternations. This account is based on the finding that an apparent delay of changes in motion direction occurs only for rapid alternations, and this delay is not accompanied by a difference in reaction time. We also found that perceptual asynchrony depends on the temporal structure of the stimuli (transitions vs turning points], rather than the attribute type (colour vs. motion).

References
[1] S. Nishida, "Direction-selective mechanism mediates identification of spatial patterns moving behind narrow slits," to be presented at Vision ScienceS Society Meeting, Sarasota, 2002.
[2] S. Nishida and A. Johnston, "Influence of motion signals on the perceived position of spatial pattern," Nature, vol. 397, pp. 610-612, 1999.
[3] S. Nishida and A. Johnston, "Marker correspondence not processing latency determines temporal binding of visual attributes," Current Biology, vol. 12, 2002.
[4] S. Nishida, I. Motoyoshi, and S. Shimojo, "Gaze modulation of visual aftereffects," presented at Vision ScienceS Society Meeting, Sarasota (Florida), 2001.
[5] S. Shimojo, Y. Kamitani, and S. Nishida, "Afterimage of perceptually filled-in surface," Science, vol. 293, pp. 1677-80, 2001.
[6] K. Moutoussis and S. Zeki, "A direct demonstration of perceptual asynchrony in vision," Proc R Soc Lond B Biol Sci, vol. 264, pp. 393-9, 1997.