Thesis Project Form

Recently, a novel approach to stereo correspondence problem has been proposed, which combines retina-like vision sensors with brain-inspired spiking neural processors to build sophisticated real-time event-based visual processing systems. The network operates using exclusively precisely-timed temporal contrast events. Conversely, the vast majority of computational neuroscience models of stereopsis are based on mean firing rates of spatial contrast and do not rely on the precise timing of spikes.

The objective of the thesis is to functionally combine the spike-timing model with the rate-based energy model. Higher order disparity detectors could integrate transient responses of energy neurons rather than events from neuromorphic vision sensors. In this case an address-event would not represent a spike of a retinal ganglion cell, but the output of a cortical simple cell, and the model could combine the best features of both approaches. The mixed spike-timing and mean-rate model will be based on V1-like binocular energy neuron tuning functions that predict the neurons’ firing rates in response to a given stimulus in their receptive fields. Such tuning functions will incorporate spatial selectivity mechanisms for orientation, frequency, and phase or position disparity. In addition, the model will make explicit use of the precise spike-timing of the neurons in the stereo correspondence process to detect correct correspondences while suppressing disparity responses to false targets.

The student will learn to employ different methodologies and instrumentation, including:
- Modeling of spiking neural networks using the Python Brian neural network simulator
- Acquisition of signals from a silicon retina producing both image frames and streams of spikes in response to temporal contrast changes (DAVIS silicon retina sensor)
- Programming convolutional networks on GPUs