ORCID
https://orcid.org/0000-0002-2741-169X
Date of Award
9-13-2023
Degree Name
Doctor of Philosophy (PhD)
Degree Type
Dissertation
Abstract
The natural world features high-dimensional retinal inputs to the visual system. In contrast, to study vision in the lab, only sparsely sampled image sets are used. Given this immense natural image manifold, what is a principled way to sample and understand the neural representations on it? In this thesis, we frame the neural coding question in a geometric way. Neural tuning can be conceptualized as a landscape on the natural image manifold, and generative models such as GANs provide a concrete instantiation of the manifolds. We hypothesized that the maximally activating images or peaks on these landscapes are critical for understanding visual neurons. Thus, we used the neuron-guided image synthesis paradigm to find these peaks, where an evolutionary algorithm iteratively optimized the images to increase the firing rate of a target neuron, gradually reaching a peak on the tuning landscape. We first characterized the Riemannian geometry of the GAN image manifolds and leveraged the geometry to develop a better evolutionary optimizer to control the neurons. We applied the closed-loop paradigm to neurons recorded in V1, V4, and posterior inferotemporal cortex (pIT) in two monkeys. Along the ventral stream, we found a few consistent trends. Going up the hierarchy, the tuning peaks became sharper, and they took more optimization iterations to find. By constraining the optimization in linear subspaces, we found the tuning peaks became higher dimensional. Further, we compared the image optimization in multiple generative spaces: a pattern-based generator, and an object-based generator. We found that going up the hierarchy, it became increasingly easy to guide optimization on the object manifold and increasingly hard on the pattern manifold. Further, on both manifolds, the optimized images for IT neurons have higher objectness scores than V4. Thus, the tuning peaks of higher visual neurons (pIT) were located closer to the object manifold, and their tuning functions were more aligned with object-based parametrization. In the future, we’d like to consider how these landscapes are combined to construct population representations, and how these maximally activating stimuli are critical to driving downstream behaviors.
Language
English (en)
Chair and Committee
Carlos Ponce
Committee Members
Timothy Holy
Recommended Citation
Wang, Binxu, "Charting the Landscape of Ventral Stream Neural Code on Generative Image Manifolds" (2023). Arts & Sciences Electronic Theses and Dissertations. 3178.
https://openscholarship.wustl.edu/art_sci_etds/3178