Brain-optimized deep neural network models of human visual areas learn non-hierarchical representations

Deep neural networks (DNNs) optimized for visual tasks learn representations that align layer depth with the hierarchy of visual areas in the primate brain. This study investigated whether hierarchical representations are necessary to accurately predict brain activity in the primate visual system by optimizing DNNs to directly predict brain activity measured with fMRI in human visual areas V1–V4. A single-branch DNN and a multi-branch DNN were trained. While the multi-branch DNN could learn hierarchical representations, only the single-branch DNN did so. This demonstrates that hierarchical representations aren't essential for accurately predicting human brain activity in V1–V4, and that DNNs encoding brain-like visual representations can vary widely in architecture.

Ghislain St-Yves, Emily J. Allen, Yihan Wu, Kendrick Kay, Thomas Naselaris