I will discuss some implications of an approach that attempts to describe the various non-linearities of neurons in the visual pathway using a geometric framework. This approach will be used to make a distinction between selectivity and hyper-selectivity. Selectivity will be defined in terms of the optimal stimulus of a neuron, while hyper-selectivity will be defined in terms of the falloff in response as one moves away from the optimal stimulus. With this distinction, I show that it is possible for a neuron to be very narrowly tuned (hyper-selective) to a broadband stimulus. We show that hyper-selectivity allows V1 neurons to break the Gabor-Heisenberg localization limit. The general approach will be used to contrast different theories of non-linear processing including sparse coding, gain control, and linear non-linear (LNL) models. Finally, I will show that the approach provides insights into the non-linearities found with overcomplete sparse codes ­ and argues that sparse coding provides the most parsimonious account of the common non-linearities found in the early visual system.