Cortical connections consist of feedforward, feedback and lateral pathways. Infragranular layers project down the cortical hierarchy to both supra- and infragranular layers at the previous processing level, while the neurons in supragranular layers are linked by extensive long-range lateral projections that cross multiple cortical columns. However, most functional models of visual cortex only account for feedforward connections. Additionally, most models of visual cortex fail to account both for the thalamic projections to non-striate areas and the reciprocal connections from extrastriate areas back to the thalamus. In this talk, I will describe how a modified Locally Competitive Algorithm (LCA; Rozell et al, Neural Comp, 2008) can be used as a unifying framework for exploring the role of top-down and lateral cortical pathways within the context of deep, sparse, generative models. I will also describe an open source software tool called PetaVision that can be used to implement and execute hierarchical LCA-based models on multi-core, multi-node computer platforms without requiring specific knowledge of parallel-programming constructs.