Analog compute-in-memory (CIM) architectures for low-power neural networks have recently been shown to achieve excellent compute efficiency and high accuracy, comparable to software-based deep neural networks. However, two primary limitations prevent them from reaching their potential: 1) resistive crossbars have difficulty scaling to large, sparse networks; and 2) analog device mismatch and variability necessitates frequent analog-to-digital conversion, which dramatically reduces power efficiency and speed. We describe two methods for solving these problems. The first solution allows for the creation of extremely large, sparse analog network layers using emerging 3D memory technology. The second enables fully analog training and inference of deep neural networks using the equilibrium propagation algorithm for gradient estimation.