The design of complete imaging systems using a joint design framework have led to significant achievements in terms of reduced system size and cost or enriched imaging features. At Ricoh Innovations we are designing the optical elements and digital processing of an imaging system through optimization that takes the optics, sensor characteristics, and image processing capabilities into account. In this talk we introduce multi-aperture systems that can be designed and optimized for a specific task, e.g. depth estimation for machine or human vision or spectral analysis. Optical design challenges as well as signal processing problems that occur when simulating and processing sensor measurements obtained from unconventional optical architectures will be discussed.