Building artificial agents that can mimic human learning and reasoning has been a longstanding objective in artificial intelligence. I will discuss some of the empirical data and computational models from neuroscience and cognitive science that could help us advance towards this goal. Specifically, I will talk about the importance of structured representations of knowledge, particularly about mental or cognitive maps for time, space, and concepts. I will present data from recent behavioral and neural studies, which suggest that the brain maintains a scale-invariant mental timeline of the past and uses it to construct a compressed mental timeline of the future. From the computational perspective, these findings illustrate how associative learning can play a role in building structured representations of knowledge. Finally, I will discuss possible strategies to incorporate these findings into building artificial agents, especially in memory-augmented and attention-based neural networks.