Computations in neural circuits are inherently flexible, allowing humans and animals to respond to sensory stimuli with actions that are appropriate in a given context. Fundamental to this flexibility is the ability to integrate only context-relevant sensory information while ignoring irrelevant, distracting information. We studied the neural mechanisms underlying such context-dependent gating in monkeys performing two different sensory discriminations on the same set of visual stimuli. A contextual cue instructed the monkeys to report either the direction of motion or the color of a noisy visual stimulus with a saccade to one of two targets. During this task, we recorded neural responses from the frontal eye fields (FEF). We found that the gating of relevant sensory signals, and their integration towards a choice, can be understood as two aspects of a single dynamical process reflected in FEF population responses. Using linear regression, we identified a multitude of task-related signals represented simultaneously in the responses of FEF neurons, including the direction of motion and the color of the stimulus, the context, and the developing choice. While these different signals are mixed at the level of single neurons, they can be separated at the level of the population by projecting the neuronal activity onto the corresponding regression vectors. To understand better the nature of the mixed signals in FEF, we trained a recurrent network model to integrate only one of two noisy input streams. We found that the network created two context-dependent, approximate line attractors to integrate the relevant sensory inputs. Surprisingly, both the relevant and irrelevant inputs drive the network activity along directions in state space that are almost orthogonal to the direction of integration. This model reproduces the observed dynamic representation of the task-related signals in FEF and reveals a previously unknown mechanism of gating and integration.