An action may lead to either a reward or a punishment. Therefore, an appropriate action needs to be chosen on the basis of the values of both expected rewards and expected punishments. To understand the underlying neural mechanisms, we conditioned monkeys using a Pavlovian procedure with two distinct contexts: one in which rewards were available and another in which punishments were feared. We found that the population of lateral habenula neurons was most strongly excited by a conditioned stimulus associated with the most unpleasant event in each context: the absence of the reward or the presence of the punishment. The population of lateral habenula neurons was also excited by the punishment itself and inhibited by the reward itself, especially when they were less predictable. These results suggest that the lateral habenula has the potential to adaptively control both reward-seeking and punishment-avoidance behaviors, presumably through its projections to dopaminergic and serotonergic systems.