Working memory is a limited-capacity form of human memory that actively holds information in mind. Which memories ought to be maintained? We approach this question by showing an equivalence between active maintenance in working memory and a Markov decision process in which, at each moment, a cognitive control mechanism selects a memory as the target of maintenance. The challenge of remembering is then finding a maintenance policy well-suited to the task at hand. We compute the optimal policy under various conditions and define plausible cognitive mechanisms that can approximate these optimal policies. Framing the problem of maintenance in this way makes it possible to capture in a single model many of the essential behavioral phenomena of memory maintenance, including directed forgetting and self-directed remembering. Finally, we consider the case of imperfect metamemory — where the current state of memory is only partially observable — and show that the fidelity of metamemory determines the effectiveness of maintenance.