Using functional magnetic resonance imaging (fMRI), we investigated prefrontal cortex (PFC) activity during remembering specific source information (format, location judgments) versus remembering that could be based on undifferentiated information, such as familiarity (old/new recognition [ON], recency judgments). A working memory (WM) paradigm with an immediate test yielded greater activation in the lateral PFC for format and location source memory (SM) tasks than ON recognition; this SM-related activity was left lateralized. The same regions of PFC were recruited in Experiment 2 when information was tested immediately and after a filled delay. Substituting recency for location judgments (Experiment 3) resulted in an overall shift in task context that produced greater right PFC activity associated with ON and recency tasks compared to the format task, in addition to left SM-related activity. These data extend to WM previous findings from long-term memory (LTM) indicating that the left and right PFC may be differentially involved in memory attributions depending on the specificity of information evaluated. The findings also provide evidence for the continuity of evaluative processes recruited in WM and LTM.