A detailed understanding of the mechanisms that establish or maintain the latent reservoir of HIV will guide approaches to eliminate persistent infection. We used a cell line and primary cell models of HIV latency to investigate viral RNA (vRNA) expression and the role of the host transcriptome using single-cell approaches. Single-cell vRNA quantitation identified distinct populations of cells expressing various levels of vRNA, including completely silent populations. Strikingly, single-cell RNA-seq of latently infected primary cells demonstrated that HIV downregulation occurred in diverse transcriptomic environments but was significantly associated with expression of a specific set of cellular genes. In particular, latency was more frequent in cells expressing a transcriptional signature that included markers of naive and central memory T cells. These data reveal that expression of HIV proviruses within the latent reservoir are influenced by the host cell transcriptional program. Therapeutic modulation of these programs may reverse or enforce HIV latency.