Responses to immunotherapy are uncommon in estrogen receptor (ER)-positive breast cancer and to date, lack predictive markers. This randomized phase II study defines safety and response rate of epigenetic priming in ER-positive breast cancer patients treated with checkpoint inhibitors as primary endpoints. Secondary and exploratory endpoints included PD-L1 modulation and T-cell immune-signatures. 34 patients received vorinostat, tamoxifen and pembrolizumab with no excessive toxicity after progression on a median of five prior metastatic regimens. Objective response was 4% and clinical benefit rate (CR + PR + SD > 6 m) was 19%. T-cell exhaustion (CD8+ PD-1+/CTLA-4+) and treatment-induced depletion of regulatory T-cells (CD4+ Foxp3+/CTLA-4+) was seen in tumor or blood in 5/5 patients with clinical benefit, but only in one non-responder. Tumor lymphocyte infiltration was 0.17%. Only two non-responders had PD-L1 expression >1%. This data defines a novel immune signature in PD-L1-negative ER-positive breast cancer patients who are more likely to benefit from immune-checkpoint and histone deacetylase inhibition (NCT02395627).