Breast cancer is classified into multiple distinct histologic types, and many of the rarer types have limited characterization. Here, we extend The Cancer Genome Atlas Breast Cancer (TCGA-BRCA) dataset with additional histologic type annotations, in a total of 1063 breast cancers. We analyze this extended dataset to define transcriptomic and genomic profiles of six rare special histologic types: cribriform, micropapillary, mucinous, papillary, metaplastic, and invasive carcinoma with medullary pattern. We show the broader applicability of our constructed special histologic type gene signatures in the TCGA Pan-Cancer Atlas dataset with a predictive model that detects mucinous histologic type across cancers of other organ systems. Using a normal mammary cell differentiation score analysis, we order histologic types into a continuum from stem cell-like to luminal progenitor-like to mature luminal-like. Finally, we classify TCGA-BRCA into 12 consensus groups based on integrated genomic and histological features. We present a rich openly accessible resource of histologic and genomic characterization of TCGA-BRCA to enable studies of the range of breast cancers.