He was warden of a castle in a province under the control of Toyotomi Hideyoshi. Hideyoshi caused Kazunari to marry Oeyo, the sister of a favorite concubine. When Kazunari's support for Tokugawa Ieyasu became known, Hideyoshi took back the woman. She later married Ieyasu's son Tokugawa Hidetada and mothered the shogun Tokugawa Iemitsu.