Born in New York City, Mazur attended the Bronx High School of Science and MIT, although he did not graduate from the latter on account of failing a then-present ROTC requirement. Regardless, he was accepted for graduate school and received his Ph.D. from Princeton University in 1959, becoming a Junior Fellow at Harvard from 1961 to 1964. He is the Gerhard Gade University Professor and a Senior Fellow at Harvard.
His observations in the 1960s on analogies between primes and knots were taken up by others in the 1990s giving rise to the field of arithmetic topology.