Calvin was born in St. Paul, Minnesota, the son of Elias Calvin and Rose Herwitz, Jewish immigrants from the Russian Empire. His father was born in Lithuania (then part of the Russian Empire) and his mother in Georgia (also part of the Russian Empire).