Frey was born in Arlesheim, Switzerland, as the son of Emil Remigius Frey. His father was a liberal separatist politician.
Frey's family provided refuge for Friedrich Hecker when he fled the repression following the revolution in Germany in 1848. Frey later emigrated to the USA arriving in Belleville, Illinois an area with many Forty-Eighters, veterans of the 1848 revolutions in Europe. For a while he worked for Hecker, but they had a falling out.
Career during the American Civil War
Frey entered the unionist24th Illinois Volunteer Infantry Regiment as a private. Hecker was his commander, and they became friends again, with Frey sharing a tent with Hecker's son. Frey was later promoted to First lieutenant but resigned on 17 June 1862. He began raising a company for the Second Hecker Regiment (The 82nd Illinois Volunteer Infantry Regiment) and became the company's captain. Captain Frey was taken prisoner at the battle of Gettysburg on 1 July 1863 and held in Libby Prison for eighteen months before being exchanged for Captain Gordon, a Confederate prisoner under sentence of death.
He became a court pianist in Bucharest after 1907. In 1908 he and Xaver Scharwenka gave a private performance on two pianos of Scharwenka's Piano Concerto No. 4 in F minor to its dedicatee Queen Elisabeth of Romania. The next day it was performed publicly with orchestra; the composer conducted and Frey was the soloist.
George Enescu dedicated his Piano Sonata No. 1 in F-sharp minor, Op. 24/1 to Emil Frey.
In 1910 Frey entered the composition section of the Anton Rubinstein Competition in St Petersburg, and won with his Piano Trio. This led to an engagement as Professor of the Virtuoso Class at the Moscow Conservatory 1912-17.