The parish was founded in 1857 as an offshoot of the Church of St. Columba and Holy Cross Church, with boundaries from 28th to 38th Street and from 6th Avenue to the banks of the Hudson. The community initially met in a chapel structure of renovated townhouses.

The original church plan was begun in 1861 and completed in 1868, with a front on 31st Street, between 9th and 10th Avenues.  It was designed by the founding pastor, Arthur J. Donnelly (1820–1890), and architect T.S. Wall. The church complex also consisted of a rectory, convent and schools, the latter designed by architect Lawrence J. O'Connor, FAIA.

During 1874–1884, the complex was the original home of the Presentation Sisters in the northeastern United States, and enrollment grew to become the largest parochial schools in New York. The Sisters and the parish also founded the Mount Saint Michael Home for destitute children, in Greenridge, Staten Island. The Manhattan church contained an 1862 Henry Erben mechanical action organ.

On May 4, 1892, a fire destroyed much of the church and the organ.

A new structure incorporated the original tower and acquired a new Indiana bluff limestone facade on 32nd Street. The blessing Mass was Sunday, January 28, 1894.  The address was listed at 408 West 32nd Street.

In 1904, the Pennsylvania Railroad Company began proceedings for the construction of the North River Tunnels and Pennsylvania Station, which would require the demolition of St. Michael's 32nd Street church and complex. At the suggestion of the pastor, John A. Gleeson, the Archdiocese sold the parish properties in exchange for a new church, school, convent, and rectory on 34th Street. The office of Napoleon LeBrun & Sons was responsible for the construction of the new complex, which involved the salvage and re-use of the altar, organ, stained glass windows, and limestone facade. The church was dedicated November 10, 1907.