Overlooking the Ohio River at a location originally known as “Bald” Hill, the park derived its historic name from its Native American associations as it was thought to have been cleared of trees by Native people. The park was later named Frederick H. Alms Memorial Park, as a memorial to Mr. Alms by his wife, Eleanora. The parkland, situated on a hilltop like many of Cincinnati's unusual parks, was purchased by the Park Board with funds from the Alms estate which resulted in nearly 94 acres of open space.
In 1929, A. D. Taylor designed the park’s entrance ensemble framing the arrival of a Picturesque road flanked with symmetrical piers and stone walls. Taylor also designed the landscape plan for the park’s centerpiece, an Italianate pavilion, designed by architects Stanley Matthews and Charles Wilkins Short, Jr., atop the former Bald Hill.
Other park features include a comfort station designed by architect Carl Freund and built in 1936 with funds from the Works Progress Administration, and a memorial statue to Stephen Foster Collins created by Arturo Ivone in 1937 and donated by Josiah Kirby Lilly, an Indianapolis-based admirer of the composer.
The pavilion is one of three in the city’s park system, along with Ault and Mount Echo Parks, which also have landscape plans designed by A.D. Taylor.