Horticulturist John Jay Smith and three partners founded the cemetery in 1836, purchasing 32 acres on the eastern banks of the Schuylkill River. They hired landscape gardener John Notman to design the cemetery on twenty acres of elevated, rolling terrain. Notman's Picturesque plan centered on a sixteen-foot wide elliptical drive which parallels the ridge, offering panoramic views of the river. Meandering gravel footpaths radiate from the drive in a geometrical pattern and converge in a gardenesque formal space called the Shrubbery. At the entrance, Notman designed a stone Doric gatehouse reminiscent of a Roman triumphal arch. Smith provided the original evergreen planting scheme, now replaced with mature deciduous trees.
Notman’s section was expanded on ten acres to the north, then followed in 1849 by the non-contiguous, 27-acre southern section, which was planned by civil engineer James C. Sidney and architect James P. W. Neff as a simpler reproduction of Notman’s scheme. The cemetery's central section, the 21-acre Fairy Hill, was acquired in 1861, bringing the total area to 78 acres. Its more formal design consists of a grid of equidistant paths punctuated by rond-ponts and encircled by a meandering loop drive. The cemetery's granite block walls and cast-iron fencing were erected between 1875 and 1900. Monuments range in styles from Roman, Egyptian, Greek and Gothic revivals through Victorian to L’Art Nouveau.
As the second oldest rural cemetery in the nation and Notman’s first known commission, Laurel Hill Cemetery was designated a National Historic Landmark in 1998.