In his book Springs and Wells of Manhattan and the Bronx (New-York Historical Society, 1938; written in 1916), James Ruel Smith documented several springs in the area near Broadway and 125th St which were still flowing at the time-- between 1890 and 1915. The Tiemann drinking fountain, Smith says, "was one stop that nearly every driver of the old Broadway stages and trolleys might be counted upon to make several times during the day..." (p. 59). The source was a large spring; a Mr. Daniel Tiemann, 73rd Mayor of NYC, had built a color works/dye factory on the site in the mid-19th century, and the water from this spring was used in the factory. The nearby "Indian Spring" was noted for its quantity and purity of water, though it was not built into a drinking fountain but remained in a bed of rocks, covered with a wooden grating. As of 1897, Smith said that the spring "is used only for drinking purposes, and the requirements of the rough shanty accomodations for several truckman's horses that are grouped about it a few yards to the south." As of 1898, the Board of Health condemned the spring as too polluted to be potable (p. 55).
Some of Smith's plates below, with their captions giving locations:
Below are images of the same area from Viele's water map.
Though it's hard to be sure which original watercourse from pre-urban times became a specific spring seen in the 19th or 20th century, I believe that the small pond outlined below, on 124th St just west of Amsterdam Ave, is the site of Ruel Smith's Indian Spring.
Prentis Hall, a Columbia University building on the south side of 125th St just west of Broadway, has seen the modern incarnation of one of these springs flowing through its basement for many years. The story i've heard is that after the building was constructed, the basement kept flooding, until eventually a tunnel was dug in a basement wall and the water now flows into a channel in the basement and out through a pipe that connects it with a sewer along 125th. The photo below is this basement river in 2006. (photo by steve duncan).