From 1951 to 1955, Davenport was in the U.S. Navy, in the Mechanic Drum and Bugle Corps. In the mid-'50s, the jazz scene was on the decline and Davenport turned to blues gigs to pay the rent. In the late '50s, he played with Billy Boy Arnold, Dusty Brown, and Freddy King. In 1960 he played with the Ernie Fields Orchestra and joined Otis Rush in 1961, working with him for about a year. He also worked with harp player Little Mack Simmons, Syl Johnson, Junior Wells, Mighty Joe Young, James Cotton, Muddy Waters, and Howlin' Wolf.

He met Paul Butterfield at Pepper's Lounge in 1964 and sat in with them at their gig at Big Johns on Chicago's northside. When Butterfield's drummer Sam Lay became ill late in 1965, Butterfield called on Davenport to join the group and tour with them. Davenport's jazz background brought a different quality to the Butterfield group. It was at this time that Bloomfield was absorbing Eastern scales, and with the more sophisticated drumming of Davenport, they began to work on a tune they called "The Raga." This resulted in the tune "East/West," the extended tune that was to have such an impact on rock music history. Much of this success was due to Davenport's ability to add color and tone to the composition, something not generally found in straight-ahead blues drummers.

Davenport retired from music in 1968 due to illness, but was playing again from 1972-1974 with Jimmy Dawkins, Willie Dixon, and Buster Benton. He again retired from 1974 to 1981, after which he joined the Pete Baron Jazztet, with whom he still works. Davenport describes his own drumming as a combination of his two idols: the two base drums of Louis Bellson and the unique drum roll of Art Blakey.