Jones's father was a Southern Pacific railroad agent. Young Lindley got his nickname by being so thin that he was compared to a railroad spike. At the age of 11 he got his first set of drums. As a teenager he played in bands that he formed himself. A railroad restaurant chef taught him how to use pots and pans, forks, knives and spoons as musical instruments. He frequently played in theater pit orchestras. In the 1930s he joined the Victor Young orchestra and thereby got many offers to appear on radio shows, including Al Jolson's Lifebuoy Program, Burns and Allen, and Bing Crosby's Kraft Music Hall.
From 1937 to 1942, he was the percussionist for the John Scott Trotter Orchestra, which played on Bing Crosby's first recording of White Christmas. Spike Jones was part of a backing band for songwriter Cindy Walker during her early recording career with Decca and Standard Transcriptions. Her song "We're Gonna Stomp Them City Slickers Down" provided the inspiration for the name of Jones’ future band, the City Slickers.
The City Slickers evolved out of the Feather Merchants, a band led by vocalist-clarinetist Del Porter, who took a back seat to Jones during the embryonic years of the group. They made experimental records for the Cinematone Corporation and performed publicly in Los Angeles, gaining a small following. The original members included vocalist-violinist Carl Grayson, banjoist Perry Botkin, trombonist King Jackson and pianist Stan Wrightsman.
The band signed a recording contract with RCA Victor in 1941 and recorded extensively for the company until 1955.
After appearing as the house band on The Bob Burns Show, Spike got his own radio show on NBC, The Chase and Sanborn Program, as Edgar Bergen's summer replacement in 1945. Frances Langford was co-host and Groucho Marx was among the guests. The guest list for Jones' 1947-49 CBS program for Coca-Cola (originally The Spotlight Revue, retitled The Spike Jones Show for its final season) included Frankie Laine, Mel Torme, Peter Lorre, Don Ameche and Burl Ives. Frank Sinatra appeared on the show in October 1948, and Lassie in May 1949.
The last City Slickers record was the LP Dinner Music For People Who Aren't Very Hungry. The whole field of comedy records changed from musical satires to spoken-word comedy (Tom Lehrer, Bob Newhart, Mort Sahl, Stan Freberg). Spike Jones adapted to this, too; most of his later albums are spoken-word comedy, including the horror-genre sendup Spike Jones in Stereo (1959).
Jones was a lifelong smoker. He was once said to have gotten through the average workday on coffee and cigarettes. Smoking may have contributed to his developing emphysema. His already thin frame deteriorated, to the point where he used an oxygen tank offstage, and onstage he was confined to a seat behind his drum set. He died at the age of 53, and is buried in Holy Cross Cemetery, Culver City, California.