Kelso was born in Fayetteville, Tennessee, on July 11, 1933. He attended public school and the University of the South in Sewanee, Tennessee, prior to entering the United States Naval Academy in 1952.

Following graduation in 1956, Kelso served on the cargo ship USS Oglethorpe before attending Submarine School in 1958.

On completion of training, Kelso was assigned to the submarine USS Sabalo before returning to Submarine School for nuclear power training in January 1960. He then served one year in the Nuclear Power Department at the school. Subsequent tours included the pre-commissioning crew of USS Pollack, Engineering Officer aboard USS Daniel Webster and Executive Officer of USS Sculpin.

From January 1969 to August 1971, Kelso served as Commanding Officer, Naval Nuclear Power School in United States Naval Training Center Bainbridge, Port Deposit, Maryland. Following tours included Commanding Officer, USS Finback; Staff of Commander, Submarine Force, United States Atlantic Fleet; and Commanding Officer, USS Bluefish. Kelso was then assigned as Executive Assistant to the Commander in Chief, United States Atlantic Command and Atlantic Fleet and Supreme Allied Commander Atlantic from September 1975 to July 1977.

Kelso served as Commander, Submarine Squadron 7 until reporting as Division Director, Submarine Distribution Division in the Naval Military Personnel Command, and Section Head of the Submarine Programs Section in the Office of the Deputy Chief of Naval Operations (Manpower, Personnel and Training) in September 1978. He was selected for promotion to the rank of rear admiral in February 1980.

Upon selection for flag rank, Kelso served as Director, Strategic Submarine Division, Office of the Chief of Naval Operations, and then was assigned as Director, Office of Program Appraisal, Office of the Secretary of the Navy. On February 8, 1985, Kelso became Commander Sixth Fleet and NATO Commander Naval Striking Force and Support Forces Southern Europe. During this tour, forces under his command launched raids on Libya in defiance of Colonel/President Muammar Gaddafi's claim that Libya's territorial waters extended 200 miles into the Gulf of Sidra. On June 30, 1986, Kelso was promoted to admiral and assumed the duties of Commander in Chief, United States Atlantic Fleet. Kelso became Supreme Allied Commander Atlantic and Commander in Chief, United States Atlantic Command on November 22, 1988. In that capacity, his forces were involved in the second Gulf of Sidra incident (1989). He succeeded Admiral Carlisle A.H. Trost to become the Navy's 24th Chief of Naval Operations (CNO) on June 29, 1990.

Shortly before his retirement, Senator Barbara Boxer attempted to punish Kelso by recommending a reduction in rank from full admiral to rear admiral (upper half). Under Congressional law, all military promotions for flag officers to have three or four stars are at the behest of the Senate, and said promotions can be revoked, as was the case of Pacific Fleet commander Husband Kimmel following the Pearl Harbor attack, who was demoted from a 4-star admiral to a 2-star, and Richard Dunleavy, Kelso's Assistant Chief of Naval Operations for Air Warfare, who was demoted from 3-star to 2-star as a result of Tailhook. Boxer claimed Kelso was deserving of punishment on the grounds that as the Navy's top officer he bore ultimate responsibility for what happened at Tailhook. Boxer's attempt failed when more Senators agreed Kelso had taken the correct and proper actions in handling the affair, and he was allowed to retire at full rank. Kelso's supporters praised his overhaul of officer training that eliminated the separate Aviation Officer Candidate School at NAS Pensacola, Florida for non-United States Naval Academy and non-Naval Reserve Officers Training Corps college graduates that had traditionally set many naval aviators and naval flight officers apart from their other officer peers, and for tough new policies on sexual harassment. Had Kelso been demoted, it would have been a significant loss in his military pension.

Kelso retired with his wife, Landess McCown Kelso (who died in 2012), to his place of birth in Fayetteville, Tennessee in 2003. He died from complications of a fall and severe head injury on June 23, 2013, in Norfolk, Virginia, where he had gone to attend his grandson's graduation. He had been married to his second wife, Georgia Robinson, for just two weeks. He was also survived by two sons (both of whom served in the Navy) and two daughters.