Elliot's father, who worked for the U.S. Fish and Wildlife Service, was skeptical of his son's career choice. Unfortunately Sam was unable to prove himself to his father, who died of a heart attack when Sam was only 18 years old. "He died thinking, 'Man, this kid is going to go down the wrong path,'" the younger Elliot said in an interview. "And I think on some levels that was either hard on me or made me more focused in my resolve to have a career." He credited his father with many of the traits that made him famous. When speaking of his father and his father's friends, "they were all men's men and outdoorsmen", Elliot reminisces, and from watching them he gained his signature personality. He also claims to have Western genes. "My family had a relative that fought at the Alamo", he says. "I've got history in Texas." He tells Parade Magazine "I'm a sixth-generation Texan, even though I was born in California."

In 1969 he gained his first television credit in Judd for the Defense as Dan Kenyon in the episode "The Crystal Maze." Later that same year he appeared in the show Lancer in an episode called "Death Bait." Also starring in that episode was Tom Selleck, his future co-star and friend. The duo would team up in 1979 as brothers in Louis L'amour's The Sacketts and again in 1982 in another Louis L'Amour adaptation The Shadow Riders.

Elliot continued on to be in several popular films and TV roles. One of his first was in Butch Cassidy and the Sundance Kid in 1969, followed by the TV series Mission Impossible from 1970-1971 where he starred for 13 episodes. Ironically, In 1981 he played the role of a wife killer in Murder in Texas alongside his future wife Katherine Ross. They co-starred together later in 1991 in Conagher, an adaptation of the Louis L'Amour novel Conagher. He also made apperances in popular shows such as Felony Squad, Gunsmoke, and Hawaii Five-O. 

Elliot had numerous roles that honored the military. He portrayed Sgt. Maj. Basil L. Plumley in the historical account We Were Soldiers Once... And Young. He also portrayed John Buford in the 1993 film Gettysburg. However, not all of his roles were so honorable or historically accurate. In 1998 he played The Stranger in the comedy The Big Lebowski, and General Thunderbolt Ross in the 2003 film Hulk. He played Carter Slade in the 2007 comic book adaptation of Ghost Rider and appeared in The Golden Compass that same year. He has also appeared three times on the television comedy Parks and Recreation as Ron Dunn, opposite Ron Swanson.