Wordorigins.org

This phrase, meaning incontrovertible evidence of guilt, is of relatively recent origin, dating to the Watergate era. From the New York Times of 14 July 1974:

The big question asked over the last few weeks in and around the House Judiciary Committee’s hearing room by committee members who are uncertain about how they felt about impeachment was, “Where’s the smoking gun?”

And there is this from the same paper on 21 July 1974:

Representative Robert F. Drinan, a Massachusetts Democrat who is among the president’s outspoken critics, said that the section of the summary that focused on alleged abuses of Presidential power contained “the smoking gun” tying Mr. Nixon directly to wrongdoing in the Ellsberg case.

First use of the phrase is widely, and falsely, credited to Republican congressman Barber Conable who on 6 August 1974 commented, as reported by the Van Nuys, California Valley News:

“I guess we have found the smoking gun, haven’t we?” asked Barber Conable (R-N.Y.) one of the most respected GOP members in the House and a constant Nixon supporter. He referred to stands taken by some Nixon supporters that they would not move against the president until they found him with the equivalent of “standing with the smoking gun in his hand.”

Conable was referring to a transcript of a 23 June 1972 conversation between Richard Nixon and his chief of staff, H.R. Haldeman, that made it clear that Nixon had approved the cover up of the Watergate break-in.  The transcript ran:

Haldeman: ...the FBI is not under control...and you think the thing to do is to get them, the FBI, to stop?
Nixon: Right, fine.

But Conable was not the first to use the phrase and not even the first to use it in the context of Watergate. Conable’s remark was in August and, as we can see from the New York Times citations above, the phrase was already in widespread use by the time of Conable’s remark. Conable simply repeated a term that had been on everyone’s lips for months and the reason his use received attention was that he had been an ardent supporter of the president. With supporters like Conable turning against it, the Nixon presidency was doomed.

It is somewhat surprising that the phrase is so recent, given that its imagery is so vivid and obvious. And indeed, the metaphor of a smoking pistol indicating guilt has been with us for much longer. Arthur Conan Doyle, in The Gloria Scott, a mystery that appears in the 1894 The Memoirs of Sherlock Holmes, used the phrase smoking pistol:

The chaplain stood with a smoking pistol in his hand.

Conan Doyle’s usage, however, was quite literal and not figurative. Also, it referred to a murder case while the current usage is usually found in a political context. Finally, there is no evidence to indicate that the phrase was used in the intervening seventy-five years.

(Sources: Oxford English Dictionary, 2nd Edition; Safire’s New Political Dictionary; Proquest Historical New York Times; Newspaperarchive.com)