1640s, "that which borders on something else, the part abutting on or against," from abut (v.) + -ment. Originally any junction; the architectural usage, "solid structure where one arch of a bridge, etc., meets another" is attested from 1793 (the notion is of the meeting-place of the arches of a bridge, etc.).
Figurative sense "chief element of a system" is from 1640s. Pennsylvania was called the Keystone State because of its position (geographical and political) in the original American confederation, occupying the middle (7th) place in the "arch" of states along the Atlantic, between eastern states and southern ones. Keystone cops were the bumbling police in the slapstick silent movies produced by Keystone Studios, formed in 1912 in Edendale, Calif., by Canadian-born U.S. film director Mack Sennett (1884-1960).
This has Germanic cognates in Gothic arhvazna, Old English earh, Old Norse ör "arrow," from Proto-Germanic *arkw-o- "belonging to a bow." It also has cognates in Greek arkeuthos, Latvian ercis "juniper," Russian rakita, Czech rokyta, Serbo-Croatian rakita "brittle willow." De Vaan sees an Italo-Germanic word for "bow" which can be connected with Balto-Slavic and Greek words for "willow" and "juniper" "under the well-founded assumption that the flexible twigs of juniper or willow were used as bows." The Balto-Slavic and Greek forms point to *arku-; "as with many plant names, this is likely to be a non-IE loanword." Electrical sense is from 1821.