"bivalve mollusk," c.1500, in clam-shell, originally Scottish, from Middle English clam "pincers, vice, clamp" (late 14c.), from Old English clamm "bond, fetter, grip, grasp," from Proto-Germanic *klam- "to press or squeeze together" (cf. Old High German klamma "cramp, fetter, constriction," German Klamm). If this is right then the original reference is to the shell.
"to dig for clams," 1630s, American English, from clam (n.). Clam up "be quiet" is 1916, American English, but clam was used in this sense as an interjection mid-14c.