My sister ... is as white as a lilly, and as small as a wand. [Shakespeare, "Two Gentlemen of Verona," 1591]
Sense of "not large, of little size" developed in Old English With many extended senses, e.g. small fry, first recorded 1690s of little fish, 1885 of insignificant people. Small potatoes first attested 1940; small change "something of little value" is from 1902; small talk "chit-chat" (1751) first recorded in Chesterfield's "Letters." Small world as a comment upon an unexpected meeting of acquaintances is recorded from 1895. Small-town (adj.) "unsophisticated, provincial" is recorded from 1824. Small arms, indicating those capable of being carried in the hand (contrasted to ordnance) is recorded from 1710.