"worthless person" (especially a young hoodlum or petty criminal), 1917, probably from punk kid "criminal's apprentice," U.S. underworld slang attested by 1904 (with overtones of "catamite"). Ultimately from punk (adj.) "inferior, bad" (q.v.), or else from punk "prostitute, harlot, strumpet," attested by 1590s, of unknown origin. Related: Punkling. For the possible sense shift from "harlot" to "homosexual," compare the possibility in gay.
By 1923 used generally for "young boy, inexperienced person" (originally in show business, as in punk day, circus slang from 1930, "day when children are admitted free"). The verb meaning "to back out of" is by 1920.
The "young criminal" sense no doubt is the inspiration in punk rock — loud, fast, aggressive, and outrageous — which is attested by 1971 (in a Dave Marsh article in Creem, referring to Rudi "Question Mark" Martinez); widely popularized in 1976.
If you looked different, people tried to intimidate you all the time. It was the same kind of crap you had to put up with as a hippie, when people started growing long hair. Only now it was the guys with the long hair yelling at you. You think they would have learned something. I had this extreme parrot red hair and I got hassled so much I carried a sign that said "FUCK YOU ASSHOLE." I got so tired of yelling it, I would just hold up the sign. [Bobby Startup, Philadelphia punk DJ, Philadelphia Weekly, Oct. 10, 2001]