1570s of armies; from 1882 in field sports; by 1905 in the political sense (compare left wing). Right-winger is attested by 1919 in U.S. politics; 1895 in sports.
1590s, in reference to the horizontal and vertical lines of soldiers marching in formation, from rank (n.) in the military sense of "number of soldiers drawn up in a line abreast" (1570s) + file (n.1). Thence generalized to "common soldiers" (1796) and "common people, general body" of any group (1860).
"one who leads a military patrol in formation in a jungle, etc.," 1944, said to be from point (n.) in military sense of "small leading party of an advance guard" (1580s) + man (n.). A more literal sense also is possible. Point (n.) in U.S. also meant "position at the front of a herd of cattle," and pointman in this sense is attested by 1903.