"pungent dried pepper," 1756, from Tupi (Brazil) kyynha "capsicum," a word mistakenly associated with the town of Cayenne in French Guyana. The town name is the French form of Guyana.

1680s, pimiento (modern form from 1718), "dried, aromatic berries of an evergreen tree native to the West Indies," cultivated mostly in Jamaica, from Spanish pimiento "pepper-plant, green or red pepper," also pimienta "black pepper," from Late Latin pigmenta, plural of pigmentum "vegetable juice," from Latin pigmentum "pigment" (see pigment (n.)). So called because it added a dash of color to food or drink.

[I]n med.L. spiced drink, hence spice, pepper (generally), Sp. pimiento, Fr. piment are applied to Cayenne or Guinea pepper, capsicum; in Eng. the name has passed to allspice or Jamaica pepper. [OED]

The tree itself so called by 1756. The piece of red sweet pepper stuffed in a pitted olive so called from 1918, earlier pimiento (1901), from Spanish. French piment is from Spanish.