It became acceptable around a thousand years ago.
The original sense of the word is simply the human chest, applied to both men and women (and metaphorically to such things as the earth and the sea). The OE is bosm.
The OED doesn’t give a sense for the singular use referring specifically to a woman’s breast. The use in the plural (bosoms) to specifically refer to a woman’s breasts is relatively recent. The OED dates it to 1959, but the entry is an old one, so I don’t trust that date.