When the sense is a love affair or a sexual relationship it’s predominately preceded by an indefinite or definite article.
Pretty much all nouns are predominantly preceded by an indefinite or definite article. That doesn’t tell you anything. The affairs of state/an affair of state, for example, or Iran-Contra was an affair involving Oliver North and Fawn Hall.
The the use of the verb to have that, absent any other marker, carries the implication of romance or sex. If you said Oliver North and Fawn Hall were having an affair, you would not be referring to Iran-Contra. (To the best of my knowledge, Oliver North and Fawn Hall did not have a romantic or sexual relationship.)