It’s from jazz, originally expressing the emotional quality of the music, gradually extended into other genres of music and aspects of Black life. First cite in the OED:
1946 Ebony Sept. 34/2 He uses a bewildering, unorthodox technique and his playing is full of what jazzmen refer to as ‘soul’.
First cite for soul food:
1960 Los Angeles Sentinel 29 Dec. a15 Claims he can’t eat the ‘soul’ food like Ham and Grits.
Green’s also has this sense from the West Indies, a form of address/term of endearment. It is a separate development but may have had an influence on the the above:
1916 [WI] J.G. Cruickshank Black Talk 74: Soley/soulie – a term of endearment [...] ‘Soley! I gone.’.
1959 [US] P. Marshall Brown Girl, Brownstones (1960) 32: Soul, I don know.
1970 [WI] F. Collymore Notes for Gloss. of Barbadian Dial. 106: Soul, soulie. [...] a nominative of address, and implying sympathy and/or familiarity, as What happen to you, soul? Also soulie gal.