The usually accepted origin for the area of London called “Elephant and Castle” (actually just the immediate area of a crossroad and now a roundabout) is that the “castle” refers to the howdah, a seat/platform placed on elephants by hunters. The site of the inn/pub (still there) used to be a smithy that made cutlery, and the sign for the cutlers craft guild used an image of an Elephant and howdah from the use of ivory for cutlery handles.
See
http://www.worldwidewords.org
for more detail and a dismantling of the “infanta” idea.
