Be careful. Dictionary.com has no entry for bludger. The info you are quoting is for bludge.
The OED has bludger, meaning a bludgeoner and more specifically a pimp, from 1856. The sense of a parasite or loafer is from 1939. This latter sense is Australian/New Zealand.
The OED has the verb bludge from 1919 in the sense of a acting the part of a parasite or shirker. The first cite is from Australian army slang. The sense of acting as a pimp is cited from 1937, but could be older albeit unattested. The noun sense meaning an easy task is attested to from 1945, again Australian.
Makes you wonder if J.K. Rowling was aware of all the negative connotations when she used the word.