do you want hotlinking to not work (as in people from external sites cannot directly use your URL in their <img> tags) or do u want hotlinking to work (people can link directly to your URL).
when hotlinking is on, people will be able to use the url.
You should block IP addresses instead, or find alternate ways to weed out the leechers.
Relying on the refer is not foolproof and it can be faked too... FYI the referer is set by the user agent (browser)