I could be wrong so happily stand corrected if I indeed would be wrong ... but I heard that some tools offer, for a fee, access to private WHOIS. So if that is true, indeed the WHOIS only protects you from the basic surfer consulting a WHOIS website.
I would however not worry too much because (from my experience having worked for several IT orientated companies) the average internet user wouldn´t know how domain registrations work, let alone what a WHOIS database is.
And if you really are extremely careful about privacy (which I understand, hence why I have private WHOIS) then I suggest to use a PO Box or an alternative address with or without private WHOIS. A family member´s address will do the trick for example, or your work address maybe?