Well, I had a couple of such situations
I usually tell the buyer that I'm living in another country/ on another continent and the call would be quite expensive. This is enough to continue the negotiation through email.
But, my last experience was very very strange and funny at the same time
I sent the emails to potential buyers regarding a great 2 word, exact match domain, City+Couriers.com, for only low XXX, and in a couple of hours got a reply with "Call me". OK, sent him the standard message and was waiting for his reply. But, no reply. After a couple of days, I sent another email asking if he is still interested in that domain. His answer amused me "No, because you didn't call me". And it was one of the biggest courier company in that city, while being at the end of the list on Google... I even sent him the link to Godaddy Auctions, where I've listed the domain, and explained him that the transaction will be automatically, and it's safe... Nothing new, "you didn't call me".
So, it all depends on the buyer. Some of them understand the simple things, some are just too paranoid
So, don't take it personal, just move forward.