In over 200 end-user sales I have overseen, in which I have always pushed before requesting payment, I have never once been robbed. I've written my thoughts on this subject too many times to count.
In my response to end-users' "how much?" e-mails, in which I quote my price tag, I also say that if they accept my offer I will transfer them the domain and expect to receive payment within 24 hours following transfer. This push-before-transfer obligement is a selling point that helps the end-user feel more comfortable moving forward with the transaction, hence they are much less likely to hesitate in deciding whether to accept your offer, and less likely to retract their acceptance upon consulting with their colleagues for advice.
So, here's how I see it: say 1 in every 100 end-users you sell to ends up "robbing" you (an over-estimate from my experience), but 5 in every 100 end-users you sell to would only agree to purchase your domain if you pushed first (a very realistic estimate from my experience).
Assuming your "agreed upon" (i.e. you and your buyer arrive at a price consensus, but no transaction has necessarily occurred yet) end-user sell-through rate is 5% across all your domain names, you own 10,000 domain names, and your average end-user sale agreement is for $400. That makes $400 x 500 = $200,000 in agreed upon end-user sales.
Based on the above parameters, if say you will not push without receiving payment first, 5% x 500 = 25 of those end-users will "disappear" and not complete the sale. You hence earn $190,000 instead of the agreed upon $200,000.
If you agree to push before receiving payment, 1% x 500 = 5 of those end-users will rob you. Assuming you fail to get those domains back, you earn $198,000 instead of the agreed upon $200,000.
So if you push before payment, you are $8,000 wealthier.
Using escrow prolongs the period between agreement and closing and hence extends the window during which the end-user could change his mind about whether to purchase your domain in consulting friends/colleagues for advice. In fact, the very decision to use escrow instead of straight-out pushing makes the end-user very slightly more likely to change his mind because he might not have heard of escrow before and might not feel like researching the subject. That or he might might make make a fuss about having to pay escrow fees which you did not account for in your price quote. I know this assertion sounds silly to some, but trust me, end-user domain sales are an extremely delicate animal because of how negatively the world at large perceives domainers. You should absolutely minimize the window between agreement and closing to the best of your ability.
That being said, it is almost certainly worth your while to use escrow for transactions of $1K+ and above and transactions with individuals from non English-speaking countries, particularly from the Middle East and southeast Asia due to high incidence of credit card fraud. I'll say it again: it's critical to call upon Escrow to cover your a** when engaging in high-volume or high-risk transactions. But for exchanges which fall within that $300-$500 quick-flip-to-enduser "sweet spot", I am absolutely convinced you'll lose money overall by not pushing first, or even by agreeing to use escrow.
Legal note: I take no responsibility, financial or otherwise, for any perceived monetary losses resulting from my advice.