A Journey into Domain Availability

[Voodoomoodoo]

I'm the developer of www.domcomp.com, a domain pice comparison site. Over christmas, I set out to add a huge new feature: domain availability checking.

Figuring out whether www.xxxx.com is available seems like such a trivial task but it actually took a huge amount of work and proved very interesting. Perhaps I was a little naive in this area but I actually learnt a lot. For instance, to get the final result, I used a combination of zone files, WhoIs checking, DNS checking, each with their own caveats and quirks.

I think the end result is pretty neat: typing a name instantly tells you whether that name is taken, across multiple tlds which is pretty cool. Check it out if you have time, I can't think of a more relevant audience that NamePros users

If you have questions on how it's implemented then happy to discuss, or if you have tips on how to improve then I'm all ears!

I PMed a mod before posting here to ask if this was too spammy/self-promotion-like but I haven't got a reply. However, if this is not allowed then please delete! I believe the new feature would be very useful to NamePros though.

 

[Name Trader]

Wow! Information overload. Good job. Well done.

 

[NameOmnia]

Thank you for sharing

 

[Nattydomain]

nice job...bookmarked

 

[ThemeRevolution]

Nice! So with that combination, what do you use for the first attempt? checking the WHOIS record? if so then how do you deal with the whois server access limitation? I believe that DNS lookup is would be your alternate method as it's not 100% accurate, for example a pending deletion domain will not shows any DNS records.

And how do you manage the data? storing into database or just simply cache the result? if it stored in database how do you update them? setting a cron jobs to check them periodically?

 

[Domainace]

Very useful! Especially for new TLDS, since their prices vary all over the place.

 

[LoneDeranger]

Well done ! very smooth experience, I like it !

do you tend to do a mobile version of it ?

 

[Voodoomoodoo]

Thanks for the feedback and response so far!

Nice! So with that combination, what do you use for the first attempt?
And how do you manage the data?

The actual implementation is fairly messy but in general we first check against zone files which is fairly authoritative. If the zone file is not available or files are old/suspicious then we move onto DNS records. You're right that DNS records are not always accurate and as such for each name you look up, we check actually cross check against multiple DNS hosts, behind the scenes. WHOIS is another form of cross-checking for results that we can't determine or not confident with for the results above. However, these are usually very slow and we may use this method in the background after getting a rough result from the previous methods.

The result's aren't persisted, all done in real-time we do some form of very short-lived caching here and there.

do you tend to do a mobile version of it ?

We designed the site to be responsive, that is, it should look good and adapt to different screensizes. You can try it on mobile/tablets too

 

[hookbox]

I tried about 10 different words and found that it's not accurate at all. Many words show as available when they are not.

I do love the format and would definitely use it if it was accurate.

 

[infosec3]

Nice tool. Thanks for letting us know.

 

[Asfas1000]

Nice job !

 

[Voodoomoodoo]

I tried about 10 different words and found that it's not accurate.

I do love the format and would definitely use it if it was accurate.

Hey, could you let me know which words you tried and which TLDs were incorrect? the tool is certainly not perfect and we're continuously looking to improve it. There are some domains that are very hard to get right - for example if the zone file is just not updated, if WHOIS record is incorrect etc etc

These should be rare cases, but if you let us know which ones are wrong then we can make improvements. Cheers!

 

[hookbox]

Hey, could you let me know which words you tried and which TLDs were incorrect? the tool is certainly not perfect and we're continuously looking to improve it. There are some domains that are very hard to get right - for example if the zone file is just not updated, if WHOIS record is incorrect etc etc

These should be rare cases, but if you let us know which ones are wrong then we can make improvements. Cheers!

I chose words that should be taken to better test the system. Words like Best, The, Buy, Free etc…..Most of these are not available but show available in your system.

I used Hexonet.net to check what is really available. I like to use them for checking on names because they show every new GTLD all very clear on one page. I type in the name and then choose "New GTLD" to see all that are available.

 

[Name Trader]

I think Hexonet Whois, whilst impressive, shows less than half the new gTLDs

 

[hookbox]

I think Hexonet Whois, whilst impressive, shows less than half the new gTLDs

I think you are right. Do you know another site that shows the entire list all on one page? I am not big on GTLD's but I like to look from time to time.

 

[vivaldi]

Hey, could you let me know which words you tried and which TLDs were incorrect? the tool is certainly not perfect and we're continuously looking to improve it. There are some domains that are very hard to get right - for example if the zone file is just not updated, if WHOIS record is incorrect etc etc

These should be rare cases, but if you let us know which ones are wrong then we can make improvements. Cheers!

Would be awesome if it actually worked, try a word like sex

 

[Name Trader]

I think you are right. Do you know another site that shows the entire list all on one page? I am not big on GTLD's but I like to look from time to time.

No. I'm looking for a good one also. And like you, I'm not big on these new gTLD's. I actually liked how Hexonet worked. It was the first time I'd seen it.

 

[hookbox]

No. I'm looking for a good one also. And like you, I'm not big on these new gTLD's. I actually liked how Hexonet worked. It was the first time I'd seen it.

I'm not sure how many GTLD's are live but Hexonet has 274 listed. Hexonet shows more domains on their "New GTLD list". On the homepage when you type in a name it then takes you to the available domains page and under "Domain Name Search Results" it gives you the option to choose "New TLDs". Here you will find all 274 domains to choose from.

I like it because it's fast. Godaddy, Name, Enom etc..are slow motion. It literally takes 1 second for the options to pop up on Hexonet. Enom and Name you have to continue to scroll for at least a minute to see all the results. Godaddy takes forever because they include Premium dot com listings in their results and that takes forever. Plus Godaddy's list is to confusing. Dynadot barely shows any names. Namebright doesn't show any. Namecheap didn't pop anything up.

I am checking them as a write and I just found that 101Domain is really good. Here is how you find the options. On the homepage under International Domain Registration click on New gTLDs which is underneath the search bar. Click on Available (General Availablity) then type whatever word you want and the whole list pops up. I didn't count them but it looks like it's a pretty complete list.

 

[equity78]

Yes @hookbox I was just going to say 101Domain.com I use them when I want to see everything fast.