NamePros
Welcome, Guest! Ready to make a name for yourself in the domain business? We welcome both the hobbyist and professional domainer to join the discussion as part of the NamePros community.

Click here to create your profile to start earning reputation for posting, and trader ratings for buying & selling in our free e-marketplace. Build your trader rating with each successful sale. Our system has tracked over 100,000 sales and counting!
FAQ & TOS Register Search Today's Posts Mark Forums Read

Go Back   NamePros.com > Website Development Discussion Forums > Programming
Reload this Page Question on Whois info extraction

Programming PHP, Perl, Ruby on Rails, AJAX, HTML, XHTML, CSS, JavaScript, MySQL and any other coding topics.

Advanced Search


Closed Thread
 
LinkBack Thread Tools
Old 03-09-2008, 03:28 PM THREAD STARTER               #1 (permalink)
NamePros Member
Join Date: Mar 2007
Posts: 130
STDomains is on a distinguished road
 



Question Question on Whois info extraction


Hi All,

Anyone have any ideas on programmable whois info extraction given a specific domain name?

What we are trying to accomplish, is to be able to extract whois info for a number of domains that we own and get it into a database.

We found a couple of good threads at NP (http://www.namepros.com/code/280961-...ity-check.html and one or 2 other threads linked from that one).. but not sure if the threads are doing exactly what we need.

Is there a "format" that all automated whois servers are supposed to conform to on their responses?

????: NamePros.com http://www.namepros.com/programming/442431-question-on-whois-info-extraction.html
Any thoughts/direction would be much appreciated!

Thanks,

ST Domains
__________________
Domains for Sale @ Sedo
STDomains is offline  
Old 03-09-2008, 05:37 PM   #2 (permalink)
NamePros Expert
 
Peter's Avatar
Join Date: Nov 2003
Location: Scotland
Posts: 5,069
Peter has a reputation beyond reputePeter has a reputation beyond reputePeter has a reputation beyond reputePeter has a reputation beyond reputePeter has a reputation beyond reputePeter has a reputation beyond reputePeter has a reputation beyond reputePeter has a reputation beyond reputePeter has a reputation beyond reputePeter has a reputation beyond reputePeter has a reputation beyond repute
 


Child Abuse Save The Children Save The Children Help The Homeless - Holiday 2009 Help The Homeless - Holiday 2009 Help The Homeless - Holiday 2009 Help The Homeless - Holiday 2009
No there is no set format returned from whois. Different registries return data slightly different.

What precise information are you after?

There should be code within the code section that does what you want as I know several people have written classes and functions for whois.
__________________
Manage your portfolio using my new Domain Portfolio Management script.
Securing Your Domain Name From Theft
Peter is offline  
Old 03-09-2008, 09:32 PM   #3 (permalink)
NamePros Regular
 
monaco's Avatar
Join Date: Jul 2005
Location: Tucson, AZ
Posts: 689
monaco will become famous soon enough
 



Certain items are pretty constant, like the Administrative Contact and Technical / Tech contact fields. Maybe some simplistic regex parsing could get the most vital information out of the whois data.
__________________
My Website | My Blog
monaco is offline  
Old 03-12-2008, 07:45 AM THREAD STARTER               #4 (permalink)
NamePros Member
Join Date: Mar 2007
Posts: 130
STDomains is on a distinguished road
 



Thanks for your responses Peter and Monaco.

Its unfortunate that there is no standard. Seems that this would be a great place for standardization!

Do most whois severs permit automated queries though ? Pretty sure the response to this will be that there is no standard (which seems to be the standard with whois servers ), but just asking for your thoughts based on your past experience ..

We are looking to extract reg date, expiry, contacts, name servers, and anything else we can "automate".
__________________
Domains for Sale @ Sedo
STDomains is offline  
Old 03-12-2008, 02:14 PM   #5 (permalink)
NamePros Expert
 
Peter's Avatar
Join Date: Nov 2003
Location: Scotland
Posts: 5,069
Peter has a reputation beyond reputePeter has a reputation beyond reputePeter has a reputation beyond reputePeter has a reputation beyond reputePeter has a reputation beyond reputePeter has a reputation beyond reputePeter has a reputation beyond reputePeter has a reputation beyond reputePeter has a reputation beyond reputePeter has a reputation beyond reputePeter has a reputation beyond repute
 


Child Abuse Save The Children Save The Children Help The Homeless - Holiday 2009 Help The Homeless - Holiday 2009 Help The Homeless - Holiday 2009 Help The Homeless - Holiday 2009
whois servers would find it hard to know if a request is an automated 1 or not. Unless of course you hammer the server. To combat this they only allow x amount of requests per hour or so.
__________________
Manage your portfolio using my new Domain Portfolio Management script.
Securing Your Domain Name From Theft
Peter is offline  
Old 03-15-2008, 09:44 PM THREAD STARTER               #6 (permalink)
NamePros Member
Join Date: Mar 2007
Posts: 130
STDomains is on a distinguished road
 



Makes sense.. Thx Peter.
__________________
Domains for Sale @ Sedo
STDomains is offline  
Old 03-15-2008, 09:48 PM   #7 (permalink)
I'll do it
 
-Nick-'s Avatar
Join Date: Dec 2005
Location: India
Posts: 6,927
-Nick- Has achieved greatness-Nick- Has achieved greatness-Nick- Has achieved greatness-Nick- Has achieved greatness-Nick- Has achieved greatness-Nick- Has achieved greatness-Nick- Has achieved greatness-Nick- Has achieved greatness-Nick- Has achieved greatness-Nick- Has achieved greatness-Nick- Has achieved greatness
 


Member of the Month
September 2007
Adoption
I think cURL would do the trick in PHP.

For a nice automation I would put the script as a function on some site of mine where there is little bit of traffic

So whenever someone visits my site the code part executes and it checks the availability.

If it is available then it goes to available.txt file and if not available then it dumps the stuff into the database

After all the data is extracted like this. You just need to find and replace junk stuff within the database like: Name etc.
-Nick- is offline  
Old 03-16-2008, 02:16 AM   #8 (permalink)
NamePros Regular
 
monaco's Avatar
Join Date: Jul 2005
Location: Tucson, AZ
Posts: 689
monaco will become famous soon enough
 



Originally Posted by -Nick-
So whenever someone visits my site the code part executes and it checks the availability.
I wonder if you could screw with the x-forwarded-for header to confuse the site and buy you more time before you got banned...maybe they'd think you were the endpoint for an entire org...
__________________
My Website | My Blog
monaco is offline  
Old 03-16-2008, 06:08 AM   #9 (permalink)
NamePros Expert
 
Peter's Avatar
Join Date: Nov 2003
Location: Scotland
Posts: 5,069
Peter has a reputation beyond reputePeter has a reputation beyond reputePeter has a reputation beyond reputePeter has a reputation beyond reputePeter has a reputation beyond reputePeter has a reputation beyond reputePeter has a reputation beyond reputePeter has a reputation beyond reputePeter has a reputation beyond reputePeter has a reputation beyond reputePeter has a reputation beyond repute
 


Child Abuse Save The Children Save The Children Help The Homeless - Holiday 2009 Help The Homeless - Holiday 2009 Help The Homeless - Holiday 2009 Help The Homeless - Holiday 2009
i doubt it. I should imagine they are actively looking for proxies as well.
__________________
Manage your portfolio using my new Domain Portfolio Management script.
Securing Your Domain Name From Theft
Peter is offline  
Old 03-16-2008, 07:02 AM   #10 (permalink)
I'll do it
 
-Nick-'s Avatar
Join Date: Dec 2005
Location: India
Posts: 6,927
-Nick- Has achieved greatness-Nick- Has achieved greatness-Nick- Has achieved greatness-Nick- Has achieved greatness-Nick- Has achieved greatness-Nick- Has achieved greatness-Nick- Has achieved greatness-Nick- Has achieved greatness-Nick- Has achieved greatness-Nick- Has achieved greatness-Nick- Has achieved greatness
 


Member of the Month
September 2007
Adoption
I am not doing it that it would get detected. I am more smarter then that.

Just trying to share some ideas.
-Nick- is offline  
Closed Thread


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools


Liquid Web Smart Servers  
All times are GMT -7. The time now is 08:17 AM.

Managed Web Hosting by Liquid Web
Domain name forum recommended by Domaining.com Powered by: vBulletin® Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.
Search Engine Friendly URLs by vBSEO 3.6.0 Ad Management plugin by RedTyger