Jonh Borin
Established Member
- Impact
- 7
hello guys Are there a tool that gives bulk nameserver to previously ended domains
Might be one soon, once Dotible is fully up (see my sig). But it won't have the ns of domains deleted BEFORE its launch.hello guys Are there a tool that gives bulk nameserver to previously ended domains
I want to know the domains that were on brandbucket and squadhelp, And her companions didn't renew her, and she fell. Now it is available again.Might be one soon, once Dotible is fully up (see my sig). But it won't have the ns of domains deleted BEFORE its launch.
What do you want to accomplish? Just asking so if I see how I can help out.
Good one, thumbs up for the idea.I want to know the domains that were on brandbucket and squadhelp, And her companions didn't renew her, and she fell. Now it is available again.
I want to know the domains that were on brandbucket and squadhelp, And her companions didn't renew her, and she fell. Now it is available again.
Sure thing! Please hit me with a DM in a couple weeks if I don't say anything.I see this happen all the time with Afternic listed domains but rarely SH (although I have caught a few). Never checked for BB domains.
I suspect there's just a lot less SH/BB names in general, as compared to Afternic.
Would definitely like to see this as a tool, @twiki!
@Jonh Borin , @okaydomains , additional question.
What is the status of the domain you search? I guess you want a specific status. Expired, pending delete, or fully deleted only?
This raises quite some special requirements so I need to know.
It might require a massive Whois live poll to determine which are still available or not... some of them will be re-taken meantime... etc. That is quite a challenge so I will be looking at ways to optimize the tasks.
Edit: And I can offer live bulk Whois for certain TLDs only, .com and .net so far. Not so much for others. So it all depends on what data to pull exactly. Might have to mix in some other data sources for it.
I'm interested in domains that are in redemption or pending delete. I guess I'm interested in fully deleted but I assume they're crap if still available for hand reg so I don't spend a lot of time on those domains myself.
I'm willing to buy from expired domain auctions.
Its a hard problem because you can't just check the nameservers. They're often replaced with the registrars' nameservers during the redemption period, and even if not plenty of people use SH's nameservers for their private label names (which I do not care about).
I have this worked out for Afternic. Its a decent signal of good domains but its not overwhelmingly good. There's a lot of crap on Afternic, even if its highly priced. I'd be more interested in SH/BB accepted domains because of the human element.
What about TLD's? I can primarily do this for .com and .net at this stage. Not so much for other tld's.
deleted only@Jonh Borin , @okaydomains , additional question.
What is the status of the domain you search? I guess you want a specific status. Expired, pending delete, or fully deleted only?
This raises quite some special requirements so I need to know.
It might require a massive Whois live poll to determine which are still available or not... some of them will be re-taken meantime... etc. That is quite a challenge so I will be looking at ways to optimize the tasks.
Edit: And I can offer live bulk Whois for certain TLDs only, .com and .net so far. Not so much for others. So it all depends on what data to pull exactly. Might have to mix in some other data sources for it.
Might be able to do this only for .coms for now. It's half of the market anyway...
Important though there is no guarantee a domain has been approved or just pointed via nameservers but never made it into SH.
A bit late to the thread but The whole bulk WHOIS model is in trouble and has been since the introduction of GDRP in May 2018. This means that a lot of gTLD registrars have now started to redact data for registrants in the EU and elsewhere. ICANN's approach to the matter has been less than optimal and it has caused serious damage to the security of the Web.The point is, I just realized, this whole thing will have to be a byproduct of another product - bulk Whois data. You cannot know what a domain status is without Whois bigdata. If this one ever gets a chance.
That is a BIG business.
Here's an example: https://www.whoisxmlapi.com/whois-database-download.php
They used to have some prices (in many hundreds or thousands and up). Now, it's just request a quote. No wonder.
So I need to build this first before doing the other (byproduct). Side note I know this well, been working on it for a while (few years) as well as other tech.
But once that is built, unsure if I (or anyone else) might be interested in doing that domain research tool anymore. It's simply... pennies vs the real money and our lifetime is limited.
Anyway, just thoughts.
Well you took that a bit too literally, and I haven't been precise. I meant domain data, including Whois data, not only Whois data.A bit late to the thread but The whole bulk WHOIS model is in trouble and has been since the introduction of GDRP in May 2018. This means that a lot of gTLD registrars have now started to redact data for registrants in the EU and elsewhere. ICANN's approach to the matter has been less than optimal and it has caused serious damage to the security of the Web.
There's also an "aging data" problem with scraped WHOIS data. The .COM gets over 20 million new registrations a year and that's an awful lot of new domain names with partial data. Domain names drop and are renewed. That's a worse problem because the number of points that can be compared is reduced to country of registrant, nameservers, and registrar. Silent transfers (where a domain name is sold and transferred) within large registrars and countries can be almost invisible unless the data from web surveys is correlated.
As a category, For Sale domain names (based on Web Usage surveys) tend to have a high non-renewal rate outside the premiums.
Regards...jmcc
It is based on years of dealing with ICANN and with WHOIS data. The WHOIS service for gTLDs is being replaced by RDAP. ( https://www.icann.org/rdap )Well you took that a bit too literally, and I haven't been precise. I meant domain data, including Whois data, not only Whois data.
The WHOIS data for a significant percentage of .COM domains has changed since 2018. I think that some of the WHOIS providers just do checks on new and tracked domain names now and use the last modified date for full updates. All that will change when RDAP is fully implemented. Some of the new gTLDs are already using it.You are right about GDPR though, and most whois is years' stale anyway not because of GDPR, but because of due to rate limits it takes years for a large Whois provider to rescan the entire web whois - just to make an update. Also, nameservers are not PII so they don't fall under GDPR.
If a user requests data on a loadbalancer IP then the number of records returned could be in the millions. It might be a good thing to allow users to limit results with a keyword to stop them going over their quota.The free version is limited to 100 max domains and up to 10k count depth, but a pro user will have access to any amount based on their quota. Also access to search with many subfilters to narrow to a given set of domains as needed via their user panel.
Better be careful what you wish for as RDAP is gated and more controlled than WHOIS. There is also talk of people being charged for access and tiered access.Sure, can't wait for RDAP. I'm sick and tired working with Whois already as each TLD is a separate nightmare and they tend to change things, so overnight now your whois isn't working anymore.
It is going away and there is a trend to limit the amount of information available. ICANN has already announced the end of WHOIS. The registrars are transitioning to RDAP and some have already listed their RDAP services.So Whois is still here to stay, with its limitations of course.
Some registrars have been transferring expired domain names to auction sites rather than moving them to a parking/expired page. (The WHOIS data can also change from the registrant's to that of the registrar.) This happens before the Pending Delete stage. Depending on the registrar, it may appear that the domain name has shifted from SH to an auction site. If a domain name does not sell then it will often go through the typical deletion process.A SH activated domain would have nameservers activated anyway, which means it's captured via resolution and cached in our archive. And once a domain has expired, it will either point to registrar parking page OR drop the resolution completely. Both will be detected by our bot network and flagged as a change to look at. Edit: A Whois call is needed for detecting neither.
It is a valuable source of data.Whois is just a bonus and actually more needed for other applications. (Edit: but useful to the end user via our live whois calls, to see when domain they want to snap in the end will drop; but they can check that on their registrar's whois page as well.)
For some sales/auction sites it will be a problem but maybe not for this particular case.Things like Cloudflare are a problem, but for this particular case, there is no problem actually.
Dan also has verification nameservers that may needed to be added. With Godaddy's takeover of Dan and Undeveloped, there may be some IP changes on the way.These domains will have the common identifier = the SH or similar platform's nameservers. Same as everything put on sale via afternic will use nsX.afternic.com or nsY.dan.com or whatnot.
Approximately 9% of .COM is on sale at the moment. It is a lot of domain names and it is best to get the scalability aspect of development built in at the start.So again you're pointing at issues, but for other things really.