We don't? Have you ever checked this on
dofo.com?
Yes. I have and you don't have that kind of data. What you do seem to have is an IP based web check to see if a domain name has a website. This is very different to what the "survey" in the OP was trying and failing to measure. It was attempting to measure how websites were being used rather than if they had a website and it failed badly.
Just because a domain name has an IP for a website does not mean that it has a website or even a webserver running on that IP or IPs (some of the larger hosters use load balancing with multiple IP addresses for the same webserver). With .COM, approximately 90% of domain names will provide an IP or IPs for a website. However, this does not mean that these IPs have active websites. Some of this is due to incorrect or non-routable IP addresses being returned. Others simply don't have any webserver running on the IP address. So between 5% and 10% could be knocked off that 90% website percentage. The .COM is perhaps one of the best in terms of website usage. The other gTLDs can have website percentages below 90% and some of them are down around 80% or so. This is because the registrants of brand protection registrations (where they own the .COM and the .gTLD version of the domain names) may not bother setting up the .gTLD in the zonefile for the domain name. The .BIZ and .INFO are good examples of this as they have flatlined in terms of new registrations in US/EU/AU markets. I see this kind of activity with the monthly website IP surveys that I run for the gTLDs and some ccTLDs in order to produce reports on how TLDs are being used in various countries.
Measuring web usage and development is a highly complex buisness when it comes to gTLDs. This is because gTLDs are rarely a single market with a single language. For a large gTLD like .COM which is effectively active in almost every country in the world, that means dealing with a lot of languages. Then there is the issue of redirects.
The incompetent "survey" of gTLDs included in a recent ICANN report on the new gTLDs used a cargo-cult methodology that stupidly categorised redirects as "parking". Approximately 6% of .COM websites will redirect to the HTTPS version of the website so in that crude methodology, those sites were "parked". The reality is that most of them were active. Wordpress and Joomla use internal redirects. As ccTLDs become more important, it is not unusual to find a .COM redirecting to the .ccTLD site. Depending on the TLD as much as 20% can be redirects.
Parked and holding sites are also difficult to measure beyond the templated holding or PPC pages. People have a wonderful way of inventing their own HTML and phrasing for "coming soon". In the Chinese market, domain names often go straight on to Gambling or Adult affiliate pages as an alternative to PPC parking. Adult affiliate sites are far more common than stand-alone porn sites especially in the Chinese market. For example, .LOAN has about 1.3 million affiliate sites alone. There's probably less than 2K developed sites in that gTLD. There are some new gTLDs that have higher rates of development and, strange as it may seem, .XYZ is one of the better ones.
The Chinese market also has high levels of scraped content being used to generate fake websites complete with PPC and affiliate adverts. The software used for this is quite sophisticated in that it actively pulls content from the original sites. To people unfamiliar with this technique these sites may appear to be genuine sites.
Then there are Clone sites. This is where a site doesn't really exist in a TLD but is actually a website from another TLD. This kind of thing happens when someone points a domain name to a website (
www .domain1. example to
www .domain2. example2) without setting up the webserver to properly redirect. The single site will be served for all domain names pointing to that server.
Single page websites are no longer an indication of parking. There are some very sophisticated single page sites which are company introductions or even an entire company's website. There are also landing pages used to gain e-mails or contacts for businesses.
There was an other ICANN report (the MEAC DNS study (185 pages - page 93 is the start of the section where it fails to measure usage)
https://www.icann.org/en/system/files/files/meac-dns-study-26feb16-en.pdf )) that tried to measure usage with what it thought that search engine developers used to categorise usage. Unfortunately, it did not understand how search engine developers categorise usage and had common holding pages (Plesk control panels and PPC advertising) as "active" websites. These were not active websites This is a common problem when people who only have web development skills (rather than search engine development expertise and web usage measurement expertise) try to measure usage. Building search engines and search engine indexes is a lot harder than it seems and there are multiple algorithms involved.
Measuring usage and development in a TLD checks every domain name in a TLD (or a statistically significant sample of domain names that represents the TLD) but it provides cumulative percentages for the TLD. It is a very complex process. There are approximately 138 million domain names in .COM TLD. The sample in the OP was not representative.
Regards...jmcc
