NameSilo

Is There Top Level Domain Bias in Search Engine Indexing and Ranking?

Labeled as SEO in Domain Industry News started by bhartzer, Jan 20, 2020.

Replies:
37
Views:
2,096

  1. bhartzer

    bhartzer VIP Member Epik.com Staff VIP ★★★★★★★★★★

    Posts:
    81
    Likes Received:
    135
    Ever since Google's Exact Match Domain (EMD) update back in September 2012, they've always said publicly that all TLDs are created equal: all of them have an equal chance to rank in their search results. I've been skeptical--especially when the New gTLDs were launched.

    So, I thought I'd test it, using several ways to look at the results: SEO Testing software, manual searches at Google, and Google Search Console data.

    I chose 15 TLDs, including New gTLDs, .COM, .NET, .ORG, and a few ccTLDs as well:
    • .ICU
    • .TOP
    • .XYZ
    • .SITE
    • .CLUB
    • .ONLINE
    • .VIP
    • .COM
    • .NET
    • .ORG
    • .DE
    • .CO.UK
    • .CO
    • .LONDON
    • .IN
    The results were interesting, and I learned a lot about how Google handles indexing and ranking of brand new sites. For example, .DE and .CO domains are both considered to be in German and in Spanish (.DE in German and .CO in Spanish), despite absolutely no indication anywhere what language the site is in. No German or Spanish content and no hreflang tags on the sites.

    Also, the .XYZ site got indexed much quicker than any of the other TLDs. Some were really difficult to get their pages indexed.

    Each site was a unique 25 page site, 375 pages total. I chose a unique word (namescon spelled backwards) to register each domain (i.e., nocseman.icu, nocseman.com, etc.). The word "nocseman" doesn't appear anywhere on any of the sites: yet Google read the keyword in the domain name and, at the end of the test, we ended up with the .NET site ranking #1 for 'nocseman'.

    Each site was also assigned a unique keyword, as well. And I followed the indexing and rankings of each site during the testing process and wrote about it all.

    The highlights are all written up here if you're interested: https://www.hartzer.com/blog/tld-bias-search-engine-indexing-rankings/

    I'm happy to answer any questions about the testing or the results.
     
    The views expressed on this page by users and staff are their own, not those of NamePros.
  2. TCK

    TCK NameOptions.com VIP ★★★★★★★★★★

    Posts:
    1,679
    Likes Received:
    772
    When you write "of the other TLDs", are you including .com? Or just from among the new TLD's?
     
    Last edited: Jan 20, 2020
  3. bhartzer

    bhartzer VIP Member Epik.com Staff VIP ★★★★★★★★★★

    Posts:
    81
    Likes Received:
    135
    TCK, yes, .COM, .NET, .ORG were all included and given an equal chance in the testing. One site per TLD was created, here's the list:
    • .ICU
    • .TOP
    • .XYZ
    • .SITE
    • .CLUB
    • .ONLINE
    • .VIP
    • .COM <<========
    • .NET
    • .ORG
    • .DE
    • .CO.UK
    • .CO
    • .LONDON
    • .IN
     
  4. TCK

    TCK NameOptions.com VIP ★★★★★★★★★★

    Posts:
    1,679
    Likes Received:
    772
    Do you have any theories why xyz got indexed much quicker than the others? Maybe because G's parent is abc.xyz?
     
  5. bhartzer

    bhartzer VIP Member Epik.com Staff VIP ★★★★★★★★★★

    Posts:
    81
    Likes Received:
    135
    That's a good question... it's only speculation. I personally don't think it's because Google's parent is .XYZ.

    I may have to do with the number of domains registered on the TLD and the overall crawling activity of domains/sites on that TLD.

    I think we can only speculate--I'd love to be able to come up with some sort of test to try to figure it out. But for now we do know that based on these test results they indexed the XYZ pages a lot quicker than any other TLD in the test.
     
  6. Internet.Domains

    Internet.Domains Top Contributor VIP

    Posts:
    2,469
    Likes Received:
    5,955
    Thank you! Very interesting. In future, I would be interested to know if .app has any extra bias, since Google owns it.
     
  7. bhartzer

    bhartzer VIP Member Epik.com Staff VIP ★★★★★★★★★★

    Posts:
    81
    Likes Received:
    135
    I'll definitely include .APP in the next round of testing.
     
  8. Bohdan

    Bohdan Established Member

    Posts:
    526
    Likes Received:
    1,670
    Did you submitted them for the indexing all at once or one-by-one?
     
    Last edited: Jan 20, 2020
  9. clarkemarketing

    clarkemarketing Top Contributor VIP

    Posts:
    1,575
    Likes Received:
    1,033
    Was every website 100% identical (other than the domain)? Were they all launched at exact same time and submitted for indexing at exact same time?
     
  10. bhartzer

    bhartzer VIP Member Epik.com Staff VIP ★★★★★★★★★★

    Posts:
    81
    Likes Received:
    135
    >> Did you submitted them for the indexing all at once or one-by-one?
    Sites were built and then waited a week to see if they would get indexed. After no pages were indexed, I verified all sites in Google Search Console and submitted manually one page per site. All at once... well it was more like "as fast as I could manually copy/paste the URLs". So yeah, essentially all at once they were hand submitted.

    All of the timeline details and the specifics are in the blog post and PDF download.

    >> Was every website 100% identical (other than the domain)?
    SEO testing software was used, it's not really a fair test if all sites are 100 percent identical. But they were 'identical' in that they were all 25 pages, all had unique text, and same content length. You can see all the sites for yourself to see if you look at the blog post and the PDF, all the data is there.
     
  11. RJ

    RJ Domain Buyer PRO VIP ★★★★★★★★★★

    Posts:
    18,834
    Likes Received:
    2,553
    Fantastic, thank you for sharing your research. This is interesting. I had a notion that, with all things equal, .ORG carried more favor with Google rankings, but this shows that is not the case.
     
  12. bhartzer

    bhartzer VIP Member Epik.com Staff VIP ★★★★★★★★★★

    Posts:
    81
    Likes Received:
    135
    >> .ORG carried more favor with Google rankings, but this shows that is not the case.
    Yes, that's not the results of the testing. In fact, the .ORG was more difficult to get pages indexed and didn't have its keyword rank as quickly as the others.
     
  13. HotKey

    HotKey Made in Canada VIP

    Posts:
    2,447
    Likes Received:
    5,662
    This is super interesting, and thanks for sharing. A couple thoughts:

    - Do you think your two weeks of testing is enough time to provide conclusive results? I mean, I waited for the good part of a year to see my main portfolio site rank #1 in results across the board (all TLDs combined) for my exact match search term, domain+TLD+site content.

    - Is it possible some TLDs will take longer for Google to properly index and rank due to a trust weighting, and number of sites registered? For example, .org is a trusted extension with a good history and reputation, with way more active sites than .xyz, so to expect proper indexing might take more than a couple of weeks.

    - To tie in to the last thought, is Google weighing relevance with a combination of TLD, domain name and content in terms of indexing and search result placement? Is it all be tied together? If not, it certainly should be, imo.

    - Selling vacuum cleaners (just a random example) on a .club extension does not make any sense in terms of relevance, and although Google says they give all TLDs a fair chance, their search and indexing algorithms must be advanced enough to weighing and applying relevant results, which at this point we've got content, domain name, and the TLD it's based on.

    - Is it possible extensions like .com or .xyz are more "catch-all" that anything to be indexed, is weighed less for indexing, versus specific TLDs like .org, or .club or .top that convey an exact keyword, thus the expectation would be building a domain and site that correlates to that TLD.

    Just some thoughts, based on your initial post without going in depth into your whole study. Again, thx for your post- its pretty cool you invested time into gathering this info.
     
  14. bhartzer

    bhartzer VIP Member Epik.com Staff VIP ★★★★★★★★★★

    Posts:
    81
    Likes Received:
    135
    >> Do you think your two weeks of testing is enough time to provide conclusive results? I mean, I waited for the good part of a year to see my main portfolio site rank #1 in results across the board (all TLDs combined) for my exact match search term, domain+TLD+site content.

    Yes, in this case, the timeline was perfectly sufficient *for this test*. The test was specifically to test indexing and initial rankings. Once the sites got pages indexed and we saw initial rankings, then that's the end of that particular test. The goal here was to see the rate of indexing and see initial rankings.

    >> Is it possible some TLDs will take longer for Google to properly index and rank due to a trust weighting, and number of sites registered? For example, .org is a trusted extension with a good history and reputation, with way more active sites than .xyz, so to expect proper indexing might take more than a couple of weeks.

    There is really no trust weighting (or is there one)? Each domain name was newly registered, all domains & sites in this test had an equal chance. How have you determined or proved that .ORG is a "trusted extension"? Even with more active sites, the testing found that it was more difficult to get pages indexed on .ORG. I don't know of another way to properly test or see the "trust weighting" and prove it, one way or the other, is if a particular TLD is trusted or not. I'm open to further testing if you have any recommendations.

    >> To tie in to the last thought, is Google weighing relevance with a combination of TLD, domain name and content in terms of indexing and search result placement? Is it all be tied together? If not, it certainly should be, imo.

    In the long run, yes, rankings are a combination of a lot of various factors. That includes content, links, trust, pages indexed, CTR, and a lot of other factors. In this case all tested TLDs and sites had the same exact chance to get pages indexed and ranked--with *No* links at all and essentially the same type of nonsensical content. That helped narrow it down to only few potential ranking factors.

    >> Selling vacuum cleaners (just a random example) on a .club extension does not make any sense in terms of relevance, and although Google says they give all TLDs a fair chance, their search and indexing algorithms must be advanced enough to weighing and applying relevant results, which at this point we've got content, domain name, and the TLD it's based on.

    Selling vacuum cleaners on something like "VacuumCleaner.Club" doesn't make sense? It could be relevant if it's a discount club, you get into the vacuum cleaner club, and you get vacuum cleaners for cheaper than you would if you're in the club. But I digress.... In this case we did give Google the same chance to index and rank brand new sites. They weighed the sites based on something--and it could have been totally random that they chose to index the .XYZ super-fast and the others took hours or days to get indexed.

    >> Is it possible extensions like .com or .xyz are more "catch-all" that anything to be indexed, is weighed less for indexing, versus specific TLDs like .org, or .club or .top that convey an exact keyword, thus the expectation would be building a domain and site that correlates to that TLD.

    I don't think that certain extensions (TLDs) are more "catch-all" or anything like that. The .COM and .NET took longer to get pages indexed. I don't see any correlation between what someone would call a "catch-all" TLD rather than a keyword rich TLD. In this case, the nonsense word (namescon spelled backwards) was used on all TLDs, and we now see how much weight Google gives to the keyword in a domain name. That word doesn't appear anywhere in the content of the sites.
     
  15. NameDeck

    NameDeck SaveDotOrg.org VIP

    Posts:
    3,541
    Likes Received:
    2,286
    Nice research.

    Regarding .org. If I were to write a ranking algorithm I would make it harder for a .org to rank so only good content matching the TLD would actually rank high.

    When it comes to trustworthiness of a TLD I think you should factor in different aspects.

    People will (subconsciously) put more trust into a website, say foundation.org than foundation.xyz. therefore when you're 'vetting' the content of both you should weigh the .org different and with higher standards.

    When the keyword is random, say domain.org and domain.xyz I see no reason not to give the .xyz preference over .org or actually make it rank higher.
     
  16. JB Lions

    JB Lions Top Contributor VIP

    Posts:
    15,122
    Likes Received:
    20,253
    You really can't base much on this test at all. You would have to repeat it a few more/many times over to get a more accurate feel. This one time .xyz got indexed faster, next time it could be some other extension. Same with ranking. So .net was ranking #1 at the end of the test, right now it's #4 for me, term nocseman:

    https://www.google.com/search?clien...VKgK0KHdVvBdkQvgUoAXoECBcQKA&biw=1536&bih=701

    Not including this thread, just the different extensions:

    .club
    .xyz
    .icu
    .net
    .top
    .co.uk
    .site
    .vip
    .london
    .org
    .online
    .in.
    .de
    .co
     
  17. bhartzer

    bhartzer VIP Member Epik.com Staff VIP ★★★★★★★★★★

    Posts:
    81
    Likes Received:
    135
    @JB Lions, the testing was completely fair and unbiased, up until the keyword(s) and sites were revealed publicly. Once the keywords and sites were made public, there are other, uncontrolled biases that come into play, and rankings change based on various search engine ranking factors.

    The indexing was done, and it is what it is. Will the results be different if the same testing was run again? Perhaps.

    The search engine rankings were recorded right before the sites and keyword(s) were made public. The .Net won for 'nocseman', which is only mentioned in the domain name. In fact, 2 pages on the .Net domain outranked any other site in the test. In the past 3 days, the rankings for that keyword have changed and they'll most likely continue to change.

    My only regret was not testing more TLDs in this test. Looking forward to eventually testing all of the TLDs. And all of these TLDs again.
     
  18. bhartzer

    bhartzer VIP Member Epik.com Staff VIP ★★★★★★★★★★

    Posts:
    81
    Likes Received:
    135
    I get what you're saying but honestly do you have any research or testing to prove that people put more trust into one particular website or TLD over another? For example, has a poll been done to ask 100 (or 1,000) people if they trust foundation.org or foundation.xyz more? An offline poll? I'm not trying to be difficult here, just honestly asking :)

    Also, what "should be done", such as vetting the content, and what a search engine actually does is completely two different things. In fact, this testing proved that there really wasn't any vetting of any content done: the .XYZ site ranked for it's designated keyword very quickly. And, for 'nocseman', there is no vetting of content: the .NET didn't even mention the nocseman keyword anywhere in content: yet it ranked #1 with only the keyword 'nocseman' in the domain name.
     
  19. JB Lions

    JB Lions Top Contributor VIP

    Posts:
    15,122
    Likes Received:
    20,253
    There are different studies out there, this is one I remember, was talked about somewhere in this forum.

    [​IMG]
     
    Last edited: Jan 21, 2020
  20. JB Lions

    JB Lions Top Contributor VIP

    Posts:
    15,122
    Likes Received:
    20,253
  21. NameDeck

    NameDeck SaveDotOrg.org VIP

    Posts:
    3,541
    Likes Received:
    2,286
    Lots when you Google for reacher on the topic,see @JB Lions posts above.

    I get what you're saying. Ranking seems to be random though.

    For me it ranks XYZ, London, co.

    Results will be different for other people. Once logged out it ranks XYZ club com top.

    The concept of your experiment is nice but I don't think it provides usable data. The dataset is way too small to be able to draw a conclusion.
     
  22. bhartzer

    bhartzer VIP Member Epik.com Staff VIP ★★★★★★★★★★

    Posts:
    81
    Likes Received:
    135
    The first study you're referring to was a good one, which polled 1,000 online shoppers. Only problem is that it was done in 2014, only one year after the new TLDs were introduced. A lot has changed since then (it's been FIVE years!)

    The Varn research is also good data--but again it was posted on 27th October 2016, *THREE* years ago. Again, good research but a lot has changed since then.

    I'm not disagreeing with you or agreeing with you here.

    I've only done my own, unbiased TLD SEO testing. I presented the results. Let's not get sidetracked in this thread, talking about public opinion on whether or not people trust TLDs or not.

    That's NOT what I tested. I simply tested how *GOOGLE* deals with indexing and ranking. This has nothing to do with public opinion on TLDs. If someone would do some *up to date* research on public opinion on TLDs that would be awesome.
     
  23. bhartzer

    bhartzer VIP Member Epik.com Staff VIP ★★★★★★★★★★

    Posts:
    81
    Likes Received:
    135
    I'm perfectly happy with this unbiased data set: 25 page sites and 375 pages of content total. The point was to see indexing rates and initial rankings data.

    My only 'regret' here (if you can call it that) is that more TLDs weren't tested. I went with the most popular TLDs based on # of domain registrations. I fully plan on testing more TLDs in the future and comparing those results with these results.
     
  24. Bob Hawkes

    Bob Hawkes formerly MetBob NameTalent Gold Account VIP Trusted Blogger

    Posts:
    5,682
    Likes Received:
    14,760
    Really interesting research, @bhartzer Thanks for sharing. I am still digesting the full report you provide by signing up. The idea of NamesCon spelled in reverse to make a nonsense word is pretty cute! :xf.cool:

    I guess I was surprised that given how smart Google is at not being fooled by content that is not really content, that they got indexed as well and quickly as they did.

    I agree it would be interesting to see how the Google new gTLDs that are secure spaces .page, .app and .dev would do in this.

    In another thread someone pointed out differences in delay time for DNS access that seemed to depend on the registry backend. Do you think that would have any impact at all in this - i.e. I think Google give preference to sites that load faster.

    It does seem that the sites that did better getting indexed fast, at least among the new extensions, were also ones like .xyz, .online and .club that are known to have a proportionately larger real world use than some of the ones that did less well like .icu, .top and even .site compared to .online.

    Thanks again for an interesting and well-designed experiment.

    Bob
     
  25. oldtimer

    oldtimer Do some good for humanity and the environment VIP ★★★★★★★★★★

    Posts:
    2,260
    Likes Received:
    2,478

    How many other sites from your experiment ranked for "nocseman" on the same results page and in what order did they appear.

    It's possible that the Google's algorithm (we better call it Google AI from now on) is smart enough to recognize that "nocseman" is backwards for NamesCon and as such associated it with Networking and found .Net to be the best fit (just guessing).

    But nowadays there is no way that Google AI is going to ignore the keyword in the domain or even in the extension itself.

    IMO
     
    Last edited: Jan 21, 2020

Want to reply or ask your own question?

It only takes a minute to sign up – and it's free!
Topics / Tags:
NameWorth
  1. NamePros uses cookies and similar technologies. By using this site, you are agreeing to our privacy policy, terms, and use of cookies.
    Dismiss Notice
Loading...