Does Google Have Trouble With Large Sites?

Whogister · 03-06-2010, 07:26 AM

I have a website that was getting about 50 visits per day from Google...
Once it hit 50,000 pages indexed in Google the traffic has hit a brick wall... nothing.
Google still adds pages from the site to their index but still no traffic.

Anyone else with 50,000 + pages ever experience this?

snowbird · 03-06-2010, 07:50 AM

In regards to power, google has no trouble indexing large sites. However, Google will only index the pages it can find on your site. It finds pages on your site through links. Once it has indexed all the pages it can find, it will slow down dramatically. This is because it only revisits pages that it feels might have changed content. When you update the content on a page, then google will revisit it.

Also if you have a site of a large size, I would also suggest looking into google's sitemaps program. This program was designed to help google find pages on large sites more easily and quickly.

Whogister · 03-06-2010, 07:58 AM

Yes, I know about getting pages indexed... as I said, I have over 50K indexed right now.
What I am wondering is why my traffic from Google has come to a screeching halt right after my indexed page count reached 50k?

snowbird · 03-06-2010, 08:08 AM

Like i said, google has no need to revisit pages unless all of those 50,000 pages have been modified or updated. Google indexes a page and normally wont visit it again unless you change the content on it. They are eliminating visits that are not needed. I also told you if you have more pages that need to be indexed besides that 50,000 it may be because google is having trouble finding all your pages. It will index what it can find but you may also need to help it with sitemaps. Which is why i mentioned you should check out the google sitemaps program. It will help it find pages on large sites more quickly and easily.

Whogister · 03-06-2010, 08:14 AM

**enlytend** · 03-06-2010, 09:57 AM

What do you get when you do a site: query? i.e.: site:example.com where example.com is your site name. Does the page count match your site? Any hints that Google finds some of the pages redundant?

Also, look at your logs. Where was traffic coming from before? What has changed? Are there external links that are no longer producing traffic or is there a decrease in search engine traffic? Has the number of different search terms used to find your site dropped? If it's a drop in search traffic, you may still be indexed but not ranking as well for the pages you ranked for previously.

Whogister · 03-06-2010, 10:09 AM

There are currently 51,200 pages of my site indexed in google...
When I do the site:example.com thing, everything is normal.

Before when I had under 50,000 pages indexed I was getting traffic from Google searchers.
As soon as I went over the 50,000 pages indexed I am no longer getting visitors from people searching on google.

My pages are indexed but just not showing up in the search results anymore.

hollywood · 03-06-2010, 11:59 AM

I'm no expert but my understanding is that Google uses velocity of new backlinks and new content pages in their ranking algos, which is in addition to counting the sheer number of backlinks and pages. I have a feeling they guard their algos much like Coca Cola guards their secret soft drink recipe so who knows what other parameters are in there?

**enlytend** · 03-06-2010, 03:13 PM

What kind of pages are these? (products? articles? user generated content?) How similar are they? Is there any duplication of page titles?

I doubt it's the magic number of 50,000 - more likely they got hit with some kind of penalty. Are we talking pages that were page 1 for certain keyword(s) that slipped back to p3-4 or are we talking pages that are no longer anywhere to be seen (like a -950 penalty)? Without knowing the specifics, it's anyone's guess.

dvduval · 03-06-2010, 03:53 PM

I wanted to add that for large site, a large concentration of traffic is usually focused on 20% of the pages or less, and for that reason I usually don't get too concerned about how many pages are indexed on muy large sites.

Whogister · 03-06-2010, 04:03 PM

They are unique product pages... for a big shopping mall type site.
Nothing duplicate about it, I spent a lot of time making the site.

Oh well, looks like I got penalized for something or another...
Can't pi$$ the wrong way these days without Google zapping you for it!

AbsoluteKC · 03-07-2010, 06:52 AM

Maybe you wanto to check your Google webmaster account to see if there is any messages from Google..

orangecopper · 03-07-2010, 06:54 AM

Hello,

I have a website that is as huge as 20,000 plus pages and all are indexed fine. Try adding a more robust Sitemap and update it through Google webmaster tool.
Also if the contents are frequently edited then only google will reindex or take a look at it in most cases.

regards
Joshu

w3bmaster · 03-07-2010, 08:03 AM

Number of indexed pages and traffic have like no connexion

Do you have rankings ?

Regards

audreysolo · 03-07-2010, 07:50 PM

I have never heared it before that Google have trouble with large sites.Maybe your keywords of your website no very hot to addict people.In addition,do some seo to improve traffics.

**enlytend** · 03-08-2010, 03:27 AM

Quote:

They are unique product pages... for a big shopping mall type site.
Nothing duplicate about it, I spent a lot of time making the site.

Unfortunately, the time you spent making the site is inconsequential

- the pages aren't showing up in searches any more, I'm trying to help you figure out why that might be.

How much content is on the pages -is there enough to outbalance the common header/footer/navigation code?
Are the product names/descriptions from a feed? If so, and if other sites may be using that feed, have you customized the descriptions, item names, etc.?
????: NamePros.com http://www.namepros.com/search-engines/643359-does-google-have-trouble-large-sites.html
How's your internal navigation? Are the pages "buried" in your site or are they easily reachable?

You might want to do a search on some of the major SEO blogs/forums - you're certainly not the first person to have this kind of problem. Good luck with it - let us know how it turns out.

shworth · 03-13-2010, 02:53 AM

Only 50 visits a day from 50,000 pages makes it seem as though there is either something wrong with your site navigation or the content you are providing.

72Reasons · 03-13-2010, 04:25 PM

Originally Posted by dreamscape

I have a website that was getting about 50 visits per day from Google...
????: NamePros.com http://www.namepros.com/showthread.php?t=643359
Once it hit 50,000 pages indexed in Google the traffic has hit a brick wall... nothing.
Google still adds pages from the site to their index but still no traffic.

Anyone else with 50,000 + pages ever experience this?

Just adding content will not likely do much to drive traffic. Pages do not equate to traffic. You need to promote a few core pages so that you might start to rank well. Then you might begin to see some visitors.

soggyindo · 03-13-2010, 04:39 PM

i agree, why so many pages and work for only 50 uniques? it's easy to get that much traffic for sites with just one or two pages. it seems like a lot of work for little... or are you waiting for it to go 'viral'? if so i'd work in social networking to the mix, too

Sitemapper · 04-06-2010, 07:24 PM

I am assuming your pages are ranking poorly? Does your main domain have lots of backlinks? Else work on that through all means possible.

And then check if your DB generated pages? (product catalog) contains unique content compared to other pages on net. Else try see if you can generate SEO'ed titles and headers with some salt to content.

s3solutions · 04-11-2010, 08:53 PM

Originally Posted by dreamscape

I have a website that was getting about 50 visits per day from Google...
Once it hit 50,000 pages indexed in Google the traffic has hit a brick wall... nothing.
Google still adds pages from the site to their index but still no traffic.

Anyone else with 50,000 + pages ever experience this?

Hi dreamscape,

I don't think the increasing indexed pages is affecting your website traffic.

The more indexed pages you have, the more keywords your website will have and the result is the more traffic you will get from that.

Just check your GA to see if it's installed correctly.
????: NamePros.com http://www.namepros.com/showthread.php?t=643359

At my side, I am in trouble with the indexed pages in Google.com and local Google search engine.

The numbers of indexed pages in these two search engine are different. I don't know what is causing this.

Has anybody here experienced this? Please help!!!

llisyspa67 · 04-16-2010, 07:56 AM

It is unknown to me that weather Google have trouble with large pages or not..
But I prefer the Google the most..
It is a very good search engine..
And I got the demerritts about google from you.
so thanks to all of you ............

____________________________
Accommodation Ballina

ePag · 04-16-2010, 02:52 PM

have you build a sitemap for your site?

Google has no problem with big sites. In fact,Google loves big sites with unique and good content, that are updated with fresh information on a regular basis.

Does you site has unique content or it's scraped ?

If it's scraped, then this might be the answer to your question.

72Reasons · 04-16-2010, 05:06 PM

Google should be able to handle any size website or at least that my understanding anyways. Getting traffic to a big site might require more than just the search engines. Our you active with social media?

andr69 · 04-22-2010, 06:34 AM

Google hardly makes mistakes, and when I have some problems with Google I begin to search for problems with my site.

03-06-2010, 07:26 AM	THREAD STARTER #1 (permalink)
Whogister NamePros Regular Join Date: Dec 2008 Location: Michigan, USA Posts: 737 Follow @whogister	Does Google Have Trouble With Large Sites? I have a website that was getting about 50 visits per day from Google... Once it hit 50,000 pages indexed in Google the traffic has hit a brick wall... nothing. Google still adds pages from the site to their index but still no traffic. Anyone else with 50,000 + pages ever experience this?

03-06-2010, 07:50 AM	#2 (permalink)
snowbird Emperor Penguin Join Date: Aug 2004 Location: North Carolina Posts: 727	In regards to power, google has no trouble indexing large sites. However, Google will only index the pages it can find on your site. It finds pages on your site through links. Once it has indexed all the pages it can find, it will slow down dramatically. This is because it only revisits pages that it feels might have changed content. When you update the content on a page, then google will revisit it. Also if you have a site of a large size, I would also suggest looking into google's sitemaps program. This program was designed to help google find pages on large sites more easily and quickly. __________________ Mike R. RegisterBird.com RegisterBird.com Star Program [Special Promotion on .IN, .COM, and .NET extensions!]

03-06-2010, 08:08 AM	#4 (permalink)
snowbird Emperor Penguin Join Date: Aug 2004 Location: North Carolina Posts: 727	Like i said, google has no need to revisit pages unless all of those 50,000 pages have been modified or updated. Google indexes a page and normally wont visit it again unless you change the content on it. They are eliminating visits that are not needed. I also told you if you have more pages that need to be indexed besides that 50,000 it may be because google is having trouble finding all your pages. It will index what it can find but you may also need to help it with sitemaps. Which is why i mentioned you should check out the google sitemaps program. It will help it find pages on large sites more quickly and easily. __________________ Mike R. RegisterBird.com RegisterBird.com Star Program [Special Promotion on .IN, .COM, and .NET extensions!]

03-06-2010, 09:57 AM	#6 (permalink)
enlytend Forum Moderator Join Date: Aug 2006 Location: USA Posts: 2,152 Follow @enlytend	What do you get when you do a site: query? i.e.: site:example.com where example.com is your site name. Does the page count match your site? Any hints that Google finds some of the pages redundant? Also, look at your logs. Where was traffic coming from before? What has changed? Are there external links that are no longer producing traffic or is there a decrease in search engine traffic? Has the number of different search terms used to find your site dropped? If it's a drop in search traffic, you may still be indexed but not ranking as well for the pages you ranked for previously. __________________ Enlytend Solutions - Internet marketing and web development Was my advice helpful? Please consider a small donation to the National Canine Cancer Foundation - a tax deductible 501(c)(3)that directly funds cancer research

03-06-2010, 11:59 AM	#8 (permalink)
hollywood NamePros Regular Join Date: Jan 2007 Location: Namepros Posts: 240	I'm no expert but my understanding is that Google uses velocity of new backlinks and new content pages in their ranking algos, which is in addition to counting the sheer number of backlinks and pages. I have a feeling they guard their algos much like Coca Cola guards their secret soft drink recipe so who knows what other parameters are in there? __________________ "Because it really is that simple imho. If you don't want sheep for users, don't treat your users like sheep." Memory Chips \| Euro ETF \| Flowers \| A1c Levels & Diabetes

03-06-2010, 07:58 AM	THREAD STARTER #3 (permalink)
Whogister NamePros Regular Join Date: Dec 2008 Location: Michigan, USA Posts: 737 Follow @whogister	Yes, I know about getting pages indexed... as I said, I have over 50K indexed right now. What I am wondering is why my traffic from Google has come to a screeching halt right after my indexed page count reached 50k?

03-06-2010, 08:14 AM	THREAD STARTER #5 (permalink)
Whogister NamePros Regular Join Date: Dec 2008 Location: Michigan, USA Posts: 737 Follow @whogister	ok

03-06-2010, 10:09 AM	THREAD STARTER #7 (permalink)
Whogister NamePros Regular Join Date: Dec 2008 Location: Michigan, USA Posts: 737 Follow @whogister	There are currently 51,200 pages of my site indexed in google... When I do the site:example.com thing, everything is normal. Before when I had under 50,000 pages indexed I was getting traffic from Google searchers. As soon as I went over the 50,000 pages indexed I am no longer getting visitors from people searching on google. My pages are indexed but just not showing up in the search results anymore.

03-06-2010, 03:13 PM	#9 (permalink)
enlytend Forum Moderator Join Date: Aug 2006 Location: USA Posts: 2,152 Follow @enlytend	What kind of pages are these? (products? articles? user generated content?) How similar are they? Is there any duplication of page titles? I doubt it's the magic number of 50,000 - more likely they got hit with some kind of penalty. Are we talking pages that were page 1 for certain keyword(s) that slipped back to p3-4 or are we talking pages that are no longer anywhere to be seen (like a -950 penalty)? Without knowing the specifics, it's anyone's guess. __________________ Enlytend Solutions - Internet marketing and web development Was my advice helpful? Please consider a small donation to the National Canine Cancer Foundation - a tax deductible 501(c)(3)that directly funds cancer research

03-06-2010, 03:53 PM	#10 (permalink)
dvduval NamePros Regular Join Date: May 2005 Posts: 433	I wanted to add that for large site, a large concentration of traffic is usually focused on 20% of the pages or less, and for that reason I usually don't get too concerned about how many pages are indexed on muy large sites. __________________ PHP Link Directory Script - seo friendly link directory script Template Forums -beautiful templates for phpLD 中国超链接脚本 - phpLD now fully translated to Chinese!

03-06-2010, 04:03 PM	THREAD STARTER #11 (permalink)
Whogister NamePros Regular Join Date: Dec 2008 Location: Michigan, USA Posts: 737 Follow @whogister	They are unique product pages... for a big shopping mall type site. Nothing duplicate about it, I spent a lot of time making the site. Oh well, looks like I got penalized for something or another... Can't pi$$ the wrong way these days without Google zapping you for it!

03-07-2010, 06:52 AM	#12 (permalink)
AbsoluteKC NamePros Regular Join Date: Apr 2006 Posts: 980	Maybe you wanto to check your Google webmaster account to see if there is any messages from Google.. __________________ SEO Tips \| One Day, One Card \| KC Tan Logo Design \| WordPress Coach Profit Blogging Workshop

03-07-2010, 06:54 AM	#13 (permalink)
orangecopper New Member Join Date: Mar 2010 Location: India Posts: 10 Follow @orangecopper	Hello, I have a website that is as huge as 20,000 plus pages and all are indexed fine. Try adding a more robust Sitemap and update it through Google webmaster tool. Also if the contents are frequently edited then only google will reindex or take a look at it in most cases. regards Joshu __________________ Learn How to make money online

03-07-2010, 08:03 AM	#14 (permalink)
w3bmaster New Member Join Date: Jun 2004 Location: Buzau/Romania Posts: 20	Number of indexed pages and traffic have like no connexion Do you have rankings ? Regards __________________ Stop panic attacks\|Forex book\|Get boyfriend back\|Stop smoking help

03-07-2010, 07:50 PM	#15 (permalink)
audreysolo First Time Poster! Join Date: Mar 2010 Posts: 1	I have never heared it before that Google have trouble with large sites.Maybe your keywords of your website no very hot to addict people.In addition,do some seo to improve traffics. __________________ website whois

03-13-2010, 02:53 AM	#17 (permalink)
shworth First Time Poster! Join Date: Mar 2010 Posts: 1	Only 50 visits a day from 50,000 pages makes it seem as though there is either something wrong with your site navigation or the content you are providing.

03-13-2010, 04:39 PM	#19 (permalink)
soggyindo Senior Member Join Date: Feb 2008 Location: ✰$479 sites✰ artistwebsite.org Posts: 1,283	i agree, why so many pages and work for only 50 uniques? it's easy to get that much traffic for sites with just one or two pages. it seems like a lot of work for little... or are you waiting for it to go 'viral'? if so i'd work in social networking to the mix, too __________________ Get a top quality domain portfolio, blog, or developed website, from $479. Quote NAMEHUGGER11 for a NamePros only discount. Artist Website \| Contemporary Art Blogs.

04-06-2010, 07:24 PM	#20 (permalink)
Sitemapper New Member Join Date: Nov 2008 Posts: 7 Follow @microsystools	I am assuming your pages are ranking poorly? Does your main domain have lots of backlinks? Else work on that through all means possible. And then check if your DB generated pages? (product catalog) contains unique content compared to other pages on net. Else try see if you can generate SEO'ed titles and headers with some salt to content. __________________ ISV Company \| Website Developer Forums \| XML Sitemaps \| HTML Sitemaps

04-16-2010, 07:56 AM	#22 (permalink)
llisyspa67 First Time Poster! Join Date: Apr 2010 Posts: 1	It is unknown to me that weather Google have trouble with large pages or not.. But I prefer the Google the most.. It is a very good search engine.. And I got the demerritts about google from you. so thanks to all of you ............ ____________________________ Accommodation Ballina

04-16-2010, 02:52 PM	#23 (permalink)
ePag NamePros Member Join Date: Apr 2010 Posts: 31	have you build a sitemap for your site? Google has no problem with big sites. In fact,Google loves big sites with unique and good content, that are updated with fresh information on a regular basis. Does you site has unique content or it's scraped ? If it's scraped, then this might be the answer to your question. __________________ ePagini

04-16-2010, 05:06 PM	#24 (permalink)
72Reasons New Member Join Date: Mar 2010 Posts: 8	Google should be able to handle any size website or at least that my understanding anyways. Getting traffic to a big site might require more than just the search engines. Our you active with social media? __________________ Tucson Dentist Cosmetic Family Dental Crowns and Tucson Dental Care Teeth Whitening Porcelain Veneers Crowns Dentist

04-22-2010, 06:34 AM	#25 (permalink)
andr69 Account Suspended Join Date: Nov 2008 Location: Russia Posts: 622	Google hardly makes mistakes, and when I have some problems with Google I begin to search for problems with my site.

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)