internal & external link count

[webmasterx]

I have noticed that many sites listed here have vastly conflicting information on their internal & external link count.....

ie i find a trade, it says there are 5x internal links and 25x external links on the site, but when you check the site physically there are many more external links than their listing states.....is there a problem with the bot??????

[vrocks]

In many other threads I stated that a mis-configuration of MYSQL settings caused the bots to timeout for an entire cycle. This has been corrected and it will take the bots about 2 weeks to fix the problem. The trade checking bot was jacked up for 2 months or possibly longer. So you should notice a lot more trades coming up as error status than normal.

Here is basically what happened in a nutshell. MYSQL got upgraded. When this happened a max_timeout setting got set to 10 seconds. Not sure why. Each bot pulls different amounts of sites from the DB to check, but lets pretend they all take 150. If that 150 sample had one site in it that took longer than 10 seconds to load then all 150 sites were dropped and not checked. So there were sites coming up in error here and there. And PR going up and down, here and there. And outbound/inbound changing here and there. Which is why I assumed the bots were working.

Then I finally had people start posting in July that something wasn't right. Counts were way off on everything. Trades that should be in error weren't in error. PR hadn't updated on sites since March or May at the latest.

So I manually tried running the bots and saw that they timed out. I enlisted Drocd and he gave me some useful info that gave me the "House" moment where I realized it wasn't a programming problem, it was a problem that occurred in MYSQL.

Again, it will take each bot a different length of time to clear things up since each one pulls a different number of sites and requires a different length of time to complete its tasks. Going to the home page: http://www.linkspun.com should show you a massive amount of PR changes. That bot will take about 2 to 2 1/2 weeks to sort it all out. Of course its been running for 1 week so about another week and a half. The outbound/inbound bot will take another week or so also and the trades bot about another 2 weeks to catch up.

[vrocks]

Some hard numbers:

There are 236,000 pages and so far 116,333 have been checked for inbound/outbound. So about 6 more days for the other HALF.

There are 254,224 trades to be checked and thus far about 178,080 have been checked. So about 4 more days.

With 236,000 pages in the system the bot has checked 83,290 for pagerank. This is the slowest bot because it cannot be allowed to trigger a lockout by Google or it will seriously hinder the bot. Could take until months end to finish checking PR.

[webmasterx]

I did read that post, sorry i did not realize that this was the root of the problem. thanks for the reply tho.

[vrocks]

All sites have now been checked for PR, inbound/outbound links and trades in error.

Things should be a lot smoother now.