Page 1 of 1

Thumblogger again!

Posted: Wed Jul 28, 2010 8:21 am
by alexweb
It`s Thumblogger again I suppose. Links cannot be found but they exist. Can you do something about it? Thank`s.

Re: Thumblogger again!

Posted: Fri Jul 30, 2010 10:11 am
by vrocks
Perhaps if url =~ 'thumblogger' { user_agent= a real user agent and not your bots info }

Re: Thumblogger again!

Posted: Fri Jul 30, 2010 3:17 pm
by drocd
vrocks wrote:Perhaps if url =~ 'thumblogger' { user_agent= a real user agent and not your bots info }
I appreciate the tip, but that's not the problem. ;)

Re: Thumblogger again!

Posted: Fri Jul 30, 2010 8:16 pm
by jdoughs
It's due to the increased spider activity, he was banning us every 4-5 days, now he is banning us daily.

As soon as I see it I go through and manually confirm them so they arent seen, but now that it is happening daily people see them down before I can get them.

It's our IP he bans, and its automatically as it see's the spider as a possible scraper. I did remove the ban when you made the post, and did so again about an hour ago.

Looks like we are lasting about 18 hrs in between bans..lol

I have filled in his forms for the permanent whitelist, but have not get an answer, and cannot submit again cause I'm already in 'que' to be dealt with.

Thumblogger has a board too, and this is an issue with his site not our spider, I will continue to try to get it whitelisted but so far efforts have been hopeless. Maybe we need more people who do use his services to email him, post on his board, ask why we can't get whitelisted.

Just do it nicely! lol

Re: Thumblogger again!

Posted: Fri Jul 30, 2010 8:43 pm
by alexweb
I have already posted on Twan`s forum regarding same problem with some other sites but with no reply. Last couple of months he hardly answers on his board. I will bump my post, mention your site and will we see if anything happens.

Re: Thumblogger again!

Posted: Fri Jul 30, 2010 10:07 pm
by jdoughs
alexweb wrote:I have already posted on Twan`s forum regarding same problem with some other sites but with no reply. Last couple of months he hardly answers on his board. I will bump my post, mention your site and will we see if anything happens.
Yeah I think he is missing in action right now. lol