A way to weed out the dead pages from the search results

[vrocks]

The search results are coming up with tons of pages where the user let the page go and never removed it from there account. When this happens the easiest way to realize it is dead is by looking at inbound/outbound links. They are often 0 on both. New pages added to the system also have 0,0 in these columns. The checker bot always scans new pages within 2 weeks. Usually much quicker.

So I am thinking about running a query on the DB to set all pages with inbound/outbound = 0/0 and a last check date older than 2 weeks ago and setting them all to hidden. This will remove about 8,000 pages from the results. I am figuring any page somebody wanted to sell something from will always have at least one internal link or one outbound link. Nobody here creates pages without them right? or wrong?

Then there are the parked pages by places like Godaddy. They usually have 0,2 with 2 outbound links. But someone might actually create a dating landing page with a similar make up... So I am thinking about creating a signature file that would search pages for certain text phrases and automatically set them to hidden when it finds them. Sedo, godaddy, etc use certain domains to host the graphics on the page that nobody would hotlink. Also when set to hidden the webmaster would receive an email letting them know why it happened.

Any ideas on this?

[CodeR70]

I think with the e-mail notification message you cant go very wrong. Cleaning up databases is always tricky and I'm sure you will always get a few pages set to hidden that should not have been. If you point to this post (or similar post) in the notification message then a webmaster has enough information and can always unhide the page again.

I applaud this effort vrocks. Services like these can die out easily if there is a lot of "noise" in the database. Keeping it clean (as possible) is always a good idea. Not easy, I'm sure.

[vrocks]

Just fearing for the guy with 120 hidden pages

Then again... Manage Pages can be sorted by headers to highlight 0,0 pages at the top and quickly unhide them.

Just wondering if anyone would ever have a 0,0 page legitimately... It would be a page you would hide because you don't want link... right? Then again, someone might have added a page and nobody ever asked for a link from it... But even then, it must be a low quality page.

[CodeR70]

I dont have that much pages (85) but I do have a few (webcam) landing pages which are 0,x (no internal links, only external). 0,0 does not make much sense to me (besides new pages that have not been scanned yet, as you mentioned). But I'm pretty sure the "120 hidden pages guy" would know…

Can you not run a few queries on the database to see if that guy actually exist?

[vrocks]

315
297
157
153
146
144
130
127
121
119
117
108
101
99
92
91
89
88
82
80
77
72
69
68
65
65
62
62
62
60

There are 1082 users with pages in 0,0 status. These are the bigger ones. The top one is an active user with a lot of domains... He doesn't seem to remove dead domains from his account. One thing that could be useful... Finding dead domains with PR by searching our DB and offering the owner $35 for the domain.

[CodeR70]

That list is not even so bad I think. If the majority of those 1000+ do not have that many pages. I think you can also consider that you will not delete any information. Sure, for some it may be an issue to unhide it again, but it's more an inconvenience (IMHO).

I think if you make a notification (you know, when people login on the homepage) and then point to a thread like this or maybe a new post explaining the situation then you should go for it. People tend to make issues about everything so it would not be a surprise if you will get a few upset users. Still, I think it's a good effort. Would be nice to have seen a little bit more feedback from other users in this thread though.

Anyway… for what it's worth, I think its a good thing if you take this action. No information will be lost. Resolving issues is rather easy by the user just unhiding it.

[vrocks]

That list is not even so bad I think. If the majority of those 1000+ do not have that many pages. I think you can also consider that you will not delete any information. Sure, for some it may be an issue to unhide it again, but it's more an inconvenience (IMHO).

I think if you make a notification (you know, when people login on the homepage) and then point to a thread like this or maybe a new post explaining the situation then you should go for it. People tend to make issues about everything so it would not be a surprise if you will get a few upset users. Still, I think it's a good effort. Would be nice to have seen a little bit more feedback from other users in this thread though.

Anyway… for what it's worth, I think its a good thing if you take this action. No information will be lost. Resolving issues is rather easy by the user just unhiding it.

LOL... When ever I use the admin alert system I get about 25 emails to support asking if it was meant specifically for them. Even after I link to the forum to explain why the alert was sent.

Notice: LinkSpun does not allow the adding of pages to your account whereby the sole purpose of the page is to link to copywrited works on file sharing web sites. If caught you will be banned forever! If you have such a web site contact [email protected] and we will work with you to stop the trades and remove the site from your account and keep you in good standing.

That alert got me about 80 people that it didn't pertain to asking me if I was banning them.

[CodeR70]

LOL, add "and if you ask me if this is only for you…. YESSSS IT IS so STFU…."

But seriously, if you do it quietly then I'm sure you get some users complaining about making changes without notification. You can never to it right

[123anddone]

i think there should be separate checker that would check if that domain have title or it has error and hide it from search result and only owner of that domain ca unhidde from his manage pages. Or other was to check if that domains is live or parked.

so you will avoid of removing domains that are just added and not crawled by your scripts of in out links of being removed.

also there are tons of sites that linkspun cant see inbound outbound links, but those sites are good for trading, using your method they will be gone.