User Tag List

Informational! Informational!:  0
Likes Likes:  0
Results 1 to 3 of 3

Thread: Yahoo! Slurp Spider, Google Spider, Google AdSense Spider

  1. #1
    Michael's Avatar
    Join Date
    Dec 2004
    Location
    Anchorage, Alaska USA
    Posts
    124
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Yahoo! Slurp Spider, Google Spider, Google AdSense Spider

    Hi--I was looking at who was on line just a bit ago and noticed that in addition to me, and a couple members, there were a number of other non-members that included:

    13 Yahoo! Slurp Spider's, 1 Google Spider, 1 Google AdSense Spider

    Forgive my ignorance--are these really users, or some sort of search engine "bot" (I think that's the term I've heard used) scanning all of the posts?

    Is this something to worry about? Curiously, Michael
    Fishing: A Perpetual Series of Occasions for Hope....

  2. #2
    Hello, I must be going... Not a Number's Avatar
    Join Date
    Nov 2006
    Location
    Los Angeles, CA
    Posts
    7,506
    Mentioned
    5 Post(s)
    Tagged
    0 Thread(s)
    Those are indeed search engine bots crawling through the postings. That's why when you do a Google search on some CP related phrase you often get hits that link to posts on this board.

    This board is also using Google's AdSense as you can see from the banners on top. Ever notice how the banner ads often are related to the topics in the current post or thread? For example you could be reading a thread about orchids and the ads will be orchid related.

    There's a search bot opt-out directive that can be added to the .htaccess file on Apache webservers but if the bot ignores this directive there's little you can do to block them. Since TerraForums is using AdSense blocking this is not really an option.

  3. #3
    Michael's Avatar
    Join Date
    Dec 2004
    Location
    Anchorage, Alaska USA
    Posts
    124
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Thanks Not a Number--I had wondered about how the ad's always matched--here and elsewhere, just hadn't quite worked out the technology of it all. That and I recently read a Wired article about 'bot' attacks... So, now I'm seeing bad-bot's everywhere...bot's in the shadows...maybe even a few bot's in the belfry's?
    Sorry--couldn't resist. At any rate, just a heightened awareness, that needs a little fine tuning so I thought I'd check...

    Happy Trails, Mike
    Fishing: A Perpetual Series of Occasions for Hope....

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •