GOOGLEBOT WON'T GO HOMEI have 'Googlebot' crawl my site every day like a dispossessed spirit that
can't leave.
It was not always like this, I would go for a month or more before he came to
my site and then would only crawl a few pages and leave again.
This has been going on now for two months or more, I've been watching the
stats on my server everyday seeing just where he goes. He will crawl a few pages
one day, and the next some other pages, and then finally leave until the morrow.
After just three or four weeks, I noticed I got an infestation of lesser bots
calling as well, some I have never heard of before, they seem to follow
'Googlebot' as though he is the Pied Piper of Hamelin.
What caused this infestation of bots?
Well, I got interested in rss feeds, I thought "What a good tool for drawing
traffic to my site."
So I studied up on rss feeds until I understood what was involved. Now I am
no 'techie', and the most programming I do is html in 'Notepad', making my own
web pages.
Rss feeds use a similar language called xml, so it was easy to understand for
me, and I was able to quickly make up an rss feed page in 'Notepad', placed it
on my server, downloaded a 'feed reader' software, opened it up, placed the
address of my feed in the 'feed reader', and up came my rss feed in the reader.
It was so easy I wrote an ebook to tell everyone how to do it, it's called
"Really Simple RSS".
There are many rss directories and syndication services that I submitted the
address of my rss feed to, so other people can look at the feed in their 'feed
reader' and see what articles I am posting.
Some of the 'subscribers' to the feed actually come and visit my site to see
what it's all about, which happened almost straight away after I submitted my
feed.
In less than 36 hours, 'Googlebot' found my feed!
In a little over two days later Yahoo's 'Slurp' bot found my feed, and from
there on they have both been crawling my site with the lesser bots every single
day.
They crawl everything, they crawl all my pdf ebooks, they look at a few pages
in 'Webmasters Corner', they even look at my 'Bonus Reports', and even descend
the staircase into my 'Secret Vault'!
When will they go home and stop prying into my little secrets?
I know I encourage them like leaving cheese for a mouse, by putting more pdf
files, software and ebooks on my site every day, so I suppose the poor bots
can't ever keep up!
I found out that like mice like cheese, that 'Googlebot' likes my pdf files,
he has been fitted with digital spectacles so he is now able to actually read
keyword rich pdf files all the way through to the end! So he likes pdf files.
So I have obliged him!
I put 85 pdf files up on my server one night just to keep him happily
reading! I have put many more up on the server since, and will continue to
create more pdf files of classic authors for free download, just to keep him
calling with his 'friends'.
Isn't this a laugh?
It's like taking in a stray dog, feeding him, housing him, and the dog losing
the ability to wander off again and finding you have got a lodger for life.
My site's ranking has risen considerably both in Yahoo and Google, but for
some reason that escapes me for the time being, my ranking rose more quickly in
Yahoo and they rank it much higher than Google. I may well find the reason for
this before long and put it in another article.
So if you are a web site owner struggling to get noticed by the web bots,
don't put it off much longer, put up an rss feed with articles about your
products or service and not only will 'Googlebot' call with his 'friends', but
actual live people will call too.
About the Author
Tony Dean runs a web site selling ebooks and software
at http://www.ebook-sales.com
|