Re: Search Engine Bots Generating Strange Queries

I'm totally no expert on this, but I'd guess that the bots are simply
trying to walk the tree.
If "http://mysite.com/directory/subdirectory/subsubdirectory" is
valid, then "http://mysite.com/directory/subdirectory", "http://mysite.com/directory
" and "http://mysite.com" are probably also valid. The GOOG doesn't
know that those directories don't actually exist. In "classic" web
development patterns there should be an index.htm file in each of
these directories, so it can't hurt to look for them.

BTW: Safari (and possibly other browsers as well) allow you to right-
click on the title bar and offer the same kind of "URL shortening
shortcuts" in a popup menu.

On 30 Oct 2008, at 15:02, MikeK wrote:

>
> In a general CMS app written in CakePHP I am noticing in my logs
> invalid queries being generated by various search engine bots
> including Google, Inktomi, and Yahoo.
>
> What I'm wondering is WHY?
>
> For example they are requesting
>
> http://mysite.com/controller/view instead of the correct
>
> http://mysite.com/controller/view/34 (ex: id 34)
>
> Nowhere on my site do I publish any links to /controllers/view without
> an id parm
>
> This is driving me slightly nuts. Why would a bot request a URI it has
> never seen?
>
> My validation code that checks for valid requests logs these
> occurences and every day I puzzle over my logs and examine the emitted
> web page source wondering where or why they are requesting these
> invalid URIs. I've been dumping $_SERVER and no clues there either.
> The referer is always '/'.
>
>
> >

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "CakePHP" group.
To post to this group, send email to cake-php@googlegroups.com
To unsubscribe from this group, send email to cake-php+unsubscribe@googlegroups.com
For more options, visit this group at http://groups.google.com/group/cake-php?hl=en
-~----------~----~----~----~------~----~------~--~---