|
|
Rank: Advanced Member
Groups: Member
Joined: 12/11/2007
Posts: 89
Points: 267
|
I'm having problems with the google bot - seems it doesn't like too many redirects and so I have tried to alter the PageHandler class by putting in a status 301 Moved Permanently: Code:
public void ProcessRequest(HttpContext context)
{
context.Response.Status = "301 Moved Permanently";
string rawUrl = context.Request.RawUrl;
log.Info("Starting request for " + rawUrl);
DateTime startTime = DateTime.Now;
// Rewrite url
UrlRewriter urlRewriter = new UrlRewriter(context);
string rewrittenUrl = urlRewriter.RewriteUrl(rawUrl);
// Obtain the handler for the current page
string aspxPagePath = rewrittenUrl.Substring(0, rewrittenUrl.IndexOf(".aspx") + 5);
IHttpHandler handler = PageParser.GetCompiledPageInstance(aspxPagePath, null, context);
// Process the page just like any other aspx page
handler.ProcessRequest(context);
// Release loaded modules. These modules are added to the HttpContext.Items collection by the ModuleLoader.
ReleaseModules();
// Log duration
TimeSpan duration = DateTime.Now - startTime;
log.Info(String.Format("Request finshed. Total duration: {0} ms.", duration.Milliseconds));
}
I have resubmitted the google sitemap.xml to see if it improves, but I think I am getting mixed up as this causes more errors.
Web Developer, Kent - Coolbytes
|
|
Rank: Newbie
Groups: Member
Joined: 5/19/2008
Posts: 6
Points: 18
|
Hi, Cuyahoga doesn't send redirects to the browser for url rewriting so I don't think you're on the right track with this. I'm pretty sure that setting the response status to 301 is only going to make things worse. Not sure if it's the same problem but we've also had issues with the bots. If you make a request with the googlebot agent string (https://addons.mozilla.org/en-US/firefox/addon/59) to a rewritten page and get a "Cannot use a leading .. to exit above the top directory." error then it can be solved by adding the following to web.config ... Quote:{authentication mode="Forms"}
{forms name="CuyahogaUser" loginUrl="Login.aspx" protection="All" timeout="600" path="/" cookieless="UseCookies" /}
{/authentication} Hope this helps, Ian
|
|
Rank: Newbie
Groups: Member
Joined: 5/19/2008
Posts: 6
Points: 18
|
oops - just read you previous posts. can't help you with this, i'm afraid. for the record, i still don't think setting the response status to 301 for every page is a good idea 
|
|
Rank: Advanced Member
Groups: Member
Joined: 12/11/2007
Posts: 89
Points: 267
|
no that's fine. Thanks for your help anyway. I did give up on that 301 thing. will see what else can be done.
Web Developer, Kent - Coolbytes
|
|
Rank: Advanced Member
Groups: Member
Joined: 12/11/2007
Posts: 89
Points: 267
|
I have altered the web.config file in the past so that forms tag includes: cookieless="UseCookies" as I think this is causing redirect errors for Google. However, I did a header check for my home page and I could see that a cookie is being set: Code:Set-Cookie:ASP.NET_SessionId=bip41v45efftrczxd5mkzb55;path=;HttpOnly could this be causing Google problems? I tried changing the sessionstate tag to cookieless = true this as well: But this caused the urls in the browser problems. Anyone have an ideas how to stop Google failing on redirects? Seems all my Cuyahoga sites have problems at the moment. I'm using Cuyahoga version 1.5.0 if that's any use? Looking at the 1.5.1 release notes, there is a section that says this: Quote:Fixed possible issues with url rewriting when Cuyahoga runs at the root
web site. (CUY-135) Is it worth upgrading?
Web Developer, Kent - Coolbytes
|
|
Rank: Advanced Member
Groups: Member
Joined: 12/11/2007
Posts: 89
Points: 267
|
Just wondering if anyone else is getting redirect errors in Google Web Master Tools for pages on their site? Below I have pasted a screenshot of my Tools page for the website www.coolbytes.co.uk. The circled links below definately exist, yet google has problems.  Any suggestions?
Web Developer, Kent - Coolbytes
|
|
Rank: Advanced Member
Groups: Member
Joined: 12/11/2007
Posts: 89
Points: 267
|
Hmm, well, I checked all my pages in Fiddler - no problems. All returned the correct header response 200. So scoured the net for similar problems and found this: http://www.google.com/support/forum/p/Webmasters/thread?tid=202f246e3f7a1d1e&hl=enAlso, it appears that they recomment putting the final URL in your google sitemap xml file, not the redirect friendly url's: http://www.google.com/support/forum/p/Webmasters/thread?tid=6b1f430954c93af1&hl=enI'll turn error logging on, record the user agent and raw url from the Error.aspx page and see what comes up. Will keep you posted....
Web Developer, Kent - Coolbytes
|
|
Rank: Advanced Member
Groups: Member
Joined: 12/11/2007
Posts: 89
Points: 267
|
Still getting redirect errors event on final destination urls!! e.g. http://www.blahblah.co.uk?default.aspx?ShortDescription=my-page What on earth could be causing google to have problems with that??
Web Developer, Kent - Coolbytes
|
|
Rank: Administration
Groups: Administration
, Member
Joined: 12/30/2004
Posts: 1,674
Points: 1,824
Location: Wageningen (NL)
|
No idea. The cookieless="UseCookies" attribute in Web.config worked for me.
|
|
Rank: Advanced Member
Groups: Member
Joined: 12/11/2007
Posts: 89
Points: 267
|
I've just had another lot of errors which makes me feel it is nothing to do with cuyahoga, or the cookieless attribute in web.config. I have new redirect errors (in Webmaster tools) on plain html files ! It must be something to do with googlebot, or my server or some other thing that's happening...but not related to the CMS or asp.net in general. I feel I have wasted everyone's time here! My apologies for that. I'll look into it further and come back to tell you what it was.
Web Developer, Kent - Coolbytes
|
|
Rank: Administration
Groups: Administration
, Member
Joined: 10/7/2008
Posts: 505
Points: 1,515
|
There is an issue with ASP.NET HTML HTML32TextWriter from what I can make out is something like this. When someone requests a page from the server hosting your site it is processed and when it comes to be dumped to the screen a specific version of HTML TextWriter is chosen to render the mark-up, based upon the browser requesting it (and it capabilities). The HTML32TextWriter renders HTML equivalent to a version 3 browsers capability. The HTML32TextWriter has a bug (that was fixed in ASP.NET 3.5 sp1). The bug involves RewritePath method to grab the sessionid from the URL. To actually get this bug would mean using a browser of such antiquity (user agent is of a 'version 3 browser', cookies turned off etc...) that the server uses the HTML32TextWriter. The search engine robots did exactly this.. Not sure this is the same issue, but I found it interesting. The advice to fix was to force cookies in web.config form authentication Use a '.browser' file Upgrade environment to ASP.NET 3.5 sp1 http://www.polymorphicpodcast.com/shows/aspnetseo/
|
|
Rank: Advanced Member
Groups: Member
Joined: 12/11/2007
Posts: 89
Points: 267
|
Thanks Constructor. I have uploaded a new .browser file. Have resubmitted sitemap xml file. I will edit this post when I get any results either way.
Web Developer, Kent - Coolbytes
|
|
Rank: Advanced Member
Groups: Member
Joined: 12/11/2007
Posts: 89
Points: 267
|
THAT WAS IT!!!!! So it was down to the Html32TextWriter bug in ASP.NET 2.0 on IIS 6. I installed the UserAgent Switcher in Firefox and went through the site. Occasionally getting the aspxerror!! It turns out I just hadn't waited long enough for it to happen. I then checked to see what the error was on the server in the event log: Quote:
Exception information:
Exception type: HttpException
Exception message: Cannot use a leading .. to exit above the top directory.
Thread information:
Thread ID: 1
Thread account name: WHVPS5106\IWAM_plesk(default)
Is impersonating: False
Stack trace: at System.Web.Util.UrlPath.ReduceVirtualPath(String path)
at System.Web.Util.UrlPath.Reduce(String path)
at System.Web.Util.UrlPath.Combine(String appPath, String basepath, String relative)
at System.Web.HttpResponse.ApplyAppPathModifier(String virtualPath)
at System.Web.UI.HtmlControls.HtmlForm.GetActionAttribute()
at System.Web.UI.HtmlControls.HtmlForm.RenderAttributes(HtmlTextWriter writer)
at System.Web.UI.HtmlControls.HtmlControl.RenderBeginTag(HtmlTextWriter writer)
at System.Web.UI.HtmlControls.HtmlForm.Render(HtmlTextWriter output)
at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)
at System.Web.UI.Control.RenderControl(HtmlTextWriter writer, ControlAdapter adapter)
at System.Web.UI.HtmlControls.HtmlForm.RenderControl(HtmlTextWriter writer)
at ASP.templates_coolbytes2008_inner_ascx.__Render__control1(HtmlTextWriter __w, Control parameterContainer)
at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)
at System.Web.UI.Control.RenderChildren(HtmlTextWriter writer)
at System.Web.UI.Control.Render(HtmlTextWriter writer)
at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)
at System.Web.UI.Control.RenderControl(HtmlTextWriter writer, ControlAdapter adapter)
at System.Web.UI.Control.RenderControl(HtmlTextWriter writer)
at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)
at System.Web.UI.Control.RenderChildren(HtmlTextWriter writer)
at System.Web.UI.Page.Render(HtmlTextWriter writer)
at Cuyahoga.Web.UI.PageEngine.Render(HtmlTextWriter writer)
at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)
at System.Web.UI.Control.RenderControl(HtmlTextWriter writer, ControlAdapter adapter)
at System.Web.UI.Control.RenderControl(HtmlTextWriter writer)
at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
So I tested the user agent in fiddler over and over again trying all of the links and now and again, the 302 error ocurred: User-Agent: Mozilla/5.0 (compatible; Googlebot/2.1; http://www.google.com/bot.html) So, I created the .browser file as suggested by Constructor which will look something like this: http://todotnet.com/archive/0001/01/01/7472.aspxUploaded the file and tested the site thoroughly through Firefox switcher agent and fiddler, trying all of the user agent strings and not one single 302 so far. I removed the .browser file to make sure - 302 errors every so often. Put it back up - nothing! Now I can sleep easy....
Web Developer, Kent - Coolbytes
|
|
Rank: Administration
Groups: Administration
, Member
Joined: 10/7/2008
Posts: 505
Points: 1,515
|
Glad to be of help.
|
|
|
Guest |