firewall Spyware removal internet speed test web security service free software and tips


Go Back   Web Security > Our Free Tools > Webmaster Tool - Sitemap Generator
FAQ Members List Search Today's Posts Mark Forums Read

Webmaster Tool - Sitemap Generator Questions about building a sitemap, including the XML Sitemap, HTML sitemap, Google and Yahoo! Sitemaps.

Advertisements

JAVA memory error

Reply
 
LinkBack Thread Tools Display Modes
  #1 (permalink)  
Old 01-24-2008,
Junior Member
 
Join Date: Jan 2008
Posts: 3
Default JAVA memory error

It seems as if several individuals are having memory issues. My sitemap generator used to work fine. Now on my PC's I get a "low memory error" when generating a large sitemap.

I can increase the java memory from the java control panal up to -Xmx185m but still get the error. If I go any higher than that I get a java environment error when I start the sitemap generator. What can I do. This happens on all 3 of my home pc's.
Reply With Quote
  #2 (permalink)  
Old 01-25-2008,
AMPC's Avatar
Administrator
 
Join Date: Jan 2007
Posts: 1,415
Default Sitemap memory

I have no other solutions expect for increasing memory or shutting down programs that are using memory in the background. The larger the sitemap, the more memory needed, but for most site, the standard works fine.

I'm working on a solution that does not tie memory to sitemap size, but it's going slow...

Regards,

Jim.
Reply With Quote
  #3 (permalink)  
Old 02-08-2008,
Junior Member
 
Join Date: Jan 2008
Posts: 3
Default

Quote:
Originally Posted by rlhs76 View Post
It seems as if several individuals are having memory issues. My sitemap generator used to work fine. Now on my PC's I get a "low memory error" when generating a large sitemap.

I can increase the java memory from the java control panal up to -Xmx185m but still get the error. If I go any higher than that I get a java environment error when I start the sitemap generator. What can I do. This happens on all 3 of my home pc's.
I'm still getting this error. Don't really understand. The system has 1Gb ram but once again when I go to the java console and enter a value of -Xmx190m or higher when I load the sitemap generator it says it cannot start the java environment.

Does anyone else have this issue or a suggestion of what to do.
Reply With Quote
  #4 (permalink)  
Old 11-25-2008,
GJM GJM is offline
Junior Member
 
Join Date: Nov 2008
Posts: 1
Unhappy Sitemap Generator peaks out???

While massively impressed with the sitemap generator we are very frustrated by the fact that when we get to over 40,000 URLs things just slow right down and grind to a halt. It seems unable to handle this volume, yet we have seen comments about adjsting the JAVA memory etc and the ability to hanlde 100,000 urls... and that the matter was also being addressed back in early 2007. We have adjusted the Javav settings, and tried doinhg the excercise on a wide variety of machines... some very well geared up, others simply office workhorses. We have cleared the Java cache, tried all sorts of settings on the Sitemap Generator... but just cannot max out. We stop and start it, it moves on a bit... crawls, then comes to a halt. Eventually the highest figure we have reached is about 55,000. We know we have north of 90,000... so have we reached the dizzy heights of the Generators abilities. We have spend dozens of hours and stuck with this Sitemap Generator in the belief that if we can get there, this is a GREAT sitemap tool.

CAN ANYONE HELP US.... WE ARE BURNING THE CANDLE AT ALL THREE ENDS AND RUNNING OUT OF IDEAS?????
Reply With Quote
  #5 (permalink)  
Old 11-25-2008,
AMPC's Avatar
Administrator
 
Join Date: Jan 2007
Posts: 1,415
Default Sitemap generation stops at 55,000 urls.

Send me your url and I'll take a look - perhaps there are files that you can filter out to bring the number down, etc. (you can right click on AMPC and send me a private message if you like).

That's all I offer - the number of urls depends on the complexity and link structure of the site.

Regards,

Jim.
Reply With Quote
  #6 (permalink)  
Old 12-03-2008,
AMPC's Avatar
Administrator
 
Join Date: Jan 2007
Posts: 1,415
Default Sitemap and number of pages in a website

I checked out your url with the goal of trying to find items that could be excluded from the sitemap, such as images, search pages, duplicates, etc.

Fortunately (or unfortunately for sitemap generation), I could not find anything. I believe your site goes beyond the limits of my generators capacity.

I plan on releasing a new version today, but I doubt that it's going to do much more for you than what it's doing now as the changes are not deep indexing related.

I wish I could give you a number and say that the generator taps out at X amount (I wish I could say it never taps out : ), but how the site is designed (complexity) has everything to do with how many pages get spidered.

Wish I could offer more! BTW - you have a great site and I'm 100% confident you'll be doing well if you're not already

Best regards,

Jim.
Reply With Quote
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off



All times are GMT -5. The time now is .


Powered by a CPU
vBulletin® v3.8.4, Copyright ©2000-2009, Jelsoft Enterprises Ltd.
LinkBacks Enabled by vBSEO 3.3.2 © 2009, Crawlability, Inc.