No. In fact, I ran 250,000 pages yesterday on my laptop (50,000, 5 page documents). It took about 1.5 hours without any issues whatsoever. We run such productions all the time, from small jobs like your 5000 pages to as large as 1,000,000+pages. I would suggest that it is not in XEP but within your own threading code. My laptop has Winmasse installed and is configured with 4 XEP processes running on four ports (as my machine is a quad-core CPU and I wanted to maximize the load across the whole CPU). The 4 processes each run in their own JVM with minimal (256MB memory each).
Are you using something like EnMasse (or WinMasse) on windows that already handles load balancing across a bank of XEP Engines? Or is this something you are writing yourself?
We have customers that have been running EnMasse/WinMasse on machines … sometimes for over a year … without doing or touching anything, producing 50,000,000+ pages – no failures, no memory leaks.
We also have VDPMill for very large production runs which has both a splitter and merger. It can split incoming XML files into chunks (for logically structured things like a batch of invoices), spread the rendering across multiple XEP engines running in separate JVMs within cores/CPUs/even different machines, and optionally combine output into a single file when formatting to PDF/PS/AFP. Single print files with over 500,000 pages have been created with this application (which sits on top of EnMasse).
Contact me to discuss how you should integrate something like our own pre-written balancer for such applications.
Kevin Brown
RenderX
From: xep-support-bounces@renderx.com [mailto:xep-support-bounces@renderx.com] On Behalf Of Justin Lipton
Sent: Monday, August 27, 2012 12:58 PM
To: RenderX Community Support List
Subject: [xep-support] OutOfMemory issues with XEP4.19
Hi,
We recently discovered some memory issues with XEP4.19 during load testing.
The test comprised of multiple (about 5000 total renders) concurrent (no more than 5) FO to PDF renders of the same two page input.
The input contains both tables and images.
We found that the memory usage slowly crept up over time a heap dump revealed the following:
71,183,440 (28.34%) [32] 5 class com/speedlegal/genericexport/XEPExport 0xdd55ba10
71,183,408 (28.34%) [16] 1 com/renderx/xep/FormatterImpl 0xde359840
71,183,392 (28.34%) [16] 1 com/renderx/xep/FormatterCore 0xdba87a10
71,183,376 (28.34%) [336] 23 com/renderx/xep/lib/Conf 0xde375fd8
67,949,056 (27.05%) [72] 10 com/renderx/xep/cmp/ImageFactory 0xdb32e790
65,786,552 (26.19%) [32] 1 com/renderx/util/Hashtable 0xdb32e9b8
65,786,520 (26.19%) [1,048,592] 91,000 array of java/lang/Object 0xe2864560
This basically means that our FormatterImpl object was holding into image references which could not be garbage collected. We were using a single FormatterImpl instance across threads to avoid the overhead of creating this object over and over. We did not and could not call the cleanup() method during this testing as we understand this cannot be called during rendering.
We made sure that TMPDIR and the appropriate MEMOIZE settings were in place but this made no difference.
Using a new instance of the FormatterImpl for each render resolved the memory issues.
Questions:
- is this a known issue, bug or limitation?
- is correct use of the FormatterImpl documented somewhere? The forums suggest that FormatterImpl should be used this way but cleanup should be called from time to time. Perhaps cleanup() could block if a render is in progress
Regards,
Justin,
-- Regards, Justin Lipton <http://exari.com/Exari/media/Exari-Resources/Email%20Signature/exari-logo.png> Email: justin.lipton@exari.com Office: +613 9621 2775 Mobile: +614 0958 9943 Demo <http://www.exari.com/demo-trial.html> Blog <http://blog.exari.com/> Twitter <http://www.twitter.com/exari> LinkedIn <http://www.linkedin.com/companies/exari-systems> Boston | London | Melbourne | Munich This e-mail message is transmitted to you by Exari Group, Inc.. This e-mail contains information which may be confidential and legally privileged, and is for the exclusive use of the intended recipient. If you are not the intended recipient, any disclosure, copying, distribution or use of this information is prohibited. If you have received this e-mail in error, please notify us immediately by telephone +1 617.938.3777 or by e-mail and then delete the message from your computer. !DSPAM:87,503bd15263731548410865!
_______________________________________________
(*) To unsubscribe, please visit http://lists.renderx.com/mailman/options/xep-support
(*) By using the Service, you expressly agree to these Terms of Service http://w
ww.renderx.com/terms-of-service.html
Received on Mon Aug 27 13:35:53 2012
This archive was generated by hypermail 2.1.8 : Mon Aug 27 2012 - 13:35:58 PDT
