Grid PDF export aborts, EOutOfMemory

VCL TMS VCL UI Pack
1

Hi all,


I have a series of automated tests which worked so far, now I fail on export of one test file, a PDF taken from a string grid. Just minutes before in the test sequence exports to PDF from other grids are working, just this one constantly fails: EOutOfMemory, yet there should be plenty of memory according to task manager. The "failing" grid is correctly exported as HTML, CSV, Excel, etc., just PDF fails.

The size of the grid is 7 columns time ~2,800 rows, abort happens in the main loop in TAdvCustomGridPDFIO.DOPDFGridExport (AdvGridPDFIO.pas), exact line I could not make out so far (looping through too often, might change the code and track it later).

Anyone experienced a similar problem, or have an idea what this could be? Will try to collect more details over the weekend where exactly this fails, some guidance appreciated.

Cheers, and a nice weekend,
Matthias
2
Was able to track it further down, though now I am getting lost in tracking. In AdvGridPDFIO.pas I break on "(r>=1488) and (c=2)" for row and column index when entering line 1575: DrawCell(Self, ...), call stack follows:

System                    4111 @NewAnsiString

System                    4111 @LStrSetLength
AdvPDFLib.General.Win      647 TAdvGeneralPDFLibFontInitializer.InternalInitializeCharWidths
AdvPDFLib.General.Win      311 TAdvGeneralPDFLibFontInitializer.InitializeCharWidths
AdvPDFLib.General         3166 TAdvGeneralPDFLib.UpdateFontList
AdvPDFLib.General         1116 TAdvGeneralPDFLib.NotifyUnicode
AdvPDFCoreLibBase         2626 TAdvPDFGraphicsLibOutputWriter.NotifyUnicode
AdvPDFGraphicsLib.General  814 TAdvGeneralPDFGraphicsLib.DrawText
System                    4111 @IntfCopy
AdvPDFGraphicsLib          289 TAdvCustomPDFGraphicsLib.GetPDFGraphicsLib
AdvGridPDFIO               632 TAdvCustomGridPDFIO.DrawCell
AdvGridPDFIO              1575 TAdvCustomGridPDFIO.DoPDFGridExport
AdvGridPDFIO              1278 TAdvCustomGridPDFIO.DoPDFExport
AdvPDFIO                   501 TAdvCustomPDFIO.Save
AdvPDFIO                   248 TAdvCustomPDFIO.Save
AdvPDFIO                   521 TAdvCustomPDFIO.Save
Dlg_BatchScan              514 TDLG_BatchScanForm.ACTITEM_ExportGridExecute

Feeling lost now, thanks in advance for any advice.
3

... a final note for today: right before entering TAdvCustomPDFIO.Save, I am at 101MB memory consumption. When aborting, task manager shows 1.5GB consumed... saving to PDF from grid is quite memory intensive?


I checked with one of the other folders I scan in batch and which do not fail, here it goes up from 50MB at start to 380MB when saving, back to 53MB when done. Was tracking memory leaks, the application does not leak...
4

Issue "solved": when creating a PDF from grid, the application becomes a memory hog. The test case I had was the 32bit version, changed to 64bit today and the auto-test went through. 2.4GB memory peak to create the 46 pages, 4.5MB PDF. This quickly limits the size and complexity of reports created from a grid.


Question: is there a way to reduce the memory required? Can the PDF be created on the fly? I fear it is rendered in memory before the file is written.

Any help appreciated, I time to time require to create reports larger than this, so I also might break on the 64bit version.
5

Hi, 


The PDF is kept in memory and is then written once to a file. We'll investigate if we can flush between pages to improve memory usage
6

Hi Pieter,

 flushing pages on the fly would be fantastic, thanks for the investigations planned.
7

Hi, 


We have tested this with our own grid export demo by changing the columncount to 7 and rowcount to 2800 and could not see an issue with performance. The 117 Page PDF is generated in a matter of seconds and memory consumption does not exceed 150 MB. Even when using our PDF Lib general demo and generating 1000 page PDF full of HTML special text, images an graphics, the maximum memory usage is 400 MB. 

In order to know exactly what is going wrong in your application, please provide a sample that demonstrates this high memory consumption so we can investigate what is going wrong.
8

Hi Pieter,


thanks for investigating. I looked into extracting the section from my project, yet that renders quite difficult due to heavy use of structures and in-depth integration in the overall large application.

 What I did instead is export a suspicious grid to a BIN file from that project, then create a small application which loads the grid and exports it to a PDF. This generates a 72 pages PDF (1.1 MB) when running 64bit, in 32bit it crashes with an out of memory exception, 64bit has high memory consumption. The BIN file from that grid is 20MB.

The BIN plus dfm+pas and the PDF created I zipped into a 1.3MB archive, please let me know how to get that package to you.

Thanks again,
Matthias
9

Hi, 


Thank you for further investigating this. You can send to project to support@tmssoftware.com with a link to this forum post. We'll investigate as soon as possible.
10

TMS Support >very< quickly came back with a result: lots of cells in the grid are filled from files where for double byte character content the appropriate font is not installed locally (the app is a log scanner, the grid shows summary of issues found from any installation around the world, these produce local output).


With fonts missing, PDF export gets into some looping to get the display correctly it seems. This is eating up the memory, in case of my 32bit example above the 2GB limit, 64bit version can handle the increased RAM requirement.

For comparison: I created a test bed with all locales working locally, took 4 times the number of files and producing a PDF that as a result is also num pages times four. In this clean setup the PDF creation consumes under 20MB...

TMS Software: thanks for your help!