Description: Memory usage goes through the roof saving document with 120k of comments Steps to Reproduce: 1. Open attachment 156113 [details] bug 128402 2. Insert a new sheet 3. Press Save Actual Results: At least 6 GB of memory usage Expected Results: Something more reasonable for a 1 MB file Reproducible: Always User Profile Reset: No Additional Info: Version: 7.0.0.0.alpha0+ (x64) Build ID: 7ae9c9572ccac55c0926b8a9779bb63c4236291c CPU threads: 4; OS: Windows 6.3 Build 9600; UI render: Skia/Raster; VCL: win; Locale: nl-NL (nl_NL); UI-Language: en-US Calc: CL
opening process takes 100% CPU. saving process takes 100% CPU, over 10 minutes of time and over 7,5Gb of memory, LO frozen and I killed it Version: 7.0.0.0.alpha0+ Build ID: 6c59c9d2b8818674640a50656ffba90f9cd3900e CPU threads: 4; OS: Mac OS X 10.15.4; UI render: default; VCL: osx; Locale: ru-RU (ru_RU.UTF-8); UI-Language: en-US Calc: threaded the same problem in 6.0 on macOS Julien, can you make your perfgraphs here for three cases with that file from description: 1. File opening 2. A new sheet adding (I saw 100% CPU in that case too) 3. File saving
(In reply to Roman Kuznetsov from comment #1) > opening process takes 100% CPU. > saving process takes 100% CPU, over 10 minutes of time and over 7,5Gb of > memory, LO frozen and I killed it > > Version: 7.0.0.0.alpha0+ > Build ID: 6c59c9d2b8818674640a50656ffba90f9cd3900e > CPU threads: 4; OS: Mac OS X 10.15.4; UI render: default; VCL: osx; > Locale: ru-RU (ru_RU.UTF-8); UI-Language: en-US > Calc: threaded > > the same problem in 6.0 on macOS > > Julien, can you make your perfgraphs here for three cases with that file > from description: > > 1. File opening > 2. A new sheet adding (I saw 100% CPU in that case too) > 3. File saving Not sure which part is covered by bug 76324 and bug 125619
See also bug 114377 and bug 131672 The issue: multiple issues stacked onto each other. * File opening/saving speed with comments should be a parsing problem * Memory usage on save with lots of comments probably a symptom * Slowdown after save bug 125619 * Insert a new sheet; slow. Probably around for a while. Not specific to comments as far i know * And we have bug 131709, which you can run into testing something else
Created attachment 159947 [details] perf flamegraph (when opening the file) Here's a Flamegraph retrieved on pc Debian x86-64 with master sources updated today. It concerns only the file opening part.
Created attachment 159948 [details] Flamegraph (when adding a new sheet) Remark: no delay here to add a sheet.
Created attachment 159949 [details] Flamegraph (when file saving)
Noel, could you look at it please?
Hello @Telesto, Roman and Julien, thanks for taking care for this problem, it is quite old and often discussed, see bugs: 131675 131672 129228 128402 127758 125619 125545 124692 123418 119650 119636 119075 114377 113599 106433 106385 105888 105499 97698 88194 76324 60418 56268 54563 54018 34519 and some more ... it is one of the bigger general problems besides - copy and paste, - autocalc / shared formulas / threading / openCL, - iterative calculations, and - floating point precision, that are probably difficult to solve. 'comments' is linked to 'copy/paste' somewhere in the drawing layer, where things have been specially arranged to allow clipboard pasting even if the source file has been closed in the meantime? Noel has already taken care of this problem and in https://bugs.documentfoundation.org/show_bug.cgi?id=119650#c11 he said: 'this is what I could do...', and pinpoints the problem. best progress I know: 6.2.7.1 and 6.2.8.2 - win - are comparatively good, catchwords I picked up: 'non linear transformation between grid and display', 'non linear transformation between grid and pointing device - accelerated moves', drawing layer (memory?) is registered by some obscure stuff, 'layout for captions / annotations / notes / comments / tracking notes is done three times', A suspicion I have, some things in calc are processed 'from bottom to top, then right to left', but the comments are stored in the files (and in memory?) from left to right, then top to bottom ... an iteration 'calculate the space for comment 10 - you need the space for comment 9 - calculate the space for comment 9 - you need 8 ... need 1 ... then coming to 9, again calculating 8 ...1, and so on ... would lead to the fact that the calculation time would increase with every further comment (don't know how many times) ... but such a thing increases unneccessarily ... [SUM(n = 1 to k+1: n*(n+1)/2)) instead of k+1 calculations for comment k+1??? observations look more like n*(n+1)/2]
@b. Thanks for reference.. this is duplicate of bug 119650; @Roman, your choice if you wanna open a new bug for file-opening.. @b. Part 2. Thanks for enthusiasm & reports A) Some advise: don't 'hijack' each and every Calc bug report, to mention the bigger general problems or note that 6.2.7.1 and 6.2.8.2 are working pretty good. Please keep it a bit on topic B) Don't write long a life story, but keep it as short as possible. Yes, there are quite a number of fundamental issues in Calc [be aware, Writer has surely the same amount of them]. Please do report them with Steps to reproduce. However, one bug in each report. Only the steps needed; not a story about OpenCL, multithreading.. And preferably not reposting a known bug (however, can happen sometimes :-). I looked at bug 131339 & bug 124577 comment 10. Both are regressions which can be bibisected. *** This bug has been marked as a duplicate of bug 119650 ***
I've closed bug #119650, as that one seemed more about speed than memory and I've improved the speed, but I doubt much has changed for memory, so I'm reopening this one. This appears to be a regression from https://cgit.freedesktop.org/libreoffice/core/commit/?id=018500a73f3b1082b6662b7c123dfe5158ae5752 .