Short update about the performance testing

I recently wrote a blog post about the performance testing in LibreOffice. This post will give a short update and show how our performance testing has helped to find a number of performance problems in LibreOffice Calc.

If we look at the performance testing page for the calc tests we can see some interesting graphs. To explain the changes that we see we will concentrate on the sum_numbers_column test case.


As can be seen in the screenshot we have 7 interesting points. The rest is just the normal variation in the test results that we have to ignore. We will look through each of them as they show different aspects of the performance testing.

The first and most obvious point is the huge jump at point 1 from about 10 million to about 1.2 billion IR cycles. That looks like a horrible performance regression as it seems that we have a 120x performance regression. Looking at the commit range however shows that I made a mistake during review and we changed the test file while adding a new test.

This shows that you should never change the test document used for your performance test as it leads to results that are not comparable anymore. As a general rule it is safer and simpler to use one test document per test case and treat the test files as immutable.

The second spike from 1.2 billion to 1.4 billion happened after switching the memory allocator from our internal allocator to the system allocator. This was done as part of a test to see if we still need our internal allocator or if we could use the system allocator. As it is obvious from looking at the graph our internal memory allocator seems to perform quite a bit better than the system allocator so we switched back to the internal allocator in point 4.

On the positive side while inspecting the first issue I noticed that about 60% of our calls are into the configuration layer reading the OpenCL setting. Luckily after talking to Michael, Tor started looking into it and came up with a simple fix for this problem. This explains the big drop from 1.4 billion to 460 million IR cycles (number 3). The quick spike (number 5 and 6) is the follow-up commit that tried to improve that patch and the following revert after it was noticed that this introduced a performance problem.

The last small drop at number 7 of about 8% is a simple fix for unnecessary memory allocations on the heap that I noticed while looking through the callgrind logs.

I hope that this story shows how useful our in-tree performance tests are and how they help us discover existing performance problems and prevent performance regressions. So the blog post shows how we fixed a serious performance problem, used the performance testing to check the performance impact of a change and how the performance tests help us prevent performance regressions.

Hopefully this motivates more people to add in-tree performance tests to LibreOffice. A special thank goes to Łukasz Hryniuk who has been adding a number of calc performance tests recently.

Posted in Libreoffice | Tagged , , | Leave a comment

Performance tests are now executed regularly in LibreOffice

Someone following the LibreOffice development closely might know that we have actually two types of performance tests. I’ll will talk mostly about the newer in-tree tests but will also give a short explanation of the out-of-tree tests to explain the difference and when to use which of the two.

Out-of-tree performance tests

The out-of-tree tests have been written by Matúš Kukan and Michael Meeks quite some time ago and they have been running regularly on a LibreOffice server. Sadly until now they have not been as visible as necessary to alert us of newly introduced regressions. Thanks to some awesome work by Norbert we have now a nice website that shows the results of all performance tests.

Technically these tests are taking a full LibreOffice (built with optimization and symbols) and execute it under callgrind. We do this for import and export of some sample documents and later analyse the generated callgrind log files.

This allows to quickly generate numbers for import and export tasks. However it always includes a full LibreOffice start-up and shut down. Additionally it tests a whole bunch of unrelated parts of the import and export and makes it nearly impossible to test the performance of any dynamic operation. The “new” tests (they are already more than half a year old) help to solve a number of these problems.


In-tree performance tests

Matúš together with Laurent Godard added another kind of performance tests to solve some of the limitations mentioned above. They can be run in our build as part of make perfcheck and use the LibreOffice testing framework.

Instead of testing a whole LibreOffice start-up, import, export, shut down cycle under callgrind these tests just test some calls that you annotate in the code. It allows us to quickly write tests that just measure the performance of the small part of code that we are really interested in. It also allows us to test more or less user input by calling the methods, that the user input would call, directly in the test. Obviously this gives us more flexibility in testing the performance of LibreOffice and opens the possibility to find performance regressions in code outside of the import and export filters.

These tests in itself are sadly not that useful unless they are executed regularly and the results are easily accessible. Norbert has been working recently on making these results more visible and run them regularly on TDF hardware. You can find the latest results now also at together with the out-of-tree tests.

To show you how easy it is to add a new performance test I’ll document here the steps necessary for adding a test case to sc/qa/perf/scperfobj.cxx.

  1. add a new class method
  2. register the new class method inside the CPPUNIT_TEST_SUITE/CPPUNIT_TEST_SUITE_END block
  3. Use callgrindStart(); before the part that you want to include in your performance test
  4. Use callgrindDump(“module:a_nice_id”); after the last call that should be included in your performance test

You can even use several callgrindStart(), callgrindDump(“”) pairs in the same cppunit test. The new test case should appear automatically on the status page once the tests are run the next time.

As can be seen it is quite easy to write a new performance test and hopefully this motivates people to add new tests.


When should I use an out-of-tree test now?

Reading my section about the in-tree tests I might sound that you should always use an in-tree test. As a general rule you should put tests where you want to test the performance of the whole import or the export into the out-of-tree tests and all other tests into the in-tree tests.

However try to test as close to user interactions as possible. Testing some code that is never exposed directly to the user might give misleading results as some of its code might have been moved to another function that is always called with it, or some code was moved from a related method to the one that you are testing.


Helping with the web development

On a only slightly related note: We are still actively looking for an experienced web developer who helps us improve the results page and the new devcentral webpage. If you are interested in that task please contact Norbert or myself.



Posted in Libreoffice | Tagged , | 6 Comments

Help for the VBA export work

So Rosemary has been working on one of my core hacks (actually she is the first to work on any of our core hacks, so congratulations for that), namely the VBA export to OOXML.

The missing VBA export is one of the old limitations of our OOXML filter. While we are able to save the unchanged stream back to an OOXML file we can’t save any modifications at the moment. With Rosemary’s work this will hopefully change in 5.1 and we will no longer loose macro changes when saving to OOXML.

To make sure that we deliver a good new feature it would be awesome if some of you who have real VBA documents can send a few to libreoffice.vba.export !at! gmail dot com. Currently we are using a few simple self generated documents but they surely miss a lot of the real world corner cases. This includes especially Excel functions defined in VBA and their use case. While I’m reading the VBA and the OOXML spec I ha to discover that MSO documents don’t completely follow the spec.

I’ll write a more detailed blog post once this feature hits master and I have an idea which features we can preserve during our export.

Posted in Libreoffice | Tagged , , | Leave a comment


So I had the pleasure of attending GUADEC for a second time and was allowed to give a talk about LibreOffice.GUADEC is always a nice conference and it was located this year in the beautiful city of Gothenburg.

My talk was shared with Pranav Kant as the talk that I handed in was about his GSoC project. So we shared the talk and he talked half of the time about the integration into Gnome Documents. I then took over and talked a bit about new LibreOffice 5.0 features and the work that the other GSoC students are doing.

As always the slides for our talk can be downloaded here.

Posted in Uncategorized | Leave a comment

New fill property names in chart2

So a quick note for all users of the chart2 UNO API.

The data point objects support now the service similar to the other chart objects. This finally allows to access all chart2(not chart1) objects with the same fill property set names.

For the data points we now have the additional property names:

  • FillColor
  • FillTransparence
  • FillTransparenceGradientName
  • FillGradientName
  • FillGradientStepCount
  • FillHatchName

This change will be available in LibreOffice 5.1, currently scheduled for the beginning of February 2016.

I also filed an EasyHack to support FillTransparenceGradient, FillGradient, FillHatch and FillBitmap which would simplify handling of the fill properties even further.

Posted in Libreoffice | Tagged , | Leave a comment

Supporting more OOXML dialects in chart import

A common problem during our OOXML import is that there are several different OOXML dialects: OOXML transitional, OOXML strict and the not specified version written by MSO 2007. The MSO 2007 version is mostly identical to OOXML transitional with the small but nasty exception that they have some differences in the default values. Recently I got a document from a Collabora customer using MSO 2007 exhibiting some bugs related to that.

A few days ago I finally managed to bring support for handling the differences between the OOXML dialect written by MSO 2007 and the one in the specification to LibreOffice. This is an important step forward for our OOXML chart import as that code was written against the MSO 2007 version and more and more documents are generated by newer MSO versions. In recent years we have changed quite a few of the default values in the code to handle OOXML specification conforming documents correctly. Sadly this introduced a number of regressions for the handling of MSO 2007 documents.

With [1] and [2] we are now able to recognize files that have been created by MSO 2007 and are able to use different default values. Currently this is only used for the flag that decides if the chart title is deleted but more cases might be fixed in the future.

Posted in Uncategorized | 4 Comments

Guadec 2014

I had the privilege to attend GUADEC this year and speak about Libreoffice. I was really impressed by the conference and enjoyed the beautiful city of Strasbourg and the nice Gnome community.

My talk was about Resuing Libreoffice in your application and centered mainly around LibreOfficeKit, “The Document Liberation Project” and new features in Libreoffice 4.3.


The next conference that I will attend will be the LibreOffice conference in Bern, Switzerland where I will give presentations about OpenGL in Libreoffice, recent development in charts and automated testing.


Posted in Libreoffice | 2 Comments