Automated import crash testing in Libreoffice

I have been working recently on finishing the work on a python script for Libreoffice that automatically imports documents and tests if we crash there. The plan is to run this script automatically against all our bugzilla documents on a regular basis.

I have been running similar tests for calc files(the TEST_BUG_FILES) case already before the 3.5 and the 4.0 releases and fixed these crashes with Eike and Kohei before the releases. However this work was done half manually inside of an “unit” test and as soon as it crashed I had to restart the test. As a result of this complex setup it took me between 4 and 6 days to import all 6000+ calc documents. Back then I already had the idea that this task could be automated and moved to a TDF server.

I already tried to convince someone at the Munich hackfest to write such a script as an Easy Hack but had to wait till December until Joren picked up the task. Based on convwatch.py he implemented the first version that has undergone several iterations now and can be found in the Libreoffice dev-tools repository. The script is still looking quite ugly as I have been only adding code and it still contains a large number of debug output for me but the current version should work fine against current Libreoffice master.

After several toolchain problems, one needs a libstdc++ created with at least Linux binutils 2.23.52.0.1 or newer, I finally published the results of the current test run at the Libreoffice developer mailing list. In my latest test run I had a collection of a bit more than 24500 documents with the file extensions cdr, doc, docx, fodg, fodp, fods, fodt, odg, odp, ods, odt, ppt, pptx, pub, rtf, vdx, vsd, wpd, xls, xlsx. While 60 crashes might sound like a lot one has to remember that many of these crashes will never be seen by users. The test is run with a dbgutil build which means that we enforce the exception specification, we switch the standard library to the gcc debug library which has additional assertions and some crashes are related to the special set up of the test. Nevertheless we are planning to fix all these crashes and use the script as part of our automatic testing.

And finally a special thanks to the amazing Libreoffice community who was incredibly supporting in realizing this crazy concept.

About these ads

About Markus Mohrhard

Hacking at Libreoffice calc
This entry was posted in Libreoffice. Bookmark the permalink.

20 Responses to Automated import crash testing in Libreoffice

  1. Pingback: Lançado o LibreOffice 4.0.3 | LibreOffice Brasil Blog

  2. Pingback: The Document Foundation announces LibreOffice 4.0.3 | The Document Foundation Blog

  3. Pingback: LibreOffice.hu — LibreOffice 4.0.3

  4. Pingback: The Document Foundation annuncia LibreOffice 4.0.3 – LibreItalia

  5. Pingback: The Document Foundation annuncia LibreOffice 4.0.3 | LibreItalia

  6. Pingback: Arriva LibreOffice 4.0.3 per Windows, OS X e Linux | Computerworld Italia

  7. Pingback: Lançado o LibreOffice 4.0.3 | LibreOffice Para Leigos

  8. Pingback: Revista Espírito Livre » Blog Archive » Lançado LibreOffice 4.0.3

  9. Pingback: Корректирующий релиз офисного пакета LibreOffice 4.0.3 | AllUNIX.ru — Всероссийский портал о UNIX-системах

  10. Pingback: LibreOffice 4.0.3, suite bureautique libre | GaLaGaNN's Blog

  11. Pingback: Siti Internet Aziendali » LibreOffice 4.0.3, release di qualita’

  12. Pingback: LibreOffice 4.0.3, release di qualita’ | SudTv Network

  13. Pingback: The Document Foundation announces LibreOffice 4.0.3 | Official Vietnamese LibreOffice Blog

  14. Pingback: Open Source Project » Корректирующий релиз офисного пакета LibreOffice 4.0.3

  15. Pingback: The Document Foundation announces LibreOffice 4.0.3 | FOSS Advcocates

  16. Mark says:

    Are those 24500 documents available in some git repository? Would be fun to run some more batch tests over those.

  17. Pingback: LibreOffice 4.0.3, release di qualità | ILCommentario.com

  18. Pingback: tb3 — more efficient tinderboxing | You can't take the sky from me.

  19. Pingback: Update about our import/export crash testing | moggi's blog about Libreoffice hacking

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s