github-starred/libxlsxwriter

Fork 0

mirror of https://github.com/jmcnamara/libxlsxwriter.git synced 2026-05-15 14:15:54 -06:00

[GH-ISSUE #364] Adding functional tests to the cmake -DBUILD_TESTS=ON configuration in libxlsxwriter #291

New issue

Closed

opened 2026-05-05 12:06:24 -06:00 by gitea-mirror · 7 comments

gitea-mirror commented

2026-05-05 12:06:24 -06:00

Owner

Originally created by @jmcnamara on GitHub (Feb 14, 2022).
Original GitHub issue: https://github.com/jmcnamara/libxlsxwriter/issues/364

Originally assigned to: @jmcnamara on GitHub.

The cmake -DBUILD_TESTS=ON configuration builds the libxlsxwriter unit tests but to date it hasn't built the functional tests.

The libxlsxwriter functional tests use Python pytest to compile complete C programs (like this), runs them, and compare the outputs file by file and element by element with the XML in files generate by Excel. Currently there are around 800 of these tests and they ensure a high level of fidelity between the files produced by libxlsxwriter and Excel.

I have been enabling GitHub actions to improve the CI since I no longer have access to travis and I made a pass at enabling these functional tests.

They require Python and Pytest. Pytest can be installed like this:

sudo apt-get install -y python-pytest

# Or this:
pip install pytest

Then you can run the entire test suite like this:

cd libxlsxwriter
mkdir build
cd build
cmake .. -DBUILD_TESTS=ON -DCMAKE_BUILD_TYPE=Release
cmake --build . --config Release
ctest -C Release -V

There are some caveats:

~~I need to add support for testing some of the compilation variants with Cmake/Ctest (they are already in the "make" tests).~~
The Cmake code is probably a bit clunky, comments/suggestions welcome: https://github.com/jmcnamara/libxlsxwriter/blob/main/CMakeLists.txt#L356
~~The check for pytest doesn't work. I've tried a lot of find_package() and find_program() variations and I can't get it to work. Suggestions welcome here as well.~~
I need to add Windows support/CI tests.

@sjmulder @Alexhuszagh @remicollet @manisandro and any other packagers/porters/cmake users: could you have a look if you get a chance and let me know if you have any feedback.

Originally created by @jmcnamara on GitHub (Feb 14, 2022). Original GitHub issue: https://github.com/jmcnamara/libxlsxwriter/issues/364 Originally assigned to: @jmcnamara on GitHub. The cmake `-DBUILD_TESTS=ON` configuration builds the libxlsxwriter unit tests but to date it hasn't built the functional tests. The libxlsxwriter functional tests use Python pytest to compile complete C programs (like [this](https://github.com/jmcnamara/libxlsxwriter/actions)), runs them, and compare the outputs file by file and element by element with the XML in files generate by Excel. Currently there are around 800 of these tests and they ensure a high level of fidelity between the files produced by libxlsxwriter and Excel. I have been enabling [GitHub actions](https://github.com/jmcnamara/libxlsxwriter/actions) to improve the CI since I no longer have access to travis and I made a pass at enabling these functional tests. They require Python and Pytest. Pytest can be installed like this: ``` sudo apt-get install -y python-pytest # Or this: pip install pytest ``` Then you can run the entire test suite like this: ``` cd libxlsxwriter mkdir build cd build cmake .. -DBUILD_TESTS=ON -DCMAKE_BUILD_TYPE=Release cmake --build . --config Release ctest -C Release -V ``` There are some caveats: - ~~I need to add support for testing some of the compilation variants with Cmake/Ctest (they are already in the "make" tests).~~ - The Cmake code is probably a bit clunky, comments/suggestions welcome: https://github.com/jmcnamara/libxlsxwriter/blob/main/CMakeLists.txt#L356 - ~~The check for pytest doesn't work. I've tried a lot of `find_package()` and `find_program()` variations and I can't get it to work. Suggestions welcome here as well.~~ - I need to add Windows support/CI tests. @sjmulder @Alexhuszagh @remicollet @manisandro and any other packagers/porters/cmake users: could you have a look if you get a chance and let me know if you have any feedback.

gitea-mirror

2026-05-05 12:06:24 -06:00

closed this issue
added the
question

cmake
labels

gitea-mirror commented

2026-05-05 12:06:29 -06:00

Author

Owner

@jmcnamara commented on GitHub (Feb 14, 2022):

The check for pytest doesn't work. I've tried a lot of find_package() and find_program() variations and I can't get it to work. Suggestions welcome here as well.

I fixed this and tested it with a clean Ubuntu install without and with pytest.

@jmcnamara commented on GitHub (Feb 14, 2022): > The check for pytest doesn't work. I've tried a lot of `find_package()` and `find_program()` variations and I can't get it to work. Suggestions welcome here as well. I fixed this and tested it with a clean Ubuntu install without and with pytest.

gitea-mirror commented

2026-05-05 12:06:31 -06:00

Author

Owner

@jmcnamara commented on GitHub (Feb 15, 2022):

I need to add support for testing some of the compilation variants with Cmake/Ctest (they are already in the "make" tests).

I've added this now as well for most/all Libxlsxwriter Cmake configuration options:

jobs:
  build:
    name:
      Cmake
    strategy:
      matrix:
        cc:          [gcc, clang]
        cmake_flags: ["",
                      "-DBUILD_EXAMPLES=ON       -DBUILD_TESTS=OFF",
                      "-DUSE_DTOA_LIBRARY=ON     -DBUILD_TESTS=ON",
                      "-DUSE_FMEMOPEN=ON         -DBUILD_TESTS=ON",
                      "-DUSE_NO_MD5=ON           -DBUILD_TESTS=ON",
                      "-DUSE_OPENSSL_MD5=ON      -DBUILD_TESTS=ON",
                      "-DUSE_STANDARD_TMPFILE=ON -DBUILD_TESTS=ON",
                      "-DUSE_SYSTEM_MINIZIP=ON   -DBUILD_TESTS=ON",
                      "-DUSE_SYSTEM_MINIZIP=ON   -DUSE_OPENSSL_MD5=ON -DBUILD_TESTS=ON"]
    runs-on: ubuntu-latest

@jmcnamara commented on GitHub (Feb 15, 2022): > I need to add support for testing some of the compilation variants with Cmake/Ctest (they are already in the "make" tests). I've added this now as well for most/all Libxlsxwriter Cmake configuration options: ``` jobs: build: name: Cmake strategy: matrix: cc: [gcc, clang] cmake_flags: ["", "-DBUILD_EXAMPLES=ON -DBUILD_TESTS=OFF", "-DUSE_DTOA_LIBRARY=ON -DBUILD_TESTS=ON", "-DUSE_FMEMOPEN=ON -DBUILD_TESTS=ON", "-DUSE_NO_MD5=ON -DBUILD_TESTS=ON", "-DUSE_OPENSSL_MD5=ON -DBUILD_TESTS=ON", "-DUSE_STANDARD_TMPFILE=ON -DBUILD_TESTS=ON", "-DUSE_SYSTEM_MINIZIP=ON -DBUILD_TESTS=ON", "-DUSE_SYSTEM_MINIZIP=ON -DUSE_OPENSSL_MD5=ON -DBUILD_TESTS=ON"] runs-on: ubuntu-latest ```

gitea-mirror commented

2026-05-05 12:06:32 -06:00

Author

Owner

@manisandro commented on GitHub (Feb 15, 2022):

I gave it a quick spin on Fedora and everything seemed to work fine, incl pytest detection.

$ cmake -DUSE_SYSTEM_MINIZIP=ON -DBUILD_TESTS=ON
[...]
$ make
[...]
$ ctest
Test project /home/sandro/Documents/Devel/Fedora/libxlsxwriter/libxlsxwriter-main/build
    Start 1: xlsxwriter_unit
1/2 Test #1: xlsxwriter_unit ..................   Passed    0.03 sec
    Start 2: functional
2/2 Test #2: functional .......................   Passed   10.92 sec

100% tests passed, 0 tests failed out of 2

@manisandro commented on GitHub (Feb 15, 2022): I gave it a quick spin on Fedora and everything seemed to work fine, incl pytest detection. $ cmake -DUSE_SYSTEM_MINIZIP=ON -DBUILD_TESTS=ON [...] $ make [...] $ ctest Test project /home/sandro/Documents/Devel/Fedora/libxlsxwriter/libxlsxwriter-main/build Start 1: xlsxwriter_unit 1/2 Test #1: xlsxwriter_unit .................. Passed 0.03 sec Start 2: functional 2/2 Test #2: functional ....................... Passed 10.92 sec 100% tests passed, 0 tests failed out of 2

gitea-mirror commented

2026-05-05 12:06:34 -06:00

Author

Owner

@jmcnamara commented on GitHub (Feb 15, 2022):

@manisandro Thanks for testing.

~~Note, in case anyone hits it, there is currently an issue with building the examples and tests together. I'm working on that.~~ Fixed it.

@jmcnamara commented on GitHub (Feb 15, 2022): @manisandro Thanks for testing. ~~Note, in case anyone hits it, there is currently an issue with building the examples and tests together. I'm working on that.~~ Fixed it.

gitea-mirror commented

2026-05-05 12:06:36 -06:00

Author

Owner

@jmcnamara commented on GitHub (Feb 17, 2022):

I've added a Windows Cmake build to the CI. It is quite slow to build OpenSSL and the test cases (~20 mins) but it may catch some issues. There are some minors warning from MSVC that other compilers don't raise so I need to look at them too.

The Windows builds are:

        cmake_flags: ["-DBUILD_EXAMPLES=ON       -DBUILD_TESTS=ON",
                      "-DUSE_DTOA_LIBRARY=ON     -DBUILD_TESTS=ON",
                      "-DUSE_OPENSSL_MD5=ON      -DBUILD_TESTS=ON",
                      "-DUSE_SYSTEM_MINIZIP=ON   -DBUILD_TESTS=ON",
                      "-DUSE_SYSTEM_MINIZIP=ON   -DUSE_OPENSSL_MD5=ON -DBUILD_TESTS=ON"]

I also had some issues with Ctest that I had to work around.

I'm going to close this issue now since this is the majority of the code tests that I want to put in the CI. I may add an automated code formatting check as well but that is non-critical.

@jmcnamara commented on GitHub (Feb 17, 2022): I've added a Windows Cmake build to the CI. It is quite slow to build OpenSSL and the test cases (~20 mins) but it may catch some issues. There are some minors warning from MSVC that other compilers don't raise so I need to look at them too. The Windows builds are: ``` cmake_flags: ["-DBUILD_EXAMPLES=ON -DBUILD_TESTS=ON", "-DUSE_DTOA_LIBRARY=ON -DBUILD_TESTS=ON", "-DUSE_OPENSSL_MD5=ON -DBUILD_TESTS=ON", "-DUSE_SYSTEM_MINIZIP=ON -DBUILD_TESTS=ON", "-DUSE_SYSTEM_MINIZIP=ON -DUSE_OPENSSL_MD5=ON -DBUILD_TESTS=ON"] ``` I also had some issues with Ctest that I had to work around. I'm going to close this issue now since this is the majority of the code tests that I want to put in the CI. I may add an automated code formatting check as well but that is non-critical.

gitea-mirror commented

2026-05-05 12:06:38 -06:00

Author

Owner

@jmcnamara commented on GitHub (Feb 18, 2022):

For the package maintainers CCed on this issue.

I'm updating the libxlsxwriter installation instructions. If you have any additional package manager instructions that I could add please let me know.

@jmcnamara commented on GitHub (Feb 18, 2022): For the package maintainers CCed on this issue. I'm updating the libxlsxwriter installation instructions. If you have any additional package manager instructions that I could add please let me know.

gitea-mirror commented

2026-05-05 12:06:39 -06:00

Author

Owner

@jmcnamara commented on GitHub (Feb 22, 2022):

For what it is worth I have restructured the Getting Started Guide to group more instructions in terms of OSes and to move cmake onto an even par with make in the instructions.

https://libxlsxwriter.github.io/getting_started.html

@jmcnamara commented on GitHub (Feb 22, 2022): For what it is worth I have restructured the Getting Started Guide to group more instructions in terms of OSes and to move cmake onto an even par with make in the instructions. https://libxlsxwriter.github.io/getting_started.html

gitea-mirror referenced this issue

2026-05-05 12:16:24 -06:00

[PR #296] [MERGED] Added addtional information on how to use the provided CocoaPod #458