I wanted to discuss, in which test phase / profile it should run. I think it is very beneficial to know it our Testcontainers setup for the integration test fails. On the other hand, there are some complications to the whole test: SQL Server and DB2 are not acting well on ARM chips. On the other hand, not every maintainer might have Docker installed or be able to run Testcontainers on their system.
What is your opinion on this? Should we add a Maven profile to specifically include the tests in the pipeline and not run them otherwise? Or will integration tests fail anyways if Testcontainers don’t work and we don’t need the test at all?
I have fixed the underlying tests, so they can be executed manually. The test is still disabled and won’t be part of any test runs.
One open point of the ticket still stands, is it okay to only run this test manually, or should it be part of pipeline builds? Personally I think having disabled unit tests which are meant to be run manually is not the best pattern, but the only thing that comes to my mind is creating yet another Maven profile.
If ever possible I would like to run in the CI pipeline. No-one thinks later on tests that have to be executed manually.
The number of Maven profiles should be kept as low as possible, but if this is the only way, then it is OK. We have that script that performs integration tests with different profiles, that should be extended then.
The execution of all tests should be done as part of the “Integration Build” workflow.
Yes, I’d definitely recommend running the Testcontainers-based tests as part of the CI pipeline.
To keep things flexible for local development, we could control their activation using a Maven profile that behaves similarly to -DskipTests. For example:
If we also need to handle platform-specific differences (like issues with SQL Server or DB2 on ARM), we can use the os-maven-plugin. It exposes properties like os.detected.arch and os.detected.name, which we can use to automatically activate or adjust profiles depending on the architecture.
To test on multiple platforms in CI, we can use GitHub Actions’ matrix strategy to run builds and tests across different operating systems and architectures (e.g., x86_64 and ARM).
Thanks for your input! For the functional Testcontainers test, I’d say this is already the case. There is a specific profile for every DB which can be activate with the build, and we are working towards matrixes with all combinations of container / database. It’s running daily, and we trigger it to run on pull requests for bigger changes.
For this simple test which just verifies the correct setup of the Container Providers. To keep it simple, I’d agree with Karsten that a single Maven profile to activate them in the pipeline is probably a pragmatic approach. If we realize that these tests are often executed locally, we can add some more logic to provide a smoother functionality