INSERT operation sets wrong data to TIMESTAMP WITH TIME ZONE column since version 3.2.0

[yoshida16729438]

Describe the bug
If I execute DatabaseTester.setDataSet() and onSetup() with the dataset which includes timestamp with time zone column, the time zone of the data are effectively ignored, and inserted as if they were local time zone value.
It seems this occurs like :

1. TimestampDataType::setSqlValue is invoked
2. The value is converted into java.sql.Timestamp with typeCast() method.
   typeCast() method shifts the timestamp value with both of time zones from dataset and local time zone, and returns Timestamp value which is translated into the same instant of local time zone.
3. Calendar for time zone from dataset is created
4. statement.setTimestamp() is called with the Timestamp and Calendar.
   setTimestamp() internally clears the time zone shift from dataset which is calculated in step 2, and that will conclude into that as if the timestamp value in dataset was treated as local time zone and time zone in dataset is ignored.

To Reproduce
I've made a small project which reproduces the problem.
Please see this pull request which shows that DBUnit version 3.1.0 successes the test but 3.2.0 fails.

Steps to reproduce the behavior:

1. dbUnit version
   3.2.0
2. Database vendor and version
   H2 Database 2.4.240 with the reproduction project, and I've also checked with Oracle DB
3. dbUnit data file rows and type
   Requires timestamp with time zone column.
   The time zone in the data must be different from the execution environment's time zone
4. How run
   Just run mvn test -Ddbunit.version=3.X.0 with the reproduction project.
   If dbunit.version is 3.1.0, it will successfully be completed.
   If dbunit.version is 3.2.0 or 3.3.0, it will fail.

Expected behavior
Timestamp data must be persisted with correct time zone.

Additional context
It seems it can be fixed by getting Calendar instance by Calendar.getInstance() with no parameter in TimestampDataType, but I'm not sure if it breaks other logics or other column types support like TIMESTAMP WITHOUT TIME ZONE or TIMESTAMP WITH LOCAL TIME ZONE.
Related issue: #711

Since this is the first time reporting the bug of OSS, and I'm not a mother tongue English speaker, so maybe this description is not enough understandable. Sorry for inconvenience.

[jeffjensen]

Thank you very much for reporting!

Since this is the first time reporting the bug of OSS, and I'm not a mother tongue English speaker, so maybe this description is not enough understandable. Sorry for inconvenience.

No, this is very clear and well written.

I am near the end of another large set of changes for bug fixes, performance, and refactoring (#801), and will review this after that completes to see if the problem still exists.

In the meantime, if you have time and want to, could see if 3.4.0-SNAPSHOT fixes it; it is a build of my in-progress branch. See this page for help in using SNAPSHOT versions: https://dbunit.github.io/dbunit-extension/repos.html

[yoshida16729438]

Thank you for the quick response!

No, this is very clear and well written.

I'm glad to hear that.

I've added SNAPSHOT support to the reproduction project, tested with dbunit-3.4.0-20260725.162802-9.jar and concluded with failure.
I've also added the workflow for testing with 3.4.0-SNAPSHOT, so you can see the log from the PR.
Please tell me if I have to run it again.

[jeffjensen]

I had time today to review it better. Agreed, #711 change is likely the cause. Thank you for trying the snapshot, after my review I realize it would not fix it tho as my work in progress only fixes a potential "off by one second" conversion error TimestampDataType.

TimestampDataTypeTest is the test class for this TimestampDataType issue. I'm wondering if that class has an existing flawed test or we have a missing test scenario?

We need at least one test in TimestampDataTypeTest that reproduces this problem so we know when we fixed it and it never happens again (the fix may reveal a flawed existing test as well).

If you have time and want to help further, adding a test to TimestampDataTypeTest is the next step so it is in the dbUnit build. Your MainTest class reproducer already has a scenario that reveals the bug, so creating a new test in TimestampDataTypeTest from MainTest is a good approach. If you want to create that, fork this repo, make the changes, commit, and open a PR to this repo. Then you have credit as the commit author in the dbUnit repo. You may even find the fix! You mentioned this is new for you so let me know if I can help with that.

Thank you again for reporting and helping with this problem, very much appreciated!

[yoshida16729438]

I want to try to fix this bug by myself, but my PC (is Windows) seems cannot accept wsl and Hyper-v since it conflicts with another app.
It seems this repository requires Docker but I can't make it run. Can I develop without docker?

And about my bug reproducer test method, yes, it will be nice to be ported to some test classes in DBUnit, but I think that its feature is integration test rather than unit test since it requires real Connection to the database.
Is it appropriate to implement it into TimestampDataTypeTest?
Or maybe I can dig into more detailed logic and make it "unit test"-nized, but I think it will take some time to define the expected behavior for all of TIMESTAMP, TIMESTAMP WITH TIME ZONE and TIMESTAMP WITH LOCAL TIME ZONE (maybe only Oracle, I don't know if it is supported or not), and check if the behavior has no contradiction with other test cases.

[jeffjensen]

You can develop without Docker, the default build uses in-memory H2 database for the integration tests, not a Dockerized database. Testing with most databases, the build uses Docker containers of them.

Please try the 3.4.0-SNAPSHOT again as it has a fix for this issue.
The commit is 1e0a37f, if you are curious.