Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -18,4 +18,4 @@ Add: Tony, 20, 0, 17.1
Add: Alice, 20, null, 18.1
NoCompact

data-multi-row-groups.parquet is the file generated by C++ paimon, where each row group contains only one row
data-multi-row-groups.parquet is the file generated by C++ paimon, where each row group contains only one row
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,4 @@ Add: 4, 11, 0, 100, 4.0, null, "four", "ddd", 123, "123.45", "1970-01-04 00:00:0
snapshot-2 (3 data files)
Add: 5, 10, 0, 100, 5.0, 2.0, null, "eee", 123, "123.45", "1970-01-05 00:00:00", "1970-01-05 00:00:00.000", "1970-01-05 00:00:00.000000", "1970-01-05 00:00:00", "1970-01-05 00:00:00.000", "1970-01-05 00:00:00.000000",[[["key",123]],[1,2,3]]
Add: 6, 10, 1, 100, 6.0, 4.0, "six", "fff", 123, "123.45", "1970-01-06 00:00:00", "1970-01-06 00:00:00.000", "1970-01-06 00:00:00.000000", "1970-01-06 00:00:00", "1970-01-06 00:00:00.000", "1970-01-06 00:00:00.000000",[[["key",123]],[1,2,3]]
Add: 7, 11, 0, 100, 7.0, 6.0, "seven", "ggg", 123, "123.45", "1970-01-07 00:00:00", "1970-01-07 00:00:00.000", "1970-01-07 00:00:00.000000", "1970-01-07 00:00:00", "1970-01-07 00:00:00.000", "1970-01-07 00:00:00.000000",[[["key",123]],[1,2,3]]
Add: 7, 11, 0, 100, 7.0, 6.0, "seven", "ggg", 123, "123.45", "1970-01-07 00:00:00", "1970-01-07 00:00:00.000", "1970-01-07 00:00:00.000000", "1970-01-07 00:00:00", "1970-01-07 00:00:00.000", "1970-01-07 00:00:00.000000",[[["key",123]],[1,2,3]]
2 changes: 1 addition & 1 deletion test/test_data/avro/append_simple.db/append_simple/README
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,4 @@ Add: 1, 2.0, "test", [[["key",123]],[1,2,3]]
Add: 1, 2.0, "test", [[["key",123]],[1,2,3]]
Add: 1, 2.0, "test", [[["key",123]],[1,2,3]]
Add: 1, 2.0, "test", [[["key",123]],[1,2,3]]
NoCompact
NoCompact
2 changes: 1 addition & 1 deletion test/test_data/db_with_blob.db/table_with_blob/README
Original file line number Diff line number Diff line change
Expand Up @@ -18,4 +18,4 @@ data-d7816e8e-6c6d-4e28-9137-837cdf706350-3.blob
blob_8_5fba0737.bin

data-d7816e8e-6c6d-4e28-9137-837cdf706350-4.blob
blob_9_f54d253c.bin
blob_9_f54d253c.bin
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,4 @@ row_id,key,is_null
4. **Edge cases**: null values, duplicate keys, boundary values

### Usage
This data can be used to verify the compatibility between the C++ BTree index implementation and the Java version.
This data can be used to verify the compatibility between the C++ BTree index implementation and the Java version.
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,4 @@ data_split-01
with snapshot-1

data_split-02
with fake external path
with fake external path
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ f1:TIMESTAMP(9):3
f2:DECIMAL(5, 2):4
f3:STRING:5
f4:DATE:6
f5:BIGINT:7
f5:BIGINT:7
partition key: key0, key1
bucket count: -1

Expand All @@ -21,14 +21,14 @@ NoCompact

schema evolution
updateColumnType("f0", INT())
updateColumnType("f1", STRING())
updateColumnType("f1", STRING())
updateColumnType("f2", DECIMAL(6, 3))
updateColumnType("f3", BOOLEAN())
updateColumnType("f4", TIMESTAMP(9))
dropColumn("f5")
dropColumn("f5")
renameColumn("f0", "f5")
renameColumn("f3", "f0")
renameColumn("f5", "f3")
renameColumn("f5", "f3")
updateColumnPosition(SchemaChange.Move.first("f4"))
addColumn("f6", INT()))

Expand Down Expand Up @@ -59,4 +59,4 @@ Recall with schema-1, result:
["1957-11-01 00:00:00", 1, 1, 130, "2024-11-26 06:38:56.031000031", "333.320", false, null],
["2091-09-07 00:00:00", 1, 1, 140, "2024-11-26 06:38:56.041000041", "444.420", true, null],
["2024-11-26 06:38:56.084000084", 1, 1, 180, "2024-11-26 15:29:01", "8.032", true, -86],
["2024-11-26 06:38:56.094000094", 1, 1, 190, "I'm strange", "-999.420", false, 96]
["2024-11-26 06:38:56.094000094", 1, 1, 190, "I'm strange", "-999.420", false, 96]
Original file line number Diff line number Diff line change
Expand Up @@ -6,39 +6,39 @@ f1:TIMESTAMP(9):3
f2:DECIMAL(5, 2):4
f3:STRING:5
f4:DATE:6
f5:BIGINT:7
f5:BIGINT:7
partition key: key0, key1
bucket count: -1

Msgs:
write "key0", "key1", "f0", "f1", "f2"
Add(0, 1, "100", Timestamp(1732603136001l, 1), "0.02")
Add(0, 1, "110", Timestamp(1732603136011l, 11), "11.12")
Add(0, 1, "120", Timestamp(1732603136021l, 21), "22.22")
Add(0, 1, "120", Timestamp(1732603136021l, 21), "22.22")
Add(0, 1, "130", Timestamp(1732603136031l, 31), "333.32")
Add(0, 1, "140", Timestamp(1732603136041l, 41), "444.42")
Add(0, 1, "140", Timestamp(1732603136041l, 41), "444.42")

write "key0", "key1", "f3", "f4", "f5"
Add(0, 1, "True", 4, 5l)
Add(0, 1, "1", -44, 25l)
Add(0, 1, "nO", 444, 35l)
Add(0, 1, "0", -4444, 45l)
Add(0, 1, "Yes", 44444, 55l)
Add(0, 1, "Yes", 44444, 55l)

set first row id to 0
commit snapshot-1
NoCompact

schema evolution
updateColumnType("f0", INT())
updateColumnType("f1", STRING())
updateColumnType("f1", STRING())
updateColumnType("f2", DECIMAL(6, 3))
updateColumnType("f3", BOOLEAN())
updateColumnType("f4", TIMESTAMP(9))
dropColumn("f5")
dropColumn("f5")
renameColumn("f0", "f5")
renameColumn("f3", "f0")
renameColumn("f5", "f3")
renameColumn("f5", "f3")
updateColumnPosition(SchemaChange.Move.first("f4"))
addColumn("f6", INT())

Expand All @@ -52,11 +52,11 @@ f0:BOOLEAN:5
f6:INT:8

write "f4", "key0", "key1", "f3", "f1"
Add(Timestamp(1732603136054l, 54), 0, 1, 150, "2024-11-26 15:28:31")
Add(Timestamp(1732603136054l, 54), 0, 1, 150, "2024-11-26 15:28:31")
Add(Timestamp(1732603136064l, 64), 0, 1, 160, "2024-11-26 15:28:41")
Add(Timestamp(1732603136074l, 74), 0, 1, 170, "2024-11-26 15:28:51")
Add(Timestamp(1732603136084l, 84), 0, 1, 180, "2024-11-26 15:29:01")
Add(Timestamp(1732603136094l, 94), 0, 1, 190, "I'm strange")
Add(Timestamp(1732603136094l, 94), 0, 1, 190, "I'm strange")

set first row id to 5
commit snapshot-2
Expand All @@ -67,7 +67,7 @@ Add(Timestamp(1732603136054l, 154), 0, 1, "55.002", true, 56)
Add(Timestamp(1732603136064l, 164), 0, 1, "666.012", false, 66)
Add(Timestamp(1732603136074l, 174), 0, 1, "-77.022", true, 76)
Add(Timestamp(1732603136084l, 184), 0, 1, "8.032", true, -86)
Add(Timestamp(1732603136094l, 194), 0, 1, "-999.420", false, 96)
Add(Timestamp(1732603136094l, 194), 0, 1, "-999.420", false, 96)

set first row id to 5
commit snapshot-3
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
key0:int:0 key1:int:1 a:int:2 b:int:3 c:int:4 d:int:5 k:int:6
key0:int:0 key1:int:1 a:int:2 b:int:3 c:int:4 d:int:5 k:int:6
partition key: key0, key1
bucket count: -1

Expand All @@ -9,7 +9,7 @@ Add(0, 1, 22, 23, 24, 25, 26)
Add(0, 1, 32, 33, 34, 35, 36)

Add(1, 1, 42, 43, 44, 45, 46)
Add(1, 1, 52, 53, 54, 55, 56)
Add(1, 1, 52, 53, 54, 55, 56)

NoCompact

Expand All @@ -26,10 +26,10 @@ key0:int:0 key1:int:1 k:int:6 c:int:3 d:int:7 a:int:5 e:int:8
snapshot-2
Add(0, 1, 66, 63, 517, 65, 618)
Add(0, 1, 76, 73, 527, 75, 628)
Add(0, 1, 86, 83, 537, 85, 638)
Add(0, 1, 86, 83, 537, 85, 638)

Add(1, 1, 96, 93, 547, 95, 648)
Add(1, 1, 106, 103, 557, 105, 658)
Add(1, 1, 106, 103, 557, 105, 658)

NoCompact

Expand All @@ -43,4 +43,4 @@ Recall with schema-1, result:
1, 1, 46, 43, null, 45, null
1, 1, 56, 53, null, 55, null
1, 1, 96, 93, 547, 95, 648
1, 1, 106, 103, 557, 105, 658
1, 1, 106, 103, 557, 105, 658
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
key0:int:0 key1:int:1 a:int:2 b:int:3 c:int:4 d:int:5 k:int:6
key0:int:0 key1:int:1 a:int:2 b:int:3 c:int:4 d:int:5 k:int:6
partition key: key0, key1
bucket count: -1

Expand All @@ -9,7 +9,7 @@ Add(0, 1, 22, 23, 24, 25, 26)
Add(0, 1, 32, 33, 34, 35, 36)

Add(1, 1, 42, 43, 44, 45, 46)
Add(1, 1, 52, 53, 54, 55, 56)
Add(1, 1, 52, 53, 54, 55, 56)

NoCompact

Expand All @@ -26,10 +26,10 @@ key0:int:0 key1:int:1 k:int:6 c:int:3 d:int:7 a:int:5 e:int:8
snapshot-2
Add(0, 1, 66, 63, 517, 65, 618)
Add(0, 1, 76, 73, 527, 75, 628)
Add(0, 1, 86, 83, 537, 85, 638)
Add(0, 1, 86, 83, 537, 85, 638)

Add(1, 1, 96, 93, 547, 95, 648)
Add(1, 1, 106, 103, 557, 105, 658)
Add(1, 1, 106, 103, 557, 105, 658)

NoCompact

Expand All @@ -43,4 +43,4 @@ Recall with schema-1, result:
1, 1, 46, 43, null, 45, null
1, 1, 56, 53, null, 55, null
1, 1, 96, 93, 547, 95, 648
1, 1, 106, 103, 557, 105, 658
1, 1, 106, 103, 557, 105, 658
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Add(0, "Alice", 11, 21, 31.1)
Add(0, "Bob", 12, 22, 32.2)

Add(1, "Cathy", 13, 23, 33.3)
Add(1, "David", 14, 24, 34.4)
Add(1, "David", 14, 24, 34.4)

NoCompact

Expand All @@ -24,9 +24,9 @@ key0:int:0 f2:int:2 f1:int:3 f3:double:4 f4:int:5

snapshot-2
Add(0, 25, 15, 35.5, 45)
Add(0, 26, 16, 36.6, 46)
Add(0, 26, 16, 36.6, 46)

Add(1, 27, 17, 37.7, 47)
Add(1, 28, 18, 38.8, 48)
Add(1, 28, 18, 38.8, 48)

NoCompact
NoCompact
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,4 @@ Add: 1, 120, 1200
Add: 2, 220, 2200

data-split-0-1: main branch fallback batch scan
data-split-pt-0: pt branch batch scan
data-split-pt-0: pt branch batch scan
Original file line number Diff line number Diff line change
Expand Up @@ -9,16 +9,16 @@ partition key: dt
4.pk table alter table setOption("primary-key", "dt, name")

5.append table write data
Add: "20240725", "apple", 5
Add: "20240725", "banana", 7
Add: "20240725", "apple", 5
Add: "20240725", "banana", 7
NoCompact
snapshot-1

6.pk table write data
Add: "20240725", "apple", 4
Add: "20240725", "peach", 10
Add: "20240726", "cherry", 3
Add: "20240726", "pear", 6
Add: "20240725", "apple", 4
Add: "20240725", "peach", 10
Add: "20240726", "cherry", 3
Add: "20240726", "pear", 6
NoCompact
snapshot-1

Expand Down
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
The directory of table `append_table_with_tag.append_table_with_tag` is copied from table `append_09.append_09,`
which adds `tag` directory to verify the tag scanned from append table.
which adds `tag` directory to verify the tag scanned from append table.
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@ NoCompact

Alter table
SchemaChange.updateColumnType("f1", DataTypes.BIGINT()),
SchemaChange.dropColumn("f2"),
SchemaChange.renameColumn("f0", "f4"),
SchemaChange.dropColumn("f2"),
SchemaChange.renameColumn("f0", "f4"),
SchemaChange.updateColumnPosition(SchemaChange.Move.first("f1")),
SchemaChange.addColumn("f5", DataTypes.INT()),
SchemaChange.setOption("file-index.bitmap.columns", "f1,f4,f5")
Expand All @@ -38,4 +38,4 @@ Add: 30, Alice, 21.1, 100
Add: 20, Emily, 22.1, 101
Add: 10, Bob, 23.1, 100
Add: 30, David, 24.1, null
NoCompact
NoCompact
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,4 @@ Add:(1, i%2, i * 3, 18000 + i, i * 2.2, 1700000000000L + i * 1000L, 170000000000
i = [0, 19]
NoCompact

snapshot-3: with global btree index for col_boolean/col_int/col_date/col_double/col_timestamp/col_timestamp_ltz/col_decimal/col_string
snapshot-3: with global btree index for col_boolean/col_int/col_date/col_double/col_timestamp/col_timestamp_ltz/col_decimal/col_string
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,4 @@ NoCompact
snapshot-1: with no index
snapshot-2: with f0 global bitmap index
snapshot-3: with f1 global bitmap index
snapshot-4: with f2 global bitmap index
snapshot-4: with f2 global bitmap index
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,4 @@ snapshot-1: with no index
snapshot-2: with f0 global bitmap index for f1=10

commit_messages-01:
f0 global bitmap index for f1=10 while commit snapshot 2
f0 global bitmap index for f1=10 while commit snapshot 2
Original file line number Diff line number Diff line change
Expand Up @@ -30,4 +30,4 @@ Add Timestamp(5000l,0), Timestamp(5l,0), null, null, Timestamp(6000l,0), null, T
---------------------------------------------


cpp_paimon_write_apache_orc_9.orc is orc file generated by CPP Paimon with orc File format with the same schema
cpp_paimon_write_apache_orc_9.orc is orc file generated by CPP Paimon with orc File format with the same schema
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,4 @@ Add: [9, 500L, 5.5f, 55.55, date('2024-01-09'), 'row4']
Add: [null, null, null, null, null, null]
Add: [null, null, null, null, null, 'null_row']
Add: [10, 600L, 6.6f, 66.66, date('2024-01-10'), 'row7']
NoCompact
NoCompact
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,4 @@ Add: [9, 500L, 5.5f, 55.55, date('2024-01-09'), 'row4']
Add: [null, null, null, null, null, null]
Add: [null, null, null, null, null, 'null_row']
Add: [10, 600L, 6.6f, 66.66, date('2024-01-10'), 'row7']
NoCompact
NoCompact
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ f1:TIMESTAMP(9):3
f2:DECIMAL(5, 2):4
f3:STRING:5
f4:DATE:6
f5:BIGINT:7
f5:BIGINT:7
blob:BLOB NOT NULL:8
partition key: key0, key1
bucket count: -1
Expand All @@ -32,11 +32,11 @@ NoCompact

schema evolution
updateColumnType("f0", INT())
updateColumnType("f1", STRING())
updateColumnType("f1", STRING())
updateColumnType("f2", DECIMAL(6, 3))
updateColumnType("f3", BOOLEAN())
updateColumnType("f4", TIMESTAMP(9))
dropColumn("f5")
dropColumn("f5")
renameColumn("f0", "f5")
renameColumn("f3", "f0")
renameColumn("f5", "f3")
Expand Down Expand Up @@ -89,4 +89,4 @@ INSERT: 2024-11-26T06:38:56.054000154, 0, 1, 150, 2024-11-26 15:28:31, 55.002, A
INSERT: 2024-11-26T06:38:56.064000164, 0, 1, 160, 2024-11-26 15:28:41, 666.012, Banana, false, 66, 6, 3
INSERT: 2024-11-26T06:38:56.074000174, 0, 1, 170, 2024-11-26 15:28:51, -77.022, Cherry, true, 76, 7, 3
INSERT: 2024-11-26T06:38:56.084000184, 0, 1, 180, 2024-11-26 15:29:01, 8.032, Durian, true, -86, 8, 3
INSERT: 2024-11-26T06:38:56.094000194, 0, 1, 190, I'm strange, -999.420, Elderberry, false, 96, 9, 3
INSERT: 2024-11-26T06:38:56.094000194, 0, 1, 190, I'm strange, -999.420, Elderberry, false, 96, 9, 3
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,4 @@ NoCompact

snapshot-3
Add: David, 10, 0, 17.1
NoCompact
NoCompact
2 changes: 1 addition & 1 deletion test/test_data/orc/pk_09.db/pk_09/README
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Add: Bob, 10, 0, 12.1
Add: Emily, 10, 0, 13.1
Add: Tony, 10, 0, 14.1
Add: Alex, 10, 0, 16.1
Add: David, 10, 0, 17.1
Add: David, 10, 0, 17.1

Add: Lucy, 20, 1, 14.1
Add: Paul, 20, 1, 18.1
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,4 @@ Delete: Tony, 10, 0, 14.1
UpdateAfter: Alice, 10, 1, 19.1

commit_messages-01: partition 10, bucket 1, snapshot 3-4
data_split-01-03: snapshot-4
data_split-01-03: snapshot-4
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,4 @@ Delete: Tony, 10, 0, 14.1
UpdateAfter: Alice, 10, 1, 19.1

commit_messages-01: partition 10, bucket 1, snapshot 3-4
data_split-01-03: snapshot-4
data_split-01-03: snapshot-4
Loading
Loading