roachtest: add split-file import test

[michae2]

Add an import roachtest that splits a dataset's data files into two halves and imports them via two separate IMPORT jobs into the same table. This verifies that successive imports produce a correct, complete dataset.

Epic: None

Release note: None

[trunk-io]

😎 Merged successfully - details.

[cockroach-teamcity]

This change is 

[mw5h]

@mw5h reviewed 1 file and all commit messages, and made 2 comments.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on michae2).

---

pkg/cmd/roachtest/tests/import.go line 896 at r1 (raw file):

	var first, second []string
	for i, url := range urls {
		if i%2 == 0 {

It might be better to put the URLs into a slice and shuffle with slices.Shuffle(), then pick some random number of them. That way you get a randomized ordering and a random number of files in each import.

---

pkg/cmd/roachtest/tests/import.go line 907 at r1 (raw file):

	importStmt := formatImportStmt(ds.getTableName(), first, false)
	if _, err := conn.ExecContext(ctx, importStmt); err != nil {
		return errors.Wrap(err, "first import")

Does conn.ExecContext include the statement in its error message? I defensively put it in the error message for runSyncImportJob(), but maybe I didn't need to.

[michae2]

@michae2 made 2 comments.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on mw5h).

---

pkg/cmd/roachtest/tests/import.go line 896 at r1 (raw file):

Previously, mw5h (Matt White) wrote…

It might be better to put the URLs into a slice and shuffle with slices.Shuffle(), then pick some random number of them. That way you get a randomized ordering and a random number of files in each import.

Good call, done.

---

pkg/cmd/roachtest/tests/import.go line 907 at r1 (raw file):

Previously, mw5h (Matt White) wrote…

Does conn.ExecContext include the statement in its error message? I defensively put it in the error message for runSyncImportJob(), but maybe I didn't need to.

Looks like it does not. Also added the statement to the error message.

[mw5h]

Thanks!

@mw5h reviewed 1 file and all commit messages, made 1 comment, and resolved 2 discussions.
Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on michae2).

[michae2]

/trunk merge

[blathers-crl]

Detected infrastructure failure on trunk-merge branch (matched: self-hosted runner lost communication with the server). Automatically resubmitting to merge queue (attempt 1 of 2). (run link)

[blathers-crl]

/trunk merge

[trunk-io]

This PR is already in the queue

[michae2]

/trunk merge