Optimize upsert performance for large datasets #7491

Job	Run time
integration-test-gcs	1m 9s
integration-test-adls	1m 8s
integration-test	1m 11s
integration-test-s3	1m 10s
lint-and-unit-test (3.13)	1m 5s
lint-and-unit-test (3.11)	1m 10s
lint-and-unit-test (3.10)	1m 3s
lint-and-unit-test (3.12)	1m 3s
integration-coverage-report	1s
	9m 0s

Provide feedback