Skip to content

GH-3482: parquet-hadoop tests to work behind a web proxy #3483

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
alexeyroytman wants to merge 6 commits into
base: master
Choose a base branch
from
+34 −2
Open

GH-3482: parquet-hadoop tests to work behind a web proxy #3483

Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
ce8533d
parquet-hadoop tests to work behind a web proxy
alexeyroytman Apr 16, 2026
b34da59
parquet-hadoop to use separate JVM propertied not to clash with exist…
alexeyroytman Apr 16, 2026
cbae73c
syntax fix
alexeyroytman Apr 17, 2026
c12b04a
parquet-hadoop: another test provisioned to run behind a web proxy
alexeyroytman Apr 17, 2026
9614fac
safer condition, catching NPE at InterOpTester.java
alexeyroytman May 6, 2026
4798a38
better proxy misconfig diagnostic for testing
alexeyroytman May 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 20 additions & 1 deletion parquet-hadoop/src/test/java/org/apache/parquet/hadoop/InterOpTester.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@
package org.apache.parquet.hadoop;

import java.io.IOException;
import java.net.InetSocketAddress;
import java.net.Proxy;
import okhttp3.OkHttpClient;
import okhttp3.Request;
import okhttp3.Response;
Expand All @@ -34,7 +36,24 @@ public class InterOpTester {
private static final String PARQUET_TESTING_REPO = "https://github.com/apache/parquet-testing/raw/";
private static final String PARQUET_TESTING_PATH = "target/parquet-testing/";
private static final Logger LOG = LoggerFactory.getLogger(InterOpTester.class);
private OkHttpClient httpClient = new OkHttpClient();
// since PARQUET_TESTING_REPO might be beyond a web proxy ...
private static OkHttpClient createOkHttpClientOptProxy() {
String proxyHost = System.getProperty("https.proxyHost");
String proxyPort = System.getProperty("https.proxyPort");
OkHttpClient client = null;
Comment thread
alexeyroytman marked this conversation as resolved.
if (proxyHost != null || proxyPort != null) {
try {
int port = Integer.valueOf(proxyPort);
Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(proxyHost, port));
client = new OkHttpClient.Builder().proxy(proxy).build();
} catch (NumberFormatException e) {
Comment thread
alexeyroytman marked this conversation as resolved.
}
Comment thread
alexeyroytman marked this conversation as resolved.
}
if (client == null) client = new OkHttpClient();
return client;
}

private OkHttpClient httpClient = createOkHttpClientOptProxy();

public Path GetInterOpFile(String fileName, String changeset) throws IOException {
return GetInterOpFile(fileName, changeset, "data");
Expand Down
Loading