Skip to content

Commit ca4bbfd

Browse files
committed
refactor: Simplify JAR filtering - filter once in download-jars job
Major simplification: - Move JAR exclusion from per-platform to single upstream filter - Filter JARs once in download-jars job (Ubuntu) before artifact upload - Remove duplicate filtering logic from build-native.sh (native builds) - Simplify setup_jars.py (Docker builds get pre-filtered JARs) Benefits: - Fixes Windows glob pattern issues automatically (no filtering needed) - Reduces artifact size by ~40MB (83 JARs vs 84 JARs) - Eliminates cross-platform compatibility issues (find/glob) - Single source of truth for JAR filtering (jar_exclusions.txt) - Cleaner, more maintainable codebase Changes: - .github/workflows/test-python-bindings.yml: - Add 'Remove excluded JARs' step after download - Filters using jar_exclusions.txt with bash glob (works on Ubuntu) - Artifact now contains pre-filtered JARs - bindings/python/build-native.sh: - Removed JAR filtering logic (~30 lines) - Simplified to just copy pre-filtered JARs from artifact - bindings/python/setup_jars.py: - Removed load_jar_exclusions() and should_exclude_jar() - Removed filtering logic from find_jar_files() - Now just copies pre-filtered JARs (~60 lines simpler) Note: Docker builds (Dockerfile.build) still filter JARs independently because they download directly from ArcadeDB image, not from artifact.
1 parent 98a012a commit ca4bbfd

3 files changed

Lines changed: 44 additions & 86 deletions

File tree

.github/workflows/test-python-bindings.yml

Lines changed: 35 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,41 @@ jobs:
5050
JAR_COUNT=$(ls -1 src/arcadedb_embedded/jars/*.jar 2>/dev/null | wc -l)
5151
echo "✅ Downloaded $JAR_COUNT JAR files"
5252
53-
- name: Upload JARs as artifact
53+
- name: Remove excluded JARs
54+
shell: bash
55+
run: |
56+
cd bindings/python
57+
58+
JARS_DIR="src/arcadedb_embedded/jars"
59+
EXCLUSIONS_FILE="jar_exclusions.txt"
60+
61+
if [[ -f "$EXCLUSIONS_FILE" ]]; then
62+
echo "🗑️ Removing excluded JARs from jar_exclusions.txt..."
63+
EXCLUSION_COUNT=0
64+
65+
while IFS= read -r pattern || [[ -n "$pattern" ]]; do
66+
# Skip empty lines and comments
67+
if [[ -n "$pattern" ]] && [[ ! "$pattern" =~ ^# ]]; then
68+
echo " Processing pattern: $pattern"
69+
70+
# Remove matching JARs
71+
for jar in "$JARS_DIR"/$pattern; do
72+
if [[ -f "$jar" ]]; then
73+
rm -f "$jar"
74+
echo " - Removed: $(basename "$jar")"
75+
((EXCLUSION_COUNT++))
76+
fi
77+
done
78+
fi
79+
done < "$EXCLUSIONS_FILE"
80+
81+
JAR_COUNT_AFTER=$(ls -1 "$JARS_DIR"/*.jar 2>/dev/null | wc -l)
82+
echo "✅ Removed $EXCLUSION_COUNT JAR(s), $JAR_COUNT_AFTER remaining"
83+
else
84+
echo "⚠️ No jar_exclusions.txt found, skipping exclusions"
85+
fi
86+
87+
- name: Upload filtered JARs as artifact
5488
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
5589
with:
5690
name: arcadedb-jars

bindings/python/build-native.sh

Lines changed: 1 addition & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -110,31 +110,9 @@ echo -e "${GREEN}✅ JRE built${NC}"
110110
JRE_SIZE=$(du -sh "$SCRIPT_DIR/temp_jre" | cut -f1)
111111
echo -e "${CYAN}📊 JRE size: ${YELLOW}${JRE_SIZE}${NC}"
112112

113-
# Step 3: Copy JRE to package (JARs already in place)
113+
# Step 3: Copy JRE to package (JARs already filtered and in place from artifact)
114114
echo -e "${CYAN}📦 Preparing package...${NC}"
115115

116-
# Remove excluded JARs based on jar_exclusions.txt
117-
EXCLUSIONS_FILE="$SCRIPT_DIR/jar_exclusions.txt"
118-
if [[ -f "$EXCLUSIONS_FILE" ]]; then
119-
echo -e "${YELLOW}🗑️ Removing excluded JARs from jar_exclusions.txt...${NC}"
120-
EXCLUSION_COUNT=0
121-
while IFS= read -r pattern || [[ -n "$pattern" ]]; do
122-
# Skip empty lines and comments
123-
if [[ -n "$pattern" ]] && [[ ! "$pattern" =~ ^# ]]; then
124-
echo -e "${CYAN} Processing pattern: $pattern${NC}"
125-
# Use find instead of glob for better cross-platform compatibility
126-
while IFS= read -r jar; do
127-
if [[ -f "$jar" ]]; then
128-
rm -f "$jar"
129-
echo -e "${YELLOW} - Removed: $(basename "$jar")${NC}"
130-
((EXCLUSION_COUNT++))
131-
fi
132-
done < <(find "$JARS_DIR" -maxdepth 1 -name "$pattern" -type f 2> /dev/null)
133-
fi
134-
done < "$EXCLUSIONS_FILE"
135-
echo -e "${GREEN}✅ Removed $EXCLUSION_COUNT JAR(s)${NC}"
136-
fi
137-
138116
# Build and copy JRE
139117
rm -rf "$SCRIPT_DIR/src/arcadedb_embedded/jre"
140118
mkdir -p "$SCRIPT_DIR/src/arcadedb_embedded/jre"

bindings/python/setup_jars.py

Lines changed: 8 additions & 62 deletions
Original file line numberDiff line numberDiff line change
@@ -4,49 +4,20 @@
44
55
This script should be run as part of the build process to ensure
66
all necessary JAR files and the bundled JRE are included in the wheel.
7+
8+
Note: JAR filtering is done upstream:
9+
- Docker builds: Filtered in Dockerfile.build (jre-builder stage)
10+
- Native builds: Filtered in download-jars GitHub Actions job
711
"""
812

9-
import fnmatch
1013
import shutil
1114
from pathlib import Path
1215

1316

14-
def load_jar_exclusions():
15-
"""Load JAR exclusion patterns from jar_exclusions.txt."""
16-
exclusions_file = Path(__file__).parent / "jar_exclusions.txt"
17-
patterns = []
18-
19-
if exclusions_file.exists():
20-
with open(exclusions_file, encoding="utf-8") as f:
21-
for line in f:
22-
line = line.strip()
23-
# Skip empty lines and comments
24-
if line and not line.startswith("#"):
25-
patterns.append(line)
26-
27-
return patterns
28-
29-
30-
def should_exclude_jar(jar_name, exclusion_patterns):
31-
"""Check if a JAR should be excluded based on exclusion patterns."""
32-
for pattern in exclusion_patterns:
33-
if fnmatch.fnmatch(jar_name, pattern):
34-
return True
35-
return False
36-
37-
3817
def find_jar_files():
39-
"""Find all necessary ArcadeDB JAR files using custom filtering."""
18+
"""Find all ArcadeDB JAR files (already filtered upstream)."""
4019
print("📦 Building arcadedb-embedded with bundled JRE")
4120

42-
# Load exclusion patterns
43-
exclusion_patterns = load_jar_exclusions()
44-
if exclusion_patterns:
45-
print(
46-
f"📋 Loaded {len(exclusion_patterns)} exclusion pattern(s) "
47-
"from jar_exclusions.txt"
48-
)
49-
5021
# Look for JAR files copied from the ArcadeDB Docker image
5122
# The Dockerfile copies JARs from the full distribution image to these locations
5223
quick_paths = [
@@ -61,43 +32,18 @@ def find_jar_files():
6132
if quick.exists():
6233
jars = list(map(str, quick.glob("*.jar")))
6334
if jars:
64-
print("✅ Found {} JAR files in: {}".format(len(jars), quick))
35+
print(f"✅ Found {len(jars)} JAR files in: {quick}")
6536
all_jars.extend(jars)
6637
break
6738

68-
# Filter JARs based on exclusion patterns
6939
if all_jars:
70-
included_jars = []
71-
excluded_jars = []
72-
40+
print("\n✅ JAR files (already filtered):")
7341
for jar in all_jars:
74-
jar_name = Path(jar).name
75-
if should_exclude_jar(jar_name, exclusion_patterns):
76-
excluded_jars.append(jar)
77-
else:
78-
included_jars.append(jar)
79-
80-
print("\n📊 JAR Filtering Results:")
81-
print(f" Total found: {len(all_jars)}")
82-
print(f" Included: {len(included_jars)}")
83-
print(f" Excluded: {len(excluded_jars)}")
84-
85-
if excluded_jars:
86-
print("\n❌ Excluded JARs:")
87-
for jar in excluded_jars:
88-
jar_name = Path(jar).name
89-
size_mb = Path(jar).stat().st_size / 1024 / 1024
90-
print(f" - {jar_name} ({size_mb:.1f}MB)")
91-
92-
print("\n✅ Included JARs:")
93-
for jar in included_jars:
9442
jar_name = Path(jar).name
9543
size_mb = Path(jar).stat().st_size / 1024 / 1024
9644
print(f" - {jar_name} ({size_mb:.1f}MB)")
9745

98-
return included_jars
99-
100-
return []
46+
return all_jars
10147

10248

10349
def copy_jars_to_package():

0 commit comments

Comments
 (0)