Skip to content

Commit 853b39a

Browse files
🔒 [security fix] Fix infinite loop in ArrowFileSystemFileIO::ReadFile
This commit fixes a security vulnerability where an infinite loop could occur in `ArrowFileSystemFileIO::ReadFile` if the underlying file read operation returns 0 bytes before the expected number of bytes has been read. The fix adds a check for `read_bytes == 0` when `remain > 0` and returns an `IOError` in that case, correctly indicating an unexpected EOF and breaking the loop. Co-authored-by: wgtmac <4684607+wgtmac@users.noreply.github.com>
1 parent e21c92a commit 853b39a

6 files changed

Lines changed: 327 additions & 204 deletions

File tree

‎cmake_modules/IcebergThirdpartyToolchain.cmake‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -260,7 +260,10 @@ function(resolve_nanoarrow_dependency)
260260
set(NANOARROW_URL "$ENV{ICEBERG_NANOARROW_URL}")
261261
else()
262262
set(NANOARROW_URL
263+
"https://www.apache.org/dyn/closer.lua?action=download&filename=/arrow/apache-arrow-nanoarrow-0.7.0/apache-arrow-nanoarrow-0.7.0.tar.gz"
263264
"https://dlcdn.apache.org/arrow/apache-arrow-nanoarrow-0.7.0/apache-arrow-nanoarrow-0.7.0.tar.gz"
265+
"https://archive.apache.org/dist/arrow/apache-arrow-nanoarrow-0.7.0/apache-arrow-nanoarrow-0.7.0.tar.gz"
266+
"https://github.com/apache/arrow-nanoarrow/releases/download/apache-arrow-nanoarrow-0.7.0/apache-arrow-nanoarrow-0.7.0.tar.gz"
264267
)
265268
endif()
266269

Lines changed: 190 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,190 @@
1+
<!--
2+
~ Licensed to the Apache Software Foundation (ASF) under one
3+
~ or more contributor license agreements. See the NOTICE file
4+
~ distributed with this work for additional information
5+
~ regarding copyright ownership. The ASF licenses this file
6+
~ to you under the Apache License, Version 2.0 (the
7+
~ "License"); you may not use this file except in compliance
8+
~ with the License. You may obtain a copy of the License at
9+
~
10+
~ http://www.apache.org/licenses/LICENSE-2.0
11+
~
12+
~ Unless required by applicable law or agreed to in writing,
13+
~ software distributed under the License is distributed on an
14+
~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
~ KIND, either express or implied. See the License for the
16+
~ specific language governing permissions and limitations
17+
~ under the License.
18+
-->
19+
20+
# Contributing Guide
21+
22+
We welcome contributions to Apache Iceberg! To learn more about contributing to Apache Iceberg, please refer to the official Iceberg contribution guidelines. These guidelines are intended as helpful suggestions to make the contribution process as seamless as possible, and are not strict rules.
23+
24+
If you would like to discuss your proposed change before contributing, we encourage you to visit our Community page. There, you will find various ways to connect with the community, including Slack and our mailing lists. Alternatively, you can open a new issue directly in the GitHub repository.
25+
26+
For first-time contributors, feel free to check out our good first issues for an easy way to get started.
27+
28+
## Contributing to Iceberg C++
29+
30+
The Iceberg C++ Project is hosted on GitHub at [https://github.com/apache/iceberg-cpp](https://github.com/apache/iceberg-cpp).
31+
32+
### Development Setup
33+
34+
#### Prerequisites
35+
36+
- CMake 3.25 or higher
37+
- C++23 compliant compiler (GCC 14+, Clang 16+, MSVC 2022+)
38+
- Git
39+
40+
#### Building from Source
41+
42+
Clone the repository for local development:
43+
44+
```bash
45+
git clone https://github.com/apache/iceberg-cpp.git
46+
cd iceberg-cpp
47+
```
48+
49+
Build the core libraries:
50+
51+
```bash
52+
cmake -S . -B build -G Ninja -DCMAKE_INSTALL_PREFIX=/path/to/install -DICEBERG_BUILD_STATIC=ON -DICEBERG_BUILD_SHARED=ON
53+
cmake --build build
54+
ctest --test-dir build --output-on-failure
55+
cmake --install build
56+
```
57+
58+
Build with bundled dependencies:
59+
60+
```bash
61+
cmake -S . -B build -G Ninja -DCMAKE_INSTALL_PREFIX=/path/to/install -DICEBERG_BUILD_BUNDLE=ON
62+
cmake --build build
63+
ctest --test-dir build --output-on-failure
64+
cmake --install build
65+
```
66+
67+
### Code Standards
68+
69+
#### C++ Coding Standards
70+
71+
We follow modern C++ best practices:
72+
73+
- **C++23 Standard**: Use C++23 features where appropriate
74+
- **Naming Conventions**:
75+
- Classes: `PascalCase` (e.g., `TableScanBuilder`)
76+
- Functions/Methods: `PascalCase` (e.g., `CreateNamespace`, `ExtractYear`)
77+
- Trivial getters: `snake_case` (e.g., `name()`, `type_id()`, `is_primitive()`)
78+
- Variables: `snake_case` (e.g., `file_io`)
79+
- Constants: `k` prefix with `PascalCase` (e.g., `kHeaderContentType`, `kMaxPrecision`)
80+
- **Memory Management**: Prefer smart pointers (`std::unique_ptr`, `std::shared_ptr`)
81+
- **Error Handling**: Use `Result<T>` types for error propagation
82+
- **Documentation**: Use Doxygen-style comments for public APIs
83+
84+
#### API Compatibility
85+
86+
It is important to keep the C++ public API compatible across versions. Public methods should have no leading underscores and should not be removed without deprecation notice.
87+
88+
If you want to remove a method, please add a deprecation notice:
89+
90+
```cpp
91+
[[deprecated("This method will be removed in version 2.0.0. Use new_method() instead.")]]
92+
void old_method();
93+
```
94+
95+
#### Code Formatting
96+
97+
We use `clang-format` for code formatting. The configuration is defined in `.clang-format` file.
98+
99+
Format your code before submitting:
100+
101+
```bash
102+
clang-format -i src/**/*.{h,cc}
103+
```
104+
105+
### Testing
106+
107+
#### Running Tests
108+
109+
Run all tests:
110+
111+
```bash
112+
ctest --test-dir build --output-on-failure
113+
```
114+
115+
Run specific test:
116+
117+
```bash
118+
ctest --test-dir build -R test_name
119+
```
120+
121+
### Linting
122+
123+
Install the python package `pre-commit` and run once `pre-commit install`:
124+
125+
```bash
126+
pip install pre-commit
127+
pre-commit install
128+
```
129+
130+
This will setup a git pre-commit-hook that is executed on each commit and will report the linting problems. To run all hooks on all files use `pre-commit run -a`.
131+
132+
### Submitting Changes
133+
134+
#### Git Workflow
135+
136+
1. **Fork the repository** on GitHub
137+
2. **Create a feature branch** from `main`:
138+
```bash
139+
git checkout -b feature/your-feature-name
140+
```
141+
3. **Make your changes** following the coding standards
142+
4. **Add tests** for your changes
143+
5. **Run tests** to ensure everything passes
144+
6. **Commit your changes** with a clear commit message
145+
7. **Push to your fork** and create a Pull Request
146+
147+
#### Commit Message Format
148+
149+
Use clear, descriptive commit messages:
150+
151+
```
152+
feat: add support for S3 file system
153+
fix: resolve memory leak in table reader
154+
docs: update API documentation
155+
test: add unit tests for schema validation
156+
```
157+
158+
#### Pull Request Process
159+
160+
1. **Create a Pull Request** with a clear description
161+
2. **Link related issues** if applicable
162+
3. **Ensure CI passes** - all tests must pass
163+
4. **Request review** from maintainers
164+
5. **Address feedback** and update the PR as needed
165+
6. **Squash commits** if requested by reviewers
166+
167+
### Community
168+
169+
The Apache Iceberg community is built on the principles described in the [Apache Way](https://www.apache.org/theapacheway/index.html) and all who engage with the community are expected to be respectful, open, come with the best interests of the community in mind, and abide by the Apache Foundation [Code of Conduct](https://www.apache.org/foundation/policies/conduct.html).
170+
171+
#### Getting Help
172+
173+
- **Submit Issues**: [GitHub Issues](https://github.com/apache/iceberg-cpp/issues/new) for bug reports or feature requests
174+
- **Mailing List**: [dev@iceberg.apache.org](mailto:dev@iceberg.apache.org) for discussions
175+
- [Subscribe](mailto:dev-subscribe@iceberg.apache.org?subject=(send%20this%20email%20to%20subscribe))
176+
- [Unsubscribe](mailto:dev-unsubscribe@iceberg.apache.org?subject=(send%20this%20email%20to%20unsubscribe))
177+
- [Archives](https://lists.apache.org/list.html?dev@iceberg.apache.org)
178+
- **Slack**: [Apache Iceberg Slack #cpp channel](https://join.slack.com/t/apache-iceberg/shared_invite/zt-1zbov3k6e-KtJfoaxp97YfX6dPz1Bk7A)
179+
180+
#### Good First Issues
181+
182+
New to the project? Check out our [good first issues](https://github.com/apache/iceberg-cpp/labels/good%20first%20issue) for an easy way to get started.
183+
184+
### Release Process
185+
186+
Releases are managed by the Apache Iceberg project maintainers. For information about the release process, please refer to the main Iceberg project documentation.
187+
188+
## License
189+
190+
Licensed under the [Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0)
Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
<!--
2+
~ Licensed to the Apache Software Foundation (ASF) under one
3+
~ or more contributor license agreements. See the NOTICE file
4+
~ distributed with this work for additional information
5+
~ regarding copyright ownership. The ASF licenses this file
6+
~ to you under the Apache License, Version 2.0 (the
7+
~ "License"); you may not use this file except in compliance
8+
~ with the License. You may obtain a copy of the License at
9+
~
10+
~ http://www.apache.org/licenses/LICENSE-2.0
11+
~
12+
~ Unless required by applicable law or agreed to in writing,
13+
~ software distributed under the License is distributed on an
14+
~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
~ KIND, either express or implied. See the License for the
16+
~ specific language governing permissions and limitations
17+
~ under the License.
18+
-->
19+
20+
# Getting Started
21+
22+
## Requirements
23+
24+
- CMake 3.25 or higher
25+
- C++23 compliant compiler
26+
27+
## Customizing Dependency URLs
28+
29+
If you experience network issues when downloading dependencies, you can customize the download URLs using environment variables.
30+
31+
The following environment variables can be set to customize dependency URLs:
32+
33+
- `ICEBERG_ARROW_URL`: Apache Arrow tarball URL
34+
- `ICEBERG_AVRO_URL`: Apache Avro tarball URL
35+
- `ICEBERG_AVRO_GIT_URL`: Apache Avro git repository URL
36+
- `ICEBERG_NANOARROW_URL`: Nanoarrow tarball URL
37+
- `ICEBERG_CROARING_URL`: CRoaring tarball URL
38+
- `ICEBERG_NLOHMANN_JSON_URL`: nlohmann-json tarball URL
39+
- `ICEBERG_CPR_URL`: cpr tarball URL
40+
41+
Example usage:
42+
43+
```bash
44+
export ICEBERG_ARROW_URL="https://your-mirror.com/apache-arrow-22.0.0.tar.gz"
45+
cmake -S . -B build
46+
```
47+
48+
## Build
49+
50+
### Build, Run Test and Install Core Libraries
51+
52+
```bash
53+
cd iceberg-cpp
54+
cmake -S . -B build -G Ninja -DCMAKE_INSTALL_PREFIX=/path/to/install -DICEBERG_BUILD_STATIC=ON -DICEBERG_BUILD_SHARED=ON
55+
cmake --build build
56+
ctest --test-dir build --output-on-failure
57+
cmake --install build
58+
```
59+
60+
### Build and Install Iceberg Bundle Library
61+
62+
#### Vendored Apache Arrow (default)
63+
64+
```bash
65+
cmake -S . -B build -G Ninja -DCMAKE_INSTALL_PREFIX=/path/to/install -DICEBERG_BUILD_BUNDLE=ON
66+
cmake --build build
67+
ctest --test-dir build --output-on-failure
68+
cmake --install build
69+
```
70+
71+
#### Provided Apache Arrow
72+
73+
```bash
74+
cmake -S . -B build -G Ninja -DCMAKE_INSTALL_PREFIX=/path/to/install -DCMAKE_PREFIX_PATH=/path/to/arrow -DICEBERG_BUILD_BUNDLE=ON
75+
cmake --build build
76+
ctest --test-dir build --output-on-failure
77+
cmake --install build
78+
```
79+
80+
### Build Examples
81+
82+
After installing the core libraries, you can build the examples:
83+
84+
```bash
85+
cd iceberg-cpp/example
86+
cmake -S . -B build -G Ninja -DCMAKE_PREFIX_PATH=/path/to/install
87+
cmake --build build
88+
```
89+
90+
If you are using provided Apache Arrow, you need to include `/path/to/arrow` in `CMAKE_PREFIX_PATH` as below.
91+
92+
```bash
93+
cmake -S . -B build -G Ninja -DCMAKE_PREFIX_PATH="/path/to/install;/path/to/arrow"
94+
```

0 commit comments

Comments
 (0)