Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
9644114
Start replication PHP extension
jakubkulhan Sep 10, 2025
17f49fc
Commit Cargo.lock for reproducible extension builds
jakubkulhan Sep 10, 2025
70deff4
Add replication SPEC
jakubkulhan Sep 10, 2025
eac5697
SPEC - only include interfaces at compile time
jakubkulhan Sep 11, 2025
a222476
Add initial interface loading upon extension startup
jakubkulhan Sep 11, 2025
7e15728
Implement Stream class with Iterator interface in Rust using ext-php-rs
jakubkulhan Sep 11, 2025
517fb68
Add final readonly event classes with comprehensive property immutabi…
jakubkulhan Sep 12, 2025
6ff5ec8
Remove compiled binary extension file from repository
jakubkulhan Sep 12, 2025
09f51ce
Implement URL-based driver system with MySQLStreamDriver
jakubkulhan Sep 12, 2025
26776ee
Configure MySQL and MariaDB containers with replication settings for …
jakubkulhan Sep 12, 2025
2d46455
Add comprehensive Stream integration test for database replication flow
jakubkulhan Sep 12, 2025
e7453d9
Separate unit and database test groups with PHPUnit group annotations
jakubkulhan Sep 12, 2025
48c28da
Implement MySQL connection with configuration validation and comprehe…
jakubkulhan Sep 12, 2025
a9a4296
Implement MySQL StreamDriver with GTID-based binlog client initializa…
jakubkulhan Sep 12, 2025
b56f058
Update SPEC.md to reflect current implementation differences
jakubkulhan Sep 12, 2025
d5d17fa
Implement PHP event object creation from Rust and refactor to unified…
jakubkulhan Sep 12, 2025
3d820ea
Add MariaDB compatibility for GTID retrieval with database detection
jakubkulhan Sep 12, 2025
8ca0306
Remove ext-php-rs/ from .gitignore
jakubkulhan Sep 12, 2025
dc2d1cd
Implement real MySQL binlog streaming with proper event handling and …
jakubkulhan Sep 12, 2025
56b43c0
Add gtid_mode=ON validation for MySQL binlog configuration
jakubkulhan Sep 12, 2025
b93e782
Use table-metadata branch for proper MySQL column name support
jakubkulhan Sep 15, 2025
1ac2286
Implement dual checkpointing strategy with prefixed format for MySQL …
jakubkulhan Sep 15, 2025
081c98c
Add checkpointer functionality and integration tests
jakubkulhan Sep 15, 2025
4217f07
Implement set_filter method with StreamFilterInterface wrapper and ev…
jakubkulhan Sep 15, 2025
265d491
Optimize MySQL stream driver to use single reusable Tokio current thr…
jakubkulhan Sep 16, 2025
fcab302
Optimize dependencies by removing unused packages and adding minimal …
jakubkulhan Sep 16, 2025
58f3917
Set up cross-platform GitHub Actions CI for data-access-kit-replicati…
jakubkulhan Sep 16, 2025
2a6927b
Fix CI to install dev dependencies for replication tests including PH…
jakubkulhan Sep 16, 2025
ebfa5cd
Add workflow name to GitHub Actions test configuration
jakubkulhan Sep 16, 2025
7866b19
Add Rust/Cargo caching to GitHub Actions for faster replication test …
jakubkulhan Sep 16, 2025
78f58e6
Rename StreamTest to StreamInterfaceTest to better reflect its testin…
jakubkulhan Sep 16, 2025
c2d84dc
Remove SPEC.md
jakubkulhan Sep 16, 2025
c54e6a0
Add comprehensive README.md with cargo-php installation and usage exa…
jakubkulhan Sep 16, 2025
b4467aa
Add data-access-kit-replication to GitHub Actions split-and-push work…
jakubkulhan Sep 16, 2025
b61b402
Add name to GitHub Actions split-and-push workflow
jakubkulhan Sep 16, 2025
5659709
Add DataAccessKit\Replication package to README packages list
jakubkulhan Sep 16, 2025
cc6a12a
Update DataAccessKit\Replication README title to match package naming…
jakubkulhan Sep 16, 2025
2905fcc
Update mysql-binlog-connector-rust to use upstream repository
jakubkulhan Sep 16, 2025
ca135a2
Remove Debug traits, position field, database field, startup function…
jakubkulhan Sep 16, 2025
2e385be
Add Remove Extension section to README installation instructions
jakubkulhan Sep 16, 2025
0d7beed
Update README
jakubkulhan Sep 17, 2025
27ce200
Add comprehensive database type conversion tests and improve ENUM/SET…
jakubkulhan Sep 17, 2025
c4b637d
Update mysql-binlog-connector-rust to fix ENUM/SET parsing with prope…
jakubkulhan Sep 17, 2025
a31d957
Convert DATETIME and TIMESTAMP values to DateTimeImmutable instances …
jakubkulhan Sep 17, 2025
43fd931
Parse JSON values to stdClass objects and arrays in MySQL while maint…
jakubkulhan Sep 17, 2025
c9419d9
Add database flavor filtering to data type tests with separate JSON c…
jakubkulhan Sep 17, 2025
1896807
Update TEXT and BLOB types to use raw binary data instead of base64 e…
jakubkulhan Sep 17, 2025
4e73dd5
Replace PHP interface loading with native Rust implementation and reo…
jakubkulhan Sep 17, 2025
31e7477
Fix MySQL stream hanging on bulk operations by implementing event que…
jakubkulhan Sep 18, 2025
18da3e0
Update mysql-binlog-connector-rust to use upstream repository
jakubkulhan Sep 18, 2025
3542ff1
Refactor Rust imports to follow consistent rules: types imported dire…
jakubkulhan Sep 18, 2025
2e38b12
Apply cargo fmt and add formatting requirements to documentation and CI
jakubkulhan Sep 18, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .docker/replication-init.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
-- MySQL/MariaDB Replication Setup for Integration Tests

-- Create replication user
CREATE USER 'replication_test'@'%' IDENTIFIED BY 'replication_test';
GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'replication_test'@'%';

FLUSH PRIVILEGES;
3 changes: 3 additions & 0 deletions .github/workflows/split-and-push.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
name: Split & Push
on:
push:
branches:
Expand All @@ -11,6 +12,7 @@ jobs:
project:
- data-access-kit
- data-access-kit-symfony
- data-access-kit-replication
steps:
- uses: actions/checkout@v4
with:
Expand All @@ -24,5 +26,6 @@ jobs:
ssh-private-key: |
${{ secrets.DATA_ACCESS_KIT_DEPLOY_KEY }}
${{ secrets.DATA_ACCESS_KIT_SYMFONY_DEPLOY_KEY }}
${{ secrets.DATA_ACCESS_KIT_REPLICATION_DEPLOY_KEY }}
- run: git subtree split --prefix=${{ matrix.project }} --branch project-branch
- run: git push --force git@github.com:${{ github.repository_owner }}/${{ matrix.project }}.git project-branch:main
92 changes: 92 additions & 0 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
name: Tests
on: [push]
jobs:
test-unit:
Expand Down Expand Up @@ -54,3 +55,94 @@ jobs:
- run: composer test:database:env
env:
DATABASE_URL: ${{ matrix.database.database_url }}

test-replication-rust-fmt:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
with:
components: rustfmt
- name: Check Rust formatting
working-directory: ./data-access-kit-replication
run: cargo fmt -- --check

test-replication-unit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
- name: Cache Rust dependencies
uses: actions/cache@v3
with:
path: |
~/.cargo/bin/
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
data-access-kit-replication/target/
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
restore-keys: |
${{ runner.os }}-cargo-
- uses: shivammathur/setup-php@v2
with:
php-version: '8.4'
- name: Install Composer dependencies
working-directory: ./data-access-kit-replication
run: composer install
- name: Run unit tests
working-directory: ./data-access-kit-replication
run: composer run test:unit

test-replication-database:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
database:
- service: mysql
database_url: mysql://root@127.0.0.1:32016
replication_database_url: mysql://replication_test:replication_test@127.0.0.1:32016
- service: mariadb
database_url: mysql://root@127.0.0.1:35098
replication_database_url: mysql://replication_test:replication_test@127.0.0.1:35098
steps:
- uses: actions/checkout@v4
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
- name: Cache Rust dependencies
uses: actions/cache@v3
with:
path: |
~/.cargo/bin/
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
data-access-kit-replication/target/
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
restore-keys: |
${{ runner.os }}-cargo-
- uses: adambirds/docker-compose-action@v1.5.0
with:
compose-file: docker-compose.yaml
services: ${{ matrix.database.service }}
- name: Wait for container health
run: |
timeout 30s bash -c 'until docker inspect --format="{{.State.Health.Status}}" data-access-kit-src-${{ matrix.database.service }}-1 | grep -q "healthy"; do
echo "Waiting for container to be healthy..."
sleep 2
done'
- uses: shivammathur/setup-php@v2
with:
php-version: '8.4'
- name: Install Composer dependencies
working-directory: ./data-access-kit-replication
run: composer install
- name: Run database tests
working-directory: ./data-access-kit-replication
run: composer run test:database:env
env:
DATABASE_URL: ${{ matrix.database.database_url }}
REPLICATION_DATABASE_URL: ${{ matrix.database.replication_database_url }}
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@

- [DataAccessKit](https://github.com/jakubkulhan/data-access-kit#readme) - Persistence layer based on Doctrine\DBAL and repository generator.
- [DataAccessKit\Symfony](https://github.com/jakubkulhan/data-access-kit-symfony#readme) - Integration with Symfony framework.
- [DataAccessKit\Replication](https://github.com/jakubkulhan/data-access-kit-replication#readme) - Real-time MySQL/MariaDB binary log replication stream for PHP.

## Contributing

Expand Down
8 changes: 8 additions & 0 deletions data-access-kit-replication/.cargo/config.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
[target.'cfg(not(target_os = "windows"))']
rustflags = ["-C", "link-arg=-Wl,-undefined,dynamic_lookup"]

[target.x86_64-pc-windows-msvc]
linker = "rust-lld"

[target.i686-pc-windows-msvc]
linker = "rust-lld"
3 changes: 3 additions & 0 deletions data-access-kit-replication/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
target/
vendor/
.phpunit.result.cache
160 changes: 160 additions & 0 deletions data-access-kit-replication/CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
# DataAccessKit Replication - Development Notes

## Running Tests

### Unit Tests
To run only unit tests (fast, no database required):
```bash
composer run test:unit
```

### Database Integration Tests
To run database tests:
```bash
composer run test:database:all
```

To run database tests against specific databases:
```bash
composer run test:database:mysql # MySQL on port 32016
composer run test:database:mariadb # MariaDB on port 35098
```

All test commands will:
1. Build the Rust extension (`cargo build`)
2. Load the extension via the local `php.ini` configuration
3. Run the specified PHPUnit test groups

### Running All Tests
**The agent should always run both test suites to ensure complete validation:**
```bash
composer run test:unit # Run all unit tests (fast, no database)
composer run test:database:all # Run database tests against MySQL and MariaDB
```

### Test Groups
- **Unit tests** (`#[Group("unit")]`): Interface validation, event property tests - no database required
- **Database tests** (`#[Group("database")]`): Integration tests requiring DATABASE_URL environment variable

The tests ensure:
- Extension builds correctly
- Interfaces load automatically on startup
- All interface definitions are valid
- Extension integrates properly with PHP 8.4
- Database replication functionality works correctly

## Test Writing Guidelines

### Test Assertions Best Practices

**For tests that don't need explicit assertions:**
```php
public function testSomeAction(): void
{
// Use expectNotToPerformAssertions() at the start of the test
$this->expectNotToPerformAssertions();

// Test code that should complete without exceptions
$stream->setCheckpointer(null);
}
```

**Avoid using `addToAssertionCount()`** - it's an internal PHPUnit method and `expectNotToPerformAssertions()` is the proper public API.

### Test Structure

- **Unit tests**: Test individual components without external dependencies
- **Database tests**: Test full integration with real database connections
- Always clean up resources in `tearDown()` methods
- Use descriptive test method names that explain what is being tested
- **Do not add comments to PHP test files** - keep test code clean and minimal
- Group related assertions with clear, descriptive assertion messages

## Rust Code Guidelines

### Formatting Requirements

**Always run `cargo fmt` after making changes to Rust code.** This ensures consistent formatting across the codebase.

```bash
cargo fmt
```

The formatter will automatically:
- Organize imports alphabetically
- Apply consistent indentation
- Format code according to Rust style guidelines
- Ensure consistent spacing and line breaks

**Important:** Run `cargo fmt` before committing any Rust code changes.

## Rust Import Rules

### Import Organization Guidelines

When writing or refactoring Rust code in this project, follow these import rules:

1. **Use statements always at the top of the file, never inside functions**
- All `use` statements must be placed at the top of the file after any comments or attributes
- Never place `use` statements inside functions, methods, or other code blocks

2. **Types should be imported directly**
- Import types (structs, enums, traits) by their full path so they can be used directly
- Examples:
```rust
use std::ffi::CString;
use ext_php_rs::types::Zval;
use ext_php_rs::zend::ClassEntry;
```

3. **Functions should be used with module prefix**
- Import modules containing functions, then call functions with module prefix
- Examples:
```rust
use std::{mem, ptr};

// Then call:
mem::zeroed()
ptr::null_mut()
```

4. **Group related imports**
- Group imports from the same crate/module using braces
- Examples:
```rust
use std::{mem, ptr};
use std::collections::{HashMap, VecDeque};
use ext_php_rs::zend::{self, ce};
```

### Examples

**Good:**
```rust
use ext_php_rs::prelude::*;
use ext_php_rs::types::Zval;
use ext_php_rs::zend::{ClassEntry, ZendType};
use std::ffi::CString;
use std::{mem, ptr};

fn example() {
let interface_ce: ffi::zend_class_entry = mem::zeroed();
let null_ptr = ptr::null_mut();
}
```

**Bad:**
```rust
use ext_php_rs::prelude::*;

fn example() {
use std::mem; // ❌ use inside function
use std::ptr; // ❌ use inside function

let interface_ce = mem::zeroed();
}
```

## Documentation

The project specification is in `SPEC.md`. **Update SPEC.md when implementation diverges from the documented design** to keep documentation accurate and current.
Loading