Skip to content

Commit 04c7f04

Browse files
author
Anna Mikhaylova
committed
[DOP-27545] fix changelog
1 parent 5da5391 commit 04c7f04

8 files changed

Lines changed: 62 additions & 62 deletions

File tree

mddocs/changelog.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
# Changelog { #changelog }
22

3-
- [0.2.5 [UNRELEASED] (2025-09-30)][DRAFT]
4-
- [0.2.4 (2026-03-05)][0.3.2]
5-
- [0.2.4 (2026-01-15)][0.3.1]
6-
- [0.2.4 (2025-12-18)][0.3.0]
7-
- [0.2.4 (2025-10-10)][0.2.5]
3+
- [0.3.3 [UNRELEASED] (2025-09-30)][DRAFT]
4+
- [0.3.2 (2026-03-05)][0.3.2]
5+
- [0.3.1 (2026-01-15)][0.3.1]
6+
- [0.3.0 (2025-12-18)][0.3.0]
7+
- [0.2.5 (2025-10-10)][0.2.5]
88
- [0.2.4 (2025-07-09)][0.2.4]
99
- [0.2.3 (2025-04-11)][0.2.3]
1010
- [0.2.2 (2025-04-11)][0.2.2]

mddocs/changelog/0.1.2.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -2,11 +2,11 @@
22

33
## Features
44

5-
- Rename `syncmaster.backend.main` module to `syncmaster.backend`. It now also accepts the same args as `uvicorn`. ([#35](https://github.com/MobileTeleSystems/syncmaster/issues/35))
6-
- Add `syncmaster.db.migrations` module to run `alembic` with proper config. ([#35](https://github.com/MobileTeleSystems/syncmaster/issues/35))
7-
- Change backend and worker images to use entrypoint. ([#35](https://github.com/MobileTeleSystems/syncmaster/issues/35))
5+
- Rename `syncmaster.backend.main` module to `syncmaster.backend`. It now also accepts the same args as `uvicorn`. [#35](https://github.com/MobileTeleSystems/syncmaster/issues/35)
6+
- Add `syncmaster.db.migrations` module to run `alembic` with proper config. [#35](https://github.com/MobileTeleSystems/syncmaster/issues/35)
7+
- Change backend and worker images to use entrypoint. [#35](https://github.com/MobileTeleSystems/syncmaster/issues/35)
88

99
## Improvements
1010

11-
- Run database migrations in the entrypoint of backend image, before starting the backend server. ([#35](https://github.com/MobileTeleSystems/syncmaster/issues/35))
12-
- Add healthchecks to example `docker-compose.yml`. ([#35](https://github.com/MobileTeleSystems/syncmaster/issues/35))
11+
- Run database migrations in the entrypoint of backend image, before starting the backend server. [#35](https://github.com/MobileTeleSystems/syncmaster/issues/35)
12+
- Add healthchecks to example `docker-compose.yml`. [#35](https://github.com/MobileTeleSystems/syncmaster/issues/35)

mddocs/changelog/0.1.5.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -3,15 +3,15 @@
33
## Breaking Changes
44

55
- Pass current `Run` to `CREATE_SPARK_SESSION_FUNCTION`. This allows using run/transfer/group information for Spark session options,
6-
like `appName` or custom ones. ([#38](https://github.com/MobileTeleSystems/syncmaster/issues/38))
6+
like `appName` or custom ones. [#38](https://github.com/MobileTeleSystems/syncmaster/issues/38)
77

88
## Improvements
99

10-
- Reduce backend image size ([#44](https://github.com/MobileTeleSystems/syncmaster/issues/44))
10+
- Reduce backend image size [#44](https://github.com/MobileTeleSystems/syncmaster/issues/44)
1111

1212
## Bug Fixes
1313

14-
- Fix 500 error in case of `PATCH v1/connections/:id` request with passed `auth_data.password` field value ([#39](https://github.com/MobileTeleSystems/syncmaster/issues/39))
15-
- Do not use `asyncio.gather` with SQLAlchemy requests ([#40](https://github.com/MobileTeleSystems/syncmaster/issues/40))
16-
- Fix 500 error while creating HDFS connection ([#41](https://github.com/MobileTeleSystems/syncmaster/issues/41))
14+
- Fix 500 error in case of `PATCH v1/connections/:id` request with passed `auth_data.password` field value [#39](https://github.com/MobileTeleSystems/syncmaster/issues/39)
15+
- Do not use `asyncio.gather` with SQLAlchemy requests [#40](https://github.com/MobileTeleSystems/syncmaster/issues/40)
16+
- Fix 500 error while creating HDFS connection [#41](https://github.com/MobileTeleSystems/syncmaster/issues/41)
1717
- Fix missing `options` field from Transfer params with `hdfs` and `s3` type

mddocs/changelog/0.2.0.md

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -13,15 +13,15 @@ Huge thanks to [Dmitry Pedchenko](https://github.com/dmitry-pedchenko), [Maxim L
1313

1414
## Breaking Changes
1515

16-
- Implement a single error handling format to improve consistency ([#95](https://github.com/MobileTeleSystems/syncmaster/issues/95))
17-
- Change response format for `GET /v1/groups` - add **current user role** for each group ([#97](https://github.com/MobileTeleSystems/syncmaster/issues/97))
18-
- Change response format for `GET /v1/groups/:id` - add **current user role** for group ([#109](https://github.com/MobileTeleSystems/syncmaster/issues/109))
19-
- Now migrations are executed in a dedicated one-off container, instead of being run as a part of `backend` container. ([#163](https://github.com/MobileTeleSystems/syncmaster/issues/163))
20-
- Delete **transfers**, **queues** and **groups** records instead of marking them as deleted ([#168](https://github.com/MobileTeleSystems/syncmaster/issues/168))
21-
- Move the `type` field from nested `connection_data` field to the root level of the `Connection` response. ([#169](https://github.com/MobileTeleSystems/syncmaster/issues/169))
22-
- Decouple `auth_data.type` from the connection type (e.g. `postgres`) and link it to the authentication type (e.g. `basic`). ([#169](https://github.com/MobileTeleSystems/syncmaster/issues/169))
23-
- Add deletion of **connections** records instead of marking them as deleted ([#170](https://github.com/MobileTeleSystems/syncmaster/issues/170))
24-
- Use PUT instead of PATCH for `Connection` and `Transfer` models ([#215](https://github.com/MobileTeleSystems/syncmaster/issues/215))
16+
- Implement a single error handling format to improve consistency [#95](https://github.com/MobileTeleSystems/syncmaster/issues/95)
17+
- Change response format for `GET /v1/groups` - add **current user role** for each group [#97](https://github.com/MobileTeleSystems/syncmaster/issues/97)
18+
- Change response format for `GET /v1/groups/:id` - add **current user role** for group [#109](https://github.com/MobileTeleSystems/syncmaster/issues/109)
19+
- Now migrations are executed in a dedicated one-off container, instead of being run as a part of `backend` container. [#163](https://github.com/MobileTeleSystems/syncmaster/issues/163)
20+
- Delete **transfers**, **queues** and **groups** records instead of marking them as deleted [#168](https://github.com/MobileTeleSystems/syncmaster/issues/168)
21+
- Move the `type` field from nested `connection_data` field to the root level of the `Connection` response. [#169](https://github.com/MobileTeleSystems/syncmaster/issues/169)
22+
- Decouple `auth_data.type` from the connection type (e.g. `postgres`) and link it to the authentication type (e.g. `basic`). [#169](https://github.com/MobileTeleSystems/syncmaster/issues/169)
23+
- Add deletion of **connections** records instead of marking them as deleted [#170](https://github.com/MobileTeleSystems/syncmaster/issues/170)
24+
- Use PUT instead of PATCH for `Connection` and `Transfer` models [#215](https://github.com/MobileTeleSystems/syncmaster/issues/215)
2525
- Return new connection and transfer object in `POST /v1/transfer/copy` and `POST /v1/connection/copy` endpoints.
2626
- Change response status from 200 to 204 for all `DELETE` endpoints.
2727

mddocs/changelog/0.3.0.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ auth:
5252
5353
## Features
5454
55-
- Added Iceberg support (([#282](https://github.com/MobileTeleSystems/syncmaster/issues/282)), ([#284](https://github.com/MobileTeleSystems/syncmaster/issues/284)), ([#294](https://github.com/MobileTeleSystems/syncmaster/issues/294)), ([#297](https://github.com/MobileTeleSystems/syncmaster/issues/297))).
55+
- Added Iceberg support ([#282](https://github.com/MobileTeleSystems/syncmaster/issues/282), [#284](https://github.com/MobileTeleSystems/syncmaster/issues/284), [#294](https://github.com/MobileTeleSystems/syncmaster/issues/294), [#297](https://github.com/MobileTeleSystems/syncmaster/issues/297)).
5656
5757
Iceberg connection currently supports only Iceberg REST Catalog with S3 warehouse.
5858

mddocs/changelog/0.3.2.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,4 @@
22

33
## Features
44

5-
Implement SQL transformations for transfer (([#327](https://github.com/MobileTeleSystems/syncmaster/issues/327)), ([#330](https://github.com/MobileTeleSystems/syncmaster/issues/255)))
5+
Implement SQL transformations for transfer ([#327](https://github.com/MobileTeleSystems/syncmaster/issues/327), [#330](https://github.com/MobileTeleSystems/syncmaster/issues/255))

mddocs/changelog/index.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
# Changelog { #changelog }
22

3-
* [0.2.5 [UNRELEASED] (2025-09-30)][DRAFT]
4-
* [0.2.4 (2026-03-05)][0.3.2]
5-
* [0.2.4 (2026-01-15)][0.3.1]
6-
* [0.2.4 (2025-12-18)][0.3.0]
7-
* [0.2.4 (2025-10-10)][0.2.5]
3+
* [0.3.3 [UNRELEASED]][DRAFT]
4+
* [0.3.2 (2026-03-05)][0.3.2]
5+
* [0.3.1 (2026-01-15)][0.3.1]
6+
* [0.3.0 (2025-12-18)][0.3.0]
7+
* [0.2.5 (2025-10-10)][0.2.5]
88
* [0.2.4 (2025-07-09)][0.2.4]
99
* [0.2.3 (2025-04-11)][0.2.3]
1010
* [0.2.2 (2025-04-11)][0.2.2]

mddocs/reference/worker/create_spark_session.md

Lines changed: 31 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -10,27 +10,27 @@ It is possible to alter default [Spark Session configuration](https://spark.apac
1010

1111
```yaml
1212
worker:
13-
spark_session_default_config:
14-
spark.master: local
15-
spark.driver.host: 127.0.0.1
16-
spark.driver.bindAddress: 0.0.0.0
17-
spark.sql.pyspark.jvmStacktrace.enabled: true
18-
spark.ui.enabled: false
13+
spark_session_default_config:
14+
spark.master: local
15+
spark.driver.host: 127.0.0.1
16+
spark.driver.bindAddress: 0.0.0.0
17+
spark.sql.pyspark.jvmStacktrace.enabled: true
18+
spark.ui.enabled: false
1919
```
2020
2121
For example, to use SyncMaster on Spark-on-K8s, you can use worker image for Spark executor containers:
2222
2323
```yaml
24-
worker:
25-
spark_session_default_config:
26-
spark.master: k8s://https://kubernetes.default.svc
27-
spark.driver.host: service-for-spark-driver
28-
spark.driver.bindAddress: 0.0.0.0
29-
spark.driver.port: 10000
30-
spark.blockManager.port: 10001
31-
spark.kubernetes.authenticate.driver.serviceAccountName: spark
32-
spark.sql.pyspark.jvmStacktrace.enabled: true
33-
spark.kubernetes.container.image: mtsrus/syncmaster-worker:{TAG}
24+
worker:
25+
spark_session_default_config:
26+
spark.master: k8s://https://kubernetes.default.svc
27+
spark.driver.host: service-for-spark-driver
28+
spark.driver.bindAddress: 0.0.0.0
29+
spark.driver.port: 10000
30+
spark.blockManager.port: 10001
31+
spark.kubernetes.authenticate.driver.serviceAccountName: spark
32+
spark.sql.pyspark.jvmStacktrace.enabled: true
33+
spark.kubernetes.container.image: mtsrus/syncmaster-worker:{TAG}
3434
```
3535
3636
!!! note
@@ -42,26 +42,26 @@ For example, to use SyncMaster on Spark-on-K8s, you can use worker image for Spa
4242
It is also possible to use custom function which returns ``SparkSession`` object:
4343
4444
```yaml
45-
worker:
46-
create_spark_session_function: my_worker.spark.create_custom_spark_session
45+
worker:
46+
create_spark_session_function: my_worker.spark.create_custom_spark_session
4747
```
4848
4949
Here is a function example:
5050
51-
```python
52-
from syncmaster.db.models import Run
53-
from syncmaster.dto.connections import ConnectionDTO
54-
from syncmaster.worker.settings import WorkerSettings
55-
from pyspark.sql import SparkSession
51+
```python title="my_workers/spark.py"
52+
from syncmaster.db.models import Run
53+
from syncmaster.dto.connections import ConnectionDTO
54+
from syncmaster.worker.settings import WorkerSettings
55+
from pyspark.sql import SparkSession
5656

57-
def create_custom_spark_session(
58-
run: Run,
59-
source: ConnectionDTO,
60-
target: ConnectionDTO,
61-
settings: WorkerSettings,
62-
) -> SparkSession:
63-
# any custom code returning SparkSession object
64-
return SparkSession.builde.config(...).getOrCreate()
57+
def create_custom_spark_session(
58+
run: Run,
59+
source: ConnectionDTO,
60+
target: ConnectionDTO,
61+
settings: WorkerSettings,
62+
) -> SparkSession:
63+
# any custom code returning SparkSession object
64+
return SparkSession.builde.config(...).getOrCreate()
6565
```
6666

6767
Module with custom function should be placed into the same Docker image or Python virtual environment used by SyncMaster worker.

0 commit comments

Comments
 (0)