Skip to content

Commit b8f9229

Browse files
author
hknokh2
committed
feat: release v5-beta with major engine redesign and migration behavior updates
New major version: - Released major version after runtime model, CSV behavior, and file pipeline redesign. - Breaking change: standalone Node.js/module execution removed; use `sf sfdmu run`. - Breaking change: `--verbose`, `--concise`, and `--usesf` runtime effect removed. These flags are now legacy compatibility no-op flags. - Breaking CSV behavior: relationship `__r` columns are now generated consistently. `excludeIdsFromCSVFiles` removes raw `Id` and lookup `...Id` columns. - Breaking file migration update: transfer was centralized in `core:ExportFiles`. This now covers `org -> org`, `org -> csvfile`, and `csvfile -> org`. - Breaking runtime defaults: `--filelog` default changed to `0`. `--diagnostic` is now the recommended mode for full troubleshooting output. - Validation hardening: default `apiVersion` moved to `65.0`. Script/object/operation validation is stricter and fails faster on invalid configs. - Major refactor: engine rebuilt into API/CSV/mapping/logging modular layers. Added dedicated Bulk v1, Bulk v2, and REST execution engines. - Major add-on platform update: custom add-on interfaces moved to `custom-addon-sdk`. Added template scaffolding and updated runtime adapters. New features: - Added Data Loader-oriented CSV and file workflow in the new engine. Includes unified handling for `ContentVersion`, `ContentDocumentLink`, `Attachment`, and `Note`. - Added expanded `core:ExportFiles` behavior for all migration directions. Improved binary-path handling for org and csvfile flows. - Added explicit CSV controls: `csvFileDelimiter`, `csvFileEncoding`, `csvInsertNulls`, `csvUseEuropeanDateFormat`, `csvWriteUpperCaseHeaders`, `csvUseUtf8Bom`. - Added `--anonymise` flag for `.log` files with deterministic per-run hashes. Helps share diagnostics without exposing secrets, domains, emails, paths. Deprecations/Removed features: - Deprecated `csvReadFileDelimiter` and `csvWriteFileDelimiter`. Use `csvFileDelimiter` for current configurations. - Removed standalone Node.js/module execution mode. Migration runs through `sf sfdmu run`. - Deprecated runtime effect of `--verbose`, `--concise`, and `--usesf`. These flags are now legacy no-op compatibility flags. Fixes: - Fixed CSV id/reference edge cases by tightening `Id`, lookup-id, and `__r`. Reduced broken parent and lookup resolution during migration. - Fixed file pipeline consistency by centralizing processing in `core:ExportFiles`. Reduced format drift between org and csvfile paths. Improvements: - Improved validation behavior with `apiVersion=65.0` and stricter checks. Invalid configs fail earlier and more predictably. - Changed diagnostic defaults: `--filelog` now defaults to `0`. `--diagnostic` is the recommended troubleshooting mode. - Improved support workflow security with complete `--anonymise` docs. Added full masking matrix of what is hashed and what stays unchanged.
1 parent 14e5d96 commit b8f9229

314 files changed

Lines changed: 53589 additions & 21777 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.eslintignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
*.cjs/

.eslintrc.cjs

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
module.exports = {
2+
extends: ['eslint-config-salesforce-typescript', 'plugin:sf-plugin/recommended'],
3+
root: true,
4+
rules: {
5+
header: 'off',
6+
'no-underscore-dangle': 'off',
7+
},
8+
};

.github/ISSUE_TEMPLATE/critical-runtime--errors-report.md

Lines changed: 0 additions & 28 deletions
This file was deleted.
Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
---
2+
name: Feature Request
3+
about: Suggest a user-facing improvement for SFDMU.
4+
title: '[FEATURE REQUEST] - Replace this placeholder with a short and specific title.'
5+
labels: feature-request
6+
assignees: hknokh
7+
---
8+
9+
**Feature title**
10+
Use a short and specific title that clearly describes the requested feature.
11+
-- Add your title here --
12+
13+
**Problem statement**
14+
Describe the current limitation or pain point this feature should solve.
15+
-- Add your problem statement here --
16+
17+
**Proposed solution**
18+
Describe the behavior you want SFDMU to support.
19+
-- Add your proposed solution here --
20+
21+
**Expected user impact**
22+
Describe how this feature improves migration results, reliability, or usability.
23+
-- Add expected user impact here --
24+
25+
**Configuration example (if applicable)**
26+
Provide a concrete `export.json` or command example for the proposed behavior.
27+
-- Add your configuration example here --
28+
29+
**Alternatives considered**
30+
Describe any workarounds or alternatives you tried and why they are insufficient.
31+
-- Add alternatives here --
32+
33+
**Additional context**
34+
Add references, screenshots, links, or related issues.
35+
-- Add additional context here --
Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
---
2+
name: Issue Report
3+
about: Report an issue with required diagnostic artifacts for investigation.
4+
title: '[ISSUE] - Replace this placeholder with a short and specific title.'
5+
labels: bug
6+
assignees: hknokh
7+
---
8+
9+
**Issue title**
10+
Use a short and specific title that clearly describes the problem.
11+
-- Add your title here --
12+
13+
**Issue summary**
14+
Describe what failed, what operation you ran, and the exact error message.
15+
-- Add your issue summary here --
16+
17+
**Steps to reproduce**
18+
List exact steps and command line used.
19+
-- Add steps to reproduce here --
20+
21+
**Expected behavior**
22+
Describe what you expected to happen.
23+
-- Add expected behavior here --
24+
25+
**Actual behavior**
26+
Describe what actually happened.
27+
-- Add actual behavior here --
28+
29+
**Log file (required)**
30+
Before submitting this issue, run SFDMU with `--diagnostic --anonymise` to generate a shareable log file.
31+
Example command: `sf sfdmu run --sourceusername source@name.com --targetusername target@name.com --diagnostic --anonymise`
32+
Attach your full generated **.log** file from that run.
33+
If `--diagnostic` was not used or the full `.log` file is missing, **we cannot review this issue**.
34+
`--anonymise` is required before sharing logs to hash sensitive values.
35+
Reference (what is hashed and what is not): https://help.sfdmu.com/full-documentation/reports/the-execution-log#what-is-masked-and-what-is-not
36+
-- Add your log file here (attach the .log file or link to it) --
37+
38+
**\_target.csv file (if relevant)**
39+
If there are failed rows, attach a dump of the **\_target.csv** file with error messages (include at least 1-2 full relevant rows).
40+
Reference: https://help.sfdmu.com/full-documentation/reports/the-target-csv-files
41+
-- Add your \_target.csv file here (attach the file or link to it) --

.github/ISSUE_TEMPLATE/question.md

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
---
2+
name: Question
3+
about: Ask a question about SFDMU behavior or usage.
4+
title: '[QUESTION] - Replace this placeholder with a short and specific title.'
5+
labels: help-wanted
6+
assignees: hknokh
7+
---
8+
9+
**Question title**
10+
Use a short and specific title that clearly describes your question.
11+
-- Add your title here --
12+
13+
**Question summary**
14+
Describe what you are trying to do and what is unclear.
15+
-- Add your question summary here --
16+
17+
**Current setup**
18+
Share the relevant command, flags, and the related `export.json` fragment.
19+
-- Add your setup details here --
20+
21+
**What you already checked**
22+
List docs/articles you already read and what did not answer your case.
23+
-- Add what you already checked here --
24+
25+
**Expected clarification**
26+
Describe what exact answer or guidance you need.
27+
-- Add expected clarification here --
28+
29+
**Log file (optional, recommended for runtime/migration questions)**
30+
For better understanding of your question, if it is related to an actual migration run,
31+
run SFDMU with `--diagnostic --anonymise` and attach the full `.log` file.
32+
Example command: `sf sfdmu run --sourceusername source@name.com --targetusername target@name.com --diagnostic --anonymise`
33+
Reference (what is hashed and what is not): https://help.sfdmu.com/full-documentation/reports/the-execution-log#what-is-masked-and-what-is-not
34+
-- Add your log file here (attach the .log file or link to it) --

.github/dependabot.yml

Lines changed: 45 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,47 @@
11
version: 2
22
updates:
3-
- package-ecosystem: npm
4-
directory: "/"
5-
schedule:
6-
interval: daily
7-
time: "07:00"
8-
allow:
9-
- dependency-type: "direct" # Optional: Focus on direct dependencies
10-
open-pull-requests-limit: 10
11-
versioning-strategy: increase
12-
ignore:
13-
- dependency-name: "csv-parse"
14-
- dependency-name: "globby"
15-
- dependency-name: "madge"
16-
- dependency-name: "mocha"
17-
- dependency-name: "nyc"
18-
- dependency-name: "@oclif/*"
19-
- dependency-name: "soql-parser-js"
20-
- dependency-name: "@babel/*"
3+
- package-ecosystem: npm
4+
directory: '/'
5+
schedule:
6+
interval: weekly
7+
day: monday
8+
time: '07:00'
9+
timezone: 'UTC'
10+
open-pull-requests-limit: 10
11+
versioning-strategy: increase
12+
rebase-strategy: auto
13+
groups:
14+
sf-cli-and-oclif:
15+
patterns:
16+
- '@salesforce/*'
17+
- '@oclif/*'
18+
- 'oclif'
19+
third-party-runtime:
20+
patterns:
21+
- 'soql-parser-js'
22+
- 'alasql'
23+
- 'casual'
24+
- 'class-transformer'
25+
- 'csv-parse'
26+
- 'csv-writer'
27+
- 'es6-shim'
28+
- 'glob'
29+
- 'promise-parallel-throttle'
30+
- 'reflect-metadata'
31+
build-and-test-tooling:
32+
patterns:
33+
- '@jsforce/jsforce-node'
34+
- 'typescript'
35+
- 'ts-node'
36+
- 'eslint*'
37+
- 'mocha'
38+
- 'nyc'
39+
- package-ecosystem: github-actions
40+
directory: '/'
41+
schedule:
42+
interval: weekly
43+
day: monday
44+
time: '07:30'
45+
timezone: 'UTC'
46+
open-pull-requests-limit: 5
47+
rebase-strategy: auto

.github/workflows/codeql-analysis.yml

Lines changed: 0 additions & 39 deletions
This file was deleted.

.github/workflows/stale.yml

Lines changed: 0 additions & 46 deletions
This file was deleted.

.gitignore

Lines changed: 63 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,26 +1,73 @@
1+
# -- CLEAN
2+
tmp/
3+
# use yarn by default, so ignore npm
4+
package-lock.json
5+
6+
# never checkin npm config
7+
.npmrc
8+
9+
# debug logs
10+
npm-error.log
11+
yarn-error.log
12+
13+
14+
# compile source
15+
lib
16+
17+
# test artifacts
18+
*xunit.xml
19+
*checkstyle.xml
20+
*unitcoverage
21+
.nyc_output
22+
coverage
23+
test_session*
24+
25+
# generated docs
26+
docs
27+
28+
# ignore sfdx-trust files
29+
*.tgz
30+
*.sig
31+
package.json.bak.
32+
33+
34+
npm-shrinkwrap.json
35+
oclif.manifest.json
36+
oclif.lock
37+
38+
# -- CLEAN ALL
39+
*.tsbuildinfo
40+
.eslintcache
41+
.wireit
42+
node_modules
43+
44+
# --
45+
# put files here you don't want cleaned with sf-clean
46+
47+
# os specific files
48+
.DS_Store
49+
.idea
50+
51+
# Project-specific rules from .gitignore-new
52+
custom-addon-sdk/dist
53+
custom-addon-sdk/custom-modules/**/dist
54+
custom-addon-sdk/custom-modules/**/*
55+
!custom-addon-sdk/custom-modules/.gitkeep
56+
!custom-addon-sdk/custom-modules/demo/
57+
!custom-addon-sdk/custom-modules/demo/**
58+
custom-addon-sdk/custom-modules/demo/dist
59+
custom-addon-sdk/custom-modules/demo/dist/**
60+
61+
tmp*
162
*-debug.log
263
*-error.log
3-
/.nyc_output
4-
/dist
5-
/dist2
6-
/lib
7-
/package-lock.json
8-
/tmp
9-
node_modules
1064
/export*.json
1165
._*
12-
.DS_Store
1366
/logs
1467
/source
1568
/target
16-
/development
17-
/documentation
18-
/src/addons/modules/sfdmu-run/custom-addons/TestModule*
19-
typedoc-sfdmu-run-addons.json
20-
/temp
2169
*-private*
2270
/.vscode
23-
/.images
71+
/development
72+
CHANGELOG-DEV.md
2473
AGENTS.md
25-
26-

0 commit comments

Comments
 (0)