Migration docs restructure#21245
Conversation
✅ Deploy Preview for cockroachdb-interactivetutorials-docs canceled.
|
Files changed:
|
✅ Deploy Preview for cockroachdb-api-docs canceled.
|
✅ Netlify Preview
To edit notification comments on pull requests, go to your Netlify project configuration. |
… molt-redux-phase-2
…draft diagram, improved readability
Migration doc rework phase 2
ryanluu12345
left a comment
ryanluu12345
left a comment
There was a problem hiding this comment.
Thanks for the iterations and refinement of the copy here!
taroface
left a comment
taroface
left a comment
There was a problem hiding this comment.
Some initial comments on Migration Overview! More as I continue to review.
| - Failback replication from CockroachDB back to source databases. | ||
| - [Performance tuning]({% link molt/molt-replicator.md %}#optimize-performance) for high-throughput workloads. | ||
| - [Performance tuning]({% link molt/molt-replicator-best-practices.md %}#optimize-performance) for high-throughput workloads. | ||
|
|
There was a problem hiding this comment.
Should we add something about userscripts here? Otherwise, these MOLT tooling feature lists feel a bit tricky to maintain. I wonder if the Overview can approach them differently, or just do without them. Perhaps something higher-level, and "for details, refer to X doc."
There was a problem hiding this comment.
@taroface I'm not so worried about maintainability, since this is all pretty high-level (and these tools won't change that drastically).
It's definitely worth thinking about though if we're moving into an IA that encourages the use of overview pages more broadly. How much detail should the overview page go into? Probably enough to be useful (to route a user to the child page that's useful to them) but not so much that it's highly redundant and/or difficult to maintain.
In this case though, userscripts feel like an anomaly (and I think it should've occurred to me in the userscript PR to update this page).
There was a problem hiding this comment.
I don't follow about why userscripts are an anomaly, but I agree with you about the purpose of the overview page. (Perhaps the pointers from overview into other pages are a thing we can automate as docs are added/removed.)
taroface
left a comment
taroface
left a comment
There was a problem hiding this comment.
Quick high-level comment on the Migration Considerations overview.
|
|
||
| ### Permissible downtime | ||
|
|
||
| How much downtime can your application tolerate during the migration? This is one of the most critical factors in determining your migration approach, and it may influence your choices for [migration granularity]({% link molt/migration-considerations-granularity.md %}), and [continuous replication]({% link molt/migration-considerations-replication.md %}). |
There was a problem hiding this comment.
This is one of the most critical factors in determining your migration approach
I'm a bit stuck on how the "Factors to consider" relate to the migration variables. The Migration Overview and intro to this page have set me up to consider the variables as the primary questions, but the above line reminds us that downtime is one of the foremost concerns (maybe the first concern) in planning a migration. Also, the Migration Overview mentions downtime in passing in a few places, but we don't get into it until this point midway down the next, nested Overview.
It's also unclear to me why downtime, timeframe, and risk tolerance are not also migration variables. Are migration variables like the nuts and bolts of the practical migration plan? If so, it seems the "factors to consider" would be higher-level.
A possible quick fix would be:
- Move "Factors to consider" to the top of this page.
- Link terms like "downtime" and "timeframe" (if it's there) in the Migration Overview directly to the subheadings on this page.
Another reason the "Factors to consider" work before the "Migration variables" is because each factor seems to impact the understanding of certain migration variables.
There was a problem hiding this comment.
@taroface In my head, "factors to consider" are basically independent variables (over which the migrators have little/no control) and the migration variables are dependent variables (downstream of those factors). I foreground the migration variables because those are the things around which decisions actually have to be made (versus e.g. permissible downtime, which is likely just a solid immovable business constraint).
Does this make sense? Does that change how you view this? I think moving "Factors to consider" above "Migration variables" could still work, but it also does make sense to me that the variables would be foregrounded. Idk I'm ambivalent and if you feel strongly I can come around to your view.
There was a problem hiding this comment.
I see now. I think we just need to spell that relationship out more. See above comment.
taroface
left a comment
taroface
left a comment
There was a problem hiding this comment.
Sorry this took me a long while! Most of my more substantial comments are in the earlier, higher-level pages. I appreciate the amount of work and thought that went into this.
taroface
left a comment
taroface
left a comment
There was a problem hiding this comment.
LGTM! Just some parting suggestions & comments.
|
|
||
| ### Permissible downtime | ||
|
|
||
| How much downtime can your application tolerate during the migration? This is one of the most critical factors in determining your migration approach, and it may influence your choices for [migration granularity]({% link molt/migration-considerations-granularity.md %}), and [continuous replication]({% link molt/migration-considerations-replication.md %}). |
There was a problem hiding this comment.
I see now. I think we just need to spell that relationship out more. See above comment.
| - Failback replication from CockroachDB back to source databases. | ||
| - [Performance tuning]({% link molt/molt-replicator.md %}#optimize-performance) for high-throughput workloads. | ||
| - [Performance tuning]({% link molt/molt-replicator-best-practices.md %}#optimize-performance) for high-throughput workloads. | ||
|
|
There was a problem hiding this comment.
I don't follow about why userscripts are an anomaly, but I agree with you about the purpose of the overview page. (Perhaps the pointers from overview into other pages are a thing we can automate as docs are added/removed.)
| @@ -0,0 +1,335 @@ | |||
| A [*Phased Bulk Load Migration*]({% link molt/migration-approach-phased-bulk-load.md %}) involves [migrating data to CockroachDB]({% link molt/migration-overview.md %}) in several phases. Data can be sliced per tenant, per service, per region, or per table to suit the needs of the migration. In this approach, you stop application traffic to the source database _only_ for the tables in a particular slice of data. You then migrate that phase of data to the target cluster using [MOLT Fetch]({% link molt/molt-fetch.md %}) during a **downtime window**. Application traffic is then cut over to those target tables after schema finalization and data verification. This process is repeated for each phase of data. | |||
There was a problem hiding this comment.
Is this relevant to the way the content reuse is scripted in this PR? Or a separate topic?
4 participants
This is a chunky PR, so here's a high-level description of the changes that have been made, so you can stay oriented.
Note that the line count that github provides for this PR (10000+ lines added) is super misleading due to how github counts .svg files (which are basically just images but which for some reason github interprets as many hundreds of new lines). The real number is far less than the one provided.
Phase 1: New guidance documentation
This is all of the content under the Migrate > Migration Considerations section of the nav. That includes:
I also moved Migration Strategies to Migrate > Migration Best Practices. It's mostly the same, but I moved some of its contents into the other guidance docs.
These pages will need the most significant tech review (though probably not from eng, probably from people in the field?) as I want to make sure that all of this strategic guidance is accurate.
Phase 2: Re-organizing the MOLT tooling docs
This is all of the content under Migrate > MOLT Tools, primarily under Fetch and Replicator. I broke up the contents by task, concept, and reference documentation. I also merged from main to include all of the Replicator userscript documentation. I did a lot of refining of the main Fetch and Replicator guides, including a new graphic in the Fetch guide. All of it is more readable and navigable now.
Phase 3: Source-db specific, step-by-step walkthroughs broken up by strategy
This is the content under Migrate > Common Migration Approaches. Mainly:
Note that each of these pages is an overview, that links to three source-DB-specific walkthroughs.
These pages re-work the content that was originally under "Migration Flows", but is more rigorously task-based, is broken up by different source DB types, tries to reduce info duplication, includes some graphics, and offers greater specificity about things like application downtime.
Other changes
A lot of files required changes to links, due to the other changes here. Various includes were removed/renamed. Therefore, many random files have minor, invisible changes (like links).