Skip to content

Commit b2a2646

Browse files
drewoldagdelucchi-cmu
drewoldag
and
delucchi-cmu
authored
Adding documentation for pulling the template into existing projects. (#124)
* Initial draft of "Incorporating the template into a prior project" documentation. * Update docs/source/update_project.rst Co-authored-by: Melissa DeLucchi <113376043+delucchi-cmu@users.noreply.github.com> * Update docs/source/existing_project.rst Co-authored-by: Melissa DeLucchi <113376043+delucchi-cmu@users.noreply.github.com> * Update docs/source/existing_project.rst Co-authored-by: Melissa DeLucchi <113376043+delucchi-cmu@users.noreply.github.com> * Responded to PR comments. * Added warning about having hardcoded and dynamic version definitions in the same pyproject.toml file. --------- Co-authored-by: Melissa DeLucchi <113376043+delucchi-cmu@users.noreply.github.com>
1 parent 46256da commit b2a2646

4 files changed

Lines changed: 296 additions & 18 deletions

File tree

‎docs/index.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ It's really neat!
1515

1616
source/overview
1717
source/new_project
18+
source/update_project
1819
source/existing_project
1920
source/contributing
2021

‎docs/source/existing_project.rst‎

Lines changed: 210 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,212 @@
1-
Adding to an existing project
1+
Incorporating the template into a prior project
22
===============================================================================
33

4-
TODO - Coming soon!
4+
These instructions are for those who want to incorporate the latest version of the
5+
LINCC Frameworks Python Project Template into an existing project that has not
6+
previously used our template.
7+
8+
If you want to update a project that already uses the template, please see
9+
:doc:`Keeping your project up to date <update_project>`.
10+
11+
.. attention ::
12+
13+
Every pre-existing project is unique - it's a snowflake - even if the developers followed
14+
accepted guidelines and best practices.
15+
16+
Because of that, we can't we enumerate all the possible sticking points that you may
17+
encounter when attempting to incorporate the LINCC Frameworks Python Project Template
18+
into a pre-existing project.
19+
20+
.. tip ::
21+
22+
The newer a pre-existing project is, the easier it will be to incorporate the
23+
LINCC Frameworks Python Project Template.
24+
25+
That being said, we have used this template to bring dormant projects up to date with
26+
modern development standards.
27+
28+
Before you begin
29+
----------------
30+
31+
Start clean
32+
...........
33+
34+
It's best to start with a clean local repository and a new git branch just for this task.
35+
Copier will not allow the user to merge a template with a git tracked local repository
36+
if there are uncommitted changes.
37+
38+
It's not explicitly necessary, but removing ``cache`` and ``build`` directories will
39+
make it easier to see what changes are being made.
40+
41+
Check your environment
42+
......................
43+
44+
Make sure your system satisfies the :doc:`prerequisites <overview>`.
45+
46+
Hydrate a copy of the template into your project
47+
................................................
48+
49+
.. code:: bash
50+
51+
>> cd <existing_project_directory>
52+
>> copier gh:lincc-frameworks/python-project-template .
53+
54+
55+
A note on directory structure
56+
.............................
57+
58+
The LINCC Frameworks template does not impose a rigid directory structure.
59+
However, some of the tools that the template includes expect that
60+
the package source code will be in a first level directory named ``src``, and
61+
that tests will be contained in a first level directory named ``tests``.
62+
63+
For example, if your project is in a directory named ``/science``, your source
64+
code and tests might look like this:
65+
66+
.. code:: bash
67+
68+
/science/src/module.py
69+
/science/tests/test_module.py
70+
71+
If your directory structure for source code and tests is significantly
72+
different than this, it might be worth reorganizing the files in order to make use
73+
of the tools (CI, linting, auto documentation) that the template provides.
74+
75+
While running Copier
76+
--------------------
77+
78+
Reasonable responses to questions
79+
.................................
80+
If your project already has a name, especially if it's published on PyPI, it's
81+
advised that you use the same name when asked "What is the name of your project?".
82+
83+
If your project has a license already, and it is not one of the licenses listed
84+
in the "What license would you like to use?", select "None".
85+
86+
Respond **no** when asked "Do you want to create some example module code?".
87+
It's fine to respond "yes", however, given that your project already has an established
88+
source code and test directory structure, there's no benefit to including an example
89+
module.
90+
91+
Copier work summary
92+
...................
93+
94+
After Copier has received all your answers, it will begin to hydrate the template
95+
with your responses and inject them into your project.
96+
As it does so, don't be scared of ``conflict`` markers - remember that everything is git tracked,
97+
and Copier does not have the ability permanently overwrite your files.
98+
99+
The following example is output from a Copier update. Note again that ``conflict`` is
100+
simply an indicator that you should review that file before committing.
101+
102+
.. code :: bash
103+
104+
Copying from template version 1.2.1
105+
identical .
106+
identical README.md
107+
conflict .copier-answers.yml
108+
overwrite .copier-answers.yml
109+
identical .gitignore
110+
identical .github/workflows
111+
conflict .github/workflows/linting.yml
112+
overwrite .github/workflows/linting.yml
113+
identical nb/README.md
114+
conflict .pre-commit-config.yaml
115+
overwrite .pre-commit-config.yaml
116+
conflict pyproject.toml
117+
overwrite pyproject.toml
118+
119+
120+
After running Copier
121+
--------------------
122+
123+
Look at what changed
124+
....................
125+
126+
You should run ``git diff`` to see what code has changed.
127+
If you don't like the new changes, you can always revert back to the previous state.
128+
129+
Additionally, if Copier encounters a merge conflict between your existing code and
130+
the updated template, it will produce ``.rej`` files that contain the unresolved diffs.
131+
If you see a ``.rej`` file, resolve the merge conflict and check that your code
132+
was updated correctly.
133+
There is no need to commit ``.rej`` files, you should remove them as
134+
the merge conflicts are resolved.
135+
136+
Confirm that your package builds
137+
................................
138+
You should attempt to use ``pip`` to build your package and install dependencies.
139+
Failure to build successfully may be an indicator of a corrupted pyproject.toml file
140+
or missing dependencies.
141+
142+
.. code:: bash
143+
144+
>> pip install -e .
145+
>> pip install .'[dev]'
146+
147+
.. note::
148+
149+
If your existing package uses a setup.py file to build, you will need to move the
150+
important definitions over to pyproject.toml.
151+
152+
It's likely that you'll only need to move the list of dependencies. But if
153+
there is a significant amount of embedded logic, then this task will become
154+
more involved.
155+
156+
After porting the definitions, remove setup.py and build with ``pip install``.
157+
This will ensure that pyproject.toml is being used for build configuration.
158+
159+
.. warning::
160+
161+
If your existing package uses a pyproject.toml file and has a hardcoded "version"
162+
line similar to ``version: "1.2.0"`` in the ``[project]`` section, please
163+
remove that line.
164+
165+
The LINCC Frameworks template makes use of dynamic versioning with
166+
``dynamic = ["version"]``.
167+
A build error will occur if both a hardcoded and dynamic version definition
168+
are present in the same pyproject.toml file.
169+
170+
171+
Run all unit tests
172+
..................
173+
174+
Once you are sure the package still builds, run all the unit tests to ensure that
175+
the built package can be imported. The Copier template should not cause any tests
176+
to fail.
177+
178+
179+
Use pre-commit
180+
..............
181+
182+
Install and use ``pre-commit``. It may seem annoying at first, but it will save
183+
you many cycles of "see a test fail on GitHub, make and push a change, hope the test passes".
184+
185+
186+
Import sorting
187+
..............
188+
189+
If your project wasn't using ``isort`` or something similar before, there's a good
190+
chance that pre-commit hook will fail. It will automatically reorder the offending
191+
imports. You'll just need to ``git add`` the modified files again.
192+
193+
194+
Linters
195+
.......
196+
197+
If your project wasn't using a linter before, and you chose to include pylint, black,
198+
or another linting tool, it's reasonable to skip the linting check on the first commit.
199+
200+
For instance if you selected ``black`` as your new linter, use the following to
201+
bypass the pre-commit linting check on the first commit.
202+
203+
.. code :: bash
204+
205+
>> SKIP=black git commit -m 'Incorporating LINCC Frameworks PPT'
206+
207+
Linters are opinionated and if your project wasn't using one before there will
208+
be a lot of linting errors that will block committing your code.
209+
210+
It's highly recommended that in the next commit after incorporating the template
211+
that you address the linting errors so that you don't have to continue to use the
212+
``SKIP=...`` command for the rest of your days.

‎docs/source/new_project.rst‎

Lines changed: 1 addition & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -82,7 +82,7 @@ Use ``pip`` to install both the standard set of dependencies as well as the ``[d
8282
Lots more output
8383
...
8484
85-
You could stop here
85+
Great, but don't stop here
8686
-------------------------------------------------------------------------------
8787

8888
At this point, your new project is hydrated and ready for you to start coding. But there's a lot more that this template has to offer. Keep reading to find out more about built in pre-commit hooks, GitHub CI, automatic documentation, and more.
@@ -114,18 +114,3 @@ Create a new repository in GitHub: (`GitHub How-to <https://docs.github.com/en/g
114114
>> git push origin <local_branch_name>
115115
116116
Notice that when you create a PR in GitHub, a set of tests for Continuous Integration starts up to verify that the project can build successfully and that all the unit tests pass. Neato!
117-
118-
Keep your project up to date
119-
-------------------------------------------------------------------------------
120-
121-
Once your project is under version control you'll be able to keep your project up to date by running the following:
122-
123-
.. code-block:: bash
124-
125-
>> copier
126-
127-
Yep. That's it.
128-
129-
Copier will automatically check to see if a newer version of the original template is available and if so the changes will be automatically applied. Neato!
130-
131-
And of course, because your project is under version control, if you don't like the new changes, you can always revert back to the previous state.

‎docs/source/update_project.rst‎

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
Keeping your project up to date
2+
===============================
3+
4+
As we release new template features, you can keep your project up to date with
5+
a single-word command:
6+
7+
.. code-block:: bash
8+
9+
>> copier
10+
11+
Copier will automatically check to see if a newer version of the original template
12+
is available and if so the changes will be automatically applied. Neato!
13+
14+
You should run ``git diff`` to see what code has changed.
15+
If you don't like the new changes, you can always revert back to the previous state.
16+
17+
Additionally, if Copier encounters a merge conflict between your existing code and
18+
the updated template, it will produce ``.rej`` files that contain the unresolved diffs.
19+
If you see a ``.rej`` file, resolve the merge conflict and check that your code
20+
was updated correctly.
21+
There is no need to commit ``.rej`` files, you should remove them as
22+
the merge conflicts are resolved.
23+
24+
There are a few additional flags that you can use to be more explicit about how
25+
and what you want to update. We've found the following to be the most useful.
26+
27+
Get the absolute latest version of the template
28+
-----------------------------------------------
29+
30+
By default, Copier will find and apply the latest tagged release of a template when creating
31+
or updating a project.
32+
However, there may be new template features that you want to incorporate into your project
33+
before they are tagged for release.
34+
Copier provides the ``--vcs-ref`` flag for this purpose.
35+
36+
.. code-block:: bash
37+
38+
>> copier --vcs-ref=HEAD update
39+
40+
Copier will use the latest template from the ``main`` branch and apply it to your project.
41+
Note that ``HEAD`` can be replaced with any reasonable git tag such as a branch name or
42+
commit hash.
43+
44+
Change your response to a single question
45+
-----------------------------------------
46+
47+
The documentation for Copier indicates that it is a bad idea to manually edit the
48+
file that records your responses to the template questions. If you decide that you
49+
answered one of the questions incorrectly, you can update it without having to edit
50+
the rest.
51+
52+
.. code-block:: bash
53+
54+
>> copier --force --data <question_name>="new answer" update
55+
56+
This tells copier to use the previous answers for all questions except the one you want to
57+
update. For instance, if you initialized your project by selected ``pylint`` as your
58+
preferred linter, but now would like to change to ``black``, you could use the following:
59+
60+
.. code-block:: bash
61+
62+
>> copier --force --data preferred_linter="black" update
63+
64+
The full list of questions can be found
65+
`here <https://github.com/lincc-frameworks/python-project-template/blob/main/copier.yml>`_.
66+
67+
More information about Copier
68+
-----------------------------
69+
70+
The maintainers of Copier have written good instructions and there's no point
71+
in reproducing it all here.
72+
For all the details about updating with Copier checkout the
73+
`original documentation <https://copier.readthedocs.io/en/latest/updating/>`_.
74+
75+
Updates to pyproject.toml
76+
-------------------------
77+
78+
The pyproject.toml file is the primary configuration file for your project.
79+
When a project is initially hydrated from the template, a custom pyproject.toml file
80+
will be created. Generally Copier updates shouldn't drastically affect this file,
81+
but if you are modifying prior answers, take a close look at any changes made to this file.
82+
83+
If something looks incorrect, remember that any changes made by Copier are only *staged for
84+
commit*, they are't permanent and can easily be unstaged.

0 commit comments

Comments
 (0)