Use dfetch for tracking upstream template

[spoorcc]

Some projects are based off off other "template" projects.

Template

• somefolder
  • someFile
• .someconfig
• README.md

Child project

• someotherfolder
  • someFile
• somefolder
  • someFile
• .someconfig
• README.md (own README)
• dfetch.yaml

It would be nice if a user could have a child project based off off certain parts of a parent template project.
In above example someFolder and .someconfig should be synced, but the README not and the own files/folders should be preserved.

This would make it possible to keep tracking the template and move along with any updates.

dfetch update template-parent

With the following manifest:

  - name: template-parent
    dst: .                                                          # explicitly tell to use root of project
    url: https://github.com/dfetch/template.git       
    src:                                                            # mark (multiple) desired files/folders
    - somefolder
    - .someconfig
    clear: false                                                    # don't clear dest folder before update

• Is the src field ok to repurpose for this?
• Is there a better alternative than clear: false? Maybe "overlaying" is a more concise description for a user?

I'm open to any suggestions!

[sach-edna]

This seems as a nice idea.
It could also be useful to add files to existing projects, which is a feature I could use from dfetch.

It allows to combine projects and sub-projects which share a common folder structure.

[spoorcc]

@sach-edna how about:

    Scenario: A repo is updated from a template
         Given MyProject with the files
            | path                   |
            | MyFolder/SomeFile.txt  |
            | dfetch.yaml            |
            | README.md              |
        And the manifest 'dfetch.yaml' in MyProject
            """
            manifest:
                version: 0.0
                projects:
                    - name: SomeTemplateProject
                      url: some-remote-server/SomeTemplateProject.git
                      src: |
                        - SomeFolder/
                        - SomeFile
                      overlay: true
            """
        And a git-repository "SomeTemplateProject.git" with the files
            | path                     |
            | SomeFolder/SomeFile.txt  |
            | SomeFolder/OtherFile.txt |
            | SomeFile.md              |
            | README.md                |
        When I run "dfetch update"
        Then the output shows
            """
            Dfetch (0.7.0)
              SomeTemplateProject: Fetched v1
            """
        And 'MyProject' looks like:
            """
            MyProject/
                MyFolder/
                    SomeFile.txt
                SomeFolder/
                    .dfetch_data.yaml
                    OtherFile.txt
                    SomeFile.txt
                SomeFile.md
                README.md
                dfetch.yaml
            """

[sach-edna]

I agree with the premise of this scenario.
It makes sense and explains the use of the feature.
The overlay keyword also makes sense.
I have a few small notes.

Correct me if I'm wrong, but the scenario doesn't clarify whether the README.md file is coming from the template or the original repository. I feel like clarification would improve the scenario for this.
I'm afraid I don't have a suggestion on how to improve this.

Further I think that it would make sense that the extension of the selected file should be set in the manifest.
I don't think SomeFile should match SomeFile.md.
This is because the differentiation of e.g. SomeFile.md and SomeFile.txt can not be made, which I feel is useful.

[spoorcc]

@sach-edna Valid points!

• Maybe the most explicit is to add an additional step or even an additional scenario:

And 'README.md' is the original 'README.md' of 'MyProject'

• The missing extension was indeed wrong, should be explicit. It does however remind me of the scenario where a wildcard may match a file in the destination. My feeling is that the project that is dfetch'ed should always have priority.

[sach-edna]

I think two additional scenarios may be used as explicit documentation for the overlay feature.

For the use case where a wildcard may match either the existing files or the template files I think the best thing would be to keep the existing files. I think this is most consistent with the overlay flag.
For normal dfetch behavior (i.e. without overlay flag) I think it would make sense that the dfetched files overwrite the existing files.

e.g.

    Scenario: A repo is updated from a template
         Given MyProject with the files
            | path                   |
            | MyFolder/SomeFile.txt  |
            | dfetch.yaml            |
            | README.md              |
        And the manifest 'dfetch.yaml' in MyProject
            """
            manifest:
                version: 0.0
                projects:
                    - name: SomeTemplateProject
                      url: some-remote-server/SomeTemplateProject.git
                      src: |
                        - SomeFolder/
                        - SomeFile
                      overlay: true
            """
        And a git-repository "SomeTemplateProject.git" with the files
            | path                     |
            | SomeFolder/SomeFile.txt  |
            | SomeFolder/OtherFile.txt |
            | SomeFile.md              |
            | README.md                |
        When I run "dfetch update"
        Then the output shows
            """
            Dfetch (0.7.0)
              SomeTemplateProject: Fetched v1
            """
        And 'MyProject' looks like:
            """
            MyProject/
                MyFolder/
                    SomeFile.txt
                SomeFolder/
                    .dfetch_data.yaml
                    OtherFile.txt
                    SomeFile.txt
                SomeFile.md
                README.md
                dfetch.yaml
            """
        And 'README.md' is the original 'README.md' of 'MyProject'

and

 Scenario: A template wildcard does not overwrite a local file
         Given MyProject with the files
            | dfetch.yaml            |
            | SomeFile.txt           |
        And the manifest 'dfetch.yaml' in MyProject
            """
            manifest:
                version: 0.0
                projects:
                    - name: SomeTemplateProject
                      url: some-remote-server/SomeTemplateProject.git
                      src: |
                        - SomeFolder/
                        - SomeFile.*
                      overlay: true
            """
        And a git-repository "SomeTemplateProject.git" with the files
            | SomeFolder/SomeFile.txt  |
            | SomeFile.md
        When I run "dfetch update"
        Then the output shows
            """
            Dfetch (0.7.0)
              SomeTemplateProject: Fetched v1
            """
        And 'MyProject' looks like:
            """
            MyProject/
                SomeFolder/
                    .dfetch_data.yaml
                    SomeFile.txt
                SomeFile.txt
                dfetch.yaml
            """

[spoorcc]

We could take some inspiration of how https://github.com/cruft/cruft handles tracking which files were from the upstream