Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: try to clean up and simplify the file adding interface #255

Merged
merged 3 commits into from
Dec 15, 2020

Conversation

djgilcrease
Copy link
Contributor

@djgilcrease djgilcrease commented Nov 15, 2020

The goal is to make this backward compatible with current config files so people can stay with the simplified format until they need more advanced features.

Internally all packagers ONLY use the Contents data which is a merging of all the old file specifications into one list. IF a file was specified in a packager specific override (eg rpm.Files) then the content.Packager field is set to the correct packager automatically which ensures files for other packagers do not get mixed together.

Example

name: "contents foo"
arch: "amd64"
version: "v1.2.3"
contents:
  - src: "./testdata/*.conf"
    dst: "/etc/foo/"
    type: "config|noreplace"
    file_info:
      mode: 0644
      mtime: 2008-01-02T15:04:05Z
      owner: me
      group: whatever
  - src: "./testdata/whatever2.conf"
    dst: "/etc/rpm/whatever.conf"
    type: "config|noreplace"
    packager: "rpm"
    file_info:
      mode: 0644
  - src: "./testdata/whatever2.conf"
    dst: "/etc/apk/whatever.conf"
    type: "config|noreplace"
    packager: "apk"
    file_info:
      mode: 0644
  - src: "./testdata/whatever2.conf"
    dst: "/etc/deb/whatever.conf"
    type: "config|noreplace"
    packager: "deb"
    file_info:
      mode: 0644

The files, config_files, symlinks, rpm.config_noreplace_files, and rpm.ghost_files are merged into contents and all packagers only the the data from contents now. This makes it simple for people to leave their config files as is and not break anything, but as they need more advanced features, like control over the file mode, owner, group, etc they can convert all their files to the contents as well.

I have also set it up so the simple mapping will work as well

contents:
  a: b
  a2: b2

Also this is setup to be ready to handle contents as a string in prep for #43 but is not implemented

contents: /path/to/a/tgz

@vercel
Copy link

vercel bot commented Nov 15, 2020

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: https://vercel.com/goreleaser/nfpm/mza41etdl
✅ Preview: https://nfpm-git-feat-file-type-enhancement.goreleaser.vercel.app

@pull-request-size pull-request-size bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Nov 15, 2020
@djgilcrease djgilcrease requested review from erikgeiser and caarlos0 and removed request for erikgeiser November 15, 2020 20:15
@erikgeiser
Copy link
Member

This is a pretty significant change and should definitely be discussed. I do think your approach is cleaner and more elegant but the old one is probably more pragmatic, since most people probably have a whole bunch of regular files and rarely any special files. I have no data to back that up though. I generate my configs automatically as a part of a bigger custom build process, but for those writing configs by hand, it will be much more to type.

I'll have a look at the exact changes when I find some time, maybe tomorrow.

@djgilcrease
Copy link
Contributor Author

Ya, that's why I am making sure to ensure backwards compatibility so most people can continue to use the simple config format if they want.

@erikgeiser
Copy link
Member

If I read your code correctly you planned to implement a custom unmarshal method so files can be both a YAML mapping and a list of FileToCopy in the YAML file. This is a great way to implement it.

Maybe we can use src and dst as YAML keys to make it more convenient for regular files in the new format.

Also I dislike the name of the type FileToCopy. If it's a glob, it represents multiple files that need to be copied. If it's a ghost file, nothing is to be copied. Maybe we could rename it to Content.

internal/files/files.go Outdated Show resolved Hide resolved
internal/files/files.go Outdated Show resolved Hide resolved
@erikgeiser
Copy link
Member

erikgeiser commented Nov 24, 2020

This is also nice for #43 because now we can then completely decide what do based on the YAML node type:

  • Mapping: src->dst mapping of regular files, no non-regular files supported (only via depecated options), this is nice for very simple packages
  • Sequence: The new rich file types you introduced here
  • String: Path to a .tar.gz file (if we want to go through with it, I only see this as relevant for repackaging)

I think then Content or Contents would indeed be a much more fitting name. Both for the config option and to replace the FilesToCopy type name.

@vercel vercel bot temporarily deployed to Preview November 25, 2020 00:25 Inactive
@vercel vercel bot temporarily deployed to Preview November 25, 2020 00:31 Inactive
@vercel vercel bot temporarily deployed to Preview November 25, 2020 00:38 Inactive
@djgilcrease
Copy link
Contributor Author

If we want a new field instead of trying to override the current files field that should actually be easier and cleaner than what I am having to do now I think. I can still have code that merges the files, config_files, symlinks, rpm.config_noreplace_files, and rpm.ghost_files into contents so all the packagers only use the contents

contents:
  - src: "something"
    dst: "/opt/pkg/something"
    type: "config"
    packager: "rpm"
    mode: 0644

give me a day or so to play with that concept to see if it turns out cleaner

@vercel vercel bot temporarily deployed to Preview November 25, 2020 05:21 Inactive
@vercel vercel bot temporarily deployed to Preview November 25, 2020 05:29 Inactive
@djgilcrease
Copy link
Contributor Author

give me a day or so to play with that concept to see if it turns out cleaner

Much cleaner code now that I am not trying to mess with existing fields

@vercel vercel bot temporarily deployed to Preview December 9, 2020 18:43 Inactive
@vercel vercel bot temporarily deployed to Preview December 9, 2020 22:21 Inactive
@pull-request-size pull-request-size bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Dec 9, 2020
@vercel vercel bot temporarily deployed to Preview December 9, 2020 23:27 Inactive
@djgilcrease djgilcrease changed the title WIP: feat: try to clean up and simplify the file adding interface feat: try to clean up and simplify the file adding interface Dec 9, 2020
@vercel vercel bot temporarily deployed to Preview December 10, 2020 00:02 Inactive
@codecov
Copy link

codecov bot commented Dec 10, 2020

Codecov Report

Merging #255 (6d9678b) into master (8535159) will increase coverage by 1.58%.
The diff coverage is 80.93%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #255      +/-   ##
==========================================
+ Coverage   68.79%   70.38%   +1.58%     
==========================================
  Files           9        9              
  Lines        1032     1138     +106     
==========================================
+ Hits          710      801      +91     
- Misses        190      209      +19     
+ Partials      132      128       -4     
Impacted Files Coverage Δ
deb/deb.go 66.22% <60.00%> (-2.35%) ⬇️
apk/apk.go 70.63% <72.97%> (-1.47%) ⬇️
rpm/rpm.go 73.86% <75.60%> (+0.47%) ⬆️
nfpm.go 84.96% <90.47%> (+6.91%) ⬆️
files/files.go 92.95% <92.95%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8535159...6d9678b. Read the comment docs.

@vercel vercel bot temporarily deployed to Preview December 10, 2020 00:42 Inactive
www/docs/configuration.md Outdated Show resolved Hide resolved
Copy link
Member

@caarlos0 caarlos0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks awesome, thanks for working on this @djgilcrease ! 🚀

@caarlos0
Copy link
Member

The goal is to make this backward compatible with current config files so people can stay with the simplified format until they need more advanced features.

I would say we deprecate the old format. The new format is simple enough for the basic use case as well, and then when users need to change it, its not that much of a change?

wdyt?

if you agree, we can copy the deprecate package from goreleaser and do basically the same thing here, and remove the old option after 6mo or so. (and we can make this all in another PR since this one is already pretty big)

Co-authored-by: Carlos Alexandro Becker <[email protected]>
@vercel vercel bot temporarily deployed to Preview December 15, 2020 16:04 Inactive
@caarlos0 caarlos0 merged commit c4ae30d into master Dec 15, 2020
@caarlos0 caarlos0 deleted the feat-file-type-enhancement branch December 15, 2020 16:47
@github-actions
Copy link
Contributor

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jan 14, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants