Removing commandline switches

[1stcall]

I have a few ideas.

1. Why not apt dist-upgrade, chroot, or anything else that has no negative impact all the time?
2. --customize --burn --burnfile etc. Do we need the -- ?
3. --1piboot, --b0script, --b1script, --bootscripts, etc. These could all be plugins with a parameter of when to run?
4. A new postimg parameter to plugins to run on the img file after its been created? This would work great for extimage and fsconvert (my next project) to convert the root filesystem to whatever you want. I would use it for btrfs obviously. Next I would use it to add data, maybe home, var, etc partitions.
5. Use environment variable for source base image and destination image? I'm sure there are a few others that could be used.

That's enough for now but I love this project and I think it will soon become the de-facto way to not have the likes of answerable running post-boot on every image.

[gitbls]

Some early feedback:

1. I was thinking about making dist-upgrade the default after the /boot/firmware screwup. Not sure what you mean by chroot?
2. Maybe not, but it would break every sdm script. Not sure if the bash parsing stuff sdm uses would handle it. And, finally, double-hyphens are your friend
3. These don't hurt anything as they are, except for a bit of documentation. I doubt anyone uses b0script/b1script, they are effectively decapitated plugins now.
4. Interesting. Will think about this. Would need to update every plugin to have the post-install code be more explicit (1-line change to the last 'else'
5. What's the compelling reason for these?

[1stcall]

1. --chroot by default. Why do we need the added systemd-containd dependency?

[gitbls]

systemd-nspawn takes care of a lot of grunge that I'm quite happy to have it do. While it's true that chroot would work everywhere, I'm pretty happy with the current implementation. I'd rather do things that create new value than churn existing code. 🤷‍♂️

[1stcall]

1. They are not really parameters, they are commands.
2. was meant to be examples of 1. Press return without realising it was adding 2. Maybe that makes 1. more relevant
3. Reducing documentation mostly.
4. PhasePostImage. So defer running on the host system until after the image has been created. I realise this would not fit in programmatically with your phases, but appear that way to the user. Maybe the way you create a script on in the image at the moment to run on first boot, but on the host (Probably in /tmp) to run after sdm has finished creating the img.

[gitbls]

1. I agree, but they are well-documented and work. I agree that --commands are intellectually ugly, but again, there's not really any added value in changing them and breaking everyone's scripts.
2. I do like the idea of a post-image phase. It would make adding certain types of features, such as whole disk encryption, much simpler and more integrated.

[1stcall]

5. I'm just getting used to using them now and they make scripting easier. Just a [ -f ./.env ] && source ./envthen less parameters on the command line in a more readable X=Y format.

Similar to @filename you currently use for apps etc.

With the added benefit of being able to use `X=Y command.sh

[gitbls]

Yea, X=Y command is a nice feature. Will consider.

[stb68]

In terms of making dist-upgrade the default I would make two observations..

1. after reading the firmware issue I tried adding dist-upgrade to my last try and it seems to hang the process.

2. I can see a case where somebody might want to keep with a particular version from an image and not do such an update

So would it be possible to switch the option off it was changed to be on by default?

I wasn't going to report this yet until I tried it out again, but I ran the command

sudo sdm --customize --plugin apps:"apps=neovim,tldr,shellcheck,exa,bat" \
 --plugin L10n:host --plugin disables:piwiz \
 --regen-ssh-host-keys --apt-dist-upgrade --restart    2023-10-10-raspios-bookworm-armhf-lite.img

and it stuck at the point where I ended up pressing CTRL-C a few times to kill it...

> Run Plugin 'disables' (/usr/local/sdm/plugins/disables) Phase 1 with arguments: 'piwiz'
* Plugin disables: Start Phase 1
* Plugin disables: Complete Phase 1
* Phase 1 post-app installation/configuration
> Start 'apt dist-upgrade'
> IMG '2023-10-10-raspios-bookworm-armhf-lite.img' has 2026708 1K-blocks (2.1GB, 1.9GiB) free at start of 'apt dist-upgrade'
^C^C% Caught CTRL/C. Exiting Phase 1...
umount: /mnt/sdm/boot/firmware unmounted

[gitbls]

I just redid your cmd on a 2023-10-10-armhf-lite IMG and it ran to completion after I extended it by 1024MB. Without the extend it died, running out of disk space Nothing to do with --apt-dist-upgrade AFAICT

[stb68]

I just double checked, the img is 4.7GB, so I did remember to do my normal 2048MB extension first.

The rest of the command is exactly as I used before, e.g. when I did my v10 tests. From my notes on my first v10 test run....

test v10
========

stb@E6530:~/Desktop/Pi/Images$ sudo sdm --customize --plugin apps:"apps=neovim,tldr,shellcheck,exa,bat" \
> --plugin L10n:host --plugin disables:piwiz \
> --regen-ssh-host-keys --restart    2023-10-10-raspios-bookworm-armhf-lite.img 
* Mount IMG '2023-10-10-raspios-bookworm-armhf-lite.img'
mount: /dev/loop0p2 mounted on /mnt/sdm.
mount: /dev/loop0p1 mounted on /mnt/sdm/boot.
* Start Configuration
> Command Line: /usr/local/bin/sdm --customize --plugin 
..
..
> Enable SSH
> Plugin phase1: Enable SSH service via /boot/ssh
> IMG '2023-10-10-raspios-bookworm-armhf-lite.img' has 1916836 1K-blocks (2.0GB, 1.8GiB) free at end of image customization
* Enter Shell Command Prompt
  'exit' to exit back to host system
root@sdm:/# 
exit
* Customization complete
umount: /mnt/sdm/boot unmounted
umount: /mnt/sdm unmounted

As I say, didn't really want to report this yet but just wanted to add it here as part of the case to make sure that the dist upgrade remains optional in one way or another.

[gitbls]

Don't worry...although I have a burning desire to change the dist upgrade default, I don't understand apt's nuances well enough to make this change. There are way too many unknowns.

[gitbls]

Here's where I'm at on the postimg phase touched on above.

Only plugins listed with the --burn-plugin switch will have the (currently called) burn-complete phase run, and these plugins will not be called for any other phases unless explicitly listed in a --plugin switch. A burndev argument is also included for each plugin when called, so they know the burn device. Burn plugins run in the context of the host.

I have a draft parted plugin that will do things like: expand root partition by a specific amount, make a new partition and it's file system, etc. A good plumbing test.

I'd be interested in hearing how the above aligns or not with your thinking. I think it will work well for fsconvert, which I assume will do a fs conversion on the root partition on the burned disk. extimage should work fine and should be callable as a regular plugin or a burn-plugin.

My sense is that this new plugin phase shouldn't be used by that many plugins. Doing it with a new switch prevents breaking every other plugin, which is a very nice sde effect.

[1stcall]

Would this include - - burnfile? The only difference I see between the two if that the - - burn destination is a block device file, and - - burnfile destination is a regular file. Or am I missing something?

[gitbls]

Doesn't work (aka hasn't been looked at) on burnfile. Hoping that it will.

[stb68]

Sounds good to me.

I'd be inclined to be able to specify a finished size for the root partition, like for the new partitions, rather than just an amount to extend by. That would also potentially mean not having to update your SDM script when newly sized images come out (or for building from different versions - lite, desktop, etc).

[gitbls]

I'm not clear how specifying a finished partition size really solves any problem, but I'll have a look at it while I'm finishing up the plugin.

[1stcall]

I guess being able to say 'imgfilesize=8gb,rootsize=3gb, datasize=remainder" could be useful. I'm not sure if that is what @stb68 meant.

[gitbls]

At the moment, I can't think of why anyone would want to specify a fixed partition size. In the good old days one could claim "Small disk", but that doesn't carry much weight these days 🤣

[1stcall]

I don't require it personally, but others may wish to add a data, backup, home, swap, etc. partition to the end of the image after the root partition.

…

________________________________ From: Benn ***@***.***> Sent: 14 December 2023 3:20 AM To: gitbls/sdm ***@***.***> Cc: Carl McAlwane ***@***.***>; Author ***@***.***> Subject: Re: [gitbls/sdm] Removing commandline switches (Discussion #146) At the moment, I can't think of why anyone would want to specify a fixed partition size. In the good old days one could claim "Small disk", but that doesn't carry much weight these days 🤣 — Reply to this email directly, view it on GitHub<#146 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ADCHHLODOXKMWKY6T3QKYNLYJJWALAVCNFSM6AAAAABANLKIG2VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TQNBZGIYTQ>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[gitbls]

Yes, and that is exactly what can be done with adding an additional partition. In a single parted plugin invocation you can extend the root partition by nnnnMB AND add another partition with a specific size and file system type.

[stb68]

I'm not clear how specifying a finished partition size really solves any problem, but I'll have a look at it while I'm finishing up the plugin.

It's only a minor point I agree, but I think it would be nice if easy to include. It's more how I think about partitioning, but maybe that shows my age!

If I was using, for example, a bunch of 16GB cards and wanted to use half for boot / root then being able to specify setting root to 7424MB would be a bit clearer and mean there's no need to change that argument based on whether I'm using the lite image or desktop, whether the latest rpi download has changed size or how much I'd ended up extending the original image to run the customize step.

I like @1stcall 's idea of having a "remainder" setting, though I might not use it myself as I tend to leave a bit of space at the end in case I want to copy the SD card to another card (i.e. not all 16GB cards are exactly the same size). Of course, an option to leave the last x% of the card empty in the process would handle that, but is probably going a bit far for most people's requirements!

[gitbls]

My curmudgeonly thinking is that it's a solution looking for a problem. That, along with a less-than-clear value proposition has me landing on "pass" for this.

And, to be clear, this is for specifying the size of the root partition (whether to extend it by nnnnMB or to extend it to a total size of nnnnMB). When adding other partitions the size is specified as the total size of the partition.