Modified plugins not used 1st time

[dshaw619]

I seem to be having a bizarre problem with plugins that are only used during a --burn operation not running the most recent version even though I run both --customize and --burn steps. If I rerun the --customize and --burn steps a second time, the modified plugin finally gets executed.

I believe this sequence is reproducible:

• Modify and save (in my local setup directory) a plugin script.
• Run --customize which does not use that script.
• Run --burn referencing my plugin by full pathname to my local setup directory.
• Observe that the plugin which ran was the previous version.
• Run --customize again (same command).
• Run --burn again (same command).
• Find that the desired updated plugin was run at the appropriate point.

[gitbls]

hmm... just to confirm, are you using --bupdate plugin on your burn command or not? By default, plugins are NOT updated on a burn unless --bupdate plugin is used.

[dshaw619]

Afraid I missed that new switch in v7.10. Should have paid more attention to the change log.
Just to clarify, the correct syntax would be "--bupdate plugin"? i.e. not the name of any specific plugin?

The Command Details page says "Best to not include it always unless you know what you're doing". What would be the downsides of doing so?

p.s. I'd suggest a reference to this be included on the Programming Plugins page.

[gitbls]

Thinking was that you would want control over it, so either make 'auto-update' or 'update-by-command-only' the default. I opted for 'by-command-only' on the principle of least suprise. Didn't think anyone would get surprised the other way 🤔

Adding it to the docs: Noted

Edit: I need to revisit this in more detail. Stay tuned.

[gitbls]

Here's what I've concluded, and am implementing for the --burn command WRT plugins:

• If --bupdate plugin is not specified, plugins on the command line specified with only a name (e.g., knockd [in plugins], or myplugin [in local-plugins]) will be checked at burn time. If there is a newer version on the host, a %message will be issued. Plugins referenced on the command line that live in /usr/local/sdm/{local-plugins,plugins} will cause a check between the version on the host and the version on the freshly-burned output
• As above, but if --bupdate plugin IS specified, an updated plugin on the host will be updated onto the burned output in /usr/local/sdm/local-plugins. So, if you modify an sdm-supplied plugin in plugins, the updated version will be in local-plugins and used before the sdm-supplied version, since all sdm code referencing plugins search local-plugins before plugins.
• OTOH, if --plugin /path/to/someplugin is specified, the plugin will ALWAYS be updated onto the burned output and placed in /usr/local/sdm/local-plugins. There is no need to specify --bupdate plugin in this case. Thinking being that you've decided that the /path/to/someplugin should be written/used in the burned output and referencing it explicitly forces a --bupdate plugin for plugins thus referenced.

In other words, to push a brand new plugin into the burned output (that's not in the source IMG), use /path/to/someplugin to force it to be copied in .

Thoughts?

[dshaw619]

Sounds great. Just to make sure I understand completely, if one specifies --plugin /path/to/someplugin AND --bupdate plugin (as is working for me now; I'll plan to take out the --bupdate plugin after the update is released, but one never knows) the behavior will be the same as if --bupdate plugin had not been specified?

[gitbls]

Exactly.

[gitbls]

This is fixed in V7.11

[dshaw619]

I was able to reproduce with greatly reduced scripts.

/usr/local/sdm/local-plugins on my laptop contains an older version (17-41) of the plugin. (the 2nd line was leftover from an even earlier version. I moved the dbgprint to the main body of the plugin so it would show up in all phases and didn't delete the dbgprint I had originally put in phase 1).

doug@dv6-Notebook-PC:~/sdir$ cat /usr/local/sdm/local-plugins/minimal-plugin.sh | grep dbg
plugin_dbgprint "v2023-03-21_17-41"
    plugin_dbgprint "Minimal plugin; v2023-03-21_16-08"

sdir is a link to my setup working directory and contains an updated version (21-34) of the plugin.

doug@dv6-Notebook-PC:~/sdir$ cat minimal-plugin.sh | grep dbg
plugin_dbgprint "v2023-03-21_21-34"

I ran significantly reduced versions of my --customize and --burn scripts. Only plugin in either is minimal-plugin.sh run during --burn.

Before exiting the --customize, I verified that the image contained the old (17-41) plugin.

root@dv6-Notebook-PC:/# cat /usr/local/sdm/local-plugins/minimal-plugin.sh | grep dbg
plugin_dbgprint "v2023-03-21_17-41"
    plugin_dbgprint "Minimal plugin; v2023-03-21_16-08"

The plugin switch in the --burn script is --plugin $setupdir/minimal-plugin.sh \
setupdir is defined in the --burn script as setupdir=$HOME/Documents/Briefcase/It/Raspberry_Pi/Setup/11-Bullseye

The older version (17-41) of the plugin was run during the --burn as seen in this excerpt from the terminal:

> Run Plugins Phase '1'
> Run Plugin 'minimal-plugin.sh' (/usr/local/sdm/local-plugins/minimal-plugin.sh) Phase 1 with 
  no arguments
D!Plugin minimal-plugin.sh: v2023-03-21_17-41
* Plugin minimal-plugin.sh: Start Phase 1
D!Plugin minimal-plugin.sh: Minimal plugin; v2023-03-21_16-08
* Plugin minimal-plugin.sh: Complete Phase 1

[dshaw619]

As a further test, I deleted /usr/local/sdm/local-plugins/minimal-plugin.sh No other changes. Reran my minimal --customize and --burn scripts and the correct new plugin (21-34) ran during the burn.

I will send you and email with the scripts and terminal logs in case there is anything there that you want to look at.

[gitbls]

Amusingly, you nailed it: The only think I see different about your test is that you ran customize before editing the plugin, but I can't see how that could possibly cause any difference.

This was precisely the problem. sdm was copying files (including itself) into the IMG at customization time using cp, which changes the file date/time. Since you modified your /fpt/plugin before doing the customize, it appeared to be older than the older plugin, which was copied into the IMG but given a newer date/time.

V7.12 updates all the appropriate cp commands in sdm and plugins to use cp -a.

Thanks for your perseverance on this!

[dshaw619]

Looks good!

• Updated sdm to v7.12.
• With older version of minimal plugin in /usr/local/sdm/local-plugins, I ran my minimal test --customize and --burn scripts (only plugin referenced was a newer copy of the minimal plugin in my working directly called by full path name during the --burn. Newer version ran as it should.
• Modified the copy of the minimal plugin in my working directory (leaving the old version still in /usr/local/sdm/local-plugins). Ran my full --customize and --burn scripts. Newest version of the minimal plugin (referenced by full path name during the --burn only) ran as it should.
• Deleted the old version of the minimal plugin in /usr/local/sdm/local-plugins. Modified the copy of the minimal plugin in my working directory. Ran my full --customize and --burn scripts. Newest version of the minimal plugin (referenced by full path name during the --burn only) ran as it should.

Thanks so much, gitbls, for your diligence in following through on this and not just brushing it off with "can't replicate" as so many do.

[gitbls]

Great! Thanks for persevering on this. I'm pretty excited about plugins, so wanted to get this working as it should.

Thanks so much, gitbls, for your diligence in following through on this and not just brushing it off with "can't replicate" as so many do.

I appreciate the kind words, and thank you for helping sort this out!