[Suggestion] Is it possible to control forced loading of "~/.avrduderc"?

[askn37]

This proposal is somewhat related to the following requirements:
Please ignore if it is deemed unnecessary.

• Add avrdude_conf_version to grammar #1547 (comment)

Background of the problem

As mentioned in the following comment, multiple board support projects that can be integrated into the Arduino IDE use different versions of "avrdude".

• Report: Incorrect SerialUPDI control of AVR64EA32 #1529 (comment)

They can be installed at the same time and use their own configuration files so that they do not interfere with each other.

Unfortunately, "~/.avrduderc" is not. There are always no or only one user-defined file. Therefore, it is always read from different versions of "avrdude" and the content that can be written differs depending on the version, so it always stops with an error.

In conclusion, the use of "~/.avrduderc" is not recommended for anyone using the Arduno IDE.

However, in that case, users may install their own "avrdude" on the OS and create "~/.avrduderc", causing various problems. This cannot be called orderly.

Suggestions or ideas

Is it possible to add the following global switch at the beginning of the new "avrdude.conf"?

default_rcfile_loading = yes|no

If this is "no", "~/.avrduderc" will not be read and will be ignored. This should prevent problems.

The default is "yes", which is the legacy compatible behavior. However, if related projects are aware of this issue, they can choose "No" in their own configuration files and distribute them.

Projects that are no longer updated or are unlikely to be updated will not be able to support this improvement and will continue to use the old "avrdude". However, we should be able to consider ongoing projects (unless it is difficult due to individual circumstances).

I'm not yet familiar enough with all of the project's source code to modify it myself, so I don't know if that's possible.

We look forward to hearing your frank opinions.

[stefanrueger]

Maybe a command-line-option is useful, eg, -N for do not load personal .avrduderc file. I face relatively often the issue that I wish to generate a canonical avrdude.conf.in file by avrdude -p\*/s -c\*/s. If I forget to temporarily rename my ~/.avrduderc file then my personal programmer definitions or test-parts creep into the output. So, yes, -N or some such might be useful.

It is generally not a great idea to tweak the avrdude.conf file so that AVRDUDE changes its behaviour to support unknown/unsupported projects, platforms or programmers. I hear that a few projects do this, eg, some of @SpenceKonde's cores. They then are at risk of being surprised if further development of AVRDUDE breaks some of the tweaky artificial behaviour that AVRDUDE had no idea was being utilised to tell bootloaders or programmers about some property. Generally, it is best to get the code base of AVRDUDE work with the project.

The idea of the avrdude.conf file is to externalise a parameter model of programmers and parts that match with the code base. In general, the system-wide avrdude.conf file should not be modified. The personal ~/.avrduderc file is meant as a means to add own programmer parameters (for example, if one rigs an FTDI chip with a certain pin mapping to the I2C signals PICO, POCI, SCK and AVR Reset) or new parts that the system-wide avrdude.conf file does not yet know about.

Enabling a system-wide configuration file to override and switch off the user's personal configuration is questionable. In my world, this is firmly something that the user should decide, not another configuration file.

[askn37]

@stefanrueger

-N

I do not know that! Because Usage currently doesn't have that. And "-?" seems to be broken in zsh.

askn@alicia avrdude_main % git log -n1                    
commit 4622a6d4291cce6e15a3c15927dacc6b2ae35d44 (HEAD -> memory-types)
Author: Stefan Rueger <[email protected]>
Date:   Tue Oct 31 11:30:39 2023 +0000

    Add sigrow sub-memories to jtagmkII_read_byte()
askn@alicia avrdude_main % ./build_darwin/src/avrdude 2>&1 | grep 'N' 
askn@alicia avrdude_main % ./build_darwin/src/avrdude 2>&1           
Usage: avrdude [options]
Options:
  -p <partno>            Specify AVR device; -p ? lists all known parts
  -p <wildcard>/<flags>  Run developer options for matched AVR devices,
                         e.g., -p ATmega328P/s or /S for part definition
  -b <baudrate>          Override RS-232 baud rate
  -B <bitclock>          Specify bit clock period (us)
  -C <config-file>       Specify location of configuration file
  -c <programmer>        Specify programmer; -c ? and -c ?type list all
  -c <wildcard>/<flags>  Run developer options for matched programmers,
                         e.g., -c 'ur*'/s for programmer info/definition
  -A                     Disable trailing-0xff removal for file/AVR read
  -D                     Disable auto erase for flash memory; implies -A
  -i <delay>             ISP Clock Delay [in microseconds]
  -P <port>              Connection; -P ?s or -P ?sa lists serial ones
  -r                     Reconnect to -P port after "touching" it; wait
                         400 ms for each -r; needed for some USB boards
  -F                     Override invalid signature or initial checks
  -e                     Perform a chip erase
  -O                     Perform RC oscillator calibration (see AVR053)
  -t                     Run an interactive terminal when it is its turn
  -T <terminal cmd line> Run terminal line when it is its turn
  -U <memstr>:r|w|v:<filename>[:format]
                         Carry out memory operation when it is its turn
                         Multiple -t, -T and -U options can be specified
  -n                     Do not write to the device whilst processing -U
  -V                     Do not automatically verify during -U
  -E <exitsp>[,<exitsp>] List programmer exit specifications
  -x <extended_param>    Pass <extended_param> to programmer, see -xhelp
  -v                     Verbose output; -v -v for more
  -q                     Quell progress output; -q -q for less
  -l logfile             Use logfile rather than stderr for diagnostics
  -?                     Display this usage

avrdude version 7.2-20231031 (4622a6d4), https://github.com/avrdudes/avrdude
askn@alicia avrdude_main % ./build_darwin/src/avrdude -? 2>&1 
zsh: no matches found: -?

If you think about it, Usage is pretty unkind right now. PM_PDI/PM_UPDI etc. can be confusing as some options can and cannot be used with other options. To filter help messages, the implementation must be modified to allow -c Programmer and -p part to be specified together.

[askn37]

@stefanrueger

It is generally not a great idea to tweak the avrdude.conf file so that AVRDUDE changes its behaviour to support unknown/unsupported projects, platforms or programmers. I hear that a few projects do this, eg, some of @SpenceKonde's cores.

When using "avrdude" in the backend of the Arduino IDE, the end user never touches the command line. Sketch builds also do not include make files. Simply choose from the options that appear in the IDE's menus. Usually only the choices that have been vetted for that project are available.

User-defined avrduderc files can override that behavior. However, adding a new programmer section will never be used by the IDE. IDE is not sensitive to it. Advanced users can directly modify the IDE's programmers.txt or preferences.txt to change the default behavior.

However, it would be useful in the future if implementations like "serialupdi", which have many unknown variants, could be controlled by user-defined avrduderc files.

[stefanrueger]

| Usage currently doesn't have [-N]

Correct. That's because I suggest implementing -N (or similar) in AVRDUDE to solve Issue #1548.

| When using "avrdude" in the backend of the Arduino IDE, the end user never touches the command line.

Correct. The Arduino IDE controls the command line and could use -N (once implemented) if they want to exclude ~/.avrduderc.

| zsh: no matches found: -?

Well, zsh interprets wildcard characters such as * and ?; if a command line wants to pass on a literal ? to any command (not just avrdude) it needs to quote the wildcard character, eg, "-?", -'?', -\? etc. The reason -? or -p*/s etc often works without quoting is that some shells (bash for example) pass on a wildcard string if no file matches the string. Create a file -e ($ touch ./-e), call avrdude -? -c ... -p ... and you are likely to erase the chip if the shell expands -? to -e. Whether or not wildcard expansion happens and in which way depends on the shell and how its globbing behaviour is set.

| [programming modes] the implementation must be modified to allow -c Programmer and -p part to be specified together

Well, if a programmer only can handle UPDI parts and it is specified with a part that cannot be programmed through UPDI, then AVRDUDE warns the user and offers to try nevertheless using -F:

$ avrdude -c jtag2updi -p m328p -t
avrdude error: programmer jtag2updi cannot program part ATmega328P as they
        lack a common programming mode; use -F to override this check

But yes, if you find an error in the programming mode specification, please raise an issue. Also note that programmers that serve bootloaders can and should specify PM_SPM in prog_modes. This is compatible with all parts that allow self-programming.

[askn37]

@stefanrueger

All right, I understand.

-N

Please implement it. In the future I can change the programmers.txt project configuration to allow '-N' via the preferences.txt user configuration.

-?

suggestion. Can Usage be changed to '-h' according to common *UNIX programming practice? Fortunately, '-h' currently appears to be unused. For the time being, '-?' can be considered equivalent to '-h'.

This probably resolves my concerns.

[mcuee]

-?

suggestion. Can Usage be changed to '-h' according to common *UNIX programming practice? Fortunately, '-h' currently appears to be unused. For the time being, '-?' can be considered equivalent to '-h'.

Indeed this may be a good improvement.

@stefanrueger
BTW, the first line has a minor mistake as well. It should be unknow option -h.

C:\work\avr\avrdude_test\avrdude_bin\avrdude.exe: unknown option -- h
Usage: avrdude [options]
Options:
  -p <partno>            Specify AVR device; -p ? lists all known parts
  -p <wildcard>/<flags>  Run developer options for matched AVR devices,
                         e.g., -p ATmega328P/s or /S for part definition
  -b <baudrate>          Override RS-232 baud rate
  -B <bitclock>          Specify bit clock period (us)
  -C <config-file>       Specify location of configuration file
  -c <programmer>        Specify programmer; -c ? and -c ?type list all
  -c <wildcard>/<flags>  Run developer options for matched programmers,
                         e.g., -c 'ur*'/s for programmer info/definition
  -A                     Disable trailing-0xff removal for file/AVR read
  -D                     Disable auto erase for flash memory; implies -A
  -i <delay>             ISP Clock Delay [in microseconds]
  -P <port>              Connection; -P ?s or -P ?sa lists serial ones
  -r                     Reconnect to -P port after "touching" it; wait
                         400 ms for each -r; needed for some USB boards
  -F                     Override invalid signature or initial checks
  -e                     Perform a chip erase
  -O                     Perform RC oscillator calibration (see AVR053)
  -t                     Run an interactive terminal when it is its turn
  -T <terminal cmd line> Run terminal line when it is its turn
  -U <memstr>:r|w|v:<filename>[:format]
                         Carry out memory operation when it is its turn
                         Multiple -t, -T and -U options can be specified
  -n                     Do not write to the device whilst processing -U
  -V                     Do not automatically verify during -U
  -E <exitsp>[,<exitsp>] List programmer exit specifications
  -x <extended_param>    Pass <extended_param> to programmer, see -xhelp
  -v                     Verbose output; -v -v for more
  -q                     Quell progress output; -q -q for less
  -l logfile             Use logfile rather than stderr for diagnostics
  -?                     Display this usage

avrdude version 7.2-20231103 (08b82ea6), https://github.com/avrdudes/avrdude

[stefanrueger]

Can Usage be changed to '-h'

Do you really want to change 23-year old behaviour of AVRDUDE that was perfectly fine and accepted all those decades? At the time everyone was using -? for usage(). I realise - like fashion - preferences for option handling changes from time to time, but we don't want users to ask the question And how is AVRDUDE's help option called this month? 😉

[mcuee]

Can Usage be changed to '-h'

Do you really want to change 23-year old behaviour of AVRDUDE that was perfectly fine and accepted all those decades? At the time everyone was using -? for usage(). I realise - like fashion - preferences for option handling changes from time to time, but we don't want users to ask the question And how is AVRDUDE's help option called this month? 😉

Probably not to change, but just to add -h as an alias for -?.

BTW, indeed people use -? or /? last time. :-)

Windows CMD prompt still uses /? now.

[askn37]

(Of course this is my personal opinion)
I think most *UNIX shells except Windows/MS-DOS treat ? as a metacharacter. Therefore, "?" is also necessary to escape, right?

"?" Certainly a traditional help command character from before CP/M. However, back then each command implemented its own wildcard and metacharacter expansion (I remember writing it in MACRO80). COMMAND.COM was just a launcher, not a shell. At some point, the shell started treating them as metacharacters, and many symbols could no longer be passed to commands without escaping them. Nowadays, this has become commonplace, so when I found the "?", I felt a strange sense of nostalgia. That may be all.

[stefanrueger]

All shells I know shells do wildcard expansion on ?. And always have done. Even the vvv old command.exe of MSDOS did and does that. So, all users have to know is how to escape ?. Then and now. Or they had to rely on the globbing behaviour of their shells: for example, bash can pass on -? to the command when there is no file that matches -?. So, -? can work with bash unless you have a file called -r or some such.

[stefanrueger]

BTW, the first line has a minor mistake as well. It should be unknow option -h.

Tell that the getopt(3) library function, which actually prints it that way. This is a standard function that is often used and that's what it prints. I wouldn't have printed that this way either, but there you go.

[askn37]

@stefanrueger

-? can work with bash unless you have a file called -r or some such.

Perhaps zsh doesn't behave the way you know.

askn@alicia avrdude_main % bash

The default interactive shell is now zsh.
To update your account to use zsh, please run `chsh -s /bin/zsh`.
For more details, please visit https://support.apple.com/kb/HT208050.

bash-3.2$ ./build_darwin/src/avrdude -?
Usage: avrdude [options]
Options:
  -p <partno>            Specify AVR device; -p ? lists all known parts
  -p <wildcard>/<flags>  Run developer options for matched AVR devices,
                         e.g., -p ATmega328P/s or /S for part definition
  -b <baudrate>          Override RS-232 baud rate
  -B <bitclock>          Specify bit clock period (us)
  -C <config-file>       Specify location of configuration file
  -C +<config-file>      Specify additional config file, can be repeated
  -N                     Do not load config file ~/.avrduderc
  -c <programmer>        Specify programmer; -c ? and -c ?type list all
  -c <wildcard>/<flags>  Run developer options for matched programmers,
                         e.g., -c 'ur*'/s for programmer info/definition
  -A                     Disable trailing-0xff removal for file/AVR read
  -D                     Disable auto erase for flash memory; implies -A
  -i <delay>             ISP Clock Delay [in microseconds]
  -P <port>              Connection; -P ?s or -P ?sa lists serial ones
  -r                     Reconnect to -P port after "touching" it; wait
                         400 ms for each -r; needed for some USB boards
  -F                     Override invalid signature or initial checks
  -e                     Perform a chip erase
  -O                     Perform RC oscillator calibration (see AVR053)
  -t                     Run an interactive terminal when it is its turn
  -T <terminal cmd line> Run terminal line when it is its turn
  -U <memstr>:r|w|v:<filename>[:format]
                         Carry out memory operation when it is its turn
                         Multiple -t, -T and -U options can be specified
  -n                     Do not write to the device whilst processing -U
  -V                     Do not automatically verify during -U
  -E <exitsp>[,<exitsp>] List programmer exit specifications
  -x <extended_param>    Pass <extended_param> to programmer, see -xhelp
  -v                     Verbose output; -v -v for more
  -q                     Quell progress output; -q -q for less
  -l logfile             Use logfile rather than stderr for diagnostics
  -?                     Display this usage

avrdude version 7.2-20231103 (ff3e65e3), https://github.com/avrdudes/avrdude

bash-3.2$ exit

askn@alicia avrdude_main % ./build_darwin/src/avrdude -?
zsh: no matches found: -?
askn@alicia avrdude_main % 

Why did Apple kill bash and make this damn shell the standard? Not at all. . .

[askn37]

By the way, zsh does not pass a simple "?". Even if it's in the middle of a string.

bash-3.2$ ./build_darwin/src/avrdude -v?v
Usage: avrdude [options]
Options:
  -p <partno>            Specify AVR device; -p ? lists all known parts
  -p <wildcard>/<flags>  Run developer options for matched AVR devices,
                         e.g., -p ATmega328P/s or /S for part definition
  -b <baudrate>          Override RS-232 baud rate
  -B <bitclock>          Specify bit clock period (us)
  -C <config-file>       Specify location of configuration file
  -C +<config-file>      Specify additional config file, can be repeated
  -N                     Do not load config file ~/.avrduderc
  -c <programmer>        Specify programmer; -c ? and -c ?type list all
  -c <wildcard>/<flags>  Run developer options for matched programmers,
                         e.g., -c 'ur*'/s for programmer info/definition
  -A                     Disable trailing-0xff removal for file/AVR read
  -D                     Disable auto erase for flash memory; implies -A
  -i <delay>             ISP Clock Delay [in microseconds]
  -P <port>              Connection; -P ?s or -P ?sa lists serial ones
  -r                     Reconnect to -P port after "touching" it; wait
                         400 ms for each -r; needed for some USB boards
  -F                     Override invalid signature or initial checks
  -e                     Perform a chip erase
  -O                     Perform RC oscillator calibration (see AVR053)
  -t                     Run an interactive terminal when it is its turn
  -T <terminal cmd line> Run terminal line when it is its turn
  -U <memstr>:r|w|v:<filename>[:format]
                         Carry out memory operation when it is its turn
                         Multiple -t, -T and -U options can be specified
  -n                     Do not write to the device whilst processing -U
  -V                     Do not automatically verify during -U
  -E <exitsp>[,<exitsp>] List programmer exit specifications
  -x <extended_param>    Pass <extended_param> to programmer, see -xhelp
  -v                     Verbose output; -v -v for more
  -q                     Quell progress output; -q -q for less
  -l logfile             Use logfile rather than stderr for diagnostics
  -?                     Display this usage

avrdude version 7.2-20231103 (ff3e65e3), https://github.com/avrdudes/avrdude

bash-3.2$ exit

askn@alicia avrdude_main % ./build_darwin/src/avrdude -v?v
zsh: no matches found: -v?v

'?' in zsh is a ternary operator. The error is thrown because the ':' pair is missing. I have a headache.

[stefanrueger]

| zsh: no matches found: -v?v

Try unsetopt nomatch (if you want that permanently, then add that line in .zshrc)

Different shells have different default behaviour if there are no files that match wildcards: tcsh and zsh say No matches found. Bash by default passes any wildcard expression on if no matching file could be expanded. Both default behaviours can be changed by the user.

[askn37]

Try unsetopt nomatch (if you want that permanently, then add that line in .zshrc)

In this case, that customization certainly makes sense. I don't like changing the user environment and ideally would like to keep everything as factory as possible. (In my job) I always make sure that all test results are reproducible. Even Emacs hasn't been fixed. So I decided to stick to my guns and accept the inconvenience. Thank you for your advice.

[dl8dtl]

It's quite normal that you have to quote special chars to the shell, ? and * belong to them.
So either use \?, or "...?...".

I always considered the Bourne-shell (and thus bash) behaviour quite inconsistent: if you have something where * or ? could expand filenames to, they expand it, otherwise they pass it literally. Thus, the actual behaviour depends on the contents of your current directory – which I think is not really a Good Thing.

[dl8dtl]

Why did Apple kill bash and make this damn shell the standard?

Btw., they didn't "kill" it. Just as in any Unix, you can simply use "chsh" to change your shell to one you'd like.
Personally, I'm used to tcsh (and will probably use it till the end of my days ;-), but I think zsh is actually quite a good default for an interactive shell.