wrong shell

[AlJohri]

@ofek userpath doesn't work for my setup locally so I'm having some trouble testing.

Here are my dotfiles in case it helps: https://github.com/AlJohri/dotfiles

Issues:

• current shell is ZSH but it appends to bash files
• appends to both ~/.bashrc and ~/.bash_profile instead of just doing the right thing based on the OS

$ userpath append /testinguserpath
Success!
$ echo "$?"
0

$ userpath verify /testinguserpath
The directory `/testinguserpath` is in PATH, pending a shell restart!

$ echo "$?"
2

$ echo "$SHELL"
/usr/local/bin/zsh

$ tail -n 2 ~/.bash_profile
# Created by `userpath` on 2019-05-10 15:39:32
export PATH="$PATH:/testinguserpath"

$ tail -n 2 ~/.bashrc
# Created by `userpath` on 2019-05-10 15:39:32
export PATH="$PATH:/testinguserpath"

~/.zprofile and ~/.zshrc were untouched

[ofek]

Yup, I'll need to add support for more shells.

@AlJohri @cs01 @jaraco On Unix, would you like me to:

1. Keep adding to every possible shell's profile and add a flag to narrow down based on OS & $SHELL
2. Narrow down based on OS & $SHELL and add a flag to add to everything

[AlJohri]

I would very much appreciate not adding the PATH to all my shell files

[AlJohri]

You can check how https://github.com/mklabs/tabtab does it

EDIT: disregard this comment. looks like they just ask the user

https://github.com/mklabs/tabtab/blob/db75a56c3935bfffb0d528286f6aa23b4035a675/lib/prompt.js#L5-L49

[jaraco]

I thought I'd tested zsh, but I see now I didn't (zsh was inheriting the path from the parent shell). Thankfully, xonsh seems to do the right thing, presumably inheriting from the .bash_profile or .bashrc (though I don't see in the docs where that's meant to happen). I think I'd assumed if userpath worked for xonsh that it likely worked for less obscure shells, but it's clear to me with a little testing that there's no obvious way to do it and may not even be a way to do it in the general case.

If only Unix had a convention where environment variables could be specified per-user without being shell-specific.

On Unix, would you like me to:

Of the two options, the second one seems preferable to me.

Sounds like you're on the right track!

[cs01]

Narrow down based on OS & $SHELL and add a flag to add to everything

What is "everything" here?

I vote to form a heuristic to determine the best single file to modify (i.e. SHELL and OS), and to only modify that single file if there is enough confidence from the heuristic.

[jaraco]

I saw this answer that purports that .profile works on many shells not zshell, but in my testing, that did not work for me under bash (maybe the presence of another file supersedes it).

[ned2]

A related issue in that it varies across OSes which I just uncounted (and which could be relevant to your situation @jaraco):

Ubuntu defaults to providing a .profile (and not a .bash_profile), which I've been using as .bash_profile might often be used. Perhaps this is not best practice, but I suspect a lot of people do this by virtue of Ubuntu's initial setup.

The catch is that if a .bash_profile is present, then .profile will not be sourced. So the default behaviour of userpath automatically adding a .bash_profile blocks an existing .profile from being sourced. Took me a little while to workout why my config was broken.

[AlJohri]

@ofek I have not been following the conversation so forgive me if this has already been covered but I still have some concerns.

as @cs01 stated:

I vote to form a heuristic to determine the best single file to modify (i.e. SHELL and OS), and to only modify that single file if there is enough confidence from the heuristic.

the latest version is still appending to both bash_profile and bashrc? I thought it should do the correct thing based on the operating system? Same for zsh. It's doing both zprofile and zshrc. I may have missed some discussion but this feels like its polluting my shell profile.

[ofek]

@AlJohri It will (usually) be 2 files per shell: 1 for login (-l/--login) & 1 for non-login shells. What #6 did was add support for more shells, and make it so it won't update files for everything (bash, zsh, fish, etc.) but rather based on your current shell (or any you select).

If you want, I could add another flag --login/--non-login which would default to both. Keep in mind if one of your shell configs does not source the other, you really do want both updated 😄

[AlJohri]

I think we should handle the cases that are known. I would also like to see a CLI option and environment variable where I can manually define the file I want the PATH to go in. For example, I personally use ~/.shprofile. It would be great to specify USERPATH_FILE=$HOME/.shprofile and have it automatically use that regardless of she shell along with cli arg such as userpath --file

Here's my thoughts so far on detection using OS+Shell:

---

general note on bash quirk:

in bash, the login-shell only runs the bash_profile or profile while a non-login shell only runs the bashrc.

this means, that an interative, login shell will not automatically source bashrc. other shells such as zsh do not have this strange behavior. in zsh, if its an interactive shell, it will run zshrc regardless of login or non-login. this is why only bash requires sourcing bashrc from the profile.

for bash, there have two approaches on non-macos systems:

1. modify BOTH bash_profile/profile and bashrc so future shells in the current login will have the PATH enabled

2. modify ONLY the bash_profile/profile, set the PATH for the current session, and tell the user the PATH will not be available in future terminal sessions until they log out and log back in

I'm personally more in favor of the latter but I don't work on linux systems everyday so open to feedback here. I think it's bad practice to populate PATHs in two places and would like to avoid that as much as possible.

---

general note on /etc/profile quirk:

on macos, arch linux, and potentially other distributions, the /etc/profile sets the initial profile https://stackoverflow.com/questions/21038903/path-variable-in-zshenv-or-zshrc

this gets run after ~/.zshenv. thus while zshenv seems like the ideal place to set the path since it is always sourced, it has quirks on some systems as the path gets overidden.

---

Rules

1. macos: on macos, every new terminal session is a login shell. the initial path gets set by /etc/profile (which is run after zshenv). thus, we set the path using bash_profile or zprofile.

   • macOS + bash = bash_profile
   • macOS + zsh = zprofile

2. ubuntu: on ubuntu, every new terminal session is a non-login shell. the default setup presents a .bashrc and .profile (where the .profile sources the .bashrc). on ubuntu the /etc/profile does not set an initial PATH so zshenv is fair game.

   • ubuntu + bash = profile/bash_profile and request user to re-login to enable in future terminal sessions
     • OR ubuntu + bash = bashrc + profile if asking to re-login is not viable
   • ubuntu + zsh = zshenv

   profile/bash_profile means use whichever file already exists, defaulting to profile. this with @ned2's issue above

3. any other linux distribution (such as arch)

   • linux + bash = profile/bash_profile and request user to re-login to enable in future terminal sessions
     • OR linux + bash = bashrc + profile/bash_profile if asking to re-login is not viable
   • linux + zsh = zprofile and request user to re-login to enable in future terminal sessions
     • OR linux + zsh = zshrc + zprofile if asking to re-login is not viable

   profile/bash_profile means use whichever file already exists, defaulting to bash_profile. this with @ned2's issue above

4. fish

   • macos + fish = ~/.config/fish/config.fish
   • OR just use set -U fish_user_paths /usr/local/bin $fish_user_paths since this always works, and you don't need to find the config file. see $PATH section http://fishshell.com/docs/current/tutorial.html

   I think the latter makes more sense for fish users.

---

Open to feedback! I'm a macOS user so I mostly care about the macOS rules but I think having a smart set of defaults that works for most users and is overridable/configurable when needed is best.

[AlJohri]

Some Example Docker Commands to Verify the Above Notes:

macOS

$ cat /etc/profile

# System-wide .profile for sh(1)

if [ -x /usr/libexec/path_helper ]; then
    eval `/usr/libexec/path_helper -s`
fi

if [ "${BASH-no}" != "no" ]; then
    [ -r /etc/bashrc ] && . /etc/bashrc
fi

Ubuntu

$ docker run -i ubuntu:18.04 ls -a /root
$ docker run -i ubuntu:18.04 cat /root/.bashrc
$ docker run -i ubuntu:18.04 cat /root/.profile
$ docker run -i ubuntu:18.04 cat /etc/profile

Arch

$ docker run -i archlinux/base ls -a /root
$ docker run -i archlinux/base cat /etc/profile

Fedora

$ docker run -i fedora:latest ls -a /root
$ docker run -i fedora:latest cat /root/.bashrc
$ docker run -i fedora:latest cat /root/.bash_profile

[ofek]

@AlJohri Ah wow thanks, I didn't know that behavior was only bash! I'll tackle this over the weekend.

[AlJohri]

no problem. I mentioned "two approaches on non-macos systems" above, one requiring the user to re-login but modifying only a single file and the other modifies two files but does not require the user to re-login.

I personally lean toward asking the user to re-login for future terminal sessions or they can manually source the bash_profile/profile in a new session until the next login.

What do you think?

[cs01]

I agree with modifying a single file and having the user re-login.

[AlJohri]

@ofek just checking in- let me know if you need any help. would love to get this wrapped up

[ofek]

I'm sorry, but I have to make a very unexpected move and therefore lack free time currently.

I would gladly accept a PR.

[AlJohri]

best of luck with the move- I don't have time to submit a PR right now but will check back in in a week or so

[AlJohri]

hey @ofek, just checking in to see if you have some more time soon

[AlJohri]

@ofek I have some time to work on this month- are you available to review a PR in the near future?

[ofek]

Yes for sure!!!

[harkabeeparolus]

bash Background

Rant mode ON. 🙀

Just FYI (Unix geek here), I think bash is broken by design when it comes to startup files. This is especially true in big university multiuser Unix installations, which is my own background. For proof, run man bash, look under the "INVOCATION" section, and prepare to have your mind blown. Compare and contrast this with man zsh and look under "STARTUP/SHUTDOWN FILES", for a much less insane approach to system and user configuration. IMHO. 👀

Red Hat Enterprise Linux, Ubuntu, and probably other OSes or distros attempt to work around the bash problem by providing clever dotfiles in "/etc/skel" -- these are dotfiles that are copied into a new user's home directories by default when the new user is created. These default dotfiles are supposed to create a complex structure where each user has a ~/.bash_profile (OR ~/.profile on Ubuntu) which always sources the user's ~/.bashrc. That way, both dotfiles will be executed for login shells, and the bulk of the configuration can be put into ~/.bashrc. On Red Hat, ~/.bashrc IN TURN also sources /etc/bashrc. To repeat; ~/.bash_profile sources ~/.bashrc sources /etc/bashrc. You could say that on Red Hat, Ubuntu et. al. it would be considered broken behavior NOT to create these default dotfiles in a new user's home directory.

Take a wild guess if Apple creates bash dotfiles in users' home directories on macOS? Of course not. 🤦 Instead, as @AlJohri noted above, they just run every Terminal window as a login shell (!). 🤦

I'm just happy Apple finally decided to switch from bash to zsh starting with macOS 10.15 Catalina, so the problem with bash will eventually go away there.

Rant mode OFF. 😉

Solution?

I believe the best way to treat bash users is probably to assume that the user will encounter at least one "login shell" somewhere during the login process before running userpath, and add our PATH definition to the first dotfile for bash login shells that we find. The reason I say this is that these files are also where Red Hat, Ubuntu and others suggest that users should add their own $PATH definitions. To quote from the manual, a bash login shell:

looks for ~/.bash_profile, ~/.bash_login, and ~/.profile, in that order, and reads and executes commands from the first one that exists and is readable.

Note — ONLY from the first one. If a ~/.bash_profile exists, that will disable the other two files, as far as bash is concerned.

What to do if none of the login dotfiles exist? That's a trickier question.

• The safest behavior is to exit with an informative error message.
  • You could also look for a ~/.bashrc file, and hope that the user or the OS has some arrangement for running it most of the time. But this is NOT guaranteed to work — especially for "login shells", since they don't read ~/.bashrc. On macOS, every Terminal window is a login shell... So that does not work. 👀
• On macOS, which is known to be "broken" for bash users (i.e. no user dotfiles exist by default), I might consider creating an empty ~/.bash_profile if none of the other login shell dotfiles exist, and THEN adding any additional PATH definitions to our newly created ~/.bash_profile.
  • Starting with macOS 10.15 Catalina, all bash users are told (in their Terminal windows) to switch to zsh, and new users have zsh by default. So this problem will decrease over time.

To sum up... This is my opinion, but I'm basing it on a lot of experience with Unix shell scripting and user support at university Unix systems. I don't believe that there's a single "correct" solution for bash. But we can try to be as helpful as possible, at least.

I'll be happy to answer any additional questions about this subject. 😄