Linux Basics 08: Environment Variables And Shell Startup Files — Why ssh Cannot Find Your PATH

Published 2022-04-16 20:30 Updated 2022-04-16 22:10 915 words 5 min read ... Page views Handwritten Every word here was typed by me.

Linux Basics 25: Wrapping Up — Self-Audit Checklists And Where To Go DeeperLinux Basics 24: Boot And Rescue — UEFI, GRUB, initramfs And Emergency ModeLinux Basics 23: Containers And Kernel Boundaries — Namespaces, Cgroups And What docker exec DoesLinux Basics 22: Logs And A Debugging Method — Six Steps From Symptom To Root CauseLinux Basics 21: Scheduled Tasks — The Traps Of crontab And systemd TimersLinux Basics 20: Shell Scripting, Part Two — trap, Logging, Idempotence And LockingLinux Basics 19: Shell Scripting, Part One — Quoting, Tests And Exit CodesLinux Basics 18: Performance And Tracing — Where Is The Machine Actually StuckLinux Basics 17: Firewalls — How iptables, nftables, firewalld And ufw RelateLinux Basics 16: SELinux — When Every Permission Is Correct And The Service Is Still DeniedLinux Basics 15: Advanced Permissions — sudoers Wildcards, ACLs And CapabilitiesLinux Basics 14: SSH And Remote Operations — Keys, Permissions And Three Kinds Of TunnelLinux Basics 13: Networking Fundamentals — Troubleshoot From The Bottom UpLinux Basics 12: Disks And Filesystems — From Block Device To Mount PointLinux Basics 11: systemd — units, systemctl And journalctl Done ProperlyLinux Basics 10: Package Management — What apt Actually Does, And The Four Common ErrorsLinux Basics 09: Users And Groups — passwd, shadow And sudoersLinux Basics 08: Environment Variables And Shell Startup Files — Why ssh Cannot Find Your PATHLinux Basics 07: Processes And Signals — The Real Difference Between TERM And KILLLinux Basics 06: Pipelines And Redirection — Three Streams, One Ordering TrapLinux Basics 05: Text Processing — Turning grep/sort/uniq Into Your Analysis ToolkitLinux Basics 04: Permissions And Ownership — rwx Is Not The Same On Files And DirectoriesLinux Basics 03: The Real Semantics Of ls, cp, mv, rm — And The * That The Shell ExpandsLinux Basics 02: Why / Looks Like This, and What An Inode Really StoresLinux Basics 01: Terminals, Shells, and How Commands Are Actually Found
Variable scope and snapshot semantics, which files each of the four shell modes loads, why running a command over ssh misses your PATH additions, and how sudo’s secure_path rewrites the environment.

What This Chapter Solves

“node -v works in my terminal, but ssh host 'node -v' says command not found.” That complaint ties together environment variables, the four ways a shell can start, and sudo’s environment policy. Once this chain is clear, the everyday mysteries — “works locally, fails remotely”, “works in my shell, fails in cron” — stop being mysteries.

1. Three Levels Of Variables

VAR=local          # 1) shell variable: only this shell knows it
export VAR=global  # 2) environment variable: inherited by children
bash -c 'echo "${VAR:-<empty>}"'      # 3) a child reads the snapshot taken at fork time

MYVAR=1; export MYVAR; bash -c 'echo $MYVAR'   # 1
MYVAR=2; bash -c 'echo $MYVAR'                 # 1 ← the running child is unaffected

Note: environment variables are copies, not shared state. Anything the parent changes afterwards is invisible to already-running children — which explains both the export experiment in chapter 01 and the classic “I changed PATH in a script but the command still isn’t found”.

env | sort | head              # environment only
printenv PATH                  # one variable
set | head                     # environment + shell variables + functions (everything)
env -i bash -c 'env'           # a clean environment — for “which variable is missing?”

2. Four Ways A Shell Starts, Four Different Files

ModeTypical caseFiles loaded
login + interactivessh login, console login, bash -l/etc/profile → the first existing of ~/.bash_profile, ~/.bash_login, ~/.profile
non-login + interactiveopening a new terminal, typing bash/etc/bash.bashrc (Debian family) → ~/.bashrc
non-login + non-interactivessh host 'cmd', scripts, bash -cnone of them; only the file named by $BASH_ENV
login + non-interactivessh host 'bash -lc cmd'same row as “login + interactive”

Note: this table is the heart of the chapter. Almost every “command not found over ssh” case is row three — your PATH extension lives in ~/.bashrc, and that file is never read.

3. Real Failure 1: ssh host 'node -v' Cannot Find node

It is usually not a missing install but two things stacking up:

  1. The PATH extension lives in ~/.bashrc (or an nvm init block);
  2. ~/.bashrc on Ubuntu starts with this:
case $- in
  *i*) ;;      # interactive: keep going
  *) return;;  # non-interactive: bail out ← this is the ssh path
esac

So ssh host 'node -v' loads neither .bashrc nor .profile, and PATH has no nvm directory.

Four fixes, best first:

ssh host 'bash -lc "node -v"'                       # 1) force a login-shell run
ssh host 'source ~/.bashrc; node -v'                # 2) load it manually
# 3) move the PATH export into ~/.profile (read by login shells, interactive or not)
# 4) restructure ~/.bashrc so PATH/aliases do not sit behind the interactive check

What I do: PATH and nvm init live in ~/.profile; only interactive niceties (aliases, prompt) stay in ~/.bashrc.

4. Real Failure 2: sudo Changes Your PATH

echo "$PATH"
sudo sh -c 'echo $PATH'      # Debian default: secure_path = /usr/local/sbin:…:/bin
sudo env | grep -i path

The cause is secure_path in sudoers: for safety, sudo resets PATH to system directories only (env_reset + secure_path). The practical fallout: a program in /opt/xxx/bin will not run under sudo.

sudo -E whoami                     # keep the current environment (works; be aware of what you carry over)
sudo visudo                        # or write rules with absolute paths for specific commands

5. Make PATH Writes Idempotent

Sourcing startup files repeatedly inflates PATH into a long chain. The safe form checks first:

case ":$PATH:" in
  *":$HOME/bin:"*) ;;                       # already present, nothing to do
  *) PATH="$HOME/bin:$PATH" ;;
esac
export PATH

Note: ~/.bashrc gets sourced once per new terminal, so idempotence saves you from debugging “why does PATH contain /usr/local/bin eight times”.

6. Hands-On Checklist

MYVAR=1; bash -c 'echo "${MYVAR:-<empty>}"'           # 1. not exported: invisible to children
export MYVAR; bash -c 'echo $MYVAR'                   # 2. visible after export
echo $-                                               # 3. is this shell interactive (contains i)?
bash -lc 'echo login: $0'; bash -c 'echo plain: $0'   # 4. login vs plain
ssh localhost 'echo "$PATH"' 2>/dev/null || true      # 5. the non-interactive PATH
sudo sh -c 'echo "$PATH"'                             # 6. secure_path
env -i bash -c 'env'                                  # 7. clean environment for comparison

Summary

  • Environment variables are a snapshot at fork time; the parent must export, and later changes do not reach running children.
  • Login shells read the profile family, interactive non-login shells read the bashrc family, and non-interactive runs (ssh host 'cmd', scripts, cron) read neither.
  • Put PATH extensions in ~/.profile; the interactive check at the top of ~/.bashrc is the usual reason remote commands cannot find your tools.
  • sudo resets the environment (secure_path); use -E or sudoers rules when you really need something else.
  • Append to PATH idempotently.

Next Up

Users and groups: the seven fields of /etc/passwd, what ! and * mean in /etc/shadow, primary versus supplementary groups, why usermod -aG must include -a, and how su, sudo and sudoers relate.

Written by Handwritten

If you enjoyed this, leave a comment~

Loading...