Unix & Linux
shell posix shebang
Updated Fri, 20 May 2022 06:10:45 GMT

Why is the behavior of the `#!` syntax unspecified by POSIX?

From the Shell Command Language page of the POSIX specification:

If the first line of a file of shell commands starts with the characters "#!", the results are unspecified.

Why is the behavior of #! unspecified by POSIX? I find it baffling that something so portable and widely used would have an unspecified behavior.


I think primarily because:

  • the behaviour varies greatly between implementation. See https://www.in-ulm.de/~mascheck/various/shebang/ for all the details.

    It could however now specify a minimum subset of most Unix-like implementations: like #! *[^ ]+( +[^ ]+)?\n (with only characters from the portable filename character set in those one or two words) where the first word is an absolute path to a native executable, the thing is not too long and behaviour unspecified if the executable is setuid/setgid, and implementation defined whether the interpreter path or the script path is passed as argv[0] to the interpreter.

  • POSIX doesn't specify the path of executables anyway. Several systems have pre-POSIX utilities in /bin//usr/bin and have the POSIX utilities somewhere else (like on Solaris 10 where /bin/sh is a Bourne shell and the POSIX one is in /usr/xpg4/bin; Solaris 11 replaced it with ksh93 which is more POSIX compliant, but most of the other tools in /bin are still ancient non-POSIX ones). Some systems are not POSIX ones but have a POSIX mode/emulation. All POSIX requires is that there be a documented environment in which a system behaves POSIXly.

    See Windows+Cygwin for instance. Actually, with Windows+Cygwin, the she-bang is honoured when a script is invoked by a cygwin application, but not by a native Windows application.

    So even if POSIX specified the shebang mechanism it could not be used to write POSIX sh/sed/awk... scripts (also note that the shebang mechanism cannot be used to write reliable sed/awk script as it doesn't allow passing an end-of-option marker).

Now the fact that it's unspecified doesn't mean you can't use it (well, it says you shouldn't have the first line start with #! if you expect it to be only a regular comment and not a she-bang), but that POSIX gives you no guarantee if you do.

In my experience, using shebangs gives you more guarantee of portability than using POSIX's way of writing shell scripts: leave off the she-bang, write the script in POSIX sh syntax and hope that whatever invokes the script invokes a POSIX compliant sh on it, which is fine if you know the script will be invoked in the right environment by the right tool but not otherwise.

You may have to do things like:

#! /bin/sh -
if : ^ false; then : fine, POSIX system by default
  # cover Solaris 10 or older. ": ^ false" returns false
  # in the Bourne shell as ^ is an alias for | there for
  # compatibility with the Thompson shell.
  PATH=`getconf PATH`:$PATH; export PATH
  exec /usr/xpg4/bin/sh - "$0" ${1+"$@"}
# rest of script

If you want to be portable to Windows+Cygwin, you may have to name your file with a .bat or .ps1 extension and use some similar trick for cmd.exe or powershell.exe to invoke the cygwin sh on the same file.

Comments (3)

  • +0 – Interestingly, from issue 5: "The construct #! is reserved for implementations wishing to provide that extension. A portable application cannot use #! as the first line of a shell script; it might not be interpreted as a comment." — Dec 18, 2018 at 07:59  
  • +0 – @muru If the script was truly portable, on a truly POSIX system running a POSIX sh, it would not need a hashbang line as it would be executed by POSIX sh. — Dec 18, 2018 at 08:11  
  • +1 – @Kusalananda that's only true if execlp or execvp were used, right? If I were to use execve, it would result in ENOEXEC? — Dec 18, 2018 at 08:18