Escaping / character in substitution expansions

[dnewhall]

While doing a naive path canonicalization I tried replacing "//" with "/", and so I wrote the following:

function canonicalize_path {
    echo "${1//\/\//\/}"
}

When I run this in Ksh, I get:

ksh$ canonicalize_path "//home////foo"
\/home\/\/foo

But when I run it in bash, I get:

bash$  canonicalize_path "//home////foo"
/home//foo

I am extremely fine with us not being bash-compatible, but is that the expected behavior? It seems to go against the default assumptions regarding strings.

[jghub]

my (limited ...) understanding is, that the replacement part is principally a verbatim string and / is not a special character per se, therefore the canonical way seems

echo "${s//\/\///}"

i.e. no quoting in the replacement string (a single /) and this works correctly and identical in ksh and bash, both producing your "bash result" in your post, /home//foo. when you do insert a backslash, the replacement string is \/ and that is what ksh uses. IIUC, bash just does removal of "redundant quoting" at parse time and therefore throws away the \ while ksh does not. ksh does the same redundant quote removal for a replacement string like, e.g., \abc and seemingly \/ is treated later/elsewhere since it contains the separator character / which does indeed have special meaning in ${parameter//pattern/string}. someone more in the know might correct me or give a clearer explanation of course :). in any case I believe

echo "${s//\/\///}"

is the "correct" way to replace two / by a single one, and for actual path normalisation the solution would be

echo ${s//+(\/)//}

[dnewhall]

After poking around a bit more, this appears inconsistent with every other variable expansion.
For example:

ksh$ echo ${PATH:+\/\/}
//
bash$ echo ${PATH:+\/\/}
//

$ echo ${PATH}
/usr/local/bin:/bin:/home/foo/bin
ksh$ echo ${PATH##*\/}
bin
bash$ echo ${PATH##*\/}
bin

Everywhere else appears to interpret \/ as simply /. While the documentation does say "string" for substitution instead of "word", I still think I'd classify this as a bug because it behaves so differently from everywhere else.

(And, of course, the code I gave wasn't intended to be an example of good, working code; it's just the dumb thing I had in front of me that illustrated the error.)

[jghub]

I am quite sure that this cannot be called a bug: I am not an authority in all things POSIX but the internet says behaviour of \ in the substitution part (the string in ${parameter//pattern/string} is not specified by POSIX and ksh93 chooses to treat it differently than bash (and others). no quoting of / is required in "string" to start with (and simply should not be there in the first place in my view). if such \-"quoting" appears, bash decides to treat occurrence of the \/ sequence in string as redundant quoting while ks93 decides to take that string literally/verbatim. this will not meet user expectation if coming from bash, but it is not "wrong".

regarding ${parameter##pattern}: there, the \/ or / appear in the pattern that is searched for and \/ quoting is possible but not mandatory while in ${parameter//pattern/string} the \/ quoting is mandatory in pattern (to disambiguate meaning of / in this construct).

so this all seems completely consistent and deterministic to me and boils down to a different choice how to interpret occurrence of \/ in the "string" part between ksh93 and bash. this is not a bug in ksh93. whether it is a feature is of course a matter of taste ;). it is convenient whenever one actually wants to replace / by \/ in ${parameter}...

I would agree that this difference between ksh93 and the rest of the bunch (including mksh, eg.) might cause problems when porting scripts. but changing the behaviour of ksh93 now would break all sort of existing ksh93 scripts (mine, anyway ;)).