Re: Revisiting Error handling (errexit)
Date:Wed, 6 Jul 2022 01:34:53 +0300
From:Yair Lenga
Message-ID: <6b7e3c85-5fcd-459e-a41c-e2803b0e7...@gmail.com>
| Function main () {
| Local x. # x is local
| For x in a b ; do process $x ; done
| }
|
| Vs.
| # x is global, all function will see it.
| For x in a b ; do process $x ; done
That is not how local variables work...
Consider:
cat /tmp/script; echo ===
func1()
{
echo In func1 x=$x
}
func2()
{
local x=foo
echo In func2 x=$x
}
func3()
{
local x=bar
echo In func3 x=$x
func1
}
x=anything
echo Globally x=$x
func1
func2
func3
echo Globally x=$x
===
Then
bash /tmp/script
Globally x=anything
In func1 x=anything
In func2 x=foo
In func3 x=bar
In func1 x=bar
Globally x=anything
Note the call of func1 from func3 and which x value it sees.
The (ideal) shell model (IMO) for variables is that all variables
are global, always. If we don't have that then it is hard to see
how to make things like OPTIND or IFS local, and work as expected,
(PATH as well) and explain how those work differently than how you
would expect other variables to work.
What local should do is arrange for any changes to a variable listed as
local to be undone as soon as the function that declared it local is no
longer active (however that happens), so when func3() makes x local,
the old value of x is saved somewhere, x is set (to bar in the above
script) and remains that way while func1 runs, when func3 is done, x
gets restored to its previous value (the effect of the local ends).
The model bash uses isn't exactly that, it is much harder to explain
(and to my mind, irrational, but that's neither here nor there), but
it isn't all that far away either.
In neither of these are local variables in bash anything like local
variables in languages designed as a programming language.
Remember that bash (and all of the other posix style shells, there are
quite a few) is primarily a shell - an environment for users to use to
type commands. It has its script ability from the genius of Ken Thompson,
who (about 50 years ago, when this idea was revolutionary) recognised that
if users could put the commands they habitually entered into a script, and
have the shell simply run the script, they wouldn't need to type the same
command sequences over and over again. Seems simple, and obvious, today,
it wasn't then - that's not how other contemporary OS's worked.
Users typing make decisions based upon what happens in earlier commands,
so the scripts needed to be able to do that as well, hence we gained
if (and later the looping commands - originally just if and goto), which
handle any anticipated decisions that users need to make, but users also
deal with unanticipated problems - things that can go wrong which weren't
planned for, that no-one had considered might happen. The typical user
reaction to one of those when it happens interactively is to just stop,
and either think, or seek help - certainly not just plow on with whatever
was supposed to happen next, which is what a script would do. That's
where -e (errexit) comes from, when something unanticipated fails, stop.
Note the unanticipated - when something that we expect might fail, fails,
we have already provided the alternate code sequence, for that, so we
don't stop.
This all sounds simple, but when (as you are discovering now I think) you
start to look at all of the weird cases, it isn't so simple - the effect
was that the implementation was a bit strange, and didn't do things quite
the way that people often thought it did - but that was the implementation,
and hence the standard, and now it is more or less locked in - other than
in the simplest cases, -e is useless.
But the solution to this isn't your idea, which will (almost inevitably)
end up being just as convoluted as -e, and just as weird (though differently)
as you try and cope with all the odd cases and make them behave as people
expect should happen. Instead (if you're writing a shell script, bash or
any other shell) the solution is as (I think) Greg suggested - simply
anticipate that anything might fail, decide what should happen in each case
if it does fail (which is anything from "I don't care" to "abandon the entire
script" with a whole range of possibilities in between).
If you do that, then neither -e, nor your proposed option, has anything
to do, they're both no-ops, and the code always behaves as implemented by
its author.
But since you seem to be attempting to emulate features of some other
languages, languages not primarily designed to be shells, the better
question is why you're not simply programming in one of those other
languages. There are lots, both fully compiled, and interpreted
(and semi-compiled) to choose from. Pick the one best suited for your
application, and use that. From the way it has been described, that's
very unlikely to be an
Re: Revisiting Error handling (errexit)
Modules can have conflicting requirements of global settings that require
weird contortions to interoperate; so starting out by making this yet
another global setting seems like a retrograde step.
So I would like to propose an alternative approach.
Firstly, I would reduce the action to simply "return from function upon
unchecked failure"; the issue is primarily with *invisible* behaviour at a
difference making this difficult to debug.
Secondly I would set this as a per-function attribute, rather than as a
global setting outside the function.
Perhaps
function -E funcname { … }
Re: Revisiting Error handling (errexit)
Hi. Thanks for proposal.
For my use case, where I have large existing code base, where I want to
improve error checking, and avoid unhandled errors, per function setting
will not work. This will also deviate from the try ... Catch pattern used
in many other scripting solution, making it hard for developer to adapt.
Asking for large set of changes (touch each function) not realistic for
many production environments.
That said, there might be some other use cases for limited scope. Probably
make sense to restore that errfail when a function exit. Practically
allowing more scoped execution.
Function foo { Set -oerrfail: ... ; }
Function bar { foo ; zap ; } ;
Function foo will run with errfail logic. But bash will revert back to no
errfail when control revert back to bar, and zap will run WITHOUT errfail.
I remember reading about other bash setting that are automatically restored
on function exits. Can not remember which one.
Another interesting option will be to limit the errfail to "blocks",
allowing something like
{ Set -o errfail ; foo ; bar ; } ; Baz ;
Instead off:
set -o errfail
{ Foo ; bar ; } || true
Set +o errfail.
Reflecting on my own usage, any will work, since I am planning to apply it
globally. Looking for community feedback. Want also to mention that I need
to change on scope of changes for each option - goal is to keep changes to
minimum, avoid risk of impact on existing scripts.
As of implementing -E, that seems like a big deviation from existing bash
style, and is likely to cause issues for developers targeting solutions
that will be somehow portable between *sh implementation.
Yair.
On Wed, Jul 6, 2022, 11:24 AM Martin D Kealey
wrote:
> Modules can have conflicting requirements of global settings that require
> weird contortions to interoperate; so starting out by making this yet
> another global setting seems like a retrograde step.
>
> So I would like to propose an alternative approach.
>
> Firstly, I would reduce the action to simply "return from function upon
> unchecked failure"; the issue is primarily with *invisible* behaviour at
> a difference making this difficult to debug.
>
> Secondly I would set this as a per-function attribute, rather than as a
> global setting outside the function.
>
> Perhaps
>
> function -E funcname { … }
>
Re: Revisiting Error handling (errexit)
2022年7月6日(水) 18:04 Martin D Kealey :
> Perhaps
>
> function -E funcname { … }
FWIW, an existing syntax for the "per-function set -o functrace / set
-T" is like
function funcname { ... }
declare -ft funcname# Note: -ft is a combination of two options -f and -t
--
Koichi
Re: Revisiting Error handling (errexit)
2022年7月6日(水) 19:05 Yair Lenga :
> Function foo will run with errfail logic. But bash will revert back to no
> errfail when control revert back to bar, and zap will run WITHOUT errfail.
> I remember reading about other bash setting that are automatically restored
> on function exits. Can not remember which one.
That should be `local -' of Bash 4.4+. E.g.,
function foo { local -; set -e; ...; }
--
Koichi
Feature request
Hey, guys I just joined to ask this please forgive me if I am in the wrong place. First of all big fan of bash, I use it a LOT 🚀. It would be fantastic if there was a nicer way to source files from the same folder instead of copy-pasting complicated logic in every script. Perhaps something similar to PHP's __DIR__ and __FILE__ constants? ``` source "$__DIR/file.ext" #$__DIR == the absolute path to the current scripts location. ``` I am aware there are existing bash mechanisms to achieve this but are cumbersome to use. USE CASE: I've been building myself a little ecosystem with hundreds of bash scripts. Its really handy to have the script's source and call each other without doing path resolving gymnastics in every single script. To make it nice to rapidly build scripts at will I have been putting common code into smaller scripts. And to solve the problem of easily sourcing them I just shoved them in my path lol. This lets me use a nice clean syntax like so: ``` #!/usr/bin/env bash source error-handler source colours #etc.. ``` While this makes including common files an absolute breeze it's not ideal to have all of them in my path because: - my sourceables also need logic to prevent them from being executed by accident (they tell you to source them instead). - I cant just share my script collection with colleagues because it will all break unless they add ALL my scripts to their system path and they might not want to do that. - all of the scripts call on each other and that also only works if the executables are in the system path. Defining a ton of functions and shoving them into peoples running environment is not an answer for me either lol. A shorthand to call them from the relative path would be immensely helpful for using bash scripting as a way to build suites of developer tools. ADDITIONAL FEATURE: This might be a little far fetch but, now that I think about it. A pseudo namespacing feature would be way handier, hear me out... Let's say you have a namespace keyword followed by the __DIR__ shorthand. And once that is done it will treat that folder as if it is at the front of the PATH var for the duration of the script. Then we can call upon all the local resources in the distributed scripts collection without jumping through hoops: ``` namespace $__DIR #$PATH = "$__DIR:$PATH" for the duration of the script. source error-handler DATA=some-common-command-in-my-library # etc.. ``` This would also mean that it can be easy to add a wrapper command to a collection of scripts named after the vendor so that, instead of directly calling from a collection of 300 scripts or adding them all to your path, you can simply add the one parent script to your path that namespaces and then calls all the others while providing bash completions etc. FOOTNOTE: I am aware I could already do this with existing bash mechanisms there is always a way. But, it's usually a hacky way I can't remember and always have to copy from an old script or look it up again and sometimes these hacks are not going to work on other systems. (talking in general not just about path resolving here now). I just feel we should be able to have some basic language features to make bash scripting better in general and being able to easily include code from your own folder would be a significant step up. Thanks for reading all this hope it makes sense.
Re: Feature request
Hello Brad!
> It would be fantastic if there was a nicer way to source files from the
> same folder instead of copy-pasting complicated logic in every script.
I'm not sure if I understand your questions correctly, but did you try
source "${0%/*}/file.ext"
?
>- my sourceables also need logic to prevent them from being executed by
>accident (they tell you to source them instead).
Just don't give them execute permission (at least when running on a unix
system).
Best regards,
Martin
Re: Feature request
On Wed, Jul 06, 2022 at 01:25:38PM +1000, Brad Hayes wrote: > source "$__DIR/file.ext" #$__DIR == the absolute path to the current > scripts location. https://mywiki.wooledge.org/BashFAQ/028
Re: Feature request
On Jul 06 2022, Brad Hayes wrote: > Perhaps something similar to PHP's __DIR__ and __FILE__ constants? You can get that from $0. -- Andreas Schwab, SUSE Labs, sch...@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different."
Re: Warning raised if trying to set LC_ALL to a locale bearing a character type other than UTF-8 and C
On 7/1/22 9:08 AM, Kerin Millar wrote: > Hello, > > I was experimenting with locale variables in bash and noticed something that > appears odd. This is with bash 5.1.16 and glibc 2.35. > > $ locale -a | grep -E '^en_(US|GB)' > en_GB.utf8 > en_US > en_US.iso88591 > en_US.utf8 > > $ locale > LANG=en_US.UTF-8 > LC_CTYPE="en_US.UTF-8" > LC_NUMERIC="en_US.UTF-8" > LC_TIME="en_US.UTF-8" > LC_COLLATE="en_US.UTF-8" > LC_MONETARY="en_US.UTF-8" > LC_MESSAGES="en_US.UTF-8" > LC_PAPER="en_US.UTF-8" > LC_NAME="en_US.UTF-8" > LC_ADDRESS="en_US.UTF-8" > LC_TELEPHONE="en_US.UTF-8" > LC_MEASUREMENT="en_US.UTF-8" > LC_IDENTIFICATION="en_US.UTF-8" > LC_ALL= > > $ ( LC_ALL=en_US.ISO-8859-1 ) > bash: warning: setlocale: LC_ALL: cannot change locale (en_US.ISO-8859-1) Bash prints a warning if setlocale(3) returns an error (a null string) to inform the user that even though they changed the shell variable, the global locale wasn't changed correspondingly. Since there are no errors defined for setlocale(), it's perfectly reasonable for it not to change errno, so there's no additional error message if errno stays 0. You can look at locale.c:set_locale_var() for the details. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: documentation string for caller builtin
On 7/2/22 6:09 AM, Ernest Adrogué wrote: > Hello, > > The documentation string for the 'caller' builtin command in bash.pot > appears to be incomplete, in particular the last 4 lines from the > snippet below which were present in previous versions are now missing: Not so. #: builtins.c:369 msgid "" "Return the context of the current subroutine call.\n" "\n" "Without EXPR, returns \"$line $filename\". With EXPR, returns\n" "\"$line $subroutine $filename\"; this extra information can be used to\n" "provide a stack trace.\n" "\n" "The value of EXPR indicates how many call frames to go back before the\n" "current one; the top frame is frame 0.\n" "\n" "Exit Status:\n" "Returns 0 unless the shell is not executing a shell function or EXPR\n" "is invalid." msgstr "" The other one is present if the caller builtin is compiled as a loadable builtin, which it is not in the distributed version of bash. I suppose I could make the two identical, but the loadable builtin version is never used, so it doesn't seem like a priority. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Revisiting Error handling (errexit)
Koichi - Thanks for highlighting this 'local -'.
This feature effectively eliminates the need to support scoped 'errfail'.
If it's needed in a limited context, the 'local -' can be used.
Yair
On Wed, Jul 6, 2022 at 1:21 PM Koichi Murase wrote:
> 2022年7月6日(水) 19:05 Yair Lenga :
> > Function foo will run with errfail logic. But bash will revert back to no
> > errfail when control revert back to bar, and zap will run WITHOUT
> errfail.
> > I remember reading about other bash setting that are automatically
> restored
> > on function exits. Can not remember which one.
>
> That should be `local -' of Bash 4.4+. E.g.,
>
> function foo { local -; set -e; ...; }
>
> --
> Koichi
>
Re: Man page section for complete's -C option is inconsistent
On 7/4/22 12:03 AM, Mark Chandler via Bug reports for the GNU Bourne Again SHell wrote: > Configuration Information [Automatically generated, do not change]: > Machine: x86_64 > OS: linux-gnu > Compiler: gcc > Compilation CFLAGS: -O2 -flto=auto -ffat-lto-objects -fexceptions -g > -grecord-gcc-switches -pipe -Wall -Werror=format-security > -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS > -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong > -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic > -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection > uname output: Linux ice-mef 5.18.7-200.fc36.x86_64 #1 SMP PREEMPT_DYNAMIC > Sat Jun 25 20:06:14 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux > Machine Type: x86_64-redhat-linux-gnu > > Bash Version: 5.1 > Patch Level: 16 > Release Status: release > > Description: > Even though the -C option of "complete" is functionally very > similar to -F, the man page > only provides detailed usage for -F. The -C description doesn't > mention parameter passing. Look at the `Programmable Completion' section: "When the function or command is invoked, the first argument ($1) is the name of the command whose arguments are be- ing completed, the second argument ($2) is the word being completed, and the third argument ($3) is the word preceding the word being com- pleted on the current command line." I can't recall why I duplicated that text in the description of `complete'. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: local -p outputs invalid declare -- -
On 7/1/22 1:16 AM, Emanuele Torre wrote: > Bash Version: 5.1 > Patch Level: 16 > Release Status: release > > Description: > `local -p' (and also `declare -p'/`typeset -p') outputs > "declare -- -" when used in a function that has declared > `local -'. > > `declare -' (and `typeset -') is not a valid command, not even > in functions. > The only valid way to localise shell options is `local -' Thanks for the report. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
$(( )): binary/unary VAR/NUM inconsistency
Hello. 'Struggling with a Dijkstra two stack parser i am thankful to be able to compare against bash that uses a completely different and correct parser! I stumbled upon this (bash 5.1.16); i find it "understandable" in respect to clarity, but inconsistent, and also in hindsight to the announcement of a forthcoming release i thought i report it quick: $ bash -c 'I=10; echo $((1++I)); echo $I' bash: line 1: 1++I: syntax error in expression (error token is "++I") $ bash -c 'I=10; echo $((1++1)); echo $I' 10 $ bash -c 'I=10;echo "<$(( +10+++I ))>"' <21> $ bash -c 'I=10;echo "<$(( +10+ ++I ))>"' <21> $ bash -c 'I=10;echo "<$(( +10+ +I ))>"' <20> I would find it more consistent if nothing would be whitespace affine. (The parsers in the wild are exactly so, very much including mine which leans upon the busybox shell parser, for example $ busybox.static sh -c 'I=10;echo "<$(( +10+ +I ))>"' <20> $ busybox.static sh -c 'I=10;echo "<$(( +10++I ))>"' <20> $ busybox.static sh -c 'I=10;echo "<$(( +10+++I ))>"' <21> $ busybox.static sh -c 'I=10;echo "<$(( +10++ + + + + +I ))>"' <20> $ busybox.static sh -c 'I=10;echo "<$(( +10++ +++I ))>"' sh: arithmetic syntax error $ busybox.static sh -c 'I=10;echo "<$(( +10++ ++ +I ))>"' sh: arithmetic syntax error $ busybox.static sh -c 'I=10;echo "<$(( +10+ + ++ +I ))>"' sh: arithmetic syntax error $ busybox.static sh -c 'I=10;echo "<$(( +10 + + ++ +I ))>"' sh: arithmetic syntax error $ busybox.static sh -c 'I=10;echo "<$(( +10 + + ++ + +I ))>"' sh: arithmetic syntax error $ busybox.static sh -c 'I=10;echo "<$(( +10 + + + + + +I ))>"' <20> But am not ready here yet. Just for comparison.) --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: $(( )): binary/unary VAR/NUM inconsistency
On Jul 06 2022, Steffen Nurpmeso wrote: > $ bash -c 'I=10; echo $((1++I)); echo $I' > bash: line 1: 1++I: syntax error in expression (error token is "++I") > $ bash -c 'I=10; echo $((1++1)); echo $I' > 10 > > $ bash -c 'I=10;echo "<$(( +10+++I ))>"' > <21> A C compiler would parse all those expressions as post-increment applied to a non-lvalue. -- Andreas Schwab, sch...@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different."
Re: $(( )): binary/unary VAR/NUM inconsistency
Andreas Schwab wrote in
<87czeiqhb2@igel.home>:
|On Jul 06 2022, Steffen Nurpmeso wrote:
|
|> $ bash -c 'I=10; echo $((1++I)); echo $I'
|> bash: line 1: 1++I: syntax error in expression (error token is "++I")
|> $ bash -c 'I=10; echo $((1++1)); echo $I'
|> 10
|>
|> $ bash -c 'I=10;echo "<$(( +10+++I ))>"'
|> <21>
|
|A C compiler would parse all those expressions as post-increment applied
|to a non-lvalue.
mumble
But i am happy the small hedgehog kid made it throught the winter,
all on his own!! And came back with a girl friend: "Frühling!"
(Spring!), as if it is the simplemost thing on this earth!!
I now call him "Riccio", "mein wunderbarer Held!" ("Hedgehog, my
wonderful hero!"). Yes!!! (I just hope he is always careful with
the many cars. You know, these two legged are a real problem.)
--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
Re: documentation string for caller builtin
2022-07- 6, 09:48 (-0400); Chet Ramey escriu: > > Hello, > > > > The documentation string for the 'caller' builtin command in bash.pot > > appears to be incomplete, in particular the last 4 lines from the > > snippet below which were present in previous versions are now missing: > > Not so. > > [...] > > The other one is present if the caller builtin is compiled as a loadable > builtin, which it is not in the distributed version of bash. I suppose I > could make the two identical, but the loadable builtin version is never > used, so it doesn't seem like a priority. I see. I didn't notice there are two versions of the same string. Thanks for clearing this up. Cheers
Re: Man page section for complete's -C option is inconsistent
Yep. It's definitely clear if you read the 'Programmable Completion' section. A brief reference to the 'Programmable Completion' section to get that explanation would be useful. Or some hint that further reading is required to understand the functionality of this option. As it stands the explanation of the -C option by itself is misleading. It doesn't mention the parameter passing at all. But it implies that all you need is a command that outputs possible completions. Particularly when you can see the following option -F explains how parameter passing is done, because that text is lacking for -C, it further implies that parameter passing is not done for -C. This is why I said that it's inconsistent. If you like, I can try and go back through the source history to see if there's a comment around the commit for the -F text, to explain why it's more detailed there and not for -C. It is curious that they're so different. Cheers, Mark C. On 07/07/2022 00:11, Chet Ramey wrote: On 7/4/22 12:03 AM, Mark Chandler via Bug reports for the GNU Bourne Again SHell wrote: Configuration Information [Automatically generated, do not change]: Machine: x86_64 OS: linux-gnu Compiler: gcc Compilation CFLAGS: -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection uname output: Linux ice-mef 5.18.7-200.fc36.x86_64 #1 SMP PREEMPT_DYNAMIC Sat Jun 25 20:06:14 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux Machine Type: x86_64-redhat-linux-gnu Bash Version: 5.1 Patch Level: 16 Release Status: release Description: Even though the -C option of "complete" is functionally very similar to -F, the man page only provides detailed usage for -F. The -C description doesn't mention parameter passing. Look at the `Programmable Completion' section: "When the function or command is invoked, the first argument ($1) is the name of the command whose arguments are be- ing completed, the second argument ($2) is the word being completed, and the third argument ($3) is the word preceding the word being com- pleted on the current command line." I can't recall why I duplicated that text in the description of `complete'.
