Skip to content

Commit

Permalink
[shopt -s strict_errexit] Allow && || ! expressions
Browse files Browse the repository at this point in the history
i.e. compound conditionals

Grouping with { } is still an issue.  This can be done with chained if
statements for now.

[doc] Intro to process model doc

[doc] YSH FAQ: "subshells by surprise" and placeholder for test --true
--false
  • Loading branch information
Andy C committed Oct 16, 2024
1 parent ed2e2ac commit 11f8dff
Show file tree
Hide file tree
Showing 5 changed files with 165 additions and 13 deletions.
71 changes: 60 additions & 11 deletions doc/process-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,34 @@
in_progress: yes
---

Process Model
The Unix Shell Process Model - When Are Processes Created?
=============

Why does a Unix shell start processes? How many processes are started?
OSH and YSH are both extensions of POSIX shell, and share its underlying "process model".

Each Unix process has its **own** memory, that is not shared with other
processes. (It's created by `fork()`, which means that the memory is
"copy-on-write".)

Understanding when a shell starts processes will make you a better shell
programmer.

As a concrete example, here is some code that behaves differently in
[bash]($xref) and [zsh]($xref):


$ bash -c 'echo hi | read x; echo x=$x'
x=

$ zsh -c 'echo hi | read x; echo x=$x'
x=hi

If you understand why they are different, then that means you understand the
process model!

(OSH behaves like zsh.)

---

Related: [Interpreter State](interpreter-state.html). These two docs are the
missing documentation for shell!
Expand All @@ -15,14 +39,14 @@ missing documentation for shell!

## Shell Constructs That Start Processes

### Pipelines
### Pipelines `myproc | wc -l`

- `shopt -s lastpipe`
- `set -o pipefail`

#### Functions Can Be Transparently Put in Pipelines
Note that functions Can Be Transparently Put in Pipelines:

Implicit subshell:
Hidden subshell:

{ echo 1; echo 2; } | wc -l

Expand All @@ -44,8 +68,31 @@ Explicit Subshells are Rarely Needed.

- prefer `pushd` / `popd`, or `cd { }` in YSH.


## FAQ: "Subshells By Surprise"

Sometimes subshells have no syntax.

Common issues:

### shopt -s lastpipe

Mentioned in the intro:

$ bash -c 'echo hi | read x; echo x=$x'
x=

$ zsh -c 'echo hi | read x; echo x=$x'
x=hi

### Other Pipelines

myproc (&p) | grep foo

## Process Optimizations - `noforklast`

Why does a Unix shell start processes? How many processes are started?

Bugs / issues

- job control:
Expand All @@ -64,11 +111,11 @@ Oils/YSH specific:
- because we don't get to test if it failed
- stats / tracing - counting exit codes


## Process State

### Redirects


## Builtins

### [wait]($help)
Expand All @@ -82,9 +129,11 @@ Oils/YSH specific:

## Appendix: Non-Shell Tools

- `xargs` and `xargs -P`
These Unix tools start processes:

- `xargs`
- `xargs -P` starts parallel processes (but doesn't buffer output)
- `find -exec`
- `make -j`
- doesn't do anything smart with output
- `ninja`
- buffers output too
- `make`
- `make -j` starts parallel processes (but doesn't buffer output)
- `ninja` (buffers output)
39 changes: 39 additions & 0 deletions doc/ysh-faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -205,6 +205,45 @@ not `${}`.
-->

## How do I combine conditional commands and expressions: `if (myvar)` versus `if test`?

TODO: `test --true --false`

This happens in `while` too.

## Why do I lose the value of `p` in `myproc (&p) | grep foo`?

In a pipeline, most components are **forked**. This means that `myproc (&p)`
runs in a different process from the main shell.

The main shell can't see the memory of a subshell.

---

In general, you have to restructure your code to avoid this. You could use a proc with multiple outputs:

myproc (&p, &grepped_output)

Or you could use a function:

var out1, out2 = myfunc(io)

---

[The Unix Shell Process Model - When Are Processes
Created?](process-model.html) may help.

This issue is similar to the `shopt -s lastpipe` issue:

$ bash -c 'echo hi | read x; echo x=$x'
x=

$ zsh -c 'echo hi | read x; echo x=$x'
x=hi

In bash, `read` runs in a subshell, but in `zsh` and OSH, it runs in the main
shell.

## Related

- [Oil Language FAQ]($wiki) on the wiki has more answers. They may be migrated
Expand Down
8 changes: 8 additions & 0 deletions osh/cmd_eval.py
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,14 @@ def _HasManyStatuses(node):
# Multiple parts like 'ls | wc' is disallowed
return True

elif case(command_e.AndOr):
node = cast(command.AndOr, UP_node)
for c in node.children:
if _HasManyStatuses(c):
return True
return False # otherwise allow 'if true && true; ...'


# - ShAssignment could be allowed, though its exit code will always be
# 0 without command subs
# - Naively, (non-singleton) pipelines could be allowed because pipefail.
Expand Down
56 changes: 56 additions & 0 deletions spec/errexit-osh.test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,62 @@ fi
yes
## END

#### strict_errexit with && || !
set -o errexit
shopt -s strict_errexit || true

if true && true; then
echo A
fi

if true || false; then
echo B
fi

if ! false && ! false; then
echo C
fi

## STDOUT:
A
B
C
## END

#### strict_errexit detects proc in && || !
set -o errexit
shopt -s strict_errexit || true

myfunc() {
echo 'failing'
false
echo 'should not get here'
}

if true && ! myfunc; then
echo B
fi

if ! myfunc; then
echo A
fi

## status: 1
## STDOUT:
## END

# POSIX shell behavior:

## OK bash/dash/mksh/ash status: 0
## OK bash/dash/mksh/ash STDOUT:
failing
should not get here
failing
should not get here
## END



#### strict_errexit without errexit proc
myproc() {
echo myproc
Expand Down
4 changes: 2 additions & 2 deletions test/runtime-errors.sh
Original file line number Diff line number Diff line change
Expand Up @@ -387,7 +387,7 @@ done

# OLD WAY OF BLAMING
# Note: most of these don't fail
test-strict_errexit_old() {
test-strict-errexit-old() {
# Test out all the location info

# command.Pipeline.
Expand All @@ -398,7 +398,7 @@ test-strict_errexit_old() {
#_strict-errexit-case 'if ! ls; then echo Pipeline; fi'

# command.AndOr
_strict-errexit-case 'if echo a && echo b; then echo AndOr; fi'
#_strict-errexit-case 'if echo a && echo b; then echo AndOr; fi'

# command.DoGroup
_strict-errexit-case '! for x in a; do echo $x; done'
Expand Down

0 comments on commit 11f8dff

Please sign in to comment.