source vs. sub-shell and environment variables scope

[MadBomber]

I have not found the secret JUST way of manipulating system environment variables from recipe to recipe. Here are three examples. They are a JUSTFILE, RUBYFILE abd a BASHFILE. Each is invoked the same way within the context of an enclosing BASH shell.

# justfile

@prefix arg="foo":
  export XYZZY={{arg}}:$XYZZY
  echo "in prefix $XYZZY"

@suffix arg="bar":
  export XYZZY=$XYZZY:{{arg}}
  echo "in suffix $XYZZY"

@xyzzy: prefix suffix
  echo "in xyzzy $XYZZY"

# Here is the result
# $ export XYZZY=No-Magic
# $ just No-Magic
# in prefix No-Magic
# in suffix No-Magic
# in xyzzy No-Magic
# $ echo $XYZZY
# No-Magic

The JUSTFILE appears to treat each RECIPE as a sub-shell. No modification via export is available between recipes.

Now lets look as a RUBYFILE where each "recipe" is implemend as a Ruby method.

#!/usr/bin/env ruby

def prefix(arg="foo")
  ENV['XYZZY'] = "#{arg}:#{ENV['XYZZY']}"
  puts "in prefix #{ENV['XYZZY']}"
end

def suffix(arg="bar")
  ENV['XYZZY'] = "#{ENV['XYZZY']}:#{arg}"
  puts "in suffix #{ENV['XYZZY']}"
end

def xyzzy
  prefix  
  suffix  
  puts "in xyzzy #{ENV['XYZZY']}"
end

xyzzy

######################################
## Result

# $ ./rubyfile.rb
# in prefix foo:No-Magic
# in suffix foo:No-Magic:bar
# in xyzzy foo:No-Magic:bar
# $ echo $XYZZY
# No-Magic

Ruby achieves the sharing of system environment variables between its methods my the Ruby process (a sub-shell) ingesting all of the variables into a global Hash called ENV. As a gkibak gasg EBV uf=s available to each Ruby method for "getting" and "setting." BUT since Ruby itself is a sub-shell no modifications to a variable inside of Ruby will be exported to the outside enclosing shell.

Now lets take a look as a BASHFILE where each "recipe" is implemented as a Bash function.

#!/usr/bin/env bash

function prefix () {
  export XYZZY="foo:$XYZZY"
  echo "in prefix $XYZZY"
}

function suffix () {
  export XYZZY="${XYZZY}:bar"
  echo "in suffix $XYZZY"
}

function xyzzy () {
  prefix
  suffix
  echo "in xyzzy $XYZZY"
}

xyzzy

####################################
## Result ...

# Executing as a sub-shell

# $ ./bashfile.sh
# in prefix foo:No-Magic
# in suffix foo:No-Magic:bar
# in xyzzy foo:No-Magic:bar
# $ echo $XYZZY
# No-Magic

# Sourcing within the context of the outter shell

# $ source bashfile.sh
# in prefix foo:No-Magic
# in suffix foo:No-Magic:bar
# in xyzzy foo:No-Magic:bar
# $ echo $XYZZY
# foo:No-Magic:bar

When the BASHFILE is executed it behaves like Ruby as a sub-shell where no modifications to a system environment variable are passed to the outside enclosing shell. However, when the BASHFILE is "sourced" it becomes part of the outter shell in context. This means that modifications to the variables are not only possible between the functions whether executed or sourced; but when sourced the modified variables are carried into the outter shell.

What am I trying to say? If a JUST recipe without a SHEBANG is treated as a shell function or simply a shell file that is "sourced" instead of executed, then the recipes can share access to modified system environment variables - maybe even the local attributes defined with the ':=' operator.

I would never expect a JUSTFILE to be able to pass its modified variables into its outter shell expect by way of the file system or pipes.

[casey]

It isn't currently possible to share variables between recipes. Just kind of has a functional, no-mutation model with variables, so it doesn't support things like modifying existing variables.

[MadBomber]

Granted that JUST has an architecture that does not support this kind of activity. Rare as it might be there are still times in which information from one recipe needs to be communicated to another recipe. If the control flow is good then this can be accomplished by passing arguments between the recipes. However with a control flow like this:

# justfile
... <snip> ...
last: one two three

Where one produces something that two and three need, etc. parameters passing does not seem to apply.

In my examples of the RUBYFILE and BASHFILE, they each had a common context - all Ruby or all Bash. For JUST we do not have that common context because each receipe is invoked within its own sub-shell and can be any script available via a shebang.

So what is the common context outside of JUST which might be used? file system. database. key-value store. Each should have a common mechanism by which information can be set and retrieved.

I've been experimenting with the key-value context. Here is what I came up with ...

# justfile_three

set shell := ["bash", "-uc"]

# Basg recipe
prefix arg="foo":
  #!/usr/bin/env bash
  xyzzy=`skate get 'xyzzy'`
  skate set "xyzzy" "{{arg}}:$xyzzy"
  echo "in prefix `skate get 'xyzzy'`"

# Ruby recipe
suffix arg="bar":
  #!/usr/bin/env ruby
  xyzzy = `skate get 'xyzzy'`
  system %Q[skate set 'xyzzy' "#{xyzzy}:{{arg}}"]
  puts "in suffix #{`skate get 'xyzzy'`}"

xyzzy: prefix suffix
  #!/usr/bin/env bash
  echo "in xyzzy `skate get 'xyzzy'`"



######################################
# Result

# $ skate set 'xyzzy' 'Magic'
# $ just -f justfile_three xyzzy
# in prefix foo:Magic
# in suffix foo:Magic:bar
# in xyzzy foo:Magic:bar
# $ skate get 'xyzzy'
# foo:Magic:bar

skate is a key-value system that I recent found. I'm testing it out but so far its CLI based getter/setter times are pretty slow. I will most likely end up using redis because I use it for everything else.

https://github.com/charmbracelet/skate

granted using a key-value store is a little heavy in the JUST environment AND I can not think of a way to easily integrate such a capability into JUST itself... but in those rare occasions where information passing between recipes is required and there is no inherent way to do it, this appears to be a viable solution.

[MadBomber]

replacing "skate" with "redis-cli" is much faster. My redis-server is running on localhost whereas the skate server is running out there in the cloud somewhere. Skate's server can run locally I just have not tried that configuration yet.

[MostHated]

I am a bit late to the party, but I use this for my day-to-day shell scripts as it is extremely fast, and I have the .kv 'datastore' write to a small 2mb ramdisk for things like Conky, eww, or other things that update often.

https://github.com/damphat/kv-bash