sr3_tools are a collection of scripts used to manage data pumps (clusters) running Sarracenia v3 (sr3).
To install, clone the repository:
git clone https://github.com/MetPX/sr3_tools.gitThen add the bin directory to your path. Generally this can be done by adding the following line to ~/.bash_profile or ~/.bashrc, where path_to_repo is substituted for the location where the repository was cloned to:
export PATH=path_to_repo/sr3_tools/bin:$PATH
# install completion script
source path_to_repo/sr3_tools/completion/sr3_tools_completion.bash
dsh is also required. On Ubuntu, it can be installed using:
sudo apt install dshsr3_tools works in conjuction with a Git repository that contains the Sarracenia configuration files for one or more data pump clusters.
The layout of the repository should be similar to the following:
config_repo_root
├── _dsh_config
│ ├── pump1.list
│ ├── pump1_ssh_config (optional)
│ └── pump2.list
├── pump1
│ ├── cpost
│ ├── credentials.conf
│ ├── default.conf
│ ├── poll
│ ├── post
│ ├── sarra
│ ├── sender
│ ├── shovel
│ ├── subscribe
│ └── winnow
├── pump2
│ ├── cpost
│ ├── credentials.conf
│ ├── default.conf
│ ├── plugins
│ ├── poll
│ ├── post
│ ├── sarra
│ ├── sender
│ ├── shovel
│ ├── subscribe
│ └── winnow
├── .git
└── .gitignore
These files define the configurations used for dsh and ssh.
-
$pump_name.listis the dsh machine file; a list of hosts to connect to, one per line.Example:
-
$pump_name_ssh_configis an SSH client config file. This is optional. When the file exists, it can be used to specify options for the SSH client. Seeman 5 ssh_configfor possible options. The example below uses a jump server to proxy connections to the nodes.Example:
Host *.example.com ProxyCommand ssh [email protected] -W %h:%p
When setting up each node, the config repository should be cloned somewhere (possibly to ~), then the correct config subdirectory (e.g. pump1) should be symlinked to ~/.config/sr3.
Plugin code can be managed in a separate Git repo from the configuration files. In this scenario, the plugins repo would be cloned to each node separately from the config repo and symlinked to ~/.config/sr3/plugins. sr3_pull will run git pull on the plugins directory if a .git directory exists inside it.
Repeat for each node in the data pump:
cd ~
git clone https://git.example.com/config_repo.git
git clone https://git.example.com/plugins_repo.git
ln -s ~/config_repo/pump1 ~/.config/sr3
ln -s ~/plugins_repo/ ~/.config/sr3/plugins
# Optional - check out a particular branch of the plugins repo
cd ~/.config/sr3/plugins
git checkout devUsage: sr3d [ -h ] (convert|declare|devsnap|dump|edit|log|restart|sanity|setup|show|status|overview|stop)
"sr3 distributed" runs sr3 on each node, with all command line arguments passed to sr3.
See man sr3.
Special Cases:
convert: converts the config from v2 to sr3 on the first node in the cluster, then copies the sr3 config to your workstation and removes the config from the node. The user needs to manually disable the v2 config, commit the sr3 config, and runsr3_pullto update the cluster.remove: runssr3 remove ...on the cluster and deletes the config from your Git repo.
Examples:
sr3d start subscribe/my_configsr3d status poll/my_poll
Usage: sr3_commit <pathspec>... [-m <msg>]
Simplifies the sequence of git pull, git add <pathspec>, git commit [-m <msg], git push origin into one step. Recommended replacement for sr(3)_push.
Examples:
sr3_commit poll/poll_some_source.conf sarra/get_some_source.conf sender/send_to_server.conf -m "Add a new data feed"sr3_commit my_config.conf
Run sr3_pull afterwards to pull the latest configs to the nodes.
Does a git pull on each node in the cluster to update the local configs.
Usage: sr3_push file_name ["Commit message"]
"Pushes" a config file by 1) commiting the file to Git and 2) running sr3_pull to update the configs on each node.
This exists for to provide a familiar workflow for people used to using sr_push, but using Git branches, merging and sr3_pull is preferred. The Git workflow supports changing multiple files at once.
The commit message is optional. If no message is passed on the command line, an editor will open where you can type the commit message.
Examples:
sr3_push new_config.conf "AA - description of change"sr3_push another_config.conf Commit messagesr3_push some_file.conf
Usage: sr3l your_command
"sr3 log" is sr3r, with cd ~/.cache/sr3/log before your command. Used for searching through logs on all nodes, typically in combination with grep or tail.
Try to be as specific as possible when grepping, e.g. search within sender*my_config*.log rather than sender*.log.
Examples:
sr3l grep looking_for_this_filename sender*my_config*.logsr3l tail -n 2 sarra*.log
Usage: sr3r your_command
"sr3 run" executes a shell command on all nodes in the cluster.
Usage: sr3_scp user@server:/remote/file /local/file
Like sr3_ssh, but for scp.
Usage: sr3_ssh user@server
"sr3_ssh" SSHes to a remote server, using the same SSH config file that is used for sr3r. This is essentially a shortcut to SSH to a server using the same proxy config that would be used for dsh with the other commands.
Specific environment variables can be used to set options for sr3_tools.
Passes additional arguments to dsh. See the dsh man page for full details.
Example: SR3TOOLS_DSH_ARGS="--remoteshellopt -vvv"
Used to force certain commands to output in colour. Replaces all instances of $cmd with $cmd --color=always.
Example: export SR3TOOLS_COLOUR_CMDS="grep ls" will always colourize the output of grep and ls.
A Bash completion script is included. This script allows automatic completion of arguments for some of the sr3_tools commands.