-
Notifications
You must be signed in to change notification settings - Fork 28
/
README
182 lines (147 loc) · 6.41 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
NAME
Pogo - Run commands on many hosts in a controlled manner
DESCRIPTION
Pogo is a highly scalable system for running arbitrary commands on many
hosts in a controlled manner.
It is mostly used for quick mass software deployments on server farms
while making sure only an allowed number of nodes are upgraded in
parallel to ensure business continuity.
Project Setup
Pogo is hosted on Github at
https://github.com/ytoolshed/pogo
The latest stable version can be found on the master branch. The project
is automatically being tested on every commit, using travis-ci's
service:
http://travis-ci.org/#!/ytoolshed/pogo
Architecture
Pogo consists of several components, which can be all running on the
same system, or, in order to scale it, be replicated and even be
installed on many distributed hosts. Those components are
Client
Users submit jobs to pogo using the client, which in turn contacts
the API.
API Takes requests via HTTP from the client and forwards them to a
dispatcher.
Dispatcher
Takes job requests from the API, figures out constraints, and
determines single tasks the job consists of. It then assigns tasks
to workers, watches their individual completion and keeps track of
overall job completion. Dispatchers can be queried by the API to
determine the status of a given job.
Worker
Takes a task (like "ssh to a host and run this command") from the
dispatcher, executes it and reports back the result. Can handle many
tasks concurrently.
Security
To make sure dispatchers and workers communicate over secure channels,
and enable them to authenticate each other (is a connecting worker
really an authorized worker, or is the dispatcher it's connecting to
really an authorized dispatcher?), Pogo uses SSL server and client
certs. See Pogo::Security for details.
NAME
pogo-one - All-In-One Pogo
SYNOPSIS
$ cat t/cfgs/one.cfg
tag:
# no tags
sequence:
- host3
- host2
- host1
# specify two hosts 1 and 2, but config says to run them in
# reverse order
$ perl -Ilib bin/pogo-one --password --targets host1,host2 \
--config t/cfgs/one.cfg --run-command 'date'
Password: ****
host2: Running target
host2: Success
host1: Running target
host1: Success
DESCRIPTION
"pogo-one" is a simple utility that loads all the usually distributed
Pogo components into one single process and runs it from the command
line. In this way, you can test Pogo functions like constraints and
sequences without having to set up the different components of an entire
production system. It is mainly used for testing, but also helps with
small deployment jobs.
PASSWORDS
By default, "pogo-one" assumes that you have access to the target boxes
without having to type your password or your ssh keyring's passphrase.
To have "pogo-one" collect the password and use it to sign in on the
target, use the "--password" options
$ pogo-one --password ...
Password: ****
...
OPTIONS
"--config file"
Location of configuration file that defineds Pogo tags, sequences,
and constraints.
"--password"
If specified, c<pogo-one> will collect the user's target password at
a command line prompt, and use it to log into the target machines.
"--targets"
All hosts to run the command on. Order and parallelism is determined
by the configuration file.
"--run-command"
The command to run on all hosts.
NAME
pogo-schedule - Print out the schedule for a job
SYNOPSIS
$ cat t/cfgs/triple.cfg
# sample configuration file
tag:
colo:
one:
- host1
- host2
- host3
two:
- host4
- host5
- host6
three:
- host7
- host8
- host9
sequence:
foo:
- $colo.one
- $colo.two
bar:
- $colo.three
$ perl -Ilib bin/pogo-schedule t/cfgs/triple.cfg
.--------------------------------------------------------------.
| Schedule |
+------------------+---------------------+---------------------+
| Thread | slot-1 | slot-2 |
+------------------+---------------------+---------------------+
| thread-000000000 | [foo.$colo.one] | [foo.$colo.two] |
| | host3, host2, host1 | host6, host5, host4 |
| thread-000000001 | [bar.$colo.three] | |
| | host9, host8, host7 | |
| unconstrained | [unconstrained] | |
'------------------+---------------------+---------------------'
DESCRIPTION
This utility pretty-prints a schedule for a job, as it would be executed
based on a Pogo configuration file. It uses the actual Pogo scheduler in
Pogo::Scheduler::Classic and asks it for an execution plan just as if we
submitted a Pogo job for all hosts present in the configuration.
LICENSE
Copyright (c) 2010-2012 Yahoo! Inc. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may
not use this file except in compliance with the License. You may obtain
a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
imitations under the License.
AUTHORS
Mike Schilli <[email protected]> Ian Bettinger <[email protected]>
Many thanks to the following folks for implementing the original version
of Pogo:
Andrew Sloane <[email protected]>, Michael Fischer
<[email protected]>, Nicholas Harteau <[email protected]>, Nick Purvis
<[email protected]>, Robert Phan <[email protected]>, Srini Singanallur
<[email protected]>, Yogesh Natarajan <[email protected]>