on_call_kit.md

On-call Kit Setup

General setup for your laptop and related equipment.

It's helpful to have a checklist that you run through prior to going on-call to make sure everthing is in order.

1. Be prepared to take your laptop with you wherever you go. Invest in a comfortable bag with ergonomic straps. If you drive most places consider getting a laptop sleeve and put your laptop in the sleeve in the map pocket behind your seat.
2. Bring a laptop and phone charger with you. Also consider purchasing a USB battery pack. They are readily available these days, and fairly inexpensive. Being able to charge your phone while on the go instead of having to be tethered to a wall socket is useful. Make sure to add an item to your checklist to charge this pack prior to going on-call. Keep a charger handy as well.
3. If you have a mifi (recommended), set it up on your laptop before you actually need to use it. If you are in the US and need to pay for your own, check out Karma. Pay-as-you-go is more convenient than monthly fees.
4. Make sure you have access to all the necessary VPNs, password trackers, and related systems. If you don't have a work-issued laptop and you don't want work-related things on your personal machine, set up a machine you can remote into. Ensure that it doesn't go to sleep if it is a Windows machine subject to corporate policy.
5. Install f.lux. It's worth it for those 3 am pages.
6. When you're home for the evening, if at all possible, go ahead and have your laptop set up and ready to go. That way you're not fumbling to get it out of your bag and plugged in, etc. at 3 A.M.
7. Consider making bookmarks for everything. Save that group of browser tabs with the recipes for restarting things and monitoring tools ordered the way you like it. If there's a set of boxes you like to connect to, save their ssh connection (instructions for macOS: part 1 + part 2, and for Ubuntu) info in your terminal and have everything auto-connect on launch -- 3 A.M. when everything's beeping at you isn't the time to test your memory.
8. A hands-free kit for your phone, whether this be the headphones that came with it or a bluetooth earpiece. Make sure you've tested it with your phone in advance of going on-call. It's good to keep your hands available in case you need to call up a colleague or join a conference call.
9. A smart watch (pebble, Apple watch, etc.) can be helpful for late night alerts. The vibration of a smart watch will wake only you up where an audible alert from your phone might also wake up your partner as well. Have your charger handy just in case.
10. A small zippable bag to keep all of your on-call kit in (other than your laptop). Your items, such as chargers, should never leave this bag unless you are using them on call. That way you never have to scramble to be ready for on-call.
11. Consider a 3rd party VPN as well as your company one. Hotels without good MiFi reception and restrictive WiFi policies can be worked round if you can VPN out on a common web port with a 3rd party, then corporate VPN inside that over regular ports.

Datacenter kit

If you might have to go into a datacenter or server room while on call, it's helpful to have some gear prepped for anything you might have to do in that room. This might go into your general on-call kit, or you may want to have a separate bag prepped and leave it at the datacenter so you don't have to carry it around.

Ideally all of these items will be already present in a datacenter cabinet, as well as other tools and spare hardware, but it's helpful to have your own set of common tools if this is a common on-call scenario for you.

1. Your preferred hearing protection. This could be a good pair of noise-canceling headphones or dedicated earmuffs, but should probably be over-the-ear muffs. Your employer may buy these for you if you work in a datacenter on a regular basis.
2. A couple of sets of disposable earplugs... for when you (or a colleague) forget your nicer earmuffs.
3. A paper notebook and pencil or pen that you've verified will work. (Trying to take notes on a laptop or phone while in a server room can sometimes be challenging!)
4. Half a dozen velcro cable ties.
5. A multi-tool like a Leatherman. Make sure it's one that includes wirecutters or scissors for taking care of non-reusable cable ties.
6. A small set of jewelers' screwdrivers (optional, but often comes in handy).
7. A decent-length ethernet cable (10+ ft), and any dongles you might need for your laptop if you need to plug into a switch for network debugging. This may include, but not limited to - usb to serial, usb extender cable, power schuko adapters
8. A small flashlight or (better!) a headlamp, so you can see into dim spaces in racks.
9. A printed or handwritten contact card, with phone numbers for your escalation path.
10. Wear proper dress/shoes for datacenter - there should be cold, so pants and sweatshirt with hoodie is always a must. No flops! This can be helpful if the room has poor cell reception but you do have access to a landline. Also make sure to get the number of the onsite security guards. After some time they'll want to check if you're still alive, or you might need their help (Access to circuit breakers, router rooms. in a larger DC you might also simply lose your way)