Skip to content

Latest commit

 

History

History
2447 lines (1988 loc) · 65.4 KB

Apps.page

File metadata and controls

2447 lines (1988 loc) · 65.4 KB
Raw

Comparisons

Media Centers

  • xbmc
    • cross platform
    • FLOSS
    • for HTPCs; plays discs, supports remotes
    • dlna often doesn't work
  • mythtv
    • cross platform
    • FLOSS
    • general streaming
    • dvr
    • dlna
  • plex
    • cross platform
    • more of a server that streams to clients
    • $5 client apps
    • forked from xbmc

Backup Systems

  • duplicity: compressions, rdiff, encryption; posix only
  • bup: smartest dedup, compression, rdiff; posix only, no encryption
    • even has ftp, http, fuse view over old data
    • no way to prune old data?
  • obnam: compression, rdiff, encryption, block dedup; slow, posix only
  • duplicati: win-friendly duplicity, easy to use; mono winforms on os x is unusable (web UI is WIP)
    • can browse files

Opinions

Excel

  • ^d on cells
  • tables
  • RC mode
  • indexing into named ranges
  • index(values, match(keys, keys)) (but use strict match)
  • goal seek

Emacs

Packages

  • ProjectSettings

Functions

  • General

    (standard-display-ascii ?\t "^I") (desktop-save-mode 1) make-auto-save-file-name auto-save-file-name-p make-backup-file-name-function point (goto-char N) point-min point-max (message STR)

  • Locating the cursor: point, point-min, point-max, bobp, eobp, bolp, eolp, current-column

  • Moving around in text: goto-char, all your favorite keyboard commands, save-excursion

  • Reading text: char-after, char-before, buffer-substring, thing-at-point

  • Searching: search-forward, re-search-forward

  • Modifying text: insert, insert-buffer, newline, delete-region

Unknown modes

transient-mark-mode hi-lock-mode column-number-mode line-number-mode

Major modes

fundamental text outline vc man shell

Minor modes

midnight flyspell flymake: syntax errors ffap auto-fill indent-tabs = noexpandtab hi-lock: highlight words set by (eg) highlight-regexp overwrite

Vars

tab-width = tabstop c-basic-offset = shiftwidth truncate-lines = wrap truncate-partial-width-windows indent-tabs-mode: expandtabs

sort-fold-case: ignore case when sorting

auto-save-file-name-transforms make-backup-files backup-directory-alist: eg ("." . ".~") version-control :: (t nil never) kept-old-versions :: int kept-new-versions :: int delete-old-versions :: (t nil never)

Commands

MACROS c-x ( start-kbd-macro c-x ) end-kbd-macro c-x e call-last-kbd-macro name-last-kbd-macro insert-kbd-macro SEARCH/REPLACE m-% query-replace query-replace-regexp grep c-x next-error m-g n next-error m-g m-n next-error occur SEARCH INTERACTION ret c-w word search c-w search for word under cursor c-y search for line under cursor m-r toggle regex m-c toggle case sensitivity m-e edit search string in minibuffer m-n search for next item in search ring m-p search for prev item in search ring m-tab complete search string using search ring c-m-i .. DISPLAY toggle-truncate-lines SPELLCHECK m-$ ispell-word ispell-region ispell-buffer EDIT c-q quoted-insert (e.g. to insert tab, use c-q tab) c-o open-line c-m-o split-line m-m back-to-indentation (jump to first non-whitespace char) c-x c-o delete-blank-lines (leaving just one) m-^ delete-indentation (join line with prev) m-spc just-one-space c-t transpose-chars m-t transpose-words c-m-t transpose-sexps c-x c-t transpose-lines transpose-sentences transpose-paragraphs m-c capitalize-word m-u upcase-word m-l downcase-word c-x c-u upcase-region c-x c-l downcase-region m-- m-c capitalize previous word m-- m-u upcase previous word m-- m-l downcase previous word center-line MARK c-@ set-mark-command m-@ mark-word m-h mark-paragraph c-m-@ mark-sexp c-m-h mark-defun c-x c-p mark-page c-x h mark-whole-buffer REGION c-x c-x exchange-point-and-mark c-x n n narrow-to-region c-x n w widen RING m-y yank-pop c-m-w append-next-kill NAV m-r move-to-window-line m-0 m-r move to top of screen m-- m-r move to bottom of screen c-m-b backward-sexp c-m-f forward-sexp c-m-u backward-up-list c-m-d down-list c-m-a backward-defun c-m-e forward-defun m-{ backward-paragraph m-} forward-paragraph m-a backward-sentence m-e forward-sentence c-x [ backward-page c-x ] forward-page C/CC MODES c-beginning-of-defun c-end-of-defun c-c c-u (up preproc conditional; leaves mark) c-c c-p (prev preproc conditional; leaves mark) c-c c-n (next preproc conditional; leaves mark) m-a c-beginning-of-statement m-e c-end-of-statement c-backward-into-nomenclature c-forward-into-nomenclature VCS vc‑resolve‑conflicts PY MODE c-c c-u py-goto-block-up (leaves mark) c-c c-p py-previous-statement (leaves mark) c-c c-n py-next-statement (leaves mark) COMPLETION/INTELLISENSE m-/ dabbrev-expand c-tab complete-symbol c-m-i .. m-. find-tag m-0 m-. go to next matching function definition m-* pop-tag-mark tags-search (looks for any occurrence of a symbol) m-, tags-loop-continue (next tags-search match) visit-tags-table tags-query-replace KILL m-z zap-to-char c-u -1 m-k backward kill-sentence m-k kill-sentence c-x del backward kill-sentence m-0 c-k backward kill-line forward-kill-paragraph backward-kill-paragraph c-c c-bs c-hungry-backspace c-c bs c-hungry-backspace c-c c-d c-hungry-delete-forward c-c c-del c-hungry-delete-forward c-c del c-hungry-delete-forward c-toggle-hungry-state m-\ delete-horizontal-space c-m-k kill-sexp m-- c-m-k backward kill-sexp SORT sort-lines sort-paragraphs sort-pages sort-fields sort-numeric-fields sort-columns reverse-region SESSIONS desktop-save desktop-change-dir desktop-revert desktop-clear ELISP find-function c-x c-e eval-last-sexp c-m-x eval-defun m-: eval-expression eval-region load-library MISC customize m-g ... set-face-... c-h w where-is c-/ undo m-0 digit-argument m-- negative-argument c-x c-q toggle readonly check-parens global-set-key load-file load-library c-x f set-fill-column c-x . set-fill-prefix c-u m-x align c-x z repeat list-command-history FILES/BUFFERS ffap-bindings c-x c-v find-alternate-file revert-buffer c-x c-w write-file (save as) c-u c-x s save all without prompting c-x k kill-buffer c-x i insert-file c-x l count-lines-page recover-file c-x b switch-to-buffer c-x c-b list-buffers list-matching-lines (:v/PAT/d) c-x # server-edit (in server, save/close and end the blocking client) WINDOWS c-x { shrink-window-horizontally c-x } enlarge-window-horizontally c-x ^ enlarge-window c-x 0 delete-window c-x 1 delete-other-windows c-x 2 split-window-vertically c-x 3 split-window-horizontally c-x o other-window c-m-v scroll-other-window c-x 4 0 kill-buffer-and-window c-x 4 b switch-to-buffer-other-window c-x 4 c-o display-buffer c-x 4 f find-file-other-window c-x 4 r find-file-read-only-other-window c-x 4 d dired-other-window c-x 4 . find-tag-other-window FRAMES c-x 5 0 delete-frame c-x 5 o other-frame c-x 5 2 make-frame-command c-x 5 b switch-to-buffer-other-frame c-x 5 f find-file-other-frame c-x 5 r find-file-read-only-other-frame c-x 5 d dired-other-frame c-x 5 . find-tag-other-frame SHELL m-! shell-command m-| shell-command-on-region c-u m-| filter region through shell command shell RECTANGLES c-x r r copy-rectangle-to-register c-x r k kill-rectangle c-x r y yank-rectangle c-x r o open-rectangle (shifts text) c-x r c clear-rectangle c-x r t string-rectangle ABBREVIATIONS c-x a g add-global-abbrev c-x a l add-mode-abbrev c-x a i g inverse-add-global-abbrev c-x a i l inverse-add-mode-abbrev c-x a e expand-abbrev I18N set-language-environment list-input-methods c-\ toggle-input-method c-x ret c universal-coding-system-argument list-coding-systems prefer-coding-system BUILD compile c-x next-error gdb VARS set-variable describe-variable FILL m-q fill-paragraph fill-region INDENT m-i tab-to-tab-stop indent-relative (indent from point to under an indentation point in the previous line) m-j c-indent-new-comment-line c-j newline-and-indent c-m-q indent-pp-sexp c-m-\ indent-region c-x tab indent-rigidly (shift block of lines rigidly right or left) c-u tab shift sexp rigidly so that first line is properly indented indent-code-rigidly (doesn't indent lines within comments and strings) c-c c-q c-indent-defun c-m-q c-indent-exp c-c . ret c-set-style c-c c-l c-toggle-electric-state (indent on special chars like { and :) c-c c-a c-toggle-auto-newline (lets electric chars trigger newline insertions) tabify untabify align-regexp HIGHLIGHT c-x w h highlight-regexp MACROS c-x ( start-kbd-macros c-x ) start-kbd-macros name-last-kbd-macro c-x e call-last-kbd-macro insert-kbd-macro c-u F3 (re-execute last macro, then append keys to it) c-x c-k c-e kmacro-edit-macro-repeat c-x c-k l kmacro-edit-lossage AWESOME artist-mode spook CUSTOM? c-K kill to end of buffer c-; scroll one line up c-' scroll one line down c-, scroll half page up c-. scroll half page down c-x g goto-line shrink-window-vertically enlarge-window-vertically HELP c-c c-b help-go-back c-h i info c-h c-i info-lookup-symbol c-h a apropos apropos-variable REGISTERS c-x r s copy-to-register c-x r i insert-register c-x r spc point-to-register c-x r j jump-to-register RECURSIVE EDITING MODES c-r enter recursive edit c-m-c exit recursive edit c-] abort: exit all nested recursive-edit sessions

Bash

  • $'hello\nworld' actually interprets the backslashed chars

Command-Line Cheatsheet

Windows

uptime + other stats

net statistics server

Linux

expose a dir somewhere else (bind mount) but w diff perms

sudo bindfs --perms=a=rX /media/yang/shared/ /mnt/shared/

force ddclient to update

ddclient -force -daemon=0

show live transfer counters for interface eth0

vnstat -l 1 -i eth0

displays information about resulting broadcast network, very very useful!

ipcalc 192.168.0.1/32

Disable alt-click in Gnome

gconftool-2 --set /apps/metacity/general/mouse_button_modifier --type string disabled

Enable alt-click in Gnome

gconftool-2 --set /apps/metacity/general/mouse_button_modifier --type string ''

Disable screen lock after suspend (in Ubuntu 11.10)

gsettings set org.gnome.desktop.lockdown disable-lock-screen 'true'

monitor CPU hogs

top -b -d 60 -c > top-cpu-processes &

monitor CPU hogs by threads

top -b -d 60 -c -H > top-cpu-threads &

monitor mem hogs

while true; do date; ps auxk-rss; sleep 60; done > top-mem-processes &

mount encrypted filesystem

sudo mount -t ecryptfs ~/data ~/data -o ecryptfs_cipher=aes,ecryptfs_key_bytes=32,ecryptfs_passthrough=n,ecryptfs_enable_filename_crypto=n

to set up a repository for multiuser access:

sudo mkdir -p /var/git/proj.git/ cd !$ sudo chown yang . git init --bare sudo chmod -R 777 . echo ' sharedRepository = 0666' >> config

now add a user (the Debian way)

sudo adduser --shell /usr/bin/git-shell joe

tweak kernel settings

sysctl kernel.shmmax # show sysctl -w kernel.shmmax=$((8<<30)) # update cat /proc/sys/kernel/shmmax # raw vim /etc/sysctl.conf # permanent changes

extract URLs from squid disk cache

grep --only-matching --no-filename --max-count 1 --text --perl-regexp 'http://[^\0]' /var/spool/squid3///

grep -ahoPm1 'http://[^\0]' /var/spool/squid3///

add/remove latency to localhost

tc qdisc add dev lo root handle 1:0 netem delay 100msec tc qdisc del dev lo root

allow X across sudo

xauth list # find the triple you want xauth add partycat/unix:10 MIT-MAGIC-COOKIE-1 73b185cea93847a27ea2d275e1e2ebf0

or: xauth add $(xauth list)

generate password

< /dev/urandom tr -cd [[:graph:]] | head -c16 | xargs echo

inspect postfix mail queues

mailq

retry queued mail

sendmail -q

set pid 123 to cpus 4-6

taskset -c 4,5,6 123

another way

cpuset ...

allow kerel to offload large packet segmentation to the network adapter

sudo ethtool -K eth1 tso on

check with

$ dmesg|tail -1 [892528.450378]0000:04:00.1:eth1:TSO is Enabled

give somewhat limited sudo

Cmnd_Alias INSTALL = aptitude apt-get yang ALL = INSTALL

trace system calls, printing up to 1024 chars; useful for finding eg what SQL

is being executed

strace -tt -s 1024

nicely align mount output

mount | column -t

repeat prev command replacing every "foo" with "bar"

!!:gs/foor/bar

watch for changes in output of df/ls commands

watch -d -n 1 'df; ls -FlAt /path'

show size of all immediate subdirectories

du -h --max-depth=1

get tlb info

x86info -c

base64 encode/decode stdin

base64 [-d]

encrypt a file or stdin to a file or stdout [-a for ascii]

openssl aes-256-cbc [-a] -salt [-in secrets.txt -out secrets.txt.enc]

decrypt a file or stdin to a file or stdout

openssl aes-256-cbc -d [-a] [-in secrets.txt.enc -out secrets.txt.new]

allow follow symlinks

$ cat > /etc/apache2/conf.d/yang <Directory /home/yang/public_html/> Options +ExecCGI +FollowSymLinks -SymLinksIfOwnerMatch AddHandler cgi-script cgi pl py bash

Show Long Format Output

ps -Al

To turn on extra full mode (it will show command line arguments passed to process): ps -AlF

To See Threads ( LWP and NLWP)

ps -AlFH

To See Threads After Processes

ps -AlLm

Print All Process On The Server

ps ax
ps axu

Print A Process Tree

ps -ejH
ps axjf
pstree

Print Security Information

ps -eo euser,ruser,suser,fuser,f,comm,label
ps axZ
ps -eM

See Every Process Running As User Vivek

ps -U vivek -u vivek u

Set Output In a User-Defined Format

ps -eo pid,tid,class,rtprio,ni,pri,psr,pcpu,stat,wchan:14,comm
ps axo stat,euid,ruid,tty,tpgid,sess,pgrp,ppid,pid,pcpu,comm
ps -eopid,tt,user,fname,tmout,f,wchan

Display Only The Process IDs of Lighttpd

ps -C lighttpd -o pid=

OR pgrep lighttpd

OR pgrep -u vivek php-cgi

Display The Name of PID 55977

ps -p 55977 -o comm=

Find Out The Top 10 Memory Consuming Process

ps -auxf | sort -nr -k 4 | head -10

Find Out top 10 CPU Consuming Process

ps -auxf | sort -nr -k 3 | head -10

check whether HD supports SMART

smartctl -i /dev/sda

enable SMART

smartctl -s on /dev/sda

estimate testing time

smartctl -c /dev/sda

extended test

smartctl -t long /dev/sda

view test stats

smartctl -l selftest /dev/sda

view detailed SMART info for IDE

sudo smartctl -a /dev/hda

view detailed SMART info for SATA

sudo smartctl -a -d ata /dev/sda

generate hashes for a word list

./genkeys -r wordlist -f words.hashes -n words.index

spoof MAC address on Mac/OS X

from http://osxdaily.com/2012/03/01/change-mac-address-os-x/

openssl rand -hex 6 | sed 's/(..)/\1:/g; s/.$//' sudo ifconfig en0 ether xx:xx:xx:xx:xx:xx

hijack HTTP/HTTPS traffic

http://blog.ksplice.com/2010/09/hijacking-http-traffic-on-your-home-subnet-using-arp-and-iptables/

scan network

nmap -sC -O 10.10.0.0/24

enable IP forwarding

cat /proc/sys/net/ipv4/ip_forward echo 1 > /proc/sys/net/ipv4/ip_forward

if i'm forwarding packets, redirect packets to go to port 8080 (if hacking SSL)

iptables -t nat -A PREROUTING -p tcp --destination-port 80 -j REDIRECT --to-port 8080

or redirect to yourself if you want to serve your own content

iptables -t nat -A PREROUTING -p tcp --destination-port 80 -j NETMAP --to 192.168.1.200 ip addr add 192.168.1.1/24 dev eth0 ip addr

ARP cache poisoning: situate ourself as the MITM between target .159 router .1

arpspoof -i eth0 -t 10.10.0.159 10.10.0.1

or use arping

arping -c 3 -A -I eth0 192.168.1.1

strip requests of SSL, and listen on port 8080

sslstrip.sh -l 8080

end hijack

play media at 1.65X speedup

mplayer -speed 1.65 file.avi

avisynth script for 1.65X speedup and 75 pitch

file = "file_name_of_video.avi"
speedup = 1.65
pitch = 75

DirectShowSource(file)

audio_rate = last.audiorate
video_rate = last.framerate

AssumeSampleRate(int(audio_rate*speedup))
AssumeFPS(video_rate*speedup)
TimeStretch(pitch = pitch)

top for files

watch -d -n 2 'df; ls -FlAt;'

replace in prev command

^foo^bar ^foo # delete

Lists all listening ports together with the PID of the associated process

netstat -tlnp

capture screencast

http://ubuntuforums.org/showthread.php?t=1392026

ffmpeg -f x11grab -s wxga -r 25 -i :0.0 -sameq /tmp/out.mpg ffmpeg -i output.mkv -acodec libvorbis -ab 128k -ac 2 -vcodec libtheora -b 1000k our-final-product.ogg

empty a file

file.txt

bring up cmd line editor (in readline)

c-x c-e

run cmd at midnight

echo cmd | at midnight

output microphone to another speaker

dd if=/dev/dsp | ssh host dd of=/dev/dsp

mount temp ram partition (fixed size, can swap)

mount -t tmpfs -o size=1024m tmpfs /mnt/tmp

mount temp ram partition (dynamic size, no swap)

mount -t ramfs -o size=1024m,maxsize=2G ramfs /mnt/ram

resize a tmpfs/ramfs

mount -o remount,size=2G /mnt/tmp

remount a partition for read-write/read-only

mount -o remount,rw mount -o remount,ro

find tips

exact

find . -perm 644

at least any of these bits are set

find . -perm /644

at least all of these bits are set

find . -perm -644

writable by someone (any of: owner, group, or anyone)

find . -perm /222 find . -perm /u+w,g+w,a+w find . -perm /u=w,g=w,a=w

writable by any of: owner, group

find . -perm /220 find . -perm /u+w,g+w find . -perm /u=w,g=w

writable by both owner and group

find . -perm -220 find . -perm -u+w,g+w find . -perm -u=w,g=w

not writable by owner

find . ! -perm /u=w find . ! -perm -u=w

disable a user account

better than passwd USER -l, which just makes no password possible

usermod --expiredate 1 dsontag

record a stream (eg asx)

use VLC, or try (i get core dumps from this):

mplayer -dumpstream -dumpfile OUTFILE URL

output 3-way merge

diff3 -m a_mine.c a_old.c a_theirs.c

resumable scp: http://www.commandlinefu.com/commands/view/1818/resume-scp-of-a-big-file

rsync --partial --progress --rsh=ssh $file_source $user@$host:$destination_file

mirror from local to remote (--dry-run, --include/exclude)

rsync -avzr --progress --delete my-www/ yz.mit.edu:/var/www/

mirror with full relative paths

rsync -vv -rlpR cabal-dev/bin/gitit cabal-dev/share/ /tmp/dest/

mirror filter rules example

rsync -vv -rlp --include /share/ --include '/share/**' --include /bin/ --include /bin/gitit --exclude * ./ /tmp/dest/

mirror only .py files (but may create extra dirs)

rsync -r --include '/' --include '.py' --exclude '*' SRC DST

mirror only .py files (with minimal new dirs)

rsync -r --files-from <( cd SRC; find . -name '*.py' ) SRC DST

mirror without permission/ownership and excluding vim files

rsync -nrltv --delete --exclude '.*.swp' . HOST:/PATH

compare two dirs (must include the trailing /!)

rsync -rvnc --delete LOCALPATH/ HOST:/PATH/

add new PPA repo

sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com
67265eb522bdd6b1c69e66ed7fb8bee0a1f196a8 echo deb http://ppa.launchpad.net/pidgin-developers/ppa/ubuntu
lsb_release --short --codename main |
sudo tee /etc/apt/sources.list.d/pidgin-ppa.list

skip tests when building a maven package

mvn -Dmaven.test.skip=true package

start a jvm to allow for (remote) attachment

java -agentlib:jdwp=transport=dt_socket,address=8765,server=y,suspend=y -jar target/server-1.0-SNAPSHOT.jar server.conf 2>&1 | tee ~/server-out3 Listening for transport dt_socket at address: 8765

compress with different compression level

GZIP=-1 tar czf ... GZIP=-9 tar czf ...

encrypt stdin with a passphrase (-a for ascii, -o for outfile)

gpg -aco OUTFILE gpg --armor --symmetric --output OUTFILE

backup/export secret/private and public keypair

gpg [-a] [-o OUTFILE] --export-secret-keys [email protected]

set x keyboard map

setxkbmap dvorak setxkbmap us

gdb c++ template names; from

http://tdistler.com/2008/11/13/debugging-c-templates-brekpoints-and-gdb

objdump -t libMyLib.so | c++filt | grep ‘BarAbstract.*Baz’

use perl as a sed-like tool

perl -pe 's///' perl -pie 's///' FILE

clear swap; this will block until there actually is enough main memory for

the swap to be cleared

sudo swapoff -a && sudo swapon -a

reset file buffer caches

sudo sync # flush dirty (clean all buffers) sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches' # drop pagecache + dentries/inodes

sudo sh -c 'echo 1 > /proc/sys/vm/drop_caches' # drop pagecache

sudo sh -c 'echo 2 > /proc/sys/vm/drop_caches' # drop dentries/inodes

max LZMA2 compression with 7z with multi-threading; must limit threads or

memory consumption will explode

7z a -mx=9 -m0=lzma2 -mmt=4 FILE.7z FILES/DIRS

convert wma to wav

mplayer -ao pcm:file=OUTPUT.wav INPUT mplayer -vc dummy -vo null -ao pcm:waveheader:file=OUTPUT.wav INPUT.wma mplayer -vo null -vc dummy -ao pcm:file=OUTPUT.wav INPUT.wma mplayer -vo null -vc dummy -af resample=44100 -ao pcm:waveheader:file=OUTPUT.wav INPUT.wma

convert flv to mp3 / extract audio from video

from http://ubuntuforums.org/archive/index.php/t-327331.html

ffmpeg -i in.flv -acodec copy out.mp3

ffmpeg -i in.flv -f mp3 -vn -acodec copy out.mp3

disable pc speakers/system beep

sudo rmmod pcskrs

permanently: edit /etc/modprobe.d/blacklist and add "blacklist pcskprs"

debugging wifi

iwlist wlan0 scan iwlist wlan0 chan iwlist wlan0 mode ifconfig wlan0 down iwconfig wlan0 essid barnacle # typical case iwconfig wlan0 ap 06:18:0a:30:00:88 # force associate with ap iwconfig wlan0 mode ad-hoc essid barnacle iwconfig wlan0 channel 56 ifconfig wlan0 up

force destroy an ap

wlanconfig wlan0 destroy

configure optional private params of a wireless network

iwpriv wlan0

cvs repository check out

CVS_RSH=ssh cvs -d :ext:[email protected]:/var/cvs co biblio

collect awstats from proftpd logs

sudo perl /usr/share/awstats/wwwroot/cgi-bin/awstats.pl -config=ftp -update

monitor proftpd activity

ftpwho ftptop

find out your Ethernet connection: 10? 100? GigE?

lshw

capture mysql queries

tshark -i 1 -T text -V -f ‘dst port 3306′ |grep -i ’statement: ‘ | cut -b20-

find out what open() syscalls are being made

strace -eopen -f

to find out if you just plugged into a usb2 port

tail -25 /var/log/messages

you should see a line saying "new usb device..." or some such. Look for

"ehci", that's the USB 2.0 driver. ohci & uhci are USB 1.1 drivers.

reformat a partition

sudo mkfs -t ext3 /dev/sdb1

from http://www.ghacks.net/2010/08/11/convert-ext23-to-ext4/

change ext2 to ext4

tune2fs -O extents,uninit_bg,dir_index,has_journal /dev/sda1

change ext3 to ext4

tune2fs -O extents,uninit_bg,dir_index /dev/sda1

check and repair an ext fs

e2fsck -pf /dev/sda1

open/close/eject CD/DVD drive

eject -t eject -t cdrom

play mp3 from command line

mplayer mpg123

select an alternative Java

sudo update-alternatives --list java sudo update-alternatives --set java

list ssh keys

ssh-keygen -l -t dsa

check for vulnerable user ssh key

ssh-vulnkey

execute something in 1 minute

sleep 60 ; echo "" | at now + 1 minute

find out which ubuntu version

lsb_release -a

find out which files/dirs were modified in the past day

find . -mtime -1

find out which files are executable

find . -type f -perm -u=x

memory address to file/line

addr2line --demangle --functions --exe ...

lock/disable a user account

sudo passwd -l

unlock/enable a user account

sudo passwd -u

sharing a screen. this requires suid on screen!

from mainuser: screen -S shared :multiuser on :acladd otheruser :acladd anotheruser from otheruser: screen -x shared

ruby gem

gem q -r -n fsxtrader gem i fsxtrader

initialize a postgresql database (at least on ubuntu, where the default run

socket directory is /var/run/postgresql/, which is sudo-only)

initdb -D /tmp/central -U postgres vim /tmp/central/postgresql.conf unix_socket_directory = '/tmp/sockets-central'>->-# (change requires restart) PGPORT=10000 pg_ctl -D /tmp/central/ -l /tmp/log-central start psql -U postgres -p 10000 -h /tmp/sockets-central/

put a process into the idle scheduling class

ionice -c3 -p[pid]

start on cpu 0

taskset -c 0

control the numa policies of a running process

numactl

turn off the bell

xset b off

which glibc?

getconf GNU_LIBC_VERSION

which thread system?

getconf GNU_LIBPTHREAD_VERSION

mount an external NTFS hard disk on usb

sudo mount -t ntfs-3g /dev/sdb1 /media/disk

if that didn't work due to 'unclean shutdown'

sudo mount -t ntfs-3g /dev/sdb1 /media/disk -o force

burn cd

k3b

enable and disable startup services on debian systems

update-rc.d apache2 [enable|disable]

same on redhat systems

chkconfig httpd [on|off] chkconfig --list httpd # see state

To work with certificates

generate a private key for the CA

openssl genrsa -des3 -out ca.key 4096

generate a CA cert (aka root cert; the -x509 makes this a self-signed cert

instead of a cert request)

openssl req -new -x509 -days 365 -key ca.key -out ca.crt # also might call it ca.pem

generate a private key for the server (-des3 requests encryption passphrase)

openssl genrsa -des3 -out server.key 4096

generate a cert signing request for the server; most clients will insist that

the Common Name field match the hostname

openssl req -new -key server.key -out server.csr

sign the cert with the CA

openssl x509 -req -days 365 -in server.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out server.crt

examine

openssl rsa -noout -text -in server.key openssl req -noout -text -in server.csr openssl rsa -noout -text -in ca.key openssl x509 -noout -text -in ca.crt

create insecure version (decrypt)

mv server.key server.key.secure openssl rsa -in server.key.secure -out server.key

by now you should have the following files; make all the .key files

accessible only to root!

-rw------- 1 yang yang 1996 2008-08-26 13:53 ca.crt

-rw------- 1 yang yang 3311 2008-08-26 13:49 ca.key

-rw------- 1 yang yang 1789 2008-08-26 13:55 server.crt

-rw------- 1 yang yang 1663 2008-08-26 13:53 server.csr

-rw------- 1 yang yang 3243 2008-08-27 22:52 server.key

-rw------- 1 yang yang 3311 2008-08-26 13:50 server.key.secure

now allow users to download and install ca.crt, then you're set!

regenerate ubuntu default self-signed certificate

make-ssl-cert generate-default-snakeoil

convert key + crt to p12

openssl pkcs12 -export -in server.crt -inkey server.key -out server.p12

test (TLS) certificate

openssl s_client -CApath /etc/ssl/certs -starttls smtp -crlf
-connect getkula.com:25

also can use CA.sh

CA.sh -newca CA.sh -newreq CA.sh -sign

End of working with certificates

enable a mod (like userdir or ssl), at least on ubuntu

sudo a2enmod userdir

enable a site (like default or ssl)

sudo a2dismod ssl

snarf data off the wire

-s 0 is for "get the full packet"

-X is for "print data (in hex)"

-w writes to log file (in tcpdump's own format)

tcpdump -i eth0 -s 0 -w /tmp/log dst host 18.138.5.240 and dst port 80

dump pure data stream from a tcpdump capture (as shown above)

tcpflow -r /tmp/log

run something at a certain time

runwhen # at is insufficient for things like alarms which need certain devices to be available; at jobs are run via the atd daemon

reverse forwarding (make sure GatewayPorts is yes)

ssh -g -R 2345:11.0.0.1:22 hades ssh -R myport:targethost:targetport somehost

run scalaint with ledit

ledit -x -h ~/.scalahistory -l 10000 scala

mount a cifs share

note: ip must be an IP address and not a hostname

mount -t cifs -o 'user=yang,pass=nutcr4ck3r,ip=127.0.0.1,port=4445' //nishachar/games /mnt/nishachar

list shares available on server

smbtree -L pbems.mit.edu -U pbe

generate tags

find . -name '.cpp' -o -name '.h' -o -name '*.c' | xargs $(ETAGS) --extra=+q --fields=+fksaiS --c++-kinds=+px --append

enable/disable certain yum repos

yum --disablerepo=livna-stable yum --enablerepo=livna-stable instal xmms-mp3

uprade all software

yum update

or

yum upgrade

rip an audio cd to flac, manually - no tagging

cdparanoia -B flac track*wav

rip an audio cd to flac with tagging from cddb/freedb

abcde

convert flac to mp3

flac -cd blah.flac | lame - blah.mp3

to create a patch

new-files recursive unified-output print-c-function

diff -N -r -u -p file.c{.orig,} > diff.patch

to apply a patch

patch -p0 < diff.patch

find out which pkg an installed file belongs to

rpm -qif /path/to/file

find out which pkg a file belongs to

yum provides xclip

find out which ps has certain sockets

lsof -i :22 lsof -i tcp:22 lsof -i @1.1.1.1 # who's connected to 1.1.1.1 lsof -i [email protected]:22

get pid of sshd command

lsof -t -c sshd

show ps with link count <1

lsof +L1

and(user=daniel, dest=1.1.1.1)

lsof -a -u daniel -i @1.1.1.1

find out which ps is using file /path/to/.file.txt.swp

lsof /path/to/.file.txt.swp

find out what files a pkg provides

sudo yum install yum-utils && repoquery -l # for uninstalled rpm -ql fuse # for installed rpm -qlp fuse-4.5.rpm # for an rpm file

list all installed packages

rpm -qa

extract rpm files into current dir

rpm2cpio livna-release-5.rpm | cpio -ivd

rip lemixx

streamripper 'http://213.251.129.38:8004' -r localhost:8000 -L lemixx.pls -d /export/home/yang/rip/pad -o never -T --xs_padding=3000:3000 --xs_search_window=3000:3000

print kernel routing table

netstat -rn

set 128.30.76.1 as the default gateway

route add default gw 128.30.76.1

processes on current terminal

[jfink@kerry jfink]$ ps -T PID TTY STAT TIME COMMAND 1668 pts/0 S 0:00 login -- jfink 1669 pts/0 S 0:00 -bash 1708 pts/0 R 0:00 ps -T

processes with command string 'identd'

[jfink@kerry jfink]$ ps -C identd PID TTY TIME CMD 535 ? 00:00:00 identd 542 ? 00:00:00 identd 545 ? 00:00:00 identd 546 ? 00:00:00 identd 550 ? 00:00:00 identd

processes in job format

[jfink@kerry jfink]$ ps -j PID PGID SID TTY TIME CMD 1669 1669 1668 pts/0 00:00:00 bash 1729 1729 1668 pts/0 00:00:00 ps

processes in job format

[jfink@kerry jfink]$ ps ae PID TTY STAT TIME COMMAND 1668 pts/0 S 0:00 login -- jfink 1669 pts/0 S 0:00 -bash TERM=ansi REMOTEHOST=172.16.14.102 HOME=/home/j 1754 pts/0 R 0:00 ps ae LESSOPEN=|/usr/bin/lesspipe.sh %s

find out nameservers

cat /etc/resolv.conf should be something like: search csail.mit.edu nameserver 128.30.2.23 nameserver 128.30.2.24 nameserver 128.30.2.25

mount an iso cd/dvd

mount -o loop -t iso9660 filename.iso /mnt/isoimage/

preferred c indenting

indent -nbad -bap -bbo -bc -bl -bli0 -bls -ci4 -cli0 -di1 -nfc1 -nfca -hnl -i4 -ip4 -l80 -npcs -prs -npsl -saf -sai -saw -nsc -sob -ss -ts4 -nut depitok2.hxx

create exact whole dvd backup (verbosely)

vobcopy -o /existing-outpath/ -m

old way: dvdbackup -M -v 2 -i /dev/dvd -o /outpath/

Create an ISO image from extracted DVD files in a <title> sub-directory of

the current directory:

mkisofs -dvd-video -udf <title> > filename.iso

Write a DVD ISO image to DVD:

growisofs -dvd-compat -Z /dev/dvd=filename.iso

mount dvd iso

sudo mount -t udf file.iso /mnt/dvd -o loop

TODO watch for http headers

tcpdump -w - 'tcp[tcpflags] & (tcp-syn|tcp-fin|tcp-ack) == 0 and tcp dst port 80'

add an svn commit hook (eg for emailing on commits)

vim /data/cartel/subversion/repos/cartel/hooks/post-commit # use a script called commit-email.pl

create a new mailing list (eg for commits)

vim /etc/aliases # add a line like: cartel-icde07: [email protected], [email protected], [email protected] newaliases

TCP ping Microsoft

nmap -PT80 -sP -d -n www.microsoft.com

benchmark disk perf

iostat -x 1

Explanation: https://forums.aws.amazon.com/thread.jspa?messageID=124044#124044

stream downloading file to another file (e.g. a youtube video)

tail -c +0 -f /proc/pgrep npviewer.bin/fd/11 > foo.flv

find out extN FS block size and other info

sudo tune2fs -l /dev/sdb1

rip stream

streamripper "http://..." -r localhost:8000 -d OUTDIR -L PLAYLIST -o never -T

restrict ssh commands by authorized_keys entries (also excludes sftp)

command="/usr/bin/...",no-port-forwarding,no-agent-forwarding,no-X11-forwarding,no-pty ssh-rsa AAAAB3NzaC1yc2EA...cBYAwXd3L user@host command="BUP_DEBUG=0 BUP_FORCE_TTY=3 bup server",no-port-forwarding,no-agent-forwarding,no-X11-forwarding,no-pty ssh-rsa AAAAB3NzaC1yc2EA...cBYAwXd3L user@host

use NFS

showmount -e 192.168.0.50 # list all the mounts available for export mount -t nfs [-o resvport] 192.168.0.50:/YGD_Repository /mnt/... # resvport for Mac if "operation not permitted"

Software

Windows

  • acrobat
  • avg free
  • avidemux
  • cdrdao
  • clamwin
  • colinux
  • cygwin
  • d-tools/daemon-tools
  • eac
  • emule
  • filezilla
  • google earth
  • google picasa
  • google sketchup
  • google talk
  • i2p
  • itunes
  • mame
  • mixminion
  • nero
  • pakexplorer
  • pdfcreator
  • peerguardian2
  • pidgin
  • putty
  • skype
  • sysinternals tools
  • tor, vidalia, privoxy
  • virtualdub
  • vlc
  • vstrip
  • winamp
  • windirstat
  • winpt

Debian/Ubuntu packages

nautilus-script-audio-convert

Linux commands

gcp # cp with progressbar

unp # extract files of any format

smem # report PSS and USS for more accurate memory accounting

dtrx # do the right extraction

smtp-source # send test mail

logger # log to syslog

glxinfo LIBGL_DEBUG=verbose glxinfo

ccomps # connected components filter for graphs; from graphviz

gsmartcontrol # GUI in smartmontools

mk-visual-explain

kinit

sslsniff sslstrip

column

monitor some process output

watch

archival

cabextract lzop pbzip2

compare words instead of lines

wdiff

convert flash video to mpeg

ffmpeg -i asdf.flv asdf.mpg ffmpeg -i asdf.flv -ab 56 -ar 22050 -b 500 -s 320×240 asdf.mpg -b bitrate: set the video bitrate in kbit/s (default = 200 kb/s) -ab bitrate: set the audio bitrate in kbit/s (default = 64) -ar sample rate: set the audio samplerate in Hz (default = 44100 Hz) -s size: set frame size. The format is WxH (default 160×128 )

slice an audio/video file starting at 10s duration 60s

ffmpeg -i input.m4a -ss 10 -t 60 -codec copy output.m4a

add audio to video, either with direct copy or (if it complains) then specifying a transcoding

more at https://stackoverflow.com/questions/11779490/how-to-add-a-new-audio-not-mixing-into-a-video-using-ffmpeg

ffmpeg -i input.webm -i input.m4a -c copy -shortest combined.webm ffmpeg -i input.webm -i input.m4a -c:v copy -c:a libvorbis -shortest combined.webm

od # octal dump hd # hex dump

git-svn show-ignore > .git/info/exclude

namei realpath readlink

suspend pulseaudio

pasuspender cat

fdupes

(c++ dev) c++filt # demangles C++/Java symbol names

(objs) addr2line # memory addresses to files and lines

(profiling) gprof oprofile valgrind

(code coverage) gcov

(X) import xsel

(scripting) dialog

look # look up spelling watch # run programs at set interval at # schedule a job od # view binary data nl # number lines stat # display file attributes chattr # change advanced file attributes lsattre # list advanced file attributes

lspci lspci -v -s 01:00.0

getconf # find various info

ionice # set IO priorities

slocate # locate using index sync # flush disk caches dump # back up a disk restore # restore a backup

mesg # prohibit 'talk'/'write'

fuser -n tcp 8767 lsof uuidgen xclipboard (or xclip) (cygwin-only) getclip putclip shred getent

ssh-copy-id

cpuid

(monitoring tools) exmap # monitors mem usage accurately (takes into account sharing) htop nmon atop glances xload xosview uptime vmstat # vmstat 1: monitor every second; vmstat -d: disk free top ps less /proc/cpuinfo bonnie time iostat # iostat 1: monitor every second dstat # combines iostat vmstat ifstat netstat sar mpstat # multiprocessor usage ss # dump socket stats

(disk monitoring) xdiskusage baobab filefrag

(bandwidth monitoring) iptraf bwm-ng ibmonitor --bits --bytes --max --avg --data iftop slurm # visualize network traffic as curses graph

(network) mtr # traceroute++ ping ping6 traceroute tcpdump ntop nmap netstat tshark socat # netcat++ netpipes # more user friendly than socat

(network apps) rsync lftp curl wget

(load testers) siege tsung

(low-level development) addr2line

(terminal tools) ttyrec ipbt

(images) pngcrush gifsicle jpegtran

(misc) multitail ack-grep

(text) shuf rev tac

Packages

  • network
    • ss: replaces netstat
  • torrent
    • qbittorrent: using; supports streaming; ugly
    • transmission: using; simple, nice interface, but no streaming yet
    • vuze: once was good, no longer
  • video
    • get-flash-video
    • ffmpeg
  • java fast primitive collections
    • fastutil
    • trove: not as comprehensive as fastutil, e.g. no SortedSets
    • hppc: nice but also not as comprehensive as fastutil, e.g. no SortedSets
  • console disk usage
    • ncdu - simple and works
    • gt5
    • tdu
  • c++ libs
    • cpp-bigint
    • pdflib-lite
  • tools
    • gnuplot: using
  • c++ general libs
    • boost: using
    • fc++: interesting
  • coro/cothr libs
    • boost coroutine: interesting
    • libco
  • java analyzers
    • esc java: interesting
    • pmd: interesting
    • findbugs: interesting
  • vim plugins
    • latexsuite
    • eclim: very hacky, noble goal, but ultimately useless
    • vimoutliner: crap
  • linux kernel modules
    • perfctr: interesting
  • devices
    • gpsd: using
  • keyloggers
    • pykeylogger: works via xlib
    • uberkey: doesn't work
    • lkl: doesn't work
  • markdowns
    • markdown
    • pandoc: using
  • dist sys mgmt
    • plush: crap
  • typesetting systems
    • lyx: using
  • alternative databases
    • schevo: interesting
  • editors
    • emacs: using
  • multimedia
    • flash: using
  • browsers
    • opera: using
    • firefox: using
    • konqueror: fast, but crashy
  • emacs modes
    • org: meh, what's the point
    • haskell: using
    • tuareg: using
  • emacs tools
    • gnuserv: unneeded
  • theorem provers
    • jahob
  • c++ static analysis tools
    • treehydra: gcc plugin for low level js binding to gcc's GIMPLE AST; built on dehydra (that bootstraps this)
    • dehydra: gcc plugin for letting js interact with gcc -
    • oink: c++ static analysis suite by karl chen et al at cal built on elsa; includes cqual++
    • pork: c++ parsing and rewriting toolchain; based on elsa/elkhound/oink
    • cqual: adds type qualifiers to C
    • elsa: c++ parser
    • elkhound: parser generator
  • schemes
    • mit scheme
    • scsh
  • cross compilation tools
    • openembedded: using, meh
  • haskell abstractions
    • arrows: interesting
    • edison: interesting
  • website interfaces
    • pyfacebook: unneeded
  • web automation
    • beautifulsoup: using
    • twill: using
    • selenium: unneeded
  • haskell metaprogramming
    • drift: interesting
  • haskell tools
    • lhs2tex: interesting
    • hare: interesting
    • hmake: obsolete by cabal
  • java profilers
    • jprofiler: using, best so far
    • netbeans: using, best free one
  • java implementations
    • sun: using
  • c++ ide
    • kdevelop: using, best so far
  • irc bots
    • lambdabot
  • remote computing
    • rdesktop: using
  • file synchronization tools
    • unison: using
    • tsyncd: interesting, can't build
  • web proxies
    • tinyproxy: using
    • squid: interesting
    • loxy: logging proxy for understanding web apps (e.g. facebook chat)
    • perl HTTP::Proxy: pure Perl web proxy
    • perl HTTP::Recorder: recorder for HTTP::Proxy
  • app frameworks
    • horde: meh
  • scheduling apps
    • kronolith: meh
    • giggler: small python tool for displaying alerts/reminders
  • mail tools
    • imap-email-monitor: sf:imap-email-mon
  • mail clients
    • thunderbird: using
    • mutt: interesting
    • pine: obsolete, was interesting for its support for custom imap labels (which
      • tbird now supports)
  • scala gui
    • scalagui: outdated
  • file sharing
    • museek: using
    • nicotine: using
    • vuze: using
    • frostwire: using
  • latex libs
    • algo: obsolete by ???
    • beamer: using
  • password generators
  • cui file managers
    • vifm: sucks
    • lfm: sucks
    • ytree
    • mc: sucks
    • sunrise commander: sucks
  • filesystems
    • fuse: using
    • fuseftp: unneeded
    • lufs: obsolete by fuse; dev stopped
  • diagram tools
    • dia: using
    • visio: interesting
  • distro enhancers
    • fedora frog: outdated
  • tutors
  • haskell ide
    • hide: doesn't exist
    • pfe: doesn't seem promising
  • python libs
    • probstat: unneeded
    • py: very interesting
    • icalendar: unneeded
    • pycrypto: unneeded
    • goopy: using
    • egenix mx
    • python-dateutil: interesting, cool parsing
    • path: using
    • pyssh: meh
    • psyco: using
    • python-gpsd: using
    • pygresql: using
    • pyx: unneeded
    • pil: using
  • python tools
    • bicyclerepair: crap
    • pychecker: outdated
    • winpdb: crap, can barely get it working
    • pyprof: thin wrapper for profiling python programs
  • rdbms
    • postgresql: using
    • mysql: crap
  • embedded rdbms
    • sqlite: interesting
    • gadfly: crap, a python rdbms
  • python ide
    • spe
    • eric
    • ipython: using
  • haskell parsing
    • alex
    • happy
    • haxml
  • unsorted
    • gadfly
  • scientific programming
    • scipy: unneeded
  • anonymity
    • mixminion: interesting
  • misc
    • calgoo: crap
    • chmlib
    • project honey pot: using
  • ripping
  • music synthesis
    • terminatorx
  • gtd
    • taskfreak: crap
  • c libs
    • prjlibs: very interesting
    • libdnet: interesting
  • haskell shell
    • hsshellscript: interesting
  • haskell debugging
    • hat: outdated
  • c++ tools
    • gstlfilt: not as good as wtf
    • wtf: using
    • cint: using
  • c++ curses
    • ctk
    • ncursesxx
    • ndk++
  • networking
  • coordination
    • vxargs: using
  • package management
    • graft: crap
    • xstow: crap
    • toast: using, awesome
  • clisp tools
    • lispbox: awesome
  • clisp libs
    • interpol: unneeded
  • haskell impls
    • helium: unneeded
    • ghc: using
  • langs
    • pyrex: interesting
    • dyna: interesting
    • scala: using
    • open quark: interesting
  • haskell curses
    • hscurses: unneeded
    • hmp3: unneeded
    • hetris: unneeded
    • riot: unneeded
    • vty: unneeded
    • yi: unneeded
  • emulation
    • bochs: using
  • pdf readers
    • evince: using
    • acrobat reader: using
  • virtualization
    • xen: interesting
    • qemu: using
    • virtualbox: interesting
    • vmware: using
  • haskell games
    • frag
    • monadius
  • im
    • pidgin: using
    • xchat: using
  • xmlrpc libs
    • xmlrpc-c
  • haskell gui
    • fg
    • fruit
    • wxfruit
    • wxhaskell
  • python
    • twisted
    • zope
  • search
    • paragrep
  • terminal
    • most: crap
  • shell tools
    • pv: pipe viewer
    • parallel: using
  • SSL terminators
    • stunnel
    • stud: higher perf than stunnel

Opinions

Ubuntu 11.10 / Unity

  • had to install ccsm to disable the f10 keybinding: http://askubuntu.com/questions/37313/deactivate-f1-and-f10-keybindings

  • had to do this to disable screen lock:

    gsettings set org.gnome.desktop.lockdown disable-lock-screen 'true'

  • had to install a PPA to get resource monitor applet

  • don't grok the alt-tab behavior (esp within same app)

  • installer made me first accept custom hw drivers

  • byobu is confusing to use/config

  • right-clicks in chrome have to be held down?

  • (still) can't alt-click without hassle

  • skype tray icon not showing up; have to pkill

  • various bugs/quirks in unity

  • vlc icon showing up to the left of the resource monitor applet

Ubuntu 12.04 / Unity

  • alt key too easy to hit; had to change that to double-alt
  • menu at the top really doesn't work for apps like gimp
  • invisible scroll bar: not a fan; you have to be to the left of it and within a certain distance for it to appear

Problems

Ubuntu 12.04

Howtos

OpenVPN

  • The official HOWTO is pretty good: http://openvpn.net/index.php/open-source/documentation/howto.html

  • Follow https://help.ubuntu.com/12.04/serverguide/openvpn.html for server and client config but remove auth-user-pass from the client config. Should look like:

    client remote vpn.slice-data.com port 1194 proto udp dev tun dev-type tun ns-cert-type server reneg-sec 86400 auth-user-pass auth-nocache auth-retry interact comp-lzo yes verb 3 ca ca.crt cert yang-xps410n-win7vm.crt key yang-xps410n-win7vm.key

  • Change the server line if necessary to avoid conflicts with the client network spaces. Basically choose a specific space within 10.0.0.0/16 or /24 that is unlikely to be covered.

  • Modify your server config

    • If you want to redirect all traffic, uncomment:

      If enabled, this directive will configure

      all clients to redirect their default

      network gateway through the VPN, causing

      all IP traffic such as web browsing and

      and DNS lookups to go through the VPN

      (The OpenVPN server machine may need to NAT

      or bridge the TUN/TAP interface to the internet

      in order for this to work properly).

      push "redirect-gateway def1 bypass-dhcp"

    • If you want to do split tunenling, uncomment (and make sure this doesn't overlap with the server directive, i.e. don't assign the client an IP in the tunneled space):

      push "route "

    • You may also want to modify the DNS server, but I've found that Tunnelblick chokes on this and Windows does fine without it; only necessary if you want to control DNS.

      Certain Windows-specific network settings

      can be pushed to clients, such as DNS

      or WINS server addresses. CAVEAT:

      http://openvpn.net/faq.html#dhcpcaveats

      The addresses below refer to the public

      DNS servers provided by opendns.com.

      push "dhcp-option DNS 208.67.222.222" push "dhcp-option DNS 208.67.220.220"

  • Set DEFAULT_FORWARD_POLICY="ACCEPT" in /etc/default/ufw and run:

    ufw disable ufw enable ufw allow 1194

  • Allow NATing, where 10.8.0.0 should be the newly created tun0 subnet:

    iptables -t nat -A POSTROUTING -s 10.8.0.0/24 -o eth0 -j MASQUERADE

  • To enable 2FA, add /etc/pam.d/openvpn:

    auth [user_unknown=ignore success=ok ignore=ignore default=bad] pam_securetty.so auth required pam_google_authenticator.so forward_pass auth include system-auth account include system-auth password include system-auth

Ruby setup

Use rbenv, ruby-build, rbenv-gemset, bundler.

From https://gist.github.com/MicahElliott/2407918:

Ensure system is in ship-shape.

aptitude install git zsh libssl-dev zlib1g-dev libreadline-dev libyaml-dev

Install rbenv for managing enabling of multiple rubies.

git clone git://github.com/sstephenson/rbenv.git ~/.rbenv echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.zshrc echo 'eval "$(rbenv init -)"' >> ~/.zshrc exec $SHELL

Install the ruby-build plugin for easy building of rubies.

mkdir -p ~/.rbenv/plugins cd !$ git clone git://github.com/sstephenson/ruby-build.git cd where/ever

Install a couple rubies

rbenv install 1.8.7-p358 rbenv install 1.9.3-p125 rbenv rehash

Try out a ruby.

rbenv shell 1.9.3-p125

Enable a newly installed ruby, persistently, globally.

rbenv global 1.9.3-p125

Enable a ruby persistently just for this directory.

rbenv local 1.8.7-p358 rbenv rehash

Install bundler, create a Gemfile, install to project.

gem install bundler

Gemfile <<<'gem "ronn"' bundle install --path vendor bundle exec gem env

(OPTIONAL: for those who maintain gems or have myriad projects)

Install rbenv-gemset, create a gemset, install a gem, use it.

cd ~/.rbenv/plugins git clone git://github.com/jamis/rbenv-gemset.git rbenv gemset create 1.9.3-p125 helloset

.rbenv-gemsets <<<helloset rbenv gemset active gem install ronn rbenv rehash rbenv gemset list

Node setup

  • nodeenv ~/.node/foo for managing node environments
  • npm init
  • npm install connect --save or --save-dev (or some other pkg)

Haskell setup

sudo aptitude install haskell-platform cabal update cabal install cabal-dev cabal-dev install FOOBAR

From http://bob.ippoli.to/archives/2013/01/11/getting-started-with-haskell/

SFTP-only accounts

Want user chrooted into her own upload directory.

Make user belong to an sftponly group, which is the group on which ssh will enforce chroots. We also have for each user a root-owned chroot target (since chroot targets must be root-owned) named after the user, containing an "upload" subdirectory which is actually writable.

One-time:

sudo groupadd sftponly

And add to /etc/ssh/sshd_config:

Match Group sftponly ChrootDirectory /mnt/upload/%u X11Forwarding no AllowTcpForwarding no AllowAgentForwarding no PermitRootLogin no ForceCommand internal-sftp

Reload the config:

sudo service ssh reload

On prod, per user (replace KEY with user's public key):

user=joe sudo useradd -m $user home=~joe sudo adduser $user sftponly

sudo chsh -s /usr/lib/sftp-server $user

sudo -u $user mkdir -p $home/.ssh/ echo KEY | sudo -u $user tee -a $home/.ssh/authorized_keys

sudo mkdir -p /mnt/upload/$user/upload/ sudo chown $user /mnt/upload/$user/upload/

Test it out---verify that you cannot cd out of this tree and that files you write show up on dev:

sftp [email protected]

Ruby

Backups

One-Time

On yz.mit.edu:

sudo ufw allow 11123/tcp ssh -fNL0.0.0.0:11123:localhost:1123 localhost

On nuthouse:

ssh -fNR1123:localhost:22 yz.mit.edu # TODO this is getting killed! mkvirtualenv bup git clone git://github.com/bup/bup cd bup/ git checkout bup-0.25-rc2 make test

Per Client

On client .ssh/config (replacing username):

ServerAliveInterval=300 TCPKeepAlive=yes

Host bup HostName yz.mit.edu Port 11123 User yang-bup IdentityFile ~/.ssh/bup

On client:

ssh-keygen -f ~/.ssh/bup # can be passphraseless

On nuthouse (replacing username):

adduser --system --shell /bin/bash yang-bup

On nuthouse as user (replacing bup path and ssh key):

mkdir ~/.ssh/ cat > ~yang-bup/.ssh/config command="BUP_DEBUG=0 BUP_FORCE_TTY=3 /.../bup server",no-port-forwarding,no-agent-forwarding,no-X11-forwarding,no-pty ssh-rsa AAAA...

the above was from https://groups.google.com/forum/#!msg/bup-list/ELicUqxeouo/rpFaCMaiHRcJ

On client, install bup (ensure same version as server):

git clone git://github.com/bup/bup cd bup/ git checkout bup-0.25-rc2 make test

On client, test it out:

ssh bup # ctrl-C ./bup index -u ~/Documents/ ./bup save -n test -r bup: ~/Documents/ ./bup ls -a

On client, install bup-runner:

git clone ...

On client, install crontab:

          • yang python .../mac.py

FreeDesktop Spec Trash Setup

If you get the following:

$ trash-put a trash-put: cannot trash regular file `a'

You can probably fix it with:

sudo mkdir --mode=1777 "$(df -P file/goes/here | tail -1 | cut -d' ' -f 1)/.Trash/"

(From https://code.google.com/p/trash-cli/issues/detail?id=63 and http://stackoverflow.com/questions/3274354/how-to-find-out-mount-partition-a-directory-or-file-is-on-linux-server)

Install Vertica

For Ubuntu, temporarily append line "6.0.1" to /etc/debian_version.

(From https://community.vertica.com/vertica/topics/how_to_install_vertica_on_an_ubuntu_box)

Then install ODBC prereqs:

sudo aptitude install odbcinst unixodbc{,-dev}

Create /etc/odbc.ini:

[vertica_prod01] Driver = /usr/local/lib64/libverticaodbc.so Servername = localhost Database = pod Port = 5433 UserName = dbadmin Password = ...

(From http://vertica-forums.com/viewtopic.php?t=62)

Create /etc/vertica.ini:

[Driver] DriverManagerEncoding=UTF-16 ODBCInstLib=/usr/lib/x86_64-linux-gnu/libodbcinst.so ErrorMessagesPath=/opt/vertica/lib64 LogLevel=4 LogPath=/tmp/vlog

(From http://stackoverflow.com/questions/9778033/unixodbc-giving-error-while-running-isql-vertica and http://vertica-forums.com/viewtopic.php?t=287)

Test with:

isql -v vertica_prod01

To get up and running with Python:

pip install pyodbc sqlalchemy

(Seems to only work with more recent versions of sqlalchemy.)

This PyPI package is broken:

git clone https://github.com/jamescasbon/vertica-sqlalchemy.git

False starts:

  • vertica-python
  • pyvertica

To load data with Vertica's COPY command, you need to write in a delimited format that pleases it. It's quite picky and unfortunately there doesn't seem to be a clean/safe way of extracting data directly in this format using psql or Postgresql's COPY. The format can't handle CSV and doesn't grok special escape literal chars like \n or \N (null), requiring e.g. \ followed by a literal newline, but still using \ to escape delimiters, quotes, and \ itself. PG doesn't natively support producing data in this format.

There are some tools in /opt/vertica/sdk/examples/ which can make importing CSVs easier but there were some compilation issues (http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56403) and it's unclear whether they can distinguish NULLs from empty strings (which PG can distinguish in its output, with the absence or presence of quotes).

Flash Android 5.0 Factory Image

Spotlight on Mac OS X stops finding Applications

Solution: https://discussions.apple.com/thread/2733345?tstart=0

Instructions:

sudo rm -R /.Spotlight-V100* sudo mdutil -i on /* sudo mduitl -E /*

Geotagging Photos from your DSLR Camera

  • (Assuming your home base is Pacific) Configure your camera to be in the Pacific timezone (and adjust the time to match). Do this even if you're about to travel.

  • Use Google Takeout to export your location history (based on your phone) as a KML file (not the default JSON)

  • Use GPSBabelFE to convert the KML ("Google Earth (Keyhole) Markup Language") to GPX XML

  • Run:

    one-time

    sudo port install p5-image-exiftool pip install pytz sudo ln -s which exiftool-5.16 /opt/local/bin/exiftool git clone [email protected]:yang/GPicSync.git

    cd ~/Pictures/...

    check current time on camera and your phone. if camera and GPS times are the same (or very close):

    python ~/proj/GPicSync/src/gpicsync.py -g PATH.gpx -d PHOTOSDIR -z US/Pacific --time-range 36000

    if they're different, then specify both camera and GPS times, and it will calculate the difference

    python ~/proj/GPicSync/src/gpicsync.py -g PATH.gpx -d PHOTOSDIR -z US/Pacific --time-range 36000 --tcam 07:04:00 --tgps 07:10:00

  • Review the results in GeoTag. If any are still missing, that may be due to sparsity in your location data. gpicsync only takes locations within an hour of the photo's timestamp - you can relax this tolerance from 3600s to (say) 36000s using --time-range 36000

    • Location History doesn't poll at a fixed frequency and considers a multitude of factors. I have seen holes on the order of a day.
    • Backitude is an app that should allow for regular location updates and eliminate this problem.

  • If you traveled outside of the Pacific timezone, then also shift the time zones of the EXIF timestamps. (Background: EXIF timestamps can't specify timezone, so Google Photos interprets them in the timezone of the geotag (source).) Run the following, replacing HOURS and possibly replacing the + with a -:

    exiftool '-DateTimeOriginal+=0:0:0 HOURS:00:0' . exiftool '-CreateDate+=0:0:0 HOURS:00:0' .

Find the App Stealing Focus on Mac OS X

http://apple.stackexchange.com/questions/123730/is-there-a-way-to-detect-what-program-is-stealing-focus-on-my-mac

#!/usr/bin/python

try: from AppKit import NSWorkspace except ImportError: print "Can't import AppKit -- maybe you're running python from brew?" print "Try running with Apple's /usr/bin/python instead." exit(1)

from datetime import datetime from time import sleep

last_active_name = None while True: active_app = NSWorkspace.sharedWorkspace().activeApplication() if active_app['NSApplicationName'] != last_active_name: last_active_name = active_app['NSApplicationName'] print '%s: %s [%s]' % ( datetime.now().strftime('%Y-%m-%d %H:%M:%S'), active_app['NSApplicationName'], active_app['NSApplicationPath'] ) sleep(1)

Create initial from-scratch Alembic revision

It's hygienic to always create such a revision before creating any other Alembic revisions.

sudo -u postgres psql
> alter database myapp rename to myapp_orig;
> create database myapp owner myuser;

alembic revision --autogenerate -m 'Initial from-scratch revision'
alembic stamp head

sudo -u postgres psql
> drop database myapp;
> alter database myapp_orig rename to myapp;

Fix "missing Java" on OS X

"No Java runtime present, requesting install."

Initially encountered this trying to import boilerpipe in Python.

There are various other solutions including installing the Apple-provided JRE 6, but the best solution IMO is tweaking Java 8 settings:

https://stackoverflow.com/questions/19563766/eclipse-kepler-for-os-x-mavericks-request-java-se-6/19594116#19594116

OS X Setup

Run:

sudo port install \
  coreutils ffmpeg git-core ncdu osxutils py-virtualenvwrapper postgresql95 \
  postgresql95-server python27 realpath the_silver_searcher wget youtube-dl \
  vim +huge +universal +cscope +python27 +python34 \
  macvim +huge +universal +cscope +python27 +python34

Install Karabiner and tweak the key repeat settings (200ms/25ms) and load your custom private.xml.

TODO merge with your Migrations GDoc.

Dev Setup

Install bash-it and run:

for i in git grunt npm pip ; do bash-it enable completion $i; done
for i in general git osx vim ; do bash-it enable alias $i; done

Install bling dotvim.

Use your custom dotfiles (in personal-cm/roles/primary).

Compare Feedly and raw versions of AudioLizard feed

  • In feedly, go to AudioLizard feed

  • View unread

  • Scroll down a bunch to load enough items

  • Run the "Feedly dump feed items (to compare with AudioLizard source)" snippet

  • Run copy(__toCopy) in the console

  • Paste into some file

  • On nuthouse, run:

    psql -h localhost -U webreader webreader -Atc 'select url from articles where converted is not null order by created desc' > /tmp/audiolizard.feed

  • Copy that file to localhost

  • Diff the two files

IntelliJ PyCharm Remote Debugging Slow

This turned out to be due to a stale project. Deleting it and creating a new project over the original sources fixed this.

Make Chrome Remote Desktop work on Ubuntu

This solves two problems:

  • Not the same session as the local (non-remote) desktop session
  • Not loading any window manager, preventing window manipulation

https://superuser.com/questions/778028/configuring-chrome-remote-desktop-with-ubuntu-gnome-14-04

Then to ensure it starts at startup, enable auto-login in Ubuntu.

Mount a LUKS-encrypted partition

If you try mounting directly, you'll see:

yang@partycat:~$ sudo mount /dev/sdb1 /media/store
mount: unknown filesystem type 'crypto_LUKS'

From https://askubuntu.com/questions/63594/mount-encrypted-volumes-from-command-line:

sudo cryptsetup luksOpen /dev/sdb1 my_encrypted_volume
sudo mount /dev/mapper/my_encrypted_volume /media/store

Install Realtek Wi-Fi drivers on Ubuntu 16.04

Bought this 802.11ac AC1200 wireless USB adapter from Central Computers (product WN688A2), but it doesn't just work out of the box on Ubuntu 16.04.

IIRC it didn't on 12.04 either, but simply downloading the drivers from the manufacturer's website running install.sh within the RTL8812AU_linux_v4.2.4_9390.20131023 dir seemed to work, although not entirely reliably (the adapter would just disconnect from the Wi-Fi after many hours and then require disconnecting and reconnecting the physical adapter to work again).

Anyway, after some brief Googling for that Realtek chipset/driver (RTL8812AU), this top hit on Google worked. For whatever reason, Realtek's official website doesn't have any associated drivers available.

Sharing in LastPass

Without a Premium account, updates to secrets don't get synced to people you previously shared it with. So you need to reshare. You just enter the email address again. Exactly the saved copy as of that time gets shared.

Move subtitles to top of screen in VLC

  • Preferences > Show All > Video > Subtitles / OSD
  • Set Force Subitle Position to 600 (positive means up)
  • Restart VLC! Can't just restart video.

Delay / Adjust subtitle timing in VLC

  • Play around with h and g keys

Create Ubuntu boot USB for PC on Mac

https://computers.tutsplus.com/tutorials/how-to-create-a-bootable-ubuntu-usb-drive-for-pc-on-a-mac--cms-21187

hdiutil convert -format UDRW -o ~/path/to/target.img ~/path/to/ubuntu.iso
diskutil list
diskutil unmountDisk /dev/diskN
sudo dd if=/path/to/ubuntu-14.04-desktop-i386.dmg of=/dev/rdiskN bs=1m
# DO NOT RESPOND TO DIALOG BOX WITH Initialize, Ignore, Eject
diskutil eject /dev/diskN
# NOW you can click Ignore

Download Youtube playlist with indexes in filename

youtube-dl -o "%(playlist_index)s-%(title)s-%(id)s.%(ext)s" <playlist-url>