Wikimedia Discovery/Command Line Tricks

This is a collection of useful *nix command line tricks and productivity tips that not everyone knows about that could improve your command-line fu. Note that not all versions of *nix are created equal, so some of these may work in some OSes or some shells but not in others. (OTOH, once you know it's possible, you can usually find a way to make it happen in your OS/shell.)

Commands

Each command here has a brief description, a use case or two, and an example or three if needed.

pushd <dir>, popd, pushd

description: controls a stack of directories. pushd <dir> moves you to <dir> and pushes <dir> onto a stack. popd pops the stack and moves you back to your previous directory. pushd (with no argument) swaps the top two items on the stack.
use case: jumping back and forth between two directories (config in <dir1>, running stuff in <dir2>), changing directories one or more times and then changing back without having to keep track of the list of directories (e.g., in a bash script)
see also https://en.wikipedia.org/wiki/Pushd_and_popd

`<cmd>` or $(<cmd>)

description: convert the output of a command into command line arguments; $(...) can be embedded—whether that is good or bad is left as an exercise for the reader.
example:
- touch `date "+%m-%d"`.txt or touch $(date "+%m-%d").txt create a file with today's date as part of the filename using date to get today's date
- vi `!!` use the output of the previous command (say, something complicated involving ls) as command line arguments for vi
- vi `grep -slR ombudsmenwiki | grep Initial` open all files in this directory tree that contain ombudsmenwiki but do not have Initial in their name.

!!, !<#>, !<cmd>, !$

description:
- !! repeats the last command, possibly in a new context;
- !-2 repeats the command two steps back in your history. !-1 is the same as !!.
- !445 repeats command 445 from the output of history.
- !ls will re-run your most recent ls command.
- !$ is the last argument from the previous command
example:
- sudo !! repeat previous command with sudo
- vi `!!` use the output of the previous command (say, something complicated involving ls) as command line arguments for vi
- !!;!! repeat the previous command twice (do it several times and you will repeat the command 4 times, 8 times, 16 times, etc.)
- vi foo/bar/baz.php followed by git add !$ adds the file you were just editing to git. Alternatively inside vim % to refer to current file. ex: :!git add %
- cat /path/to/some/file/that/is/very/far/away.txt followed by cd $(dirname !$) to move to the directory of the file referenced in the previous command.
- See also: https://jaysoo.ca/2009/09/16/unix-history-and-bang-commands/

bash scripting

description: shell scripts can be really powerful. Simple loops are super useful...
example:
- for cfg in $(ls *.cfg); do cp $cfg bk/; done copy every .cfg file into directory bk/
- for cfg in $(ls *.cfg); do cp $x $x.bk; done copy every .cfg file to a .cfg.bk backup in the same directory
Note though that parsing ls output can be error prone. What if a file has a space in it's name? Use find with null delimited strings:
- find *.cfg -print0 | while read -r -d $'\0' line; do cp "$x" "$x.bk"; done
But really find is pretty powerful all on its own and will handle oddities like files with spaces without thinking about it
- find *.cfg -exec cp {} {}.bk \;
see http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO-7.html

grep -R

description: recursive grep, searches all files in this directory and every directory below it. Not as powerful as find but easier to remember
example: vi `grep -slR ombudsmenwiki | grep Initial` open all files in this directory tree that contain ombudsmenwiki but do not have Initial in their name.

typing the hard-to-type

CTRL+SHIFT+u + <unicode_number> + <Enter> Type a random unicode char (Mac and Windows instructions, more)
Commonly used characters:

°	00b0
á	00e1
é	00e9
î	00ee
–	2013
—	2014
^♫	266b
Δ	0394
✔	2714
❏	274f
←	2190
→	2192
²	00b2
³	00b3
ℵ	2135
α	00b1
µ	00b5

CTRL+v <char> interpret <char> (most usefully for tab or return) as just a character. Also works in vi/vim

tmux and screen

tmux and screen both allow you to create a terminal session that will persist after logging out. tmux also allows you to split screens and do all sorts of other cool stuff. To learn more—use the Google-fu, Luke.

vi(m) shortcuts

A few favorite shortcuts in vi. No discussion allowed about whether vi or emacs or some other lame editor is best.

ciw = replace current word
ci( = replace inside parenthesis
diw = delete current word
xp = swap current character and the next one
~ = toggle case of current character; 10~ toggles ten characters
w = move forward one word with a restrictive definition of word (ex: stops on each / of a path)
W = move forward one word with a generous definition of word (ex: a whole path is considered a single word)
J = remove newline at end of line, merging current and next line
} = Move to next blank line. Useful for jumping large distances in source files. { does same backwards.

Vim prettiness (adds symbols for eol, tab and spaces at the end of the line (tralling) that shouldn't be there) (vim command line or .vimrc or equivalent). This can make it easy to recognize mixed tabs/spaces for example:

set list

set listchars=eol:⏎,tab:▸·,trail:␠,nbsp:⎵

To turn back off:

set nolist

Config

These are cool things you can add to your .bashrc or .bash_profile to make your command line more efficient

alias

alias lets you rename a command (including command line options and piping commands together).

alias q=exit quit with one keystroke
alias ll="ls -l" just two keystrokes to get a long directory listing
alias h="history | grep" now h mwscript will list only lines in your history that include mwscript
alias stats="ssh stat1002.eqiad.wmnet" log into stat1002 much more easily
alias stuff="cd /really/long/harDtOType/or-hard-to-remember/path/to/useful/stuff" if you type it more than twice a day, make an alias!
alias li='find . -maxdepth 1 -iname' makes lia case-insensitive version of ls
alias freq="sort | uniq -c | sort -n" pipe into freq to get a frequency sorted list of lines; use sort -nr as the last command to get the most frequent item first; e.g., cat list_of_terms | freq
alias vo=vi or alias grpe=grep alias a common typo to what you intended
alias cdcirrus="cd ~/mediawiki/vagrant/mediawiki/extensions/CirrusSearch" Stop typing out this crazy long path you move into several times a day. Now you can cdc<tab><return> and go there.
alias duh='du --max-depth=1 -h' a quick list of subdirectories with sizes.
alias df='df -h' How much space is left?

history config

export HISTCONTROL=erasedups Only keep one copy of any give command in your history. Very useful is you repeat the same command or set of commands over and over without filling up your history
shopt -s histappend
- If the histappend shell option is enabled, the lines are appended to the history file, otherwise the history file is over-written.
export PROMPT_COMMAND='history -a; history -r' synch up history across all terminals
- PROMPT_COMMAND is run every time before a command line prompt is displayed
- -aappends this terminal's history to the history file; -r appends the history file to this terminal's history
- See also https://www.gnu.org/software/bash/manual/bashref.html#Bash-History-Builtins
hold on to more history
- export HISTSIZE=1000000 # Number of lines in history. Arbitrarily large so histfilesize takes over
- export HISTFILESIZE=1000000 # Number of bytes in history

export LC_ALL=en_US.UTF-8

Makes lots of stuff work better with Unicode

custom bash prompt (PS1)

You can add all sorts of useful information into your bash prompt, including emoji, directories, the time, color, and more. Add some version of these commands to your .bashrc or equivalent. It can be useful to have one color/emoji/etc on one (real/virtual) machine and a different config on another, so you can always tell where you are.

Using emojis for local sessions: export PS1="👾 \[\033[35m\]\u\[\033[35m\]@\[\033[35m\]\h:\[\033[33;1m\]\w\[\033[m\]\$ "
Add color (green) and current time: export PS1='\[\033[1m\033[32m\][\t] \u@\h \w$ \[\033[0m\]'

Show current git branch in your prompt:

function parse_git_branch {
  git branch --no-color 2> /dev/null | sed -e '/^[^*]/d' -e 's/* \(.*\)/(\1)/'
}
local LIGHT_RED="\[\033[1;31m\]"
local BLACK="\[\033[0;00m\]"
PS1=\"\[\033[01;32m\]\u\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]$LIGHT_RED\$(parse_git_branch)$BLACK\$ "

See also:
- https://www.cyberciti.biz/tips/howto-linux-unix-bash-shell-setup-prompt.html
- https://www.cyberciti.biz/faq/bash-shell-change-the-color-of-my-shell-prompt-under-linux-or-unix/

SSH config

See https://wikitech.wikimedia.org/wiki/Managing_multiple_SSH_agents and https://office.wikimedia.org/wiki/Discovery_Analytics#SSH_Configuration

For Mac-specific SSH key config, see https://github.com/jirsbek/SSH-keys-in-macOS-Sierra-keychain

Other Specific Tools

These are some specific tools that people have configured for themselves. They may or may not be useful, but may also provide inspiration.

up

A bash function (goes in .bashrc or equivalent) to navigate around deep directory trees:

up() {
    local p= i=${1:-1}
    while ((i--)); do
        p+=../ 
    done
    cd "$p" && pwd
}

Usage: up 3 goes up three directories

Auto-magic tmux into place on first opened console from .bashrc (with irc):

if [ -z "$TMUX" ]; then
    base_session='my_session'
    if ! tmux has-session -t $base_session; then
        tmux new -d -s $base_session 'irssi' \; split-window  \; attach \;
    fi
fi

Turns on tmux and opens an irc session using irssi on login. tmux is generally awesome and has many cool features.

These are some useful command-line navigation shortcuts.

up-arrow & down-arrow

Move through previous command line commands; edit or just re-run them

CTRL-r <string>

Search back through your history for a command matching what you type. CTRL-r mwscript will take you to your last command with mwscript in it. CTRL-r again will jump back to the matching command before that!

emacs-style navigation & cut/paste

CRTL-e: go to end of line
CRTL-a: go to beginning of line
CRTL-k: cut to the end
CTRL-u: cut to the beginning
CRTL-w: cut the previous word
CRTL-y: paste
ALT-f: forward one word
ALT-b: backwards one word