Network traffic monitoring with tcpdump

tcpdump is a command-line network traffic analyzer. It allows capture of network packets from a host and subsequent filtering and display.

Sample applications

Show traffic into port 2003 from a given host

sudo tcpdump -A -nn 'port 2003 and src host 10.60.35.49'

Must run as root (sudo)
-A print ASCII payload
-nn numeric IP and ports

Snippets

# Sudo a long command line
sudo bash -c "
# Get 16th field
cut -f16 -d\| /data/sample.csv | 
# Render unicode as ASCII
iconv -f utf8 -t ascii//TRANSLIT | 
# Drop empty lines
sed '/^$/d' | 
# Filter for lines longer than 30 characters
awk 'length(\$0) > 30' > /data/sample.out"

NUMA, affinity and OS tuning

NUMA tuning and affinity (RHEL)
NUMA and optimization (Elasticsearch, Riak)

IRC cheat sheet

Register IRC nick

/msg NickServ REGISTER password youremail@example.com

Identify to primary IRC account:

/msg NickServ IDENTIFY account password

Get nick info

/msg NickServ INFO account

Reset password

/msg NickServ SET PASSWORD mynewpassword

Reset command hash

To flush the hash table used by the shell to remember the full path names of executable files:

hash -r

git recipes

Git: prune local branches without remotes

git branch -vv | grep ': gone]' | awk '{print $1}' | xargs git branch -D 

Cache git credentials (Linux)

While it is possible to store git credentials locally via .netrc, that requires storing them in plain text. Even with 400 permissions, it is an unnecessary risk. I have read references about using GPG for encryption within this file, which increases security, but introduces a new dependency. When changing hosts frequently, local dependencies become highly impractical. Instead, to cache the credentials in memory with an expiration timeout, one hour, for example:

git config --global credential.helper 'cache --timeout=3600'

Fix brightness controls in Ubuntu 14

In some instances of Ubuntu 14, brightness control keys do not work. To correct this, first determine the type  of video card adapter controlling the display backlight; use:

$ ls /sys/class/backlight
intel_backlight

sudo vim /usr/share/X11/xorg.conf.d/20-intel.conf

Add the following lines to this file:

Section "Device"

        Identifier  "card0"
        Driver      "intel"
        Option      "Backlight"  "intel_backlight"
        BusID       "PCI:0:2:0"
EndSection

Save it. Log out and log in back. The brightness control should be working through function keys now:

Setting up Scalding

Twitter does not publish Scalding binaries. To set up Scalding locally, the following is needed:

1. Clone: git clone https://github.com/twitter/scalding.git
2. sbt assembly
3. If necessary, adjust sbt-assembly version accoridng to Scala/sbt versions (use sbt about). Under project/plugins.sbt, e.g.:

-addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.10.2")
+addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.11.2")
 
4. sbt publishLocal

Using netcat

Send a string (e.g. to a Graphite server)

echo "metrics.test.random $RANDOM $(date +%s)" | nc 10.60.35.50 2003

Test status of a TCP daemon

nc -vz someserver port

-v: verbose
-z: scan for listening daemons but send no data

Disable IPv6 on CentOS

echo 'install ipv6 /bin/true' > /etc/modprobe.d/ipv6.conf
init 6

This process disables the IPv6 kernel module entirely. It can be considered good form, but by no means necessity, to do the following:

chkconfig ip6tables off

Also in /etc/sysconfig/network-scripts/ifcfg-*:

IPV6INIT=no
IPV6_AUTOCONF=no

To verify, the following should produce empty output:

lsmod | grep v6

Set static host name (Ubuntu)

sudo hostname myhost #

echo 'myhost' | sudo tee /etc/hostname

VirtualBox video resolution: guest only allows small video resolutions

Symptom: Guest OS (Kubuntu) only allows 640x480 video resolution

Solution: Increase video memory allocation on virtual machine - on VirtualBox: Settings > Display > Video > Video Memory (set to >64MB)