13 May 2017

Systemd service for IP aliasing

In this post I'll show how to make a Systemd service for easy addition and removal of local IP addresses.

Problem

The service may be useful when working with emulators such as Android Emulator, when one needs to configure network communication between the emulated environment and a local server. For example, the problem with Android Emulator is that it is isolated from the host's private network, as it has its own virtualised address space.

But it somehow manages to get access to addresses of the interfaces configured on the host. And we are going to exploit this feature.

I have tried different ways, and eventually came to making a Systemd service.

NetworkManager

We can easily overcome the problem by creating a virtual (VLAN) connection for an existing wired connection with the help of a popular network manager such as NetworkManager. For example, NetworkManager's interface allows to create a VLAN connection based on real wired or wireless connections and assign different IP addresses to it. However, for some reason it didn't work for my Wi-Fi connection, and wired connection was not available. So I was forced to search for another solution.

Systemd service

The following solution is based on the ip tool from the iproute2 package.

Actually we can add address to an interface with a single command without the need for a Systemd service or something, e.g.:

ip address add 10.100.100.10/24 dev wlp3s0 label wlp3s0:1
or
ip address change 10.100.100.10/24 dev wlp3s0 label wlp3s0:1
where 10.100.100.10/24 specifies IP address and the network (24-bit mask, particularly), wlp3s0 points to existing wireless interface, and wlp3s0:1 is a tag for the address being added (will appear as separate interface in the output of ifconfig). Removal is just as easy:
ip address del 10.100.100.10/24 dev wlp3s0

But I don't like the idea of typing the commands over and over. So I created the following script and saved it as /usr/local/sbin/ip-alias:

#!/bin/bash -
# Configures extra IP addresses for a network interface.
#
# Configuration files are located at /usr/local/etc/ip-alias.IFNAME.conf
# where IFNAME is the network interface name.
#
# Ruslan Osmanov 2017

# Prints an error message
err()
{
  printf 'Error: %s\n' "$1" >&2
}

# Prints an warning message
warn()
{
  printf 'Warning: %s\n' "$1" >&2
}

# Prints an error message $1 (if any), then exits
die()
{
  [ $# -gt 0 ] && err "$1"
  exit 1
}

# Prints usage info and exits with status $1
usage()
{
  printf 'Usage: %s IFNAME [add|remove]\n' "$0"
  exit $1
}

# Adds all addresses specified in the configuration file for device $1
add()
{
  local device="$1"
  local address
  local new_device

  # Use the command group '(...)' in order to prevent modifying variables in
  # the global scope when 'source'ing the configuration file
  (
    load_config "$device" || die "failed to read configuration file"

    [ ! -v addresses -o ${#addresses[*]} -eq 0 ] && \
      die "required configuration 'addresses' is invalid/missing"

    for index in "${!addresses[@]}" ; do
      address="${addresses[$index]}"
      label="${device}:${index}"

      printf 'adding %s for %s as %s\n' "$address" "$device" "$label"
      # We might used ip address add instead, but it complicates consecutive calls to the script
      /bin/ip address change "$address" dev "$device" label "$label"
      [ $? -eq 0 ] || die "failed to add $address for $label"
    done
  )
}

# Removes address $1 for device $2
remove_address()
{
  local address="$1"
  local device="$2"

  printf 'removing address %s for device %s\n' "$address" "$device"
  /bin/ip address del "$address" dev "$device"

  if [ $? -ne 0 ]; then
    err "Failed to remove device $device"
    return 1
  fi
}

# Removes all addresses specified in the configuration file for device $1
remove()
{
  local device="$1"
  local address

  # Use the command group '(...)' in order to prevent modifying variables in
  # the global scope when 'source'ing the configuration file
  (
    load_config "$device" || die "failed to read configuration file"

    [ ! -v addresses -o ${#addresses[*]} -eq 0 ] && \
      die "required configuration 'addresses' is invalid/missing"

    for index in "${!addresses[@]}" ; do
      address="${addresses[$index]}"
      remove_address "$address" "$device"
    done
  )
}

# Loads configuration file for device $1
load_config()
{
  local filename="/usr/local/etc/ip-alias.${1}.conf"

  printf 'loading configuration from %s\n' "$filename"
  source "$filename"
}


if [ $# -lt 1 ]; then
  err "Invalid number of arguments"
  usage 1
fi

device="$1"
[ $# -gt 1 ] && action="$2"
: ${action:='add'}


case "$action" in
  add)
    add "$device"
    ;;
  remove)
    remove "$device"
    ;;
  *)
    die "action $action didn't match anything"
esac

Configuration file is just a shell script with declaration of addresses array variable. For example, /usr/local/etc/ip-alias.wlp3s0.conf file might look like the following:

addresses=("10.100.100.10/24" "10.100.101.10/24")

Having this script we don't need to remember the syntax of the ip command. All we need is to pass an interface name and an action name to the command:

/usr/local/sbin/ip-alias wlp3s0 add
/usr/local/sbin/ip-alias wlp3s0 remove

But we can do better with the help of an init system. For example, we can create /etc/systemd/system/ip-alias@.service Systemd service file with the following content:

[Unit]
Description=Setup IP aliases for %i network interface
After=network.target

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/local/sbin/ip-alias %i add
ExecStop=/usr/local/sbin/ip-alias %i remove

[Install]
WantedBy=multi-user.target

If you are using NetworkManager, you may want to add it as dependency:

[Unit]
Description=Setup IP aliases for %i network interface
Wants=NetworkManager.service NetworkManager-wait-online.service
After=network.target NetworkManager.service NetworkManager-wait-online.service
BindsTo=NetworkManager.service NetworkManager-wait-online.service
PartOf=NetworkManager.service NetworkManager-wait-online.service

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/local/sbin/ip-alias %i add
ExecStop=/usr/local/sbin/ip-alias %i remove
ExecReload=/usr/local/sbin/ip-alias %i remove
ExecReload=/usr/local/sbin/ip-alias %i add

[Install]
WantedBy=multi-user.target
# systemctl enable NetworkManager-wait-online.service

Then we can enable, start and stop the service for specific network interface(s), e.g.:

# systemctl enable ip-alias@wlp3s0
# systemctl start ip-alias@wlp3s0
# systemctl stop ip-alias@wlp3s0

With this service we don't need to remember anything, except the use of systemctl.

Sample ifconfig output when the service is running:

wlp3s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.10.0.102  netmask 255.255.255.0  broadcast 10.10.0.255
        inet6 fe80::2ac2:ddff:fec7:16d5  prefixlen 64  scopeid 0x20<link>
        ether 28:c2:dd:c7:16:d5  txqueuelen 1000  (Ethernet)
        RX packets 173600  bytes 166406125 (158.6 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 96319  bytes 13015624 (12.4 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wlp3s0:0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.100.100.10  netmask 255.255.255.0  broadcast 0.0.0.0
        ether 28:c2:dd:c7:16:d5  txqueuelen 1000  (Ethernet)

wlp3s0:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.100.101.10  netmask 255.255.255.0  broadcast 0.0.0.0
        ether 28:c2:dd:c7:16:d5  txqueuelen 1000  (Ethernet)

30 March 2016

Making charts from iostat output

I've decided to make some charts from the output of the iostat command. Just for fun =)

The idea is simple: a script named `report.sh` runs periodically in background(via `cron`) collecting `iostat` data into $REPORT_DIR/$param/$device files, where

  • $REPORT_DIR is a directory with the reports;
  • $param is the `iostat` parameter name;
  • $device is a device name from the same `iostat` output.

When we need to update the charts, we call `plot.sh`.

Required components

Scripts

util.sh
#!/bin/bash -

# $1 - Optional error message.
function die
{
  [ $# -gt 0 ] && echo >&2 $1
  exit 1
}

function warning
{
  echo >&2 "Warning: $1"
}


function getSafeFilename
{
  r=${1//\//_}
  echo ${r/\%/}
}

#vim: ts=2 sts=2 sw=2 et
report.sh
#!/bin/bash -
# Generates iostat data files for processing with the GNU plot utility.

HEADER=
TIME_FORMAT="%Y/%m/%d/%H:%M"
TIME=$(date "+$TIME_FORMAT")
DIR=$(cd $(dirname "$0"); pwd)
REPORT_DIR="${DIR}/r"


#####################################################################
# Functions

source $DIR/util.sh


# $1 - exit code
function usage
{
  echo "
Generates iostat data files for processing with the GNU plot utility(plot.sh).

Usage: $0 OPTIONS

OPTIONS:
-h, --help            Display help message
-t, --time-format     Time format. Default: ${TIME_FORMAT}.
-i, --report-dir      Directory with reports. Default: ${REPORT_DIR}.
"
  exit $1
}

#####################################################################
# Parsing CLI options

OPTS=$(getopt -o t:i:h -l time-format:,report-dir:,help -- "$@")
[ $? -eq 0 ] || usage 1
eval set -- "$OPTS"

while true
do
  case "$1" in
    -t|--time-format)
      TIME_FORMAT="$2"
      shift 2;;
    -i|--report-dir)
      REPORT_DIR="$2"
      shift 2;;
    -h|--help)
      usage 0
      shift;;
    --)
      shift
      break;;
    ?)
      usage 1;;
    *)
      die "Internal error: failed to parse CLI args"
  esac
done

echo TIME_FORMAT: $TIME_FORMAT REPORT_DIR: $REPORT_DIR

#####################################################################

mkdir -p $REPORT_DIR
[ $? -eq 0 ] || die "Report directory '$REPORT_DIR' is inaccessible"

iostat -dpxk | while read -r line
do
  if [[ -z $HEADER && $line == Device* ]]; then
    HEADER=( $line )
  elif [[ -n $HEADER ]]; then
    [[ -z $line ]] && continue

    columns=( $line )

    [[ ${#HEADER[@]} != ${#columns[@]} ]] && continue


    i=1
    for h in ${HEADER[@]:1}
    do
      param_report_dir="${REPORT_DIR}/"$(getSafeFilename $h)
      mkdir -p $param_report_dir
      [ $? -eq 0 ] || die "Parameter directory '$param_report_dir' is inaccessible"

      # $REPORT_DIR/param/device
      echo $TIME$'\t'${columns[$i]} >> $param_report_dir'/'${columns[0]}
      (( ++i ))
    done
  fi
done

echo 'Done'
plot.sh
#!/bin/bash -
# Creates visual representation of data files generated by report.sh

TIME_FORMAT="%Y/%m/%d/%H:%M"
# Report files location $REPORT_DIR/param/device
DIR=$(cd $(dirname "$0"); pwd)
REPORT_DIR="${DIR}/r"
OUTPUT_DIR="${DIR}/o"
XRANGE=
PLOT_TERMINAL="png"

#####################################################################
# Functions

source $DIR/util.sh

function getPlots
{
  param_path="$1"

  find $param_path/ -maxdepth 1 -mindepth 1 -type f | while read -r device_path; do
    # $device_path = /*/*/.../param/device
    device=$(basename $device_path)

    printf "'%s' using 1:2 t '%s' with lp pt 5, " \
      "$device_path" $device
    printf "'%s' using 1:2 notitle with impulses, " \
      "$device_path"
  done
}


# Returns default value for GNU plot xrange
function getXRange
{
  y=$(date +%Y)
  m=$(date +%m)

  (( m -= 1 ))

  if [ $m -lt 1 ]; then
    m=1
    (( y -= 1 ))
  fi

  d=$(date "+$TIME_FORMAT")

  echo "[ '$y/$m/1/00:00' : '$d' ]"
}


# $1 - exit code
function usage
{
  echo "
Creates charts from files generated by report.sh

Usage: $0 OPTIONS

OPTIONS:
-h, --help            Display help message
-x, --xrange          GNU Plot xrange value. Default: $(getXRange).
-t, --time-format     Time format. Default: ${TIME_FORMAT}.
-i, --report-dir      Directory with reports. Default: ${REPORT_DIR}.
-o, --output-dir      Output directory. Default: ${OUTPUT_DIR}.
-f, --term            GNU Plot terminal. Default: ${PLOT_TERMINAL}. See 'gnuplot set terminal'.
"
  exit $1
}

#####################################################################
# Parsing CLI options

OPTS=$(getopt -o x:t:i:o:f:h -l xrange:,time-format:,report-dir:,output-dir:,term:,help -- "$@")
[ $? -eq 0 ] || usage 1
eval set -- "$OPTS"

while true
do
  case "$1" in
    -x|--xrange)
        XRANGE="$2"
        shift 2;;
    -t|--time-format)
      TIME_FORMAT="$2"
      shift 2;;
    -i|--report-dir)
      REPORT_DIR="$2"
      shift 2;;
    -o|--output-dir)
      OUTPUT_DIR="$2"
      shift 2;;
    -f|--term)
      PLOT_TERMINAL="$2"
      shift 2;;
    -h|--help)
      usage 0
      shift;;
    --)
      shift
      break;;
    ?)
      usage 1;;
    *)
      die "Internal error: failed to parse CLI args"
  esac
done

echo XRANGE: $XRANGE TIME_FORMAT: $TIME_FORMAT OUTPUT_DIR: $OUTPUT_DIR REPORT_DIR: $REPORT_DIR

#####################################################################

[ -d $REPORT_DIR ] || die "$REPORT_DIR doesn't exist"

find $REPORT_DIR/ -maxdepth 1 -mindepth 1 -type d | while read -r param_path
do
  # $param_path = /*/*/.../param
  param=$(basename $param_path)
  plots=$(getPlots "$param_path")
  plots=${plots/%, /}

  mkdir -p $OUTPUT_DIR
  [ $? -eq 0 ] || die "Output directory '$OUTPUT_DIR' is inaccessible"

  outfile="${OUTPUT_DIR}/${param}.${PLOT_TERMINAL}"

  [[ -z $XRANGE ]] && XRANGE="$(getXRange)"

  gnuplot <<EOS
TIME_FORMAT = "$TIME_FORMAT"
set terminal $PLOT_TERMINAL
set grid nopolar
set xtics rotate
set xlabel "Date/Time"
set ylabel "Value"
set yrange [ 0 : ]
set format x TIME_FORMAT timedate
set timefmt TIME_FORMAT
set xdata time
set xrange $XRANGE
set output "$outfile"

plot $plots
EOS
done

echo 'Done'

Sample chart

Gnuplot Produced by GNUPLOT 5.0 patchlevel 1 (Gentoo revision r1) 0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 2016/03/30/16:00 2016/03/30/16:30 2016/03/30/17:00 2016/03/30/17:30 2016/03/30/18:00 2016/03/30/18:30 Value Date/Time sda4 sda4 gnuplot_plot_2 sda3 sda3 gnuplot_plot_4 sda1 sda1 gnuplot_plot_6 loop1 loop1 gnuplot_plot_8 loop0 loop0 gnuplot_plot_10 sda sda gnuplot_plot_12 sda2 sda2 gnuplot_plot_14 loop2 loop2 gnuplot_plot_16