Basic Zookeeper in Clojure

Add [zookeeper-clj "0.9.1"] to dependencies

lein repl

(require '[zookeeper :as zk])

(.getChildren client "/" false) ;; watcher=false

#<ArrayList [hbase-unsecure, storm, zookeeper]>

(def ZK_HOSTS "127.0.0.1:2181,node1,node2,node3)

(def ZK_ROOT "/twitter-demo")(def client (zk/connect ZOOKEEPER_HOSTS :watcher (fn [event] (println event))))

(zk/create client ZK_ROOT :persistent? false)

(def version (:version (zk/exists client ZK_ROOT)))

(zk/create client ZK_ROOT :data (.getBytes "123143kdjkds") :persistent? false)

Dump Java class loaders including JAR physical paths

private static Iterator list(ClassLoader CL)

        throws NoSuchFieldException, SecurityException,

        IllegalArgumentException, IllegalAccessException {

    Class CL_class = CL.getClass();

    while (CL_class != java.lang.ClassLoader.class) {

        CL_class = CL_class.getSuperclass();

    }

    java.lang.reflect.Field ClassLoader_classes_field = CL_class

            .getDeclaredField("classes");

    ClassLoader_classes_field.setAccessible(true);

    Vector classes = (Vector) ClassLoader_classes_field.get(CL);

    return classes.iterator();

}

 

private void dumpClasses() {

    ClassLoader myCL = Thread.currentThread().getContextClassLoader();

    while (myCL != null) {

        log.warn("===> ClassLoader: " + myCL);

        try {

            for (Iterator iter = list(myCL); iter.hasNext();) {

                Class clazz = (Class) iter.next();

                log.warn("======> " + clazz.getName() + " @ "

                         + clazz.getProtectionDomain().getCodeSource().getLocation());

            }

        } catch (NoSuchFieldException e) {

            e.printStackTrace();

        } catch (IllegalAccessException e) {

            e.printStackTrace();

        }

        myCL = myCL.getParent();

    }

}

Magit

kudu: magit-20170913.1424

JACK Audio/ALSA on Ubuntu

Most Linux distributions (except audio-oriented distros such as Ubuntu Studio) have active memory limits too restrictive to operate JACK.

First, try ulimit -l. If the output is not 'unlimited' you will need to make it so by changing some configurations.

1. Your login user must be part of the groups 'audio' and 'realtime'. Typically:

groupadd realtime
usermod -a -G audio <my user>
usermod -a -G realtime <my user>

2. Increase memory limits and I/O priority for the audio group. Edit /etc/security/limits.d/audio.conf and make sure it contains:

@audio   -  rtprio     95
@audio   -  memlock    unlimited

3. Give the JACK daemon high priority:

dpkg-reconfigure -p high jackd

4. Enable PAM limits. Edit /etc/pam.d/common-session and add:

session required pam_limits.so

5. Log out and log back in to activate the changes

6. Start JACK. Make sure your output device is correctly referenced by hw:<n>.

jackd -R -d alsa -d hw:1 -r 44100

To list local devices:

aplay -l

Verify configuration in your software of choice, e.g. Ardour, LMMS, Audacity.

Using Docker behind a proxy

Proxies directly affect three independent aspects of Docker's operation: 

1. Docker daemon: for interacting with public repositories (e.g. `docker pull`)
2. Image building: building images via the Docker client almost always requires remote artifacts (e.g. package managers, build tools, curl)
3. Container runtime: running containers requiring internet access

Each of these aspects requires particular measures to operate correctly behind a proxy. The following sections explain configuration procedures for each of them.

Docker daemon

Docker daemon requires network access for pulling and pushing images from/to public repositories. To operate behind a proxy, the daemon must receive proxy settings as described below (applies to CentOS and related distributions -- for others, see applicable documentation).

Despite some claims in StackOverflow and other sites, Docker does not accept proxy configs in `/etc/docker/daemon.json` (https://docs.docker.com/engine/admin/systemd/#httphttps-proxy). Proxy variables must be set in the systemd startup configs for the service.

To update the daemon's proxy configuration, edit/create `proxy.conf` with your proxy settings, reload the configuration and restart the daemon:

sudo mkdir /etc/systemd/system/docker.service.d # not present by default

cat <<EOF > /etc/systemd/system/docker.service.d/proxy.conf
[Service]
Environment="HTTP_PROXY=http://your.proxy:3128/" "HTTPS_PROXY=https://your.proxy:3128/" "NO_PROXY=localhost,127.0.0.1,docker-registry.yourdomain.net"
EOF

sudo systemctl daemon-reload && sudo systemctl restart docker

The changes above will allow the daemon to access public registries to perform pull operations to retrieve images.

Image building

Image builds specified in Dockerfiles virtually always contain access operations on public resources, be it through curl, yum, npm or similar tools. For these operations to work behind a proxy, the container's build context must provide correct proxy settings.

The simplest but least flexible option is to hardcode proxy settings in your Dockerfile:

ENV http_proxy http://<PROXY_HOST>:<PORT>
ENV https_proxy https://<PROXY_HOST>:<PORT>

Notice both HTTP and HTTPS values are specified to ensure both protocols are configured. Replace <PROXY_HOST> and <PORT> with your network's proxy settings, e.g. `https://proxy.my-domain:3128`.

Many Linux applications will honor the `http(s?)_proxy` variable (e.g. curl, wget, many http client libraries for Python, Ruby and other scripted languages). There are, however, exceptions and irregularities in this area. See the documentation for the particular tool you intend to use. The CentOS package manager, yum, for example, requires explicit configuration via `/etc/yum.conf`. To allow yum to access packages during image building, include the following entries in your Dockerfile before the first invocation of yum:

RUN echo "http_proxy=http://proxy.my-domain:3128" >> /etc/yum.conf
RUN echo "https_proxy=https://proxy.my-domain:3128" >> /etc/yum.conf
# Make sure YUM proxy is set up first
RUN yum update -y && yum install curl

 

Plotting 3D sets in Jupyter

%matplotlib inline
import matplotlib.pyplot as plt
from matplotlib import cm
from mpl_toolkits.mplot3d import Axes3D
import numpy as np

fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
S = 50

def f(x,y):
    return np.exp(- x**2 - y**2)

def plot(x, y, f):
    xx, yy = np.meshgrid(X,Y)
    Z = map(lambda x: map(lambda y: f(x,y), Y), X)
    p = ax.plot_surface(xx, yy, Z, rstride=1, cstride=1, cmap=cm.jet,
                        linewidth=0.01, antialiased=True, shade=True)

X = np.linspace(-2,2,S)
Y = np.linspace(-2,2,S)
plot(X, Y, f)

Orbiter 2016 - Delta Glider IV autopilot

ALT+1  PRO400SPEC01  Taxiing hold speed (maintain a speed of 10 m/s)
ALT+2  PRO400SPEC18  Aproach hold speed (maintain a speed of 180 m/s)
ALT+3  PRO400SPEC25  Flight hold speed (maintain a speed of 250 m/s)
ALT+4  PRO300SPEC0   Docking auto (Automatically dock your DGIV)
ALT+5  PRO110SPEC0   Atmospheric flight autopilot (heading, altitude &speed)
ALT+6  PRO903SPEC40  Earth ascent to low orbit. (normal take-off)
ALT+7  PR105SPEC40   Automatic reentry

PRO104SPEC40 Manual rentry (Hold attitude with selected AOA)
PRO105SPEC40 Auto reentry (keep a low temperature reentry profile)
PRO200SPEC7  Manual Hover
PRO200SPEC8  Auto Hover
PRO500SPEC0  Null relative speed in regard of a close object (in space only)
PRO904SPECnn Earth ascent autopilot with automatic hover take-off.
PRO905SPECnn Moon ascent autopilot with automatic hover take-off.
PRO906SPECnn Mars ascent autopilot with automatic hover take-off.

Launch Azimuth = arcsin (cos (desired_orbital_inclination) / cos (launch_ latitude))

KSC(28.591) to ISS (51.56) => 45.075°(PRO904SPEC45)

Merci beaucoup Dansteph.

Numpad thrusters (translation mode):

• 0: hover +
• .: hover -
• 1: left
• 2: up
• 3: right
• 4: (inactive)
• 5: killrot
• 6: forward
• 7: (inactive)
• 8: down
• 9: retro

Set a breakpoint in Jupyter (iPython Notebook)

from IPython.core.debugger import Tracer; Tracer()()

In the Spring Java framework, referencing autowired beans with session or request scope requires CGLIB proxies, e.g.:

@Scope(value = "request", proxyMode = ScopedProxyMode.TARGET_CLASS)

This in turn requires that unit and integration tests (e.g. JUnit or Spock) use
@WebAppConfiguration in their class declarations.

Get full command line of a running process

ps -ww -fp <pid>

Quick and dirty: one-line FTP server (Python)

sudo pip install pyftpdlib

python -m pyftpdlib -w #(deprecated) 

sudo python -m pyftpdlib.ftpserver

Fast port scanning with nmap

Use nmap to test for listening services on a range of ports

nmap -sS --reason -T4 -p32000-32100 10.10.10.10/32

Explanation of arguments

-sS: TCP SYN scan. Also known as half-open scan. It requires root privileges (see -sT if this is an issue). It is fast, enabling testing of large numbers of ports (assuming high network bandwidth and absence of rate-limiting firewalls). It is unobtrusive since it does not complete TCP connections: it sends a SYN packet and checks the response. It allows clear differentiation between listening (a SYN/ACK response), not listening (a RST/reset response) or filtered (no response after retries).

--reason: Shows details regarding why each port is reported in the given state

-T<0-5>: Timing template.  The respective, equivalent text flags are instructive: paranoid (0), sneaky (1), polite (2), normal (3), aggressive (4), insane (5). Higher is faster. The default is normal (3), but it is often too slow in practice for large port ranges. Higher values must be used cautiously, as they can either stress or crash target systems or easily trigger intrusion detection systems. I have found T4 to be practical for routine systems diagnostics work. T4 is the equivalent of --max-rtt-timeout 1250ms --min-rtt-timeout 100ms --initial-rtt-timeout 500ms --max-retries 6. When scanning many ports on a reliable network, I override maximum retries to accelerate the scan with --max-retries=0.

-p: Port range

Hosts: Target hosts, represented as IP addresses, with a given host mask. In the example, 10.10.10.10/32, a single host is requested. A mask shorter than 32 bits designates ranges, e.g. 10.10.10.0/24, includes 10.10.10.0 through 10.10.10.255

More jq magic for JSON wrangling

Count number of JSON elements in an embedded array

Data out of an Elasticsearch aggregation result set: 

{
  "aggregations": {
    "flightdestination": {
      "buckets": [
        {
          "sentiment": {
            "value": 2.1706734237483767
          },
          "doc_count": 6931,
          "key": "newark"
        },
        {
          "sentiment": {
            "value": 2.17875893247776
          },
          "doc_count": 6857,
          "key": "houston"

        },

...

curl -s 'http://10.60.35.34:9200/1$1_0/sentence/_search' -XGET --data "$(cat ~/query.json)" | jq '.aggregations.flightdestination.buckets | length'