tag:blogger.com,1999:blog-246241622024-02-22T02:18:14.045-08:00Allan's BlogAllan Riordan Boll's blogAllanhttp://www.blogger.com/profile/11850694461776134997noreply@blogger.comBlogger43125tag:blogger.com,1999:blog-24624162.post-53803683557171208312023-03-30T21:01:00.004-07:002023-03-30T21:05:53.585-07:00Peer-to-peer voice over IP call with pjsua<p>I was recently playing around with some voice over IP (VoIP) applications. SIP is a beast of a complex protocol, and I found surprisingly little good simple information on the web.</p>
<p>I found the pjsua CLI app to be the least bad SIP client that works on Linux. Unfortunately a prebuilt version isn't available in Ubuntu's repos as of writing. Luckily, it's pretty easy to compile pjsua:</p>
<pre name="code" class="bash">
# A few deps off the top of my head, but your configure run may reveal you need more...
sudo apt install build-essential libasound2-dev libssl-dev
git clone --depth 1 --branch 2.13 https://github.com/pjsip/pjproject.git
cd pjproject
./configure
make dep -j8
make -j8
# Copy the binary to a convenient location of your choice.
cp pjsip-apps/bin/pjsua-x86_64-unknown-linux-gnu ~/bin/pjsua
</pre>
<p>For fun, I wanted to make a secure peer-to-peer call between two machines, without any external registration server involved. Here's how I finally managed to do so:</p>
<pre name="code" class="bash">
# Listen for peer-to-peer calls on local ports UDP 5060, TCP 5060, TLS 5061.
openssl genrsa -out server.key 2048
openssl req -new -key server.key -subj "/CN=server" -x509 -days 3650 -out server.crt
pjsua \
--use-tls \
--tls-ca-file=server.crt \
--tls-cert-file=server.crt \
--tls-privkey-file=server.key \
--use-srtp=2
# Usage within the CLI app:
# "Half-answer" call (let the other side know that it's ringing "beep... beep...").
a
180
# Answer call (can be either done directly, or after a 180).
a
200
# Hang up.
h
# Show main menu at any time.
<enter>
# Dump status.
d
# Outgoing call to SIP address.
m
sip:1234@sip.example.com
# Connect peer-to-peer call with SIP meta data secure via TLS, and voice RTP stream secure via SRTP.
pjsua \
--use-tls \
--use-srtp=0 \
--id sip:a \
--outbound="sips:192.168.1.123:5061;transport=tls" \
sip:b
</pre>
<p>Some key parameters worth noting are:</p>
<ul>
<li>Enable support for TLS in the process. Without this you can neither initiate outgoing SIP over TLS connections, nor receive incomming SIP over TLS connection: <pre>--use-tls</pre></li>
<li>Secure the actual audio stream via SRTP (0=off, 1=optional, 2=required): <pre>--use-srtp=2</pre></li>
<li>For playing around in Wireshark use the simplest possible codec: <pre>--dis-codec="*" --add-codec=pcm</pre></li>
</ul>
<p>If you are not trying to do a peer-to-peer call, but want to connecto to an actual SIP provider such as antisip.com, then you are probably looking for something like the following:</p>
<pre name="code" class="bash">
pjsua \
--use-tls \
--use-srtp=2 \
--id sip:somebody@sip.antisip.com \
--registrar "sip:sip.antisip.com:9091;transport=tls" \
--realm sip.antisip.com \
--username somebody \
--password somepassword
# See "Usage within the CLI app" above.
</pre>
Allanhttp://www.blogger.com/profile/11850694461776134997noreply@blogger.com0tag:blogger.com,1999:blog-24624162.post-52265628849867855582021-08-03T11:13:00.002-07:002021-08-03T21:28:32.302-07:00Download OpenStreetMap bounding box PNG<p>For a project where I was plotting some locations using Matplotlib, I needed a way to get a PNG map from a bounding box lat-long pair. Here's a Python function I came up with.</p>
<pre name="code" class="python">
from os import listdir, mkdir
from os.path import exists
from PIL import Image
from random import uniform
from time import sleep
from urllib.request import Request, urlopen
import math
# Similar to https://wiki.openstreetmap.org/wiki/Slippy_map_tilenames#Python .
def deg2float(lat_deg, lon_deg, zoom):
lat_rad = math.radians(lat_deg)
n = 2.0 ** zoom
xtile = (lon_deg + 180.0) / 360.0 * n
ytile = (1.0 - math.asinh(math.tan(lat_rad)) / math.pi) / 2.0 * n
return (xtile, ytile)
def download_map(zoom, lat1, lon1, lat2, lon2):
lon_start, lon_end = min(lon1, lon2), max(lon1, lon2)
lat_start, lat_end = max(lat1, lat2), min(lat1, lat2)
# Top left corner of bounding box.
x1, y1 = deg2float(lat_start, lon_start, zoom)
x1i, y1i = math.floor(x1), math.floor(y1)
# Bottom right corner of bounding box.
x2, y2 = deg2float(lat_end, lon_end, zoom)
x2i, y2i = math.ceil(x2), math.ceil(y2)
x_cnt, y_cnt = abs(x1i - x2i), abs(y1i - y2i)
if x_cnt*y_cnt > 250:
err = "Too many tiles. Probably too big an area at too high a zoom level."
err += " See https://operations.osmfoundation.org/policies/tiles/ ."
raise Exception(err)
if not exists("maptiles"):
mkdir("maptiles")
for x in range(x_cnt):
for y in range(y_cnt):
xt, yt = x + x1i, y + y1i
path = "maptiles/{}_{}_{}.png".format(zoom, xt, yt)
if not exists(path):
sleep(uniform(0.5, 1.5))
url = "https://tile.openstreetmap.org/{}/{}/{}.png".format(zoom, xt, yt)
print("Downloading tile {}".format(url))
req = Request(url)
ua = "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:90.0) Gecko/20100101 Firefox/90.0"
req.add_header("User-Agent", ua) # OSM seems to not like Python's default UA.
resp = urlopen(req)
body = resp.read()
with open(path, "wb") as f:
f.write(body)
im = Image.open("maptiles/{}_{}_{}.png".format(zoom, x1i, y1i))
tile_w, tile_h = im.size
total_w = x_cnt*tile_w
total_h = y_cnt*tile_h
new_im = Image.new("RGB", (total_w, total_h))
for x in range(x_cnt):
for y in range(y_cnt):
xt, yt = x + x1i, y + y1i
im = Image.open("maptiles/{}_{}_{}.png".format(zoom, xt, yt))
new_im.paste(im, (x*tile_w, y*tile_h))
cropped_w = round((x2 - x1)*tile_w)
cropped_h = round((y2 - y1)*tile_h)
cropped_im = Image.new("RGB", (cropped_w, cropped_h))
translate_x = round(-(x1 - x1i)*tile_w)
translate_y = round(-(y1 - y1i)*tile_h)
cropped_im.paste(new_im, (translate_x, translate_y))
cropped_im.save("map.png")
# Download a map of the SF Bay Area at zoom level 12. Approx 3000*3000px.
download_map(12, 38, -122.7, 37.2, -121.7)
</pre>
Allanhttp://www.blogger.com/profile/11850694461776134997noreply@blogger.com0tag:blogger.com,1999:blog-24624162.post-85426180143262939692021-04-27T00:45:00.001-07:002021-04-27T14:08:49.917-07:00Geohash Python example<p>Here's a simple Geohash encode/decode implementation. The decoding function pretty closely follows the technique layed out on the <a href+"https://en.wikipedia.org/wiki/Geohash">Geohash Wikipedia entry</a>.</p>
<pre name="code" class="python">
from fractions import Fraction
"""
First decode the special geohash variant of base32 encoding.
Each encoded digit (0-9-b..z) (not continuous abc) is a 5 bit val 0,1,2...,30,31.
In the resulting bitstream, every second bit is now for latitude and longtitude.
Initially the latitutde range is -90,+90.
When a latitude bit is 1, then it now starts at the mid of these.
Else if 0 it now ends at the mid of these.
Same for longtitude but with range -180,+180.
"""
def decode_geohash(s):
alphabet_32ghs = "0123456789bcdefghjkmnpqrstuvwxyz"
dec_from_32ghs = dict()
for i, c in enumerate(alphabet_32ghs):
dec_from_32ghs[c] = i
bits = 0 # Integer representation of hash.
bit_cnt = 0
for c in s:
bits = (bits << 5) | dec_from_32ghs[c]
bit_cnt += 5
# Every second bit is longtitude and latitude. Digits in even positions are latitude.
lat_bits, lon_bits = 0, 0
lat_bit_cnt = bit_cnt // 2
lon_bit_cnt = lat_bit_cnt
if bit_cnt % 2 == 1:
lon_bit_cnt += 1
for i in range(bit_cnt):
cur_bit_pos = bit_cnt - i
cur_bit = (bits & (1 << cur_bit_pos)) >> cur_bit_pos
if i % 2 == 0:
lat_bits |= cur_bit << (cur_bit_pos//2)
else:
lon_bits |= cur_bit << (cur_bit_pos//2)
lat_start, lat_end = Fraction(-90), Fraction(90)
for cur_bit_pos in range(lat_bit_cnt-1, -1, -1):
mid = (lat_start + lat_end) / 2
if lat_bits & (1 << cur_bit_pos):
lat_start = mid
else:
lat_end = mid
lon_start, lon_end = Fraction(-180), Fraction(180)
for cur_bit_pos in range(lon_bit_cnt-1, -1, -1):
mid = (lon_start + lon_end) / 2
if lon_bits & (1 << cur_bit_pos):
lon_start = mid
else:
lon_end = mid
return float(lat_start), float(lat_end), float(lon_start), float(lon_end)
# Inspired by https://www.factual.com/blog/how-geohashes-work/
def encode_geohash(lat, lon, bit_cnt):
if bit_cnt % 5 != 0:
raise ValueError("bit_cnt must be divisible by 5")
bits = 0
lat_start, lat_end = Fraction(-90), Fraction(90)
lon_start, lon_end = Fraction(-180), Fraction(180)
for i in range(bit_cnt):
if i % 2 == 0:
mid = (lon_start + lon_end) / 2
if lon < mid:
bits = (bits << 1) | 0
lon_end = mid
else:
bits = (bits << 1) | 1
lon_start = mid
else:
mid = (lat_start + lat_end) / 2
if lat < mid:
bits = (bits << 1) | 0
lat_end = mid
else:
bits = (bits << 1) | 1
lat_start = mid
print("bits: {:>b}".format(bits))
# Do the special geohash base32 encoding.
s = ""
alphabet_32ghs = "0123456789bcdefghjkmnpqrstuvwxyz"
for i in range(bit_cnt // 5):
idx = (bits >> i*5) & (1 | 2 | 4 | 8 | 16)
s += alphabet_32ghs[idx]
return s[::-1]
print(decode_geohash("ezs42"))
print(decode_geohash("9q8y"))
print(encode_geohash(37.7, -122.5, 20))
</pre>
Allanhttp://www.blogger.com/profile/11850694461776134997noreply@blogger.com0tag:blogger.com,1999:blog-24624162.post-50037207594973660202021-04-25T14:48:00.003-07:002021-04-25T19:00:52.165-07:00Lossy Counting Algorithm Python example<p>This is an implementation of the Lossy Counting Algorithm described in <a href="http://www.mathcs.emory.edu/~cheung/Courses/584/Syllabus/papers/SlidingWindows/2002-Appr-freq-counts.pdf">Manku and Motwani's 2002 paper "Approximate Frequency Counts over Data Streams"</a>.</p>
<p>It is an algorithm for estimate elements in a stream whose frequency count exceeds a threshold, while using only limited memory. For example for video view counts on something like YouTube, finding which videos that each constitute more than 3% of the views.</p>
<p>Please let me know if you spot any bugs.</p>
<pre name="code" class="python">
from math import ceil
class LossyCount:
def __init__(self, max_error=0.005): # max_error is the parameter they call epsilon in the paper.
self.max_error = max_error
self.bucket_width = ceil(1/max_error)
self.entries = dict()
self.n = 0
def put(self, x):
self.n += 1
current_bucket = ceil(self.n / self.bucket_width)
freq, delta = 1, current_bucket-1
if x in self.entries:
freq, delta = self.entries[x]
freq += 1
self.entries[x] = (freq, delta)
# If at bucket boundary then prune low frequency entries.
if self.n % self.bucket_width == 0:
prune = []
for key in self.entries:
freq, delta = self.entries[key]
if freq + delta <= current_bucket:
prune.append(key)
for key in prune:
del self.entries[key]
def get(self, support_threshold=0.001): # support_threshold is the parameter they call s in the paper.
res = []
for key in self.entries:
freq, delta = self.entries[key]
if freq >= (support_threshold - self.max_error)*self.n:
res.append(key)
return res
# Generate test data.
from math import log
from random import random, randint
view_cnt = 500000
videos_cnt = 100000
x = [random() for _ in range(view_cnt)]
# y = [1/v for v in x] # This distribution is too steep...
y = [(1/v)*0.01 -log(v) for v in x] # A distribution that is reasonably steep and has a very long tail.
m = max(y)
y = [v/m for v in y]
# ids = [i for i in range(videos_cnt)] # Easy to read IDs, but unrealistic. Most popular video will have ID 0, second most popular ID 1, etc.
ids = [randint(1000000, 9000000) for _ in range(videos_cnt)] # More realistic video IDs.
idxs = [int(v*(videos_cnt-1)) for v in y]
views = [ids[v] for v in idxs]
# import matplotlib.pyplot as plt
# plt.hist(views, bins=200) # Only works when the IDs are 1,2,3,4...
# plt.show()
threshold = 0.03 # We are interested in videos that each constitute more than 3% of the views.
# Generate exact results using a counter, to compare with.
from collections import Counter
c = Counter(views)
r = []
for k in c:
r.append((c[k], k))
r2 = []
for cnt, id in r:
if cnt >= view_cnt*threshold:
r2.append(id)
print(sorted(r2))
# Test the LossyCount class. Should give similar (but not exact) results to the above.
lc = LossyCount()
for v in views:
lc.put(v)
print(sorted(lc.get(threshold)))
</pre>
Allanhttp://www.blogger.com/profile/11850694461776134997noreply@blogger.com0tag:blogger.com,1999:blog-24624162.post-3606348153963299522021-02-12T23:34:00.002-08:002021-02-13T00:17:00.201-08:00Create a local Kubernetes cluster and deploy your code end to endThese are the commands accompanying the video tutorial I've made on how to create a local Kubernetes cluster and deploy a simple stateless app from code, end to end. You can find the video tutorial here: <a href="https://youtu.be/MZr9Ls38uPw">https://youtu.be/MZr9Ls38uPw</a> .
<pre class="code">
#
# Install k3s
#
curl -sfL https://get.k3s.io | K3S_CLUSTER_INIT=1 INSTALL_K3S_EXEC="--disable=servicelb" sh -
cat /var/lib/rancher/k3s/server/node-token
# Run on other nodes to join the cluster
curl -sfL https://get.k3s.io | \
INSTALL_K3S_EXEC=server \
K3S_URL=https://192.168.50.203:6443 \
K3S_TOKEN=... \
sh -
kubectl get nodes --watch
journalctl --unit=k3s
kubectl get all --all-namespaces
#
# Install dashboard
#
GITHUB_URL=https://github.com/kubernetes/dashboard/releases
VERSION_KUBE_DASHBOARD=$(curl -w '%{url_effective}' -I -L -s -S ${GITHUB_URL}/latest -o /dev/null | sed -e 's|.*/||')
echo $VERSION_KUBE_DASHBOARD
kubectl create -f https://raw.githubusercontent.com/kubernetes/dashboard/${VERSION_KUBE_DASHBOARD}/aio/deploy/recommended.yaml
cat>dashboard.admin-user.yml<<"EOF"
apiVersion: v1
kind: ServiceAccount
metadata:
name: admin-user
namespace: kubernetes-dashboard
EOF
cat>dashboard.admin-user-role.yml<<"EOF"
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: admin-user
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: admin-user
namespace: kubernetes-dashboard
EOF
kubectl create -f dashboard.admin-user.yml -f dashboard.admin-user-role.yml
kubectl get all --all-namespaces
# Connect to node again, but with your port 8001 forwarded.
ssh -L 8001:127.0.0.1:8001 user1@ubuntutest1
kubectl proxy
# Open in browser: http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/
kubectl --namespace kubernetes-dashboard describe secret admin-user-token | grep ^token
#
# Install MetalLB
#
GITHUB_URL=https://github.com/metallb/metallb/releases
VERSION=$(curl -w '%{url_effective}' -I -L -s -S ${GITHUB_URL}/latest -o /dev/null | sed -e 's|.*/||')
echo $VERSION
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/$VERSION/manifests/namespace.yaml
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/$VERSION/manifests/metallb.yaml
kubectl create secret generic -n metallb-system memberlist --from-literal=secretkey="$(openssl rand -base64 128)"
cat>metallb-configmap.yml<<"EOF"
apiVersion: v1
kind: ConfigMap
metadata:
namespace: metallb-system
name: config
data:
config: |
address-pools:
- name: default
protocol: layer2
addresses:
- 192.168.50.20-192.168.50.40
EOF
kubectl apply -f metallb-configmap.yml
# Check logs for errors
kubectl logs -lapp=metallb --namespace metallb-system --all-containers=true --prefix -f
#
# Install docker registry
#
apt-get install docker-compose
docker run -d -p 5000:5000 --restart=always --name registry registry
docker ps -a
# On each node:
mkdir -p /etc/rancher/k3s/
cat>/etc/rancher/k3s/registries.yaml<<"EOF"
mirrors:
"ubuntutest1:5000":
endpoint:
- "http://ubuntutest1:5000"
EOF
systemctl restart k3s
#
# Create containerized app
#
mkdir -p /home/user1/dev/myapp1
cd /home/user1/dev/myapp1
cat>main.go<<"EOF"
package main
import (
"github.com/gorilla/mux"
"html/template"
"log"
"net/http"
"os"
"time"
)
var helloTemplate, _ = template.New("").Parse(`<!DOCTYPE html>
<html>
<head><title>testapp</title></head>
<body>
<h1>Test 1</h1>
<p>Now: {{.now.Format "2006-01-02 15:04:05" }}</p>
<p>Served from node: {{ .node }}</p>
<p>Served from pod: {{ .pod }}</p>
</body>
</html>
`)
type Msg struct {
Ts time.Time `json:"ts"`
}
func helloGet(w http.ResponseWriter, r *http.Request) {
v := map[string]interface{}{
"now": time.Now(),
"node": os.Getenv("NODE_NAME"),
"pod": os.Getenv("POD_NAME"),
}
helloTemplate.Execute(w, v)
}
func main() {
log.Println("Starting")
router := mux.NewRouter()
router.HandleFunc("/", helloGet).Methods("GET")
handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
log.Println(r.Method + " " + r.URL.String())
router.ServeHTTP(w, r)
})
log.Fatal(http.ListenAndServe(":8080", handler))
}
EOF
cat>go.mod<<"EOF"
module myapp1
EOF
touch go.sum
cat>Dockerfile<<"EOF"
FROM golang as builder
WORKDIR /app
COPY . ./
RUN go mod download
RUN CGO_ENABLED=0 GOOS=linux go build -v -o server
FROM alpine
RUN apk add --no-cache ca-certificates
COPY --from=builder /app/server /server
CMD ["/server"]
EOF
docker build -t "myapp1" .
docker run -it --rm -p 8080:8080 myapp1
docker tag myapp1 ubuntutest1:5000/myapp1:v1
docker push ubuntutest1:5000/myapp1:v1
curl -X GET ubuntutest1:5000/v2/_catalog
curl -X GET ubuntutest1:5000/v2/myapp1/tags/list
#
# Create and deploy our app in k8s
#
mkdir -p /home/user1/dev/myapp1/k8s
cd /home/user1/dev/myapp1
cat>k8s/deployment.yml<<"EOF"
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp1-deployment
spec:
selector:
matchLabels:
app: myapp1
replicas: 3
template:
metadata:
labels:
app: myapp1
spec:
containers:
- name: myapp1
image: ubuntutest1:5000/myapp1:v1
ports:
- containerPort: 8080
env:
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
topologySpreadConstraints:
- maxSkew: 1
topologyKey: "kubernetes.io/hostname"
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
app: myapp1
EOF
kubectl apply -f k8s/deployment.yml
cat>k8s/service.yml<<"EOF"
kind: Service
apiVersion: v1
metadata:
name: myapp1-service
spec:
type: ClusterIP
selector:
app: myapp1
ports:
- name: http-myapp1
protocol: TCP
port: 8080
EOF
kubectl apply -f k8s/service.yml
# Expose app via HTTP proxy (aka. "ingress")
cat>k8s/ingress.yml<<"EOF"
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: myapp1-ingress
spec:
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: myapp1-service
port:
number: 8080
EOF
kubectl apply -f k8s/ingress.yml
kubectl logs -lapp=myapp1 --all-containers=true --prefix -f
#
# Upgrading to a new image. Either use this command or update the same field in deployment.yml.
#
cd /home/user1/dev/myapp1
docker build -t "myapp1" .
docker tag myapp1 ubuntutest1:5000/myapp1:v2
docker push ubuntutest1:5000/myapp1:v2
kubectl set image deployments/myapp1-deployment myapp1=ubuntutest1:5000/myapp1:v2
kubectl rollout status deployments/myapp1-deployment
# Rolling back (calling repeatedly switches between two newest versions).
kubectl rollout undo deployments/myapp1-deployment
#
# Testing HA
#
kubectl get nodes ; kubectl get pod -o=custom-columns=NAME:.metadata.name,STATUS:.status.phase,NODE:.spec.nodeName --all-namespaces
kubectl drain ubuntutest1 --ignore-daemonsets --delete-emptydir-data
kubectl uncordon ubuntutest1
</pre>
Allanhttp://www.blogger.com/profile/11850694461776134997noreply@blogger.com0tag:blogger.com,1999:blog-24624162.post-14613908181818203602021-02-07T18:59:00.005-08:002021-02-07T19:00:21.657-08:00k3s and MetalLB "destination unreachable" issue<p>I was trying out MetalLB with a bare-metal Kubernetes cluster (using the k3s distro), and was hitting issues with requests to the cluster IP seemingly randomly giving me "destination unreachable". I noticed with Wireshark that I was getting a ton of "gratuitous" ARP packets. Every few seconds.</p>
<p>I inspected MetalLB's logs with the following command:</p>
<pre class="code">
kubectl logs -lapp=metallb --namespace metallb-system --all-containers=true --prefix -f
</pre>
<p>It was logging the message "IP allocated by controller not allowed by config" every few seconds.</p>
<p>Turned out to be k3s's builtin internal load balancer that was interferring with MetalLB. Disabling it with the "--disable=servicelb" flag during k3s's installation fixed the issue.</p>
<pre class="code">
curl -sfL https://get.k3s.io | K3S_CLUSTER_INIT=1 INSTALL_K3S_EXEC="--disable=servicelb" sh -
</pre>
Allanhttp://www.blogger.com/profile/11850694461776134997noreply@blogger.com0tag:blogger.com,1999:blog-24624162.post-41078956750979332892020-07-13T16:46:00.002-07:002020-07-13T16:46:39.030-07:00A "super ping" command to run while diagnosing network issues<p>Here's a bash function for your bash profile that continuously prints some basic checks I usually do while diagnosing network issues:</p>
<ul>
<li>Show my LAN IPs</li>
<li>Show the default gateway IP</li>
<li>Ping the default gateway IP</li>
<li>Ping 8.8.8.8 (Google's public DNS servers)</li>
<li>DNS resolve amazon.com</li>
<li>Ping amazon.com</li>
<li>Get my WAN IP by curl'ing ifconfig.me</li>
</ul>
<p>Tested on Ubuntu 20.04 and MacOS Catalina</p>
<pre class="code">
function pingg() {
while true; do
printf "%s " $(TZ=UTC date "+%Y-%m-%dT%H:%M:%S")
if [[ "$OSTYPE" == "darwin"* ]]; then
lanIps=$(ifconfig | grep "inet " | grep -v 127.0.0.1 | awk '{print $2}' | paste -d, -s -)
gw=$(netstat -rn -f inet | grep default | awk '{print $2}')
else
lanIps=$(ip addr | grep "inet " | grep -v 127.0.0.1 | awk '{print $2}' | paste -d, -s -)
gw=$(ip route | grep default | awk '{print $3}')
fi
printf "lanIp=%-15s " $lanIps
printf "gw=%-15s " $gw
pingGw="-------"
if [ "$gw" != "" ]; then
pingGw=$(ping -W1 -c1 $gw 2>/dev/null | tail -1 | grep -v "0 packets received" | cut -d '=' -f 2 | cut -d '/' -f 2)
if [ "$pingGw" == "" ]; then
pingGw="-------"
fi
fi
printf "pGw=%-7s " $pingGw
ping8888=$(ping -W1 -c1 8.8.8.8 2>/dev/null | tail -1 | grep -v "0 packets received" | cut -d '=' -f 2 | cut -d '/' -f 2)
if [ "$ping8888" == "" ]; then
ping8888="-------"
fi
printf "p8=%-7s " $ping8888
dnsAmzn=$(host -4 -W 1 -t a amazon.com | head -n1 | grep "has address" | awk '{print $4}')
if [ "$dnsAmzn" == "" ]; then
dnsAmzn="-------"
fi
printf "dnsAmz=%-15s " $dnsAmzn
pingAmzn="-------"
if [ "$dnsAmzn" != "-------" ]; then
pingAmzn=$(ping -W1 -c1 $dnsAmzn 2>/dev/null | tail -1 | grep -v "0 packets received" | cut -d '=' -f 2 | cut -d '/' -f 2)
if [ "$pingAmzn" == "" ]; then
pingAmzn="-------"
fi
fi
printf "pAmz=%-7s " $pingAmzn
wanIp="-------"
dnsIfconfigMe=$(host -4 -W 1 -t a ifconfig.me | head -n1 | grep "has address" | awk '{print $4}')
if [ "$dnsIfconfigMe" != "" ]; then
wanIp=$(curl -s --max-time 2 --connect-timeout 2 -H "Host: ifconfig.me" $dnsIfconfigMe | head -n 1 | awk '{print $1'})
if [ "$wanIp" == "" ]; then
wanIp="-------"
fi
fi
printf "wanIp=%-15s " $wanIp
printf "\n"
sleep 0.7
done
}
</pre>
<p>Example output:</p>
<pre class="code">
2020-07-13T23:43:59 lanIp=192.168.1.6/24 gw=192.168.1.1 pGw=0.264 p8=53.338 dnsAmz=176.32.98.166 pAmz=127.532 wanIp=1.2.3.4
2020-07-13T23:44:01 lanIp=192.168.1.6/24 gw=192.168.1.1 pGw=0.190 p8=47.107 dnsAmz=176.32.103.205 pAmz=111.309 wanIp=1.2.3.4
2020-07-13T23:44:02 lanIp=192.168.1.6/24 gw=192.168.1.1 pGw=0.290 p8=------- dnsAmz=176.32.103.205 pAmz=118.834 wanIp=1.2.3.4
2020-07-13T23:44:04 lanIp=192.168.1.6/24 gw=192.168.1.1 pGw=0.328 p8=143.387 dnsAmz=176.32.103.205 pAmz=135.333 wanIp=1.2.3.4
2020-07-13T23:44:06 lanIp=192.168.1.6/24 gw=192.168.1.1 pGw=0.313 p8=17.944 dnsAmz=176.32.103.205 pAmz=105.977 wanIp=1.2.3.4
2020-07-13T23:44:07 lanIp=192.168.1.6/24 gw=192.168.1.1 pGw=0.223 p8=22.505 dnsAmz=176.32.103.205 pAmz=97.623 wanIp=1.2.3.4
2020-07-13T23:44:08 lanIp=192.168.1.6/24 gw=192.168.1.1 pGw=0.250 p8=23.917 dnsAmz=176.32.103.205 pAmz=98.403 wanIp=1.2.3.4
2020-07-13T23:44:09 lanIp=192.168.1.6/24 gw=192.168.1.1 pGw=0.220 p8=41.262 dnsAmz=176.32.103.205 pAmz=100.828 wanIp=1.2.3.4
</pre>
Allanhttp://www.blogger.com/profile/11850694461776134997noreply@blogger.com0tag:blogger.com,1999:blog-24624162.post-69850805041967638252020-06-23T00:32:00.002-07:002020-06-23T00:33:48.411-07:00Upgrading Alpine Linux diskless setup on a Raspberry Pi<p>Here's a procedure I've come up with to fully upgrade a diskless setup of Alpine Linux on a running Raspberry Pi. It's not pretty, but seems gets the job done, based on the tests I did.</p>
<p>I tested this procedure on each version from 3.3 to to 3.12. The system I tested on had a working setup of Apache2 with PHP, as well as Samba. Here's the result of the testing.</p>
<ul>
<li>Started with v3.3.</li>
<li>Confirmed 3.3 to 3.4. Package "php-apache2" package had disappeared and had to be manually replaced by "php5-apache2".</li>
<li>Confirmed 3.4 to 3.5.</li>
<li>Confirmed 3.5 to 3.6.</li>
<li>Confirmed 3.6 to 3.7.</li>
<li>Confirmed 3.7 to 3.8.</li>
<li>Could not boot after upgrading from 3.8 to 3.9. But a fresh installation of 3.9 was also unbootable, so assuming 3.9 is broken. Seems to be a <a href="https://gitlab.alpinelinux.org/alpine/aports/-/issues/10155">known issue</a>.</li>
<li>Confirmed 3.8 to 3.10. Package "php5-apache2" package had disappeared and had to be manually replaced by "php7-apache2".</li>
</ul>
<p>The process I came up with can be summarized to:</p>
<ol>
<li>Remount the SD card in read-write mode.</li>
<li>Move all files in the SD card FAT32 root to a subdirectory such as old/2020-06-23T04_22_54.</li>
<li>Download a fresh alpine-rpi-x.y.z-armhf.tar.gz file and untar it to the SD card root. This will give you Alpine Linux's newest Raspberry compatible Linux kernel image.</li>
<li>Remount the SD card back to read-only mode.</li>
<li>Update the /etc/apk/repositories (in the in-memory file system) to point to the target version.</li>
<li>Do an "apk upgrade --available" to download the updated versions of the packages you used, so they will be ready on the device after rebooting.</li>
<li>Reboot into the new setup.</li>
</ol>
<p>If anything goes wrong, you can put the SD card in another PC, delete the broken setup, and copy your old working setup back out from the "old" directory.</p>
<p>Here are the commands. As I don't know if this process will keep working in future versions, I recommend running them one line at a time and ensuring the line does what you expect.</p>
<pre class="code">
targetVersion=v3.12
# Move everything to a safe place as it is now.
mount -oremount,rw /media/mmcblk0p1
cd /media/mmcblk0p1
olddir=/media/mmcblk0p1/old/$(date --utc "+%Y-%m-%dT%H_%M_%S")
mkdir -p $olddir
mv * $olddir # This will give a warning that it cannot move old into old. That's ok.
mv .* $olddir # This will give a warning that it cannot move . and .. . That's ok.
# Download fresh Alpine Linux image and replace existing.
newImgFile=$(curl http://dl-cdn.alpinelinux.org/alpine/$targetVersion/releases/armhf/latest-releases.yaml --silent | grep "file: alpine-rpi-" | sed "s/ *file: //g")
echo $newImgFile
mount -oremount,rw /media/mmcblk0p1
wget http://dl-cdn.alpinelinux.org/alpine/$targetVersion/releases/armhf/$newImgFile
wget http://dl-cdn.alpinelinux.org/alpine/$targetVersion/releases/armhf/${newImgFile}.sha512
sha512sum -c ${newImgFile}.sha512 # Ensure it says OK
tar -xzf $newImgFile
rm ${newImgFile}*
cp $olddir/usercfg.txt .
mount -oremount,ro /media/mmcblk0p1
# Update packages to the target distro.
cat > /etc/apk/repositories <<EOF
/media/mmcblk0p1/apks
http://dl-cdn.alpinelinux.org/alpine/$targetVersion/main
http://dl-cdn.alpinelinux.org/alpine/$targetVersion/community
EOF
setup-apkcache /media/mmcblk0p1/cache
apk update
apk upgrade --available
apk add
apk cache download
apk -v cache clean
lbu commit -d
reboot
# After reboot, ensure there aren't packages that no longer exist in this version.
apk add
</pre>
Allanhttp://www.blogger.com/profile/11850694461776134997noreply@blogger.com0tag:blogger.com,1999:blog-24624162.post-89726675023059201002020-06-21T13:10:00.012-07:002021-04-07T13:54:40.845-07:00Add shortcut keys to Google Drawing<p>Here's a quick and dirty way to add shortcut keys to Google Drawing. It probably only works in Chrome.</p>
<p>It's a JavaScript snippet that can be run in the context of the Google Drawing page in order to automate clicking on the toolbar buttons.<p>
<p>The keys this code adds are:<p>
<ul>
<li>Alt-A: Arrow</li>
<li>Alt-L: Line</li>
<li>Alt-B: Rectangle (box)</li>
<li>Alt-C: Cylinder</li>
<li>Alt-X: Text box</li>
</ul>
<p>Here's a bookmarklet with the script: <a href="javascript:(function()%7Bfunction%20newMouseEvent(eventType%2C%20x%2C%20y)%20%7Breturn%20new%20MouseEvent(eventType%2C%20%7B%22view%22%3A%20window%2C%22bubbles%22%3A%20true%2C%22cancelable%22%3A%20true%2C%22screenX%22%3A%20x%2C%22screenY%22%3A%20y%7D)%3B%7Dfunction%20myclick(x%2C%20y)%20%7Bvar%20el%20%3D%20document.elementFromPoint(x%2C%20y)%3Bel.dispatchEvent(newMouseEvent(%22mouseover%22%2C%20x%2C%20y))%3Bel.dispatchEvent(newMouseEvent(%22mousedown%22%2C%20x%2C%20y))%3Bel.dispatchEvent(newMouseEvent(%22mouseup%22%2C%20x%2C%20y))%3Bel.dispatchEvent(newMouseEvent(%22click%22%2C%20x%2C%20y))%3B%7Dfunction%20clickLineButton()%20%7Bvar%20r%20%3D%20document.getElementById(%22lineMenuButton%22).getBoundingClientRect()%3Bmyclick(r.x%2B3%2C%20r.y%2B3)%3BsetTimeout(function()%20%7Bmyclick(r.x%2B3%2C%20r.y%2B50)%3B%7D%2C%20200)%3B%7Dfunction%20clickArrowButton()%20%7Bvar%20r%20%3D%20document.getElementById(%22lineMenuButton%22).getBoundingClientRect()%3Bmyclick(r.x%2B3%2C%20r.y%2B3)%3BsetTimeout(function()%20%7Bmyclick(r.x%2B3%2C%20r.y%2B80)%3B%7D%2C%20200)%3B%7Dfunction%20clickBoxButton()%20%7Bvar%20r%20%3D%20document.getElementById(%22shapeButton%22).getBoundingClientRect()%3Bmyclick(r.x%2B3%2C%20r.y%2B3)%3BsetTimeout(function()%20%7Bmyclick(r.x%2B3%2C%20r.y%2B40)%3BsetTimeout(function()%20%7Bmyclick(r.x%2B172%2C%20r.y%2B56)%3B%7D%2C%20200)%3B%7D%2C%20200)%3B%7Dfunction%20clickCylinderButton()%20%7Bvar%20r%20%3D%20document.getElementById(%22shapeButton%22).getBoundingClientRect()%3Bmyclick(r.x%2B3%2C%20r.y%2B3)%3BsetTimeout(function()%20%7Bmyclick(r.x%2B3%2C%20r.y%2B40)%3BsetTimeout(function()%20%7Bmyclick(r.x%2B197%2C%20r.y%2B223)%3B%7D%2C%20200)%3B%7D%2C%20200)%3B%7Dfunction%20clickTextboxButton()%20%7Bvar%20r%20%3D%20document.getElementById(%22textboxButton%22).getBoundingClientRect()%3Bmyclick(r.x%2B3%2C%20r.y%2B3)%3B%7Dfunction%20myKeydownHandler(e)%20%7B%2F*%20A%20key%20*%2Fif%20(e.keyCode%20%3D%3D%2065%20%26%26%20e.altKey%20%3D%3D%20true)%20%7BclickArrowButton()%3Be.preventDefault()%3B%7D%2F*%20L%20key%20*%2Fif%20(e.keyCode%20%3D%3D%2076%20%26%26%20e.altKey%20%3D%3D%20true)%20%7BclickLineButton()%3Be.preventDefault()%3B%7D%2F*%20B%20key%20*%2Fif%20(e.keyCode%20%3D%3D%2066%20%26%26%20e.altKey%20%3D%3D%20true)%20%7BclickBoxButton()%3Be.preventDefault()%3B%7D%2F*%20C%20key%20*%2Fif%20(e.keyCode%20%3D%3D%2067%20%26%26%20e.altKey%20%3D%3D%20true)%20%7BclickCylinderButton()%3Be.preventDefault()%3B%7D%2F*%20X%20key%20*%2Fif%20(e.keyCode%20%3D%3D%2088%20%26%26%20e.altKey%20%3D%3D%20true)%20%7BclickTextboxButton()%3Be.preventDefault()%3B%7D%7Dwindow.addEventListener(%22keydown%22%2C%20myKeydownHandler%2C%20true)%3Bvar%20elements%20%3D%20document.getElementsByTagName(%22iframe%22)%3Bfor%20(var%20i%20%3D%200%3B%20i%20%3C%20elements.length%3B%20i%2B%2B)%20%7Bif%20(elements%5Bi%5D.src%20%3D%3D%20%22%22%20%7C%7C%20elements%5Bi%5D.src%20%3D%3D%20%22about%3Ablank%22)%20%7Belements%5Bi%5D.contentWindow.addEventListener(%22keydown%22%2C%20myKeydownHandler%2C%20true)%3B%7D%7D%7D)()">GDraw shortcuts</a>. <b>Drag this hyperlink to your bookmarks bar and click it when you are inside Google Drawing in order to enable the shortcut keys</b>. (Thanks to Peter Coles for his bookmarklet creation <a href="https://mrcoles.com/bookmarklet/">tool</a>)</p>
<p>And here's the code, in case you want to extend it with additional shortcut keys.</p>
<pre class="code">
function newMouseEvent(eventType, x, y) {
return new MouseEvent(eventType, {
"view": window,
"bubbles": true,
"cancelable": true,
"screenX": x,
"screenY": y
});
}
function myclick(x, y) {
var el = document.elementFromPoint(x, y);
el.dispatchEvent(newMouseEvent("mouseover", x, y));
el.dispatchEvent(newMouseEvent("mousedown", x, y));
el.dispatchEvent(newMouseEvent("mouseup", x, y));
el.dispatchEvent(newMouseEvent("click", x, y));
}
function clickLineButton() {
var r = document.getElementById("lineMenuButton").getBoundingClientRect();
myclick(r.x+3, r.y+3);
setTimeout(function() {
myclick(r.x+3, r.y+50);
}, 200);
}
function clickArrowButton() {
var r = document.getElementById("lineMenuButton").getBoundingClientRect();
myclick(r.x+3, r.y+3);
setTimeout(function() {
myclick(r.x+3, r.y+80);
}, 200);
}
function clickBoxButton() {
var r = document.getElementById("shapeButton").getBoundingClientRect();
myclick(r.x+3, r.y+3);
setTimeout(function() {
myclick(r.x+3, r.y+40);
setTimeout(function() {
myclick(r.x+172, r.y+56);
}, 200);
}, 200);
}
function clickCylinderButton() {
var r = document.getElementById("shapeButton").getBoundingClientRect();
myclick(r.x+3, r.y+3);
setTimeout(function() {
myclick(r.x+3, r.y+40);
setTimeout(function() {
myclick(r.x+197, r.y+223);
}, 200);
}, 200);
}
function myKeydownHandler(e) {
/* A key */
if (e.keyCode == 65 && e.altKey == true) {
clickArrowButton();
e.preventDefault();
}
/* L key */
if (e.keyCode == 76 && e.altKey == true) {
clickLineButton();
e.preventDefault();
}
/* B key */
if (e.keyCode == 66 && e.altKey == true) {
clickBoxButton();
e.preventDefault();
}
/* C key */
if (e.keyCode == 67 && e.altKey == true) {
clickCylinderButton();
e.preventDefault();
}
}
window.addEventListener("keydown", myKeydownHandler, true);
var elements = document.getElementsByTagName("iframe");
for (var i = 0; i < elements.length; i++) {
if (elements[i].src == "" || elements[i].src == "about:blank") {
elements[i].contentWindow.addEventListener("keydown", myKeydownHandler, true);
}
}
</pre>
Allanhttp://www.blogger.com/profile/11850694461776134997noreply@blogger.com0tag:blogger.com,1999:blog-24624162.post-65316997357295527872020-06-18T22:54:00.001-07:002020-06-18T22:57:21.092-07:00Reach servers behind NAT with WireGuard and a VPS, while preserving source IP<p>This post describes an approach to let your Linux machine behind a NAT be reachable from the public internet, using a cheap VPS and Wireguard.</p>
<p>It's similar to <a href="https://golb.hplar.ch/2019/01/expose-server-vpn.html">this post in Ralph's Blog</a>, but with with one distinction: it preserves the source IP of the incoming connection. This may be important depending on your use. For me, I was running a PHP web app for which I needed the $_SERVER["REMOTE_ADDR"] field to be the true and correct IP of the client, and not just the IP of the gateway that forwarded the traffic.</p>
<p>Values used in this example:</p>
<ul>
<li>192.168.51.0/24 is the private VPN network between the cloud VPS host and the server you want to expose.</li>
<li>192.168.51.1 is the VPN network IP of the cloud VPS.</li>
<li>192.168.51.2 is the VPN network IP of the server you want to expose.</li>
<li>50.40.30.20 is the public IP of the cloud VPS.</li>
<li>59014 the UDP port number I randomly chose for WireGuard on the VPS.</li>
<li>80 and 443 are the TCP port numbers I want to forward from my VPS to my local server.</li>
</ul>
<p>On the cloud VPS:</p>
<pre class="code">
mkdir -p /etc/wireguard
cd /etc/wireguard
wg genkey | tee privatekey | wg pubkey > publickey
chown 600 privatekey
cat privatekey
cat publickey
# Replace the PrivateKey in the [Interface] section with the actual privatekey value generated above.
# Replace the PublicKey in the [Peer] section with the publickey value you generate ON YOUR LOCAL SERVER in the later steps.
cat > /etc/wireguard/wg0.conf <<EOF
[Interface]
Address = 192.168.51.1
PrivateKey = KF+X0SO2lkBFnxTCK4R5PCv/FC6+wgH6rt6wWHkVVHE=
ListenPort = 59014
[Peer]
PublicKey = y700qaTgjDjMrLc9tGF7+PKFrX7uHyIJJ/s2zvadKVY=
AllowedIPs = 192.168.51.2/32
EOF
chmod 600 wg0.conf
# Start the VPN server
wg-quick up wg0
echo 0 > /proc/sys/net/ipv4/ip_forward
iptables -P FORWARD DROP
iptables -A FORWARD -i eth0 -o wg0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -A FORWARD -i wg0 -o eth0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -A FORWARD -i wg0 -o eth0 -m conntrack --ctstate NEW -j ACCEPT
iptables -A FORWARD -i eth0 -o wg0 -p tcp --syn --dport 80 -m conntrack --ctstate NEW -j ACCEPT
iptables -A FORWARD -i eth0 -o wg0 -p tcp --syn --dport 443 -m conntrack --ctstate NEW -j ACCEPT
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j DNAT --to-destination 192.168.51.2
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 443 -j DNAT --to-destination 192.168.51.2
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
echo 1 > /proc/sys/net/ipv4/ip_forward
</pre>
<p>On your local server, instead of using wg-quick, we will set up the interface a bit more manually. This is in order to have finer control over routing. We want connections originating on this server to exit via this server's normal internet connection, and not via the WireGuard VPN connection. However, packets part of TCP connections that got forwarded to us on the VPN from the VPS should be routed back via the VPS.</p>
<p>On your local server:</p>
<pre class="code">
mkdir -p /etc/wireguard
cd /etc/wireguard
wg genkey | tee privatekey | wg pubkey > publickey
chown 600 privatekey
cat privatekey
cat publickey
# Replace the PrivateKey in the [Interface] section with the actual privatekey value generated above.
# Replace the PublicKey in the [Peer] section with the publickey value you generated ON YOUR CLOUD VPS in the earlier steps.
cat > /etc/wireguard/wg0.conf <<EOF
[Interface]
PrivateKey = YIOy80YJdcMqI1WCMROoyUfsfkc6GhP0KqRS3BfcoEs=
[Peer]
PublicKey = i1VQnnODEBbf+P9cyd9XmB9G58qEpM53TbMXn1UAJx8=
Endpoint = 50.40.30.20:59014
AllowedIPs = 0.0.0.0/0
PersistentKeepalive = 25
EOF
chmod 600 wg0.conf
# Set up VPN connection
ip link add wg0 type wireguard
wg setconf wg0 /etc/wireguard/wg0.conf
ip address add 192.168.51.2 dev wg0
ip link set mtu 1420 up dev wg0
ip rule add from 192.168.51.2 table 51820
ip rule add iif wg0 table 51820
ip route add 192.168.51.0/24 dev wg0
ip route add 0.0.0.0/0 via 192.168.51.1 dev wg0 table 51820
# Shut down VPN connection (don't run these steps now, but keep them handy)
ip rule delete from 192.168.51.2 table 51820
ip rule delete iif wg0 table 51820
ip link delete dev wg0
</pre>
<p>You should now be able to receive TCP traffic on port 80 and 443 on your local server, and the src IP address of the actual real client will be preserved all the way to your local server.</p>Allanhttp://www.blogger.com/profile/11850694461776134997noreply@blogger.com0tag:blogger.com,1999:blog-24624162.post-41315550885970395672020-06-18T20:09:00.002-07:002020-06-18T21:20:29.745-07:00WireGuard kernel module for Alpine Linux on diskless Raspberry PiAs of writing (Alpine Linux v3.12), the WireGuard kernel module is not part of the Alpine Linux Raspberry Pi kernel by default. There's a community module called wireguard-rpi2, but it doesn't work in a system set up in diskless mode. The reason it doesn't work, is because in diskless mode on a Raspberry Pi, Alpine Linux uses a read-only loopback file system to store its kernel modules. This filesystem is mounted on /.modloop .<br />
<br />
The workaround I came up with, is to use an overlay file system to make the kernel modules directory writable.<br />
<br />
Here's the workaround:<br />
<pre name="code">
# Remount the kernel module directory /.modloop as an overlay, to allow writing
modprobe overlay
mkdir -p /.modloop.lower /.modloop.upper /.modloop.workdir
mount /dev/loop0 /.modloop.lower
umount /.modloop/
mount -t overlay -o lowerdir=/.modloop.lower,upperdir=/.modloop.upper,workdir=/.modloop.workdir none /.modloop
lbu include /.modloop.upper
lbu commit -d
# Manually get the Wireguard kernel module to avoid installing the wireguard-rpi2 which does not work with diskless systems
cd /tmp
pkgname=$(apk list | grep wireguard-rpi2 | cut -d " " -f 1)
wget http://dl-cdn.alpinelinux.org/alpine/v3.12/community/armhf/$pkgname.apk
mkdir /tmp/$pkgname
tar -xzf $pkgname.apk -C /tmp/$pkgname
mkdir -p /lib/modules/$(uname -r)/extra/
cp /tmp/$pkgname/lib/modules/$(uname -r)/extra/wireguard.ko /lib/modules/$(uname -r)/extra/
rm -fr /tmp/$pkgname /tmp/$pkgname.apk
depmod
# Create an init script to remount the /.modloop overlay on next boot
cat > /etc/init.d/modloopoverlay <<EOF
#!/sbin/openrc-run
depend() {
before networking
need modules
}
start() {
ebegin "Starting modloop overlay"
modprobe overlay
mkdir -p /.modloop.lower /.modloop.upper /.modloop.workdir
if [ ! -d /.modloop.lower/modules ]; then
mount /dev/loop0 /.modloop.lower
fi
umount /.modloop
mount -t overlay -o lowerdir=/.modloop.lower,upperdir=/.modloop.upper,workdir=/.modloop.workdir none /.modloop
eend 0
}
EOF
chmod +x /etc/init.d/modloopoverlay
/etc/init.d/modloopoverlay restart
rc-update add modloopoverlay boot
lbu include /etc/init.d/modloopoverlay
apk add wireguard-tools
# Create an init script to start wg0
cat > /etc/init.d/wg0 <<EOF
#!/sbin/openrc-run
depend() {
need networking ntpd modloopoverlay
}
start() {
ebegin "Starting Wireguard tunnel wg0"
ntpd -n -q -p pool.ntp.org
date
wg-quick up wg0
eend $?
}
stop() {
ebegin "Stopping Wireguard tunnel wg0"
wg-quick down wg0
eend 0
}
EOF
chmod +x /etc/init.d/wg0
/etc/init.d/wg0 restart
rc-update add wg0 default
lbu include /etc/init.d/wg0
lbu commit -d
# inspect the overlay file
tar -tvf /media/mmcblk0p1/localhost.apkovl.tar.gz
</pre>
Allanhttp://www.blogger.com/profile/11850694461776134997noreply@blogger.com2tag:blogger.com,1999:blog-24624162.post-4832755235081130292020-01-10T13:42:00.000-08:002020-01-10T13:51:53.796-08:00Alternative to negative lookbehinds in regular expressions<p>I recently faced the problem of requiring negative lookbehinds in a regex engine that does not support them (Golang's Regexp package, RE2, and Hyperscan).</p>
<p>For example, say you want to match the string "def" except if it is preceeded by "abc". In PCRE using negative lookbehinds you could achieve this with the regex:</p>
<pre name="code">
(?<!abc)def
</pre>
<p>But this will not work in all regex engines, so I needed an alternative.</p>
<p>I found <a href="https://stackoverflow.com/questions/38014387/negative-lookbehind-alternative/38014871#38014871">this Stackoverflow answer by user Sebastian Proske</a>. The approach is clever, but tricky to comprehend and write for long and complex negative lookbehinds. I will try to explain the general approach here, and then present a script to help use this approach for longer and more complex cases.</p>
<p>Continuing the example above, the approach works by having alternatives for each possible prefix that is not "abc". These alternatives are:</p>
<ul>
<li>The beginning of the string, or a character that is not "c", followed by "def"</li>
<li>The beginning of the string, or a character that is not "b", followed by "cdef"</li>
<li>The beginning of the string, or a character that is not "a", followed by "bcdef"</li>
</ul>
<p>Written as a regex (line breaks and indentation for readability are not part of the regex):</p>
<pre name="code">
(
(^|[^c])def
|
(^|[^b])cdef
|
(^|[^a])bcdef
)
</pre>
<p>We can extract the common suffix out:</p>
<pre name="code">
(
(^|[^c])
|
(^|[^b])c
|
(^|[^a])bc
)
def
</pre>
<p>To support multiple alternative negative lookbehinds, such as both "abc" and "1234", which in PCRE syntax this can be written as "(?<!abc|1234)def", we can write:</p>
<pre name="code">
(
(^|[^c4])
|
(^|[^b])c
|
(^|[^a])bc
|
(^|[^3])4
|
(^|[^2])34
|
(^|[^1])234
)
def
</pre>
<p>To support character classes in the negative lookbehinds, such as "[bB]", which in PCRE syntax can be written as "(?<!a[bB]c)def", we can write:</p>
<pre name="code">
(
(^|[^c])
|
(^|[^bB])c
|
(^|[^a])[bB]c
)
def
</pre>
<p>For better equivalence to the negative lookbehinds, we can additionally use non-capturing groups:</p>
<pre name="code">
(?:
(?:^|[^c])
|
(?:^|[^b])c
|
(?:^|[^a])bc
)
def
</pre>
<p>Here's a quick and dirty Python script to help construct regexes following this approach. The example input here is equivalent to "(?<!a[bB]c|1234)".</p>
<p>The script is also available to conveniently run in your browser on repl.it: <a href="https://repl.it/@allanrbo/regexnegativelookbehindalternative1">https://repl.it/@allanrbo/regexnegativelookbehindalternative1</a></p>
<pre class="python" name="code">
negativePrefixes = [
"a[bB]c",
"1234",
]
def removeDuplicateChars(s):
return "".join([c for i,c in enumerate(s) if c not in s[:i]])
def removeChars(s, charsToRemove):
return "".join([c for i,c in enumerate(s) if c not in charsToRemove])
# Split into arrays of strings. Each string is either a single char, or a char class.
negativePrefixesSplit = []
for np in negativePrefixes:
npSplit = []
curCc = ""
inCc = False
for c in np:
if c == "[":
inCc = True
elif c == "]":
npSplit.append(removeDuplicateChars(curCc))
curCc = ""
inCc = False
else:
if inCc:
if c in "-\\":
raise "Only really simply char classes are currently supported. No ranges or escapes, sorry."
curCc += c
else:
npSplit.append(c)
negativePrefixesSplit.append(npSplit)
allexprs = []
class Expr():
pass
suffixLength = 0
while True:
suffixes = []
for np in negativePrefixesSplit:
if suffixLength < len(np):
suffixes.append(np[len(np)-suffixLength-1:])
if len(suffixes) == 0:
break
exprs = []
for suffix in suffixes:
curChar = suffix[0]
remainder = suffix[1:]
expr = Expr()
expr.curChar = curChar
expr.remainder = remainder
exprs.append(expr)
# Is the remainder a subset of any other suffixes remainders?
for i in range(len(exprs)):
e1 = exprs[i]
for j in range(len(exprs)):
e2 = exprs[j]
isSubset = True
for k in range(len(e1.remainder)):
if not set(e1.remainder[k]).issubset(set(e2.remainder[k])):
isSubset = False
break
if isSubset:
if e1.curChar == e2.curChar:
e1.remainder = e2.remainder
continue
e1.curChar += e2.curChar
e1.curChar = removeDuplicateChars(e1.curChar)
for k in range(len(e1.remainder)):
if len(set(e2.remainder[k]) - set(e1.remainder[k])) > 0:
charsInCommon = "".join(set(e2.remainder[k]) & set(e1.remainder[k]))
e2.remainder[k] = removeChars(e2.remainder[k], charsInCommon)
# Remove duplicate expressions
exprsFiltered = []
for i in range(len(exprs)):
e1 = exprs[i]
alreadyExists = False
for j in range(len(exprs)):
if i == j:
break
e2 = exprs[j]
sameC = set(e1.curChar) == set(e2.curChar)
sameR = True
for k in range(len(e1.remainder)):
if set(e1.remainder[k]) != set(e2.remainder[k]):
sameR = False
break
if sameC and sameR:
alreadyExists = True
break
if not alreadyExists:
exprsFiltered.append(e1)
allexprs.extend(exprsFiltered)
suffixLength += 1
continue
out = "(?:\n"
for i in range(len(allexprs)):
e = allexprs[i]
out += ("(?:^|[^" + e.curChar + "])")
for c in e.remainder:
if len(c) > 1:
out += "[" + c + "]"
else:
out += c
if i != len(allexprs)-1:
out += "|"
out += "\n"
out += ")"
print("Human readable:")
print(out)
print()
print("Single line:")
print(out.replace("\n",""))
</pre>
Example output:
<pre name="code">
Human readable:
(?:
(?:^|[^c4])|
(?:^|[^bB])c|
(?:^|[^3])4|
(?:^|[^a])[bB]c|
(?:^|[^2])34|
(?:^|[^1])234
)
Single line:
(?:(?:^|[^c4])|(?:^|[^bB])c|(?:^|[^3])4|(?:^|[^a])[bB]c|(?:^|[^2])34|(?:^|[^1])234)
</pre>
Allanhttp://www.blogger.com/profile/11850694461776134997noreply@blogger.com0tag:blogger.com,1999:blog-24624162.post-85454490701283802882018-06-20T00:17:00.000-07:002018-06-20T00:18:26.461-07:00Foreflight in DenmarkI enjoy using the Foreflight app when flying in the US, and recently discovered that it can work very well in Europe as well. They have a <a href="https://foreflight.com/support/user-content/">guide on how to import custom charts</a>, and it turned out to be very easy. PDFs of aviation charts for Denmark can be downloaded from <a href="https://aim.naviair.dk/">the AIM</a> and converted via the tool described in Foreflights guide.<br />
<br />
Here are a few charts current as of June 2018. <a href="http://acoby.com/aviationcharts/EK_Chart_ANC_DENMARK_front_en%202018.mbtiles">Denmark</a>, <a href="http://acoby.com/aviationcharts/EK_Chart_ANC_COPENHAGEN_AREA_en.mbtiles">Copenhagen Area</a>, <a href="http://acoby.com/aviationcharts/EK_AD_2_EKRK_VAC_en.mbtiles">Roskilde Visual Approach</a>. Download these from Safari on your iPad, and it should prompt you to import them in Foreflight.Allanhttp://www.blogger.com/profile/11850694461776134997noreply@blogger.com0tag:blogger.com,1999:blog-24624162.post-72894741838053406752018-05-01T14:46:00.000-07:002018-08-08T20:05:33.195-07:00Remote Ubuntu dev setup with multi screen VNCGoal: A VM in the cloud or similar running an Ubuntu desktop with your dev tools (Sublime Text, Visual Studio Code, etc.), that you can remotely connect to and comfortably work on, spanning all your local monitors. Like this:
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiS5KqaW-Gh0OJu5zUwSBZ_txCOvdSKkmzt3ULIvTqOlFTyBBhl8Lt5zNJBpyku2JTtMf00vd87Gx13wRqDdo311hpED5OZAB5R9Epq_9xIWrDmvO4GYd5BAqCqHtyA9iGeZezPJA/s1600/desktop.jpg.small.jpg" imageanchor="1"><img border="0" data-original-height="281" data-original-width="900" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiS5KqaW-Gh0OJu5zUwSBZ_txCOvdSKkmzt3ULIvTqOlFTyBBhl8Lt5zNJBpyku2JTtMf00vd87Gx13wRqDdo311hpED5OZAB5R9Epq_9xIWrDmvO4GYd5BAqCqHtyA9iGeZezPJA/s1600/desktop.jpg.small.jpg" /></a>
<br />
<br />
We'll use TigerVNC. RealVNC 4 that comes with Ubuntu 16.04 can also be used, but it lacks the X extensions that are required for a well behaved multi monitor setup. Also, Visual Studio Code can't run on RealVNC 4 without a special patched version of libxcb.so.1.<br />
<br />
<h3>
Ubuntu 18.04</h3>
<h4>
Desktop edition</h4>
<pre>sudo apt update
sudo apt install --no-install-recommends openssh-server tigervnc-standalone-server tigervnc-common
sudo su
cat /etc/X11/Xvnc-session
mv /etc/X11/Xvnc-session /etc/X11/Xvnc-session.orig
cat > /etc/X11/Xvnc-session << "EOF"
#!/bin/sh
export GNOME_SHELL_SESSION_MODE=ubuntu
export XDG_CURRENT_DESKTOP=ubuntu:GNOME
vncconfig -nowin &
exec /etc/X11/Xsession
vncserver -kill $DISPLAY
EOF
chmod +x /etc/X11/Xvnc-session
exit
# Start the VNC session (this must be run as your user from an SSH session after each reboot)
vncserver
# To stop the VNC session
vncserver -kill</pre>
<h4>
Server edition</h4>
<pre>sudo apt-get update
sudo apt install --no-install-recommends tigervnc-standalone-server tigervnc-common ubuntu-desktop</pre>
<pre></pre>
And then followed by the same procedure with /etc/X11/Xvnc-session and vncserver as for desktop.
<br />
<br />
For additional desktop software, I recommend using the Ubuntu Software Center, which will install Snap packages for popular desktop software.
<br />
<pre># Install the Ubuntu Software Center
sudo apt install gnome-software</pre>
<br />
<br />
<h3>
Ubuntu 16.04</h3>
I only tried with server edition on 16.04, so adapt as needed.
<br />
<pre>sudo apt-get update
sudo apt-get install --no-install-recommends ubuntu-desktop gnome-terminal unity-lens-applications unity-lens-files gnome-settings-daemon
sudo apt-get install --no-install-recommends software-center
wget https://bintray.com/tigervnc/stable/download_file?file_path=ubuntu-16.04LTS%2Famd64%2Ftigervncserver_1.8.0-1ubuntu1_amd64.deb -O tigervncserver_1.8.0-1ubuntu1_amd64.deb
sudo dpkg -i tigervncserver_1.8.0-1ubuntu1_amd64.deb
sudo apt-get install -f
mkdir $HOME/.vnc/
cat > $HOME/.vnc/xstartup <<"EOF"
#!/bin/sh
unset SESSION_MANAGER
unset DBUS_SESSION_BUS_ADDRESS
/etc/X11/xinit/xinitrc &
vncconfig -nowin &
/usr/lib/x86_64-linux-gnu/unity/unity-panel-service &
/usr/lib/x86_64-linux-gnu/indicator-datetime/indicator-datetime-service &
/usr/lib/x86_64-linux-gnu/indicator-keyboard/indicator-keyboard-service &
unity &
EOF
chmod +x $HOME/.vnc/xstartup
# some kde or qt apps seem to look ugly without this package
sudo apt-get install kde-style-breeze-qt4
# Start the VNC session (this must be run as your user from an SSH session after each reboot)
vncserver
# To stop the VNC session
vncserver -kill :1
</pre>
For Additional desktop software, I recommend using Snap:
<br />
<pre>sudo apt install snap
# For example install Visual Studio Code, Chrome, and Sublime, like this
sudo snap install vscode --classic
sudo snap install chromium
sudo snap install sublime-text --classic
</pre>
<br />
<h3>
Connecting</h3>
Use the TigerVNC client to take advantage of the dynamic resizing and multi-screen features. Binaries for Win/Mac/Linux can be found <a href="http://tigervnc.org/">on their website</a>.
<br />
<br />
Open the TigerVNC menu by pressing F8, and note the shortcut keys here for full screen, minimize, etc. Copy the remote .vnc/passwd file locally and pass it to the TigerVNC client with the -PasswordFile= option to avoid having to type your password repeatedly. For example, I connect from my Windows box using a cmd file with this content:<br />
<pre>start "" "C:\utils\vnc\vncviewer64-1.8.0.exe" "yourRemoteServer:5901" -PasswordFile=C:\utils\vnc\passwd</pre>
<br />
<br />
<h3>
Secure connection</h3>
If you are connecting over the internet, you'll want to use an SSH tunnel to secure the unencrypted VNC. In that case you can start the VNC server in a mode where it only listens on localhost, and then use SSH port forwarding.
<br />
<pre>vncserver -localhost=yes -nolisten tcp</pre>
<div>
Connect using your preferred SSH client, and set up port forwarding to localhost:5901, for example from your local port 5901.</div>
<div>
</div>
<br />Allanhttp://www.blogger.com/profile/11850694461776134997noreply@blogger.com4tag:blogger.com,1999:blog-24624162.post-75386160716856419642017-11-22T15:20:00.000-08:002018-01-31T14:22:45.848-08:00Delete the undeletable on WindowsI had trouble deleting C:\ProgramData\Docker on Docker for Windows using Windows containers. This is because it contains a full Windows directory layout, with all system files and special permissions and flags. Here's a Powershell snippet to get a file hierarchy on Windows into a state where it can be deleted.<br />
<br />
<pre name="code">
$ErrorActionPreference = "stop"
function reallyDelete($d) {
function renameToNumbersRecursive($dir) {
$i = 0
foreach($f in (dir $dir)) {
while(Test-Path ($dir + "\" + $i)) {
$i++
}
try {
Rename-Item $f.FullName ($dir + "\" + $i)
} catch {
takeown /f $dir | Out-Null
icacls $dir /reset | Out-Null
takeown /f $f.FullName | Out-Null
icacls $f.FullName /reset | Out-Null
Rename-Item $f.FullName ($dir + "\" + $i)
}
}
foreach($f in (dir $dir)) {
if($f -is [System.IO.DirectoryInfo]) {
renameToNumbersRecursive($f.FullName)
}
}
}
# paths might become too long for icacls, so rename paths to short numbers
renameToNumbersRecursive $d
takeown /f $d /r | Out-Null
icacls $d /reset /T | Out-Null
attrib -s -h -r "$d\*.*" /s /d | Out-Null
cmd /c del /s /q $d | Out-Null
cmd /c rmdir /s /q $d | Out-Null
}
reallyDelete "C:\ProgramData\Docker"
</pre>Allanhttp://www.blogger.com/profile/11850694461776134997noreply@blogger.com0tag:blogger.com,1999:blog-24624162.post-77680011957439410742017-05-30T00:27:00.003-07:002022-10-27T09:46:52.809-07:00Utility to remind you to keep a task diaryHere's a program that asks you every hour what you are doing, so you end up with a diary at the end of the day. It saves your input to a text file WhatAmIDoing.txt in the same folder you put the exe.<br />
<br />
The "ask every hour" button creates a Windows Scheduled Task. Note that if you move the location of the exe file, you will need to click the "ask every hour" button again.<br />
<br />
<a href="https://acoby.com/utils/WhatAmIDoing.exe">https://acoby.com/utils/WhatAmIDoing.exe</a><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjoNtQYrS6bevmtpf9nOa8CSmEeSr3ahOXvMUQ2XzVRIsnMLvl7yfnvy6ssWwGXHxSXf0F89chyphenhyphenye-ZUUTtObZvWuiM7ClE1ytFp6c4dMcXEFUhUXUgvA2sXPSRSELoE7LzgjOYWA/s1600/whatamdoing.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="540" data-original-width="786" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjoNtQYrS6bevmtpf9nOa8CSmEeSr3ahOXvMUQ2XzVRIsnMLvl7yfnvy6ssWwGXHxSXf0F89chyphenhyphenye-ZUUTtObZvWuiM7ClE1ytFp6c4dMcXEFUhUXUgvA2sXPSRSELoE7LzgjOYWA/s1600/whatamdoing.png" /></a></div>
<br />Allanhttp://www.blogger.com/profile/11850694461776134997noreply@blogger.com0tag:blogger.com,1999:blog-24624162.post-62490549029619030302016-09-29T18:36:00.000-07:002016-09-29T18:40:57.017-07:00Mailing list managerI needed a simple mailing list manager for a sports club I'm in. We wanted it to be of the type where an administrator manages the lists (not the type where any random person can subscribe him- or herself). Couldn't find any decent looking free service offering this, so I decided to make one: <a href="http://mailgroup.io/">mailgroup.io</a> .<br />
<br />
These are its features:<br />
<ul>
<li>Free for noncommercial use.</li>
<li>Your own custom domain.</li>
<li>Or @mailgroup.io as domain if you prefer.</li>
<li>Your choice of members-only mailing lists, or everyone-may-write.</li>
<li>Detailed Postfix delivery reports for each mail for each recipient.</li>
<li>Max 50 recipients per account (negotiable).</li>
<li>Mail triggers. These are special email addresses that trigger HTTP POSTs with the email content to a defined URL when they receive email. Useful if you want some logic on your website to be triggered by email.</li>
</ul>
<br />
<br />Allanhttp://www.blogger.com/profile/11850694461776134997noreply@blogger.com1tag:blogger.com,1999:blog-24624162.post-47335515876455783962016-09-14T23:29:00.001-07:002016-09-14T23:29:13.147-07:00GLaDOS-like sound pack for TaranisHere's a soundpack for the FrSky Taranis, based on the original, but run through Melodyne so it sounds like GLaDOS from Portal: <a href="https://acoby.com/fpv/gladosStyleTaranisSounds.zip">https://acoby.com/fpv/gladosStyleTaranisSounds.zip</a>Allanhttp://www.blogger.com/profile/11850694461776134997noreply@blogger.com0tag:blogger.com,1999:blog-24624162.post-33945703117160962052016-05-01T16:33:00.000-07:002016-05-01T16:33:43.303-07:00Heap sort implementation in C#Here's a heap sort implementation in C#:<br />
<br />
<pre class="csharp" name="code">
class HeapSort
{
public void Sort(int[] a)
{
// Turn the array into a max heap
for (int i = (a.Length / 2) - 1; i >= 0; i--)
{
MaxHeapify(a, i, a.Length);
}
// Remove the largest element from the max heap and put it on the end,
// and then max heapify the heap which is now 1 smaller
for (int i = a.Length - 1; i >= 1; i--)
{
Swap(a, 0, i);
MaxHeapify(a, 0, i);
}
}
private void MaxHeapify(int[] a, int i, int n)
{
int idxLeft = (i + 1) * 2 - 1;
int idxRight = (i + 1) * 2 - 1 + 1;
int largest = i;
if (idxLeft < n && a[idxLeft] > a[largest])
{
largest = idxLeft;
}
if (idxRight < n && a[idxRight] > a[largest])
{
largest = idxRight;
}
if (largest != i)
{
Swap(a, i, largest);
MaxHeapify(a, largest, n);
}
}
private void Swap(int[] a, int i, int j)
{
int tmp = a[i];
a[i] = a[j];
a[j] = tmp;
}
}
</pre>Allanhttp://www.blogger.com/profile/11850694461776134997noreply@blogger.com0tag:blogger.com,1999:blog-24624162.post-73535241372515155462016-03-25T12:35:00.000-07:002016-03-25T12:35:31.007-07:00Segment tree implementation in JavaInspired by <a href="https://kartikkukreja.wordpress.com/2014/11/09/a-simple-approach-to-segment-trees/">Kartik Kukreja's implementation</a>, but storing all data in a tree node class, rather than an array. I find it a lot easier to comprehend this way.
<br />
<pre class="java" name="code">
class SegmentTreeNode {
public int aggregateValue;
public SegmentTreeNode left;
public SegmentTreeNode right;
public int origLowIndex;
public int origHighIndex;
}
class SegmentTreeTest {
public static SegmentTreeNode buildSegmentTree(int[] a, int low, int high) {
SegmentTreeNode n = new SegmentTreeNode();
n.origLowIndex = low;
n.origHighIndex = high;
if(low == high) {
n.aggregateValue = a[low];
return n;
}
int mid = (high - low)/2 + low;
n.left = buildSegmentTree(a, low, mid);
n.right = buildSegmentTree(a, mid+1, high);
// This segment tree is for summation. Could also be min, max, or any other associative func.
n.aggregateValue = n.left.aggregateValue + n.right.aggregateValue;
return n;
}
public static SegmentTreeNode getAggregateValue(SegmentTreeNode n, int low, int high) {
if(n.origLowIndex == low && n.origHighIndex == high) {
return n;
}
// The interval is fully contained within the left node
if(low >= n.left.origLowIndex && high <= n.left.origHighIndex) {
return getAggregateValue(n.left, low, high);
}
// The interval is fully contained within the right node
if(low >= n.right.origLowIndex && high <= n.right.origHighIndex) {
return getAggregateValue(n.right, low, high);
}
// Split into queries on the left subtree and the right subtree
SegmentTreeNode leftResult = getAggregateValue(n.left, low, n.left.origHighIndex);
SegmentTreeNode rightResult = getAggregateValue(n.right, n.right.origLowIndex, high);
SegmentTreeNode result = new SegmentTreeNode();
result.origLowIndex = low;
result.origHighIndex = high;
// This segment tree is for summation. Could also be min, max, or any other associative func.
result.aggregateValue = leftResult.aggregateValue + rightResult.aggregateValue;
return result;
}
public static void update(SegmentTreeNode n, int index, int val) {
if(n.origLowIndex == index && n.origHighIndex == index) {
n.aggregateValue = val;
return;
}
if(n.left.origLowIndex <= index && index <= n.left.origHighIndex) {
update(n.left, index, val);
} else {
update(n.right, index, val);
}
// This segment tree is for summation. Could also be min, max, or any other associative func.
n.aggregateValue = n.left.aggregateValue + n.right.aggregateValue;
}
public static void main(String[] args) {
// 0 1 2 3 4 5 6 7 8 9 10
int[] a = new int[] { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 };
SegmentTreeNode r = buildSegmentTree(a, 0, a.length-1);
System.out.println(r.aggregateValue);
System.out.println(getAggregateValue(r, 1, 3).aggregateValue);
System.out.println(getAggregateValue(r, 4, 7).aggregateValue);
update(r, 2, 10);
System.out.println(r.aggregateValue);
System.out.println(getAggregateValue(r, 4, 7).aggregateValue);
System.out.println(getAggregateValue(r, 1, 3).aggregateValue);
}
}
</pre>
Allanhttp://www.blogger.com/profile/11850694461776134997noreply@blogger.com1tag:blogger.com,1999:blog-24624162.post-52775912416784417622015-06-19T06:56:00.001-07:002015-06-24T03:52:36.460-07:00Windows scheduled task to auto logoffHere's some PowerShell to remotely schedule a task to log out 2+ hours idle users:
<pre>function InstallAutoLogoff($computer)
{
invoke-command -computer $computer `
{
schtasks.exe /delete /tn AutoLogOff /f 2>&1 | Out-Null
set-content c:\windows\temp\AutoLogOffTask.xml `
'<?xml version="1.0" encoding="UTF-16"?>
<Task version="1.2" xmlns="http://schemas.microsoft.com/windows/2004/02/mit/task">
<Triggers>
<TimeTrigger>
<Repetition>
<Interval>PT30M</Interval>
</Repetition>
<StartBoundary>2015-01-01T00:00:00</StartBoundary>
</TimeTrigger>
</Triggers>
<Actions Context="Author">
<Exec>
<Command>PowerShell.exe</Command>
<Arguments>-Command $r1 = quser.exe; $idPos = $r1[0].IndexOf(''ID ''); foreach($row in $r1) { $r2 = $row.Substring($idPos).Trim() -split '' +''; if($r2[2].Contains('':'')) { $t = $r2[2].Replace('':'', ''.''); if($t.Contains(''+'') -or ($t -as [double] -ge 2)) { rwinsta.exe $r2[0] } } }</Arguments>
</Exec>
</Actions>
</Task>'
schtasks.exe /create /tn AutoLogOff /ru system /xml c:\windows\temp\AutoLogOffTask.xml
del c:\windows\temp\AutoLogOffTask.xml
}
}
InstallAutoLogoff 'myserver1'
InstallAutoLogoff 'myserver2'
InstallAutoLogoff 'myserver3'
</pre>
Allanhttp://www.blogger.com/profile/11850694461776134997noreply@blogger.com0tag:blogger.com,1999:blog-24624162.post-47076698254211397002015-03-23T16:55:00.000-07:002015-03-23T17:43:53.937-07:00VoCore OpenWRT as NAT access point<p>I recently bought a <a href="http://vocore.io/">VoCore</a>. It came pre-flashed with OpenWRT Chaos Calmer (as of 2015-03-16). OpenWRT was configured as a wireless access point, bridging with the ethernet port. Here is what I did to get it to a configuration where it acts as a NAT'ing wireless access point, similar to most consumer routers/access points:</p>
<p>SSH'ed to its default IP of 192.168.61.1.</p>
<p>Edited /etc/config/network:</p>
<pre>...
#config interface 'lan'
config 'interface' 'wan'
option macaddr 'b8:d8:12:60:00:01'
option proto 'dhcp'
config interface 'lan'
option force_link '1'
option macaddr 'b8:d8:12:60:00:02'
option proto 'static'
option ipaddr '192.168.61.1'
option netmask '255.255.255.0'
...
</pre>
<p>Edited /etc/config/wireless:</p>
<pre>...
config wifi-iface
option device radio0
option network lan
option mode ap
option ssid gaffel8080
option encryption psk2
option key palle123
...
</pre>
<p>Edited /etc/config/dhcp:</p>
<pre>...
#config odhcpd 'odhcpd'
# option maindhcp '0'
# option leasefile '/tmp/hosts/odhcpd'
# option leasetrigger '/usr/sbin/odhcpd-update'
...
</pre>
<p>To enable SSH from the ethernet port, I also edited /etc/config/firewall:</p>
<pre>...
config rule
option src wan
option proto tcp
option dst_port 22
option target ACCEPT
</pre>
<p>This was all that was needed for the basic scenario of using VoCore as an accesspoint combined with a NAT'ing router.</p>
<p>I went a little further and installed some nice to have packages like openssh-sftp-server, nano, htop, ip, etc. However, in order to do this, I first had to fix /etc/opkg.conf:</p>
<pre>...
#src/gz cheese_base http://downloads.openwrt.org/snapshots/trunk/ramips/packages/base
#src/gz cheese_luci http://downloads.openwrt.org/snapshots/trunk/ramips/packages/luci
src/gz chaos_calmer_base http://downloads.openwrt.org/snapshots/trunk/ramips/generic/packages/base/
src/gz chaos_calmer_luci http://downloads.openwrt.org/snapshots/trunk/ramips/generic/packages/luci/
src/gz chaos_calmer_management http://downloads.openwrt.org/snapshots/trunk/ramips/generic/packages/management/
src/gz chaos_calmer_packages http://downloads.openwrt.org/snapshots/trunk/ramips/generic/packages/packages/
src/gz chaos_calmer_routing http://downloads.openwrt.org/snapshots/trunk/ramips/generic/packages/routing/
src/gz chaos_calmer_telephony http://downloads.openwrt.org/snapshots/trunk/ramips/generic/packages/telephony/
</pre>
<p>After that I could install the extra packages by running the following:</p>
<pre>opkg update
opkg install openssh-sftp-server
opkg install nano
opkg install htop
opkg install ip
</pre>
<p>Also, I recommend disabling Luci, as it is buggy, and a security concern:</p>
<pre>rm /www/cgi-bin/luci
echo "A private box" > /www/index.html
</pre>
Allanhttp://www.blogger.com/profile/11850694461776134997noreply@blogger.com0tag:blogger.com,1999:blog-24624162.post-45619073818684471542014-08-13T17:10:00.002-07:002014-08-13T17:34:53.298-07:00A simple TCP proxy forwarder in .NETHere's a way of doing a super simple TCP proxy in C#.<br />
<br />
Warning: this should only be used for testing purposes, as it <i>will</i> leak threads.
<br />
<br />
<pre class="csharp" name="code">using System;
using System.Net;
using System.Net.Sockets;
using System.Threading;
class Program
{
static void ProxyStream(string stream1name, NetworkStream stream1, string stream2name, NetworkStream stream2)
{
var buffer = new byte[65536];
try
{
while (true)
{
var len = stream1.Read(buffer, 0, 65536);
stream2.Write(buffer, 0, len);
}
}
catch (Exception)
{
Console.WriteLine("Stream from " + stream1name + " to " + stream2name + " closed");
}
}
static void Main(string[] args)
{
if (args.Length != 3)
{
Console.WriteLine("Usage: program.exe localport remoteServerHost remoteServerPort");
Console.WriteLine("Example: program.exe 13389 10.1.2.3 3389");
return;
}
var localPort = int.Parse(args[0]);
var remoteServerHost = args[1];
var remoteServerPort = int.Parse(args[2]);
var l = new TcpListener(IPAddress.Any, localPort);
l.Start();
while (true)
{
var client1 = l.AcceptTcpClient();
var remoteAddress = (client1.Client.RemoteEndPoint as IPEndPoint).Address.ToString();
Console.WriteLine("Accepted session from " + remoteAddress);
var client2 = new TcpClient(remoteServerHost, remoteServerPort);
Console.WriteLine("Created connection to " + remoteServerHost + ":" + remoteServerPort);
new Thread(() => { ProxyStream(remoteAddress, client1.GetStream(), remoteServerHost, client2.GetStream()); }).Start();
new Thread(() => { ProxyStream(remoteServerHost, client2.GetStream(), remoteAddress, client1.GetStream()); }).Start();
}
}
}
</pre>
<br />
Or, if you need this from PowerShell, I would suggest wrapping it (as threads in PowerShell are painful):
<br />
<pre>$source = '
using System;
using System.Net;
using System.Net.Sockets;
using System.Threading;
class Program
{
static void ProxyStream(string stream1name, NetworkStream stream1, string stream2name, NetworkStream stream2)
{
var buffer = new byte[65536];
try
{
while (true)
{
var len = stream1.Read(buffer, 0, 65536);
stream2.Write(buffer, 0, len);
}
}
catch (Exception)
{
Console.WriteLine("Stream from " + stream1name + " to " + stream2name + " closed");
}
}
static void Main(string[] args)
{
if (args.Length != 3)
{
Console.WriteLine("Usage: program.exe localport remoteServerHost remoteServerPort");
Console.WriteLine("Example: program.exe 13389 10.1.2.3 3389");
return;
}
var localPort = int.Parse(args[0]);
var remoteServerHost = args[1];
var remoteServerPort = int.Parse(args[2]);
var l = new TcpListener(IPAddress.Any, localPort);
l.Start();
while (true)
{
var client1 = l.AcceptTcpClient();
var remoteAddress = (client1.Client.RemoteEndPoint as IPEndPoint).Address.ToString();
Console.WriteLine("Accepted session from " + remoteAddress);
var client2 = new TcpClient(remoteServerHost, remoteServerPort);
Console.WriteLine("Created connection to " + remoteServerHost + ":" + remoteServerPort);
new Thread(() => { ProxyStream(remoteAddress, client1.GetStream(), remoteServerHost, client2.GetStream()); }).Start();
new Thread(() => { ProxyStream(remoteServerHost, client2.GetStream(), remoteAddress, client1.GetStream()); }).Start();
}
}
}
'
Add-Type `
-TypeDefinition $source `
-Language CSharp `
-OutputAssembly 'c:\windows\temp\tcpproxy.exe' `
-OutputType 'ConsoleApplication'
</pre>
Allanhttp://www.blogger.com/profile/11850694461776134997noreply@blogger.com0tag:blogger.com,1999:blog-24624162.post-32406924712006771862014-08-05T16:56:00.000-07:002014-08-26T02:24:00.639-07:00Command prompt running as NT AUTHORITY\Network Service <p>Here's a convenient way to get a command prompt running as Network Service. This is useful for troubleshooting authentication issues for services running as Network Service.</p>
<p>Get Psexec from <a href="http://live.sysinternals.com/psexec.exe">http://live.sysinternals.com/psexec.exe</a>.</p>
<p>Run:
<pre>psexec -u "NT AUTHORITY\Network Service" -i cmd</pre>
</p>
<p>
Or, if you prefer PowerShell:
<pre>psexec -u "NT AUTHORITY\Network Service" -i powershell</pre>
</p>
<p>
Or, if you prefer PowerShell, and a bigger than standard window with some distinguishable color:
<pre>psexec -u "NT AUTHORITY\Network Service" -i powershell -noexit "cd c:\ ; mode con:cols=170 lines=60 ; (Get-Host).UI.RawUI.BufferSize = New-Object System.Management.Automation.Host.Size(170,3000) ; (Get-Host).UI.RawUI.BackgroundColor='DarkCyan' ; clear"</pre>
</p>
<br/>
<br/>Allanhttp://www.blogger.com/profile/11850694461776134997noreply@blogger.com0tag:blogger.com,1999:blog-24624162.post-79967148194034932802014-08-05T16:47:00.002-07:002015-07-06T03:39:39.034-07:00WCF net.tcp test client in PowerShellHere's a convenient way of creating a WCF net.tcp client for a quick test. It uses PowerShell to compile C# code to an exe on the fly. Note that the interface can be partial (only containing the method you want to call).
<pre>
$source = '
using System;
using System.ServiceModel;
[ServiceContract(Namespace = "http://example.com/SomeSchema")]
public interface ISomeService
{
[OperationContract]
bool SomeMethod();
}
public class Wcftest
{
public static void Main(string[] args)
{
var binding = new NetTcpBinding();
var address = new EndpointAddress("net.tcp://" + args[0] + ":808/SomeService");
var factory = new ChannelFactory<ISomeService>(binding, address);
var proxy = factory.CreateChannel();
Console.WriteLine(proxy.SomeMethod());
}
}
'
Add-Type `
-ReferencedAssemblies 'System.ServiceModel.dll' `
-TypeDefinition $source `
-Language CSharp `
-OutputAssembly 'c:\windows\temp\wcftest.exe' `
-OutputType 'ConsoleApplication'
c:\windows\temp\wcftest.exe "example.com"
del c:\windows\temp\wcftest.*
</pre>Allanhttp://www.blogger.com/profile/11850694461776134997noreply@blogger.com0