Monday, July 13, 2020

A "super ping" command to run while diagnosing network issues

Here's a bash function for your bash profile that continuously prints some basic checks I usually do while diagnosing network issues:

  • Show my LAN IPs
  • Show the default gateway IP
  • Ping the default gateway IP
  • Ping 8.8.8.8 (Google's public DNS servers)
  • DNS resolve amazon.com
  • Ping amazon.com
  • Get my WAN IP by curl'ing ifconfig.me

Tested on Ubuntu 20.04 and MacOS Catalina

function pingg() {
    while true; do
        printf "%s  " $(TZ=UTC date "+%Y-%m-%dT%H:%M:%S")

        if [[ "$OSTYPE" == "darwin"* ]]; then
            lanIps=$(ifconfig | grep "inet " | grep -v 127.0.0.1 | awk '{print $2}' | paste -d, -s -)
            gw=$(netstat -rn -f inet | grep default | awk '{print $2}')
        else
            lanIps=$(ip addr | grep "inet " | grep -v 127.0.0.1 | awk '{print $2}' | paste -d, -s -)
            gw=$(ip route | grep default | awk '{print $3}')
        fi
        printf "lanIp=%-15s " $lanIps
        printf "gw=%-15s " $gw

        pingGw="-------"
        if [ "$gw" != "" ]; then
            pingGw=$(ping -W1 -c1 $gw 2>/dev/null | tail -1 | grep -v "0 packets received" | cut -d '=' -f 2 | cut -d '/' -f 2)
            if [ "$pingGw" == "" ]; then
                pingGw="-------"
            fi
        fi
        printf "pGw=%-7s " $pingGw

        ping8888=$(ping -W1 -c1 8.8.8.8 2>/dev/null | tail -1 | grep -v "0 packets received" | cut -d '=' -f 2 | cut -d '/' -f 2)
        if [ "$ping8888" == "" ]; then
            ping8888="-------"
        fi
        printf "p8=%-7s " $ping8888

        dnsAmzn=$(host -4 -W 1 -t a amazon.com | head -n1 | grep "has address" | awk '{print $4}')
        if [ "$dnsAmzn" == "" ]; then
            dnsAmzn="-------"
        fi
        printf "dnsAmz=%-15s " $dnsAmzn

        pingAmzn="-------"
        if [ "$dnsAmzn" != "-------" ]; then
            pingAmzn=$(ping -W1 -c1 $dnsAmzn 2>/dev/null | tail -1 | grep -v "0 packets received" | cut -d '=' -f 2 | cut -d '/' -f 2)
            if [ "$pingAmzn" == "" ]; then
                pingAmzn="-------"
            fi
        fi
        printf "pAmz=%-7s " $pingAmzn

        wanIp="-------"
        dnsIfconfigMe=$(host -4 -W 1 -t a ifconfig.me | head -n1 | grep "has address" | awk '{print $4}')
        if [ "$dnsIfconfigMe" != "" ]; then
            wanIp=$(curl -s --max-time 2 --connect-timeout 2 -H "Host: ifconfig.me" $dnsIfconfigMe | head -n 1 | awk '{print $1'})
            if [ "$wanIp" == "" ]; then
                wanIp="-------"
            fi
        fi
        printf "wanIp=%-15s " $wanIp

        printf "\n"

        sleep 0.7
    done
}

Example output:

2020-07-13T23:43:59  lanIp=192.168.1.6/24 gw=192.168.1.1    pGw=0.264   p8=53.338  dnsAmz=176.32.98.166   pAmz=127.532 wanIp=1.2.3.4
2020-07-13T23:44:01  lanIp=192.168.1.6/24 gw=192.168.1.1    pGw=0.190   p8=47.107  dnsAmz=176.32.103.205  pAmz=111.309 wanIp=1.2.3.4
2020-07-13T23:44:02  lanIp=192.168.1.6/24 gw=192.168.1.1    pGw=0.290   p8=------- dnsAmz=176.32.103.205  pAmz=118.834 wanIp=1.2.3.4
2020-07-13T23:44:04  lanIp=192.168.1.6/24 gw=192.168.1.1    pGw=0.328   p8=143.387 dnsAmz=176.32.103.205  pAmz=135.333 wanIp=1.2.3.4
2020-07-13T23:44:06  lanIp=192.168.1.6/24 gw=192.168.1.1    pGw=0.313   p8=17.944  dnsAmz=176.32.103.205  pAmz=105.977 wanIp=1.2.3.4
2020-07-13T23:44:07  lanIp=192.168.1.6/24 gw=192.168.1.1    pGw=0.223   p8=22.505  dnsAmz=176.32.103.205  pAmz=97.623  wanIp=1.2.3.4
2020-07-13T23:44:08  lanIp=192.168.1.6/24 gw=192.168.1.1    pGw=0.250   p8=23.917  dnsAmz=176.32.103.205  pAmz=98.403  wanIp=1.2.3.4
2020-07-13T23:44:09  lanIp=192.168.1.6/24 gw=192.168.1.1    pGw=0.220   p8=41.262  dnsAmz=176.32.103.205  pAmz=100.828 wanIp=1.2.3.4

Tuesday, June 23, 2020

Upgrading Alpine Linux diskless setup on a Raspberry Pi

Here's a procedure I've come up with to fully upgrade a diskless setup of Alpine Linux on a running Raspberry Pi. It's not pretty, but seems gets the job done, based on the tests I did.

I tested this procedure on each version from 3.3 to to 3.12. The system I tested on had a working setup of Apache2 with PHP, as well as Samba. Here's the result of the testing.

  • Started with v3.3.
  • Confirmed 3.3 to 3.4. Package "php-apache2" package had disappeared and had to be manually replaced by "php5-apache2".
  • Confirmed 3.4 to 3.5.
  • Confirmed 3.5 to 3.6.
  • Confirmed 3.6 to 3.7.
  • Confirmed 3.7 to 3.8.
  • Could not boot after upgrading from 3.8 to 3.9. But a fresh installation of 3.9 was also unbootable, so assuming 3.9 is broken. Seems to be a known issue.
  • Confirmed 3.8 to 3.10. Package "php5-apache2" package had disappeared and had to be manually replaced by "php7-apache2".

The process I came up with can be summarized to:

  1. Remount the SD card in read-write mode.
  2. Move all files in the SD card FAT32 root to a subdirectory such as old/2020-06-23T04_22_54.
  3. Download a fresh alpine-rpi-x.y.z-armhf.tar.gz file and untar it to the SD card root. This will give you Alpine Linux's newest Raspberry compatible Linux kernel image.
  4. Remount the SD card back to read-only mode.
  5. Update the /etc/apk/repositories (in the in-memory file system) to point to the target version.
  6. Do an "apk upgrade --available" to download the updated versions of the packages you used, so they will be ready on the device after rebooting.
  7. Reboot into the new setup.

If anything goes wrong, you can put the SD card in another PC, delete the broken setup, and copy your old working setup back out from the "old" directory.

Here are the commands. As I don't know if this process will keep working in future versions, I recommend running them one line at a time and ensuring the line does what you expect.

targetVersion=v3.12

# Move everything to a safe place as it is now.
mount -oremount,rw /media/mmcblk0p1
cd /media/mmcblk0p1
olddir=/media/mmcblk0p1/old/$(date --utc "+%Y-%m-%dT%H_%M_%S")
mkdir -p $olddir
mv *  $olddir   # This will give a warning that it cannot move old into old. That's ok.
mv .* $olddir   # This will give a warning that it cannot move . and .. . That's ok.

# Download fresh Alpine Linux image and replace existing.
newImgFile=$(curl http://dl-cdn.alpinelinux.org/alpine/$targetVersion/releases/armhf/latest-releases.yaml --silent | grep "file: alpine-rpi-" | sed "s/ *file: //g")
echo $newImgFile
mount -oremount,rw /media/mmcblk0p1
wget http://dl-cdn.alpinelinux.org/alpine/$targetVersion/releases/armhf/$newImgFile
wget http://dl-cdn.alpinelinux.org/alpine/$targetVersion/releases/armhf/${newImgFile}.sha512
sha512sum -c ${newImgFile}.sha512    # Ensure it says OK
tar -xzf $newImgFile
rm ${newImgFile}*
cp $olddir/usercfg.txt .
mount -oremount,ro /media/mmcblk0p1

# Update packages to the target distro.
cat > /etc/apk/repositories <<EOF
/media/mmcblk0p1/apks
http://dl-cdn.alpinelinux.org/alpine/$targetVersion/main
http://dl-cdn.alpinelinux.org/alpine/$targetVersion/community
EOF
setup-apkcache /media/mmcblk0p1/cache
apk update
apk upgrade --available
apk add
apk cache download
apk -v cache clean
lbu commit -d

reboot

# After reboot, ensure there aren't packages that no longer exist in this version.
apk add

Sunday, June 21, 2020

Add shortcut keys to Google Drawing

Here's a quick and dirty way to add shortcut keys to Google Drawing. It probably only works in Chrome.

It's a JavaScript snippet that can be run in the context of the Google Drawing page in order to automate clicking on the toolbar buttons.

The keys this code adds are:

  • Alt-A: Arrow
  • Alt-L: Line
  • Alt-B: Rectangle (box)
  • Alt-C: Cylinder
  • Alt-X: Text box

Here's a bookmarklet with the script: GDraw shortcuts. Drag this hyperlink to your bookmarks bar and click it when you are inside Google Drawing in order to enable the shortcut keys. (Thanks to Peter Coles for his bookmarklet creation tool)

And here's the code, in case you want to extend it with additional shortcut keys.

function newMouseEvent(eventType, x, y) {
    return new MouseEvent(eventType, {
        "view": window,
        "bubbles": true,
        "cancelable": true,
        "screenX": x,
        "screenY": y
    });
}

function myclick(x, y) {
    var el = document.elementFromPoint(x, y);

    el.dispatchEvent(newMouseEvent("mouseover", x, y));
    el.dispatchEvent(newMouseEvent("mousedown", x, y));
    el.dispatchEvent(newMouseEvent("mouseup", x, y));
    el.dispatchEvent(newMouseEvent("click", x, y));
}

function clickLineButton() {
  var r = document.getElementById("lineMenuButton").getBoundingClientRect();
  myclick(r.x+3, r.y+3);

  setTimeout(function() {
    myclick(r.x+3, r.y+50);
  }, 200);
}

function clickArrowButton() {
  var r = document.getElementById("lineMenuButton").getBoundingClientRect();
  myclick(r.x+3, r.y+3);

  setTimeout(function() {
    myclick(r.x+3, r.y+80);
  }, 200);
}

function clickBoxButton() {
  var r = document.getElementById("shapeButton").getBoundingClientRect();
  myclick(r.x+3, r.y+3);

  setTimeout(function() {
    myclick(r.x+3, r.y+40);

    setTimeout(function() {
      myclick(r.x+172, r.y+56);
    }, 200);

  }, 200);
}

function clickCylinderButton() {
  var r = document.getElementById("shapeButton").getBoundingClientRect();
  myclick(r.x+3, r.y+3);

  setTimeout(function() {
    myclick(r.x+3, r.y+40);

    setTimeout(function() {
      myclick(r.x+197, r.y+223);
    }, 200);

  }, 200);
}

function myKeydownHandler(e) {
  /* A key */
  if (e.keyCode == 65 && e.altKey == true) { 
    clickArrowButton();
    e.preventDefault();
  }

  /* L key */
  if (e.keyCode == 76 && e.altKey == true) { 
    clickLineButton();
    e.preventDefault();
  }

  /* B key */
  if (e.keyCode == 66 && e.altKey == true) {
    clickBoxButton();
    e.preventDefault();
  }

  /* C key */
  if (e.keyCode == 67 && e.altKey == true) { 
    clickCylinderButton();
    e.preventDefault();
  }
}

window.addEventListener("keydown", myKeydownHandler, true);

var elements = document.getElementsByTagName("iframe");
for (var i = 0; i < elements.length; i++) {
  if (elements[i].src == "" || elements[i].src == "about:blank") {
    elements[i].contentWindow.addEventListener("keydown", myKeydownHandler, true);
  }
}

Thursday, June 18, 2020

Reach servers behind NAT with WireGuard and a VPS, while preserving source IP

This post describes an approach to let your Linux machine behind a NAT be reachable from the public internet, using a cheap VPS and Wireguard.

It's similar to this post in Ralph's Blog, but with with one distinction: it preserves the source IP of the incoming connection. This may be important depending on your use. For me, I was running a PHP web app for which I needed the $_SERVER["REMOTE_ADDR"] field to be the true and correct IP of the client, and not just the IP of the gateway that forwarded the traffic.

Values used in this example:

  • 192.168.51.0/24 is the private VPN network between the cloud VPS host and the server you want to expose.
  • 192.168.51.1 is the VPN network IP of the cloud VPS.
  • 192.168.51.2 is the VPN network IP of the server you want to expose.
  • 50.40.30.20 is the public IP of the cloud VPS.
  • 59014 the UDP port number I randomly chose for WireGuard on the VPS.
  • 80 and 443 are the TCP port numbers I want to forward from my VPS to my local server.

On the cloud VPS:

mkdir -p /etc/wireguard
cd /etc/wireguard
wg genkey | tee privatekey | wg pubkey > publickey
chown 600 privatekey
cat privatekey
cat publickey

# Replace the PrivateKey in the [Interface] section with the actual privatekey value generated above.
# Replace the PublicKey in the [Peer] section with the publickey value you generate ON YOUR LOCAL SERVER in the later steps.
cat > /etc/wireguard/wg0.conf <<EOF
[Interface]
Address = 192.168.51.1
PrivateKey = KF+X0SO2lkBFnxTCK4R5PCv/FC6+wgH6rt6wWHkVVHE=
ListenPort = 59014

[Peer]
PublicKey = y700qaTgjDjMrLc9tGF7+PKFrX7uHyIJJ/s2zvadKVY=
AllowedIPs = 192.168.51.2/32
EOF

chmod 600 wg0.conf

# Start the VPN server
wg-quick up wg0

echo 0 > /proc/sys/net/ipv4/ip_forward
iptables -P FORWARD DROP
iptables -A FORWARD -i eth0 -o wg0  -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -A FORWARD -i wg0  -o eth0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -A FORWARD -i wg0  -o eth0 -m conntrack --ctstate NEW -j ACCEPT
iptables -A FORWARD -i eth0 -o wg0  -p tcp --syn --dport 80  -m conntrack --ctstate NEW -j ACCEPT
iptables -A FORWARD -i eth0 -o wg0  -p tcp --syn --dport 443 -m conntrack --ctstate NEW -j ACCEPT
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80  -j DNAT --to-destination 192.168.51.2
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 443 -j DNAT --to-destination 192.168.51.2
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
echo 1 > /proc/sys/net/ipv4/ip_forward

On your local server, instead of using wg-quick, we will set up the interface a bit more manually. This is in order to have finer control over routing. We want connections originating on this server to exit via this server's normal internet connection, and not via the WireGuard VPN connection. However, packets part of TCP connections that got forwarded to us on the VPN from the VPS should be routed back via the VPS.

On your local server:

mkdir -p /etc/wireguard
cd /etc/wireguard
wg genkey | tee privatekey | wg pubkey > publickey
chown 600 privatekey
cat privatekey
cat publickey

# Replace the PrivateKey in the [Interface] section with the actual privatekey value generated above.
# Replace the PublicKey in the [Peer] section with the publickey value you generated ON YOUR CLOUD VPS in the earlier steps.
cat > /etc/wireguard/wg0.conf <<EOF
[Interface]
PrivateKey = YIOy80YJdcMqI1WCMROoyUfsfkc6GhP0KqRS3BfcoEs=

[Peer]
PublicKey = i1VQnnODEBbf+P9cyd9XmB9G58qEpM53TbMXn1UAJx8=
Endpoint = 50.40.30.20:59014
AllowedIPs = 0.0.0.0/0
PersistentKeepalive = 25
EOF

chmod 600 wg0.conf

# Set up VPN connection
ip link add wg0 type wireguard
wg setconf wg0 /etc/wireguard/wg0.conf
ip address add 192.168.51.2 dev wg0
ip link set mtu 1420 up dev wg0
ip rule add from 192.168.51.2 table 51820
ip rule add iif wg0 table 51820
ip route add 192.168.51.0/24 dev wg0
ip route add 0.0.0.0/0 via 192.168.51.1 dev wg0 table 51820

# Shut down VPN connection (don't run these steps now, but keep them handy)
ip rule delete from 192.168.51.2 table 51820
ip rule delete iif wg0 table 51820
ip link delete dev wg0

You should now be able to receive TCP traffic on port 80 and 443 on your local server, and the src IP address of the actual real client will be preserved all the way to your local server.

WireGuard kernel module for Alpine Linux on diskless Raspberry Pi

As of writing (Alpine Linux v3.12), the WireGuard kernel module is not part of the Alpine Linux Raspberry Pi kernel by default. There's a community module called wireguard-rpi2, but it doesn't work in a system set up in diskless mode. The reason it doesn't work, is because in diskless mode on a Raspberry Pi, Alpine Linux uses a read-only loopback file system to store its kernel modules. This filesystem is mounted on /.modloop .

The workaround I came up with, is to use an overlay file system to make the kernel modules directory writable.

Here's the workaround:
# Remount the kernel module directory /.modloop as an overlay, to allow writing
modprobe overlay
mkdir -p /.modloop.lower /.modloop.upper /.modloop.workdir
mount /dev/loop0 /.modloop.lower
umount /.modloop/
mount -t overlay -o lowerdir=/.modloop.lower,upperdir=/.modloop.upper,workdir=/.modloop.workdir none /.modloop
lbu include /.modloop.upper
lbu commit -d

# Manually get the Wireguard kernel module to avoid installing the wireguard-rpi2 which does not work with diskless systems
cd /tmp
pkgname=$(apk list | grep wireguard-rpi2 | cut -d " " -f 1)
wget http://dl-cdn.alpinelinux.org/alpine/v3.12/community/armhf/$pkgname.apk
mkdir /tmp/$pkgname
tar -xzf $pkgname.apk -C /tmp/$pkgname
mkdir -p /lib/modules/$(uname -r)/extra/
cp /tmp/$pkgname/lib/modules/$(uname -r)/extra/wireguard.ko /lib/modules/$(uname -r)/extra/
rm -fr /tmp/$pkgname /tmp/$pkgname.apk
depmod

# Create an init script to remount the /.modloop overlay on next boot
cat > /etc/init.d/modloopoverlay <<EOF
#!/sbin/openrc-run

depend() {
    before networking
    need modules
}

start() {
    ebegin "Starting modloop overlay"
    modprobe overlay
    mkdir -p /.modloop.lower /.modloop.upper /.modloop.workdir
    if [ ! -d /.modloop.lower/modules ]; then
        mount /dev/loop0 /.modloop.lower
    fi
    umount /.modloop
    mount -t overlay -o lowerdir=/.modloop.lower,upperdir=/.modloop.upper,workdir=/.modloop.workdir none /.modloop
    eend 0
}
EOF
chmod +x /etc/init.d/modloopoverlay
/etc/init.d/modloopoverlay restart
rc-update add modloopoverlay boot
lbu include /etc/init.d/modloopoverlay

apk add wireguard-tools

# Create an init script to start wg0
cat > /etc/init.d/wg0 <<EOF
#!/sbin/openrc-run

depend() {
    need networking ntpd modloopoverlay
}

start() {
    ebegin "Starting Wireguard tunnel wg0"
    ntpd -n -q -p pool.ntp.org
    date
    wg-quick up wg0
    eend $?
}

stop() {
    ebegin "Stopping Wireguard tunnel wg0"
    wg-quick down wg0
    eend 0
}
EOF
chmod +x /etc/init.d/wg0
/etc/init.d/wg0 restart
rc-update add wg0 default
lbu include /etc/init.d/wg0

lbu commit -d

# inspect the overlay file
tar -tvf /media/mmcblk0p1/localhost.apkovl.tar.gz

Friday, January 10, 2020

Alternative to negative lookbehinds in regular expressions

I recently faced the problem of requiring negative lookbehinds in a regex engine that does not support them (Golang's Regexp package, RE2, and Hyperscan).

For example, say you want to match the string "def" except if it is preceeded by "abc". In PCRE using negative lookbehinds you could achieve this with the regex:

(?<!abc)def

But this will not work in all regex engines, so I needed an alternative.

I found this Stackoverflow answer by user Sebastian Proske. The approach is clever, but tricky to comprehend and write for long and complex negative lookbehinds. I will try to explain the general approach here, and then present a script to help use this approach for longer and more complex cases.

Continuing the example above, the approach works by having alternatives for each possible prefix that is not "abc". These alternatives are:

  • The beginning of the string, or a character that is not "c", followed by "def"
  • The beginning of the string, or a character that is not "b", followed by "cdef"
  • The beginning of the string, or a character that is not "a", followed by "bcdef"

Written as a regex (line breaks and indentation for readability are not part of the regex):

(
    (^|[^c])def
    |
    (^|[^b])cdef
    |
    (^|[^a])bcdef
)

We can extract the common suffix out:

(
    (^|[^c])
    |
    (^|[^b])c
    |
    (^|[^a])bc
)
def

To support multiple alternative negative lookbehinds, such as both "abc" and "1234", which in PCRE syntax this can be written as "(?<!abc|1234)def", we can write:

(
    (^|[^c4])
    |
    (^|[^b])c
    |
    (^|[^a])bc
    |
    (^|[^3])4
    |
    (^|[^2])34
    |
    (^|[^1])234
)
def

To support character classes in the negative lookbehinds, such as "[bB]", which in PCRE syntax can be written as "(?<!a[bB]c)def", we can write:

(
    (^|[^c])
    |
    (^|[^bB])c
    |
    (^|[^a])[bB]c
)
def

For better equivalence to the negative lookbehinds, we can additionally use non-capturing groups:

(?:
    (?:^|[^c])
    |
    (?:^|[^b])c
    |
    (?:^|[^a])bc
)
def

Here's a quick and dirty Python script to help construct regexes following this approach. The example input here is equivalent to "(?<!a[bB]c|1234)".

The script is also available to conveniently run in your browser on repl.it: https://repl.it/@allanrbo/regexnegativelookbehindalternative1

negativePrefixes = [
"a[bB]c",
"1234",
]

def removeDuplicateChars(s):
  return "".join([c for i,c in enumerate(s) if c not in s[:i]])

def removeChars(s, charsToRemove):
  return "".join([c for i,c in enumerate(s) if c not in charsToRemove])

# Split into arrays of strings. Each string is either a single char, or a char class.
negativePrefixesSplit = []
for np in negativePrefixes:
  npSplit = []
  curCc = ""
  inCc = False
  for c in np:
    if c == "[":
      inCc = True
    elif c == "]":
      npSplit.append(removeDuplicateChars(curCc))
      curCc = ""
      inCc = False
    else:
      if inCc:  
        if c in "-\\":
          raise "Only really simply char classes are currently supported. No ranges or escapes, sorry."
        curCc += c
      else:
        npSplit.append(c)
  negativePrefixesSplit.append(npSplit)

allexprs = []

class Expr():
  pass

suffixLength = 0
while True:
  suffixes = []
  for np in negativePrefixesSplit:
    if suffixLength < len(np):
      suffixes.append(np[len(np)-suffixLength-1:])

  if len(suffixes) == 0:
    break

  exprs = []
  for suffix in suffixes:
    curChar = suffix[0]
    remainder = suffix[1:]
    expr = Expr()
    expr.curChar = curChar
    expr.remainder = remainder
    exprs.append(expr)

  # Is the remainder a subset of any other suffixes remainders?
  for i in range(len(exprs)):
    e1 = exprs[i]
    for j in range(len(exprs)):
      e2 = exprs[j]
      isSubset = True
      for k in range(len(e1.remainder)):
        if not set(e1.remainder[k]).issubset(set(e2.remainder[k])):
          isSubset = False
          break
      if isSubset:
        if e1.curChar == e2.curChar:
          e1.remainder = e2.remainder
          continue

        e1.curChar += e2.curChar
        e1.curChar = removeDuplicateChars(e1.curChar)
        for k in range(len(e1.remainder)):
          if len(set(e2.remainder[k]) - set(e1.remainder[k])) > 0:
            charsInCommon = "".join(set(e2.remainder[k]) & set(e1.remainder[k]))
            e2.remainder[k] = removeChars(e2.remainder[k], charsInCommon)

  # Remove duplicate expressions
  exprsFiltered = []
  for i in range(len(exprs)):
    e1 = exprs[i]
    alreadyExists = False
    for j in range(len(exprs)):
      if i == j:
        break

      e2 = exprs[j]

      sameC = set(e1.curChar) == set(e2.curChar)
      sameR = True
      for k in range(len(e1.remainder)):
        if set(e1.remainder[k]) != set(e2.remainder[k]):
          sameR = False
          break
      if sameC and sameR:
        alreadyExists = True
        break

    if not alreadyExists:
      exprsFiltered.append(e1)
  
  allexprs.extend(exprsFiltered)

  suffixLength += 1
  continue

out = "(?:\n"
for i in range(len(allexprs)):
  e = allexprs[i]
  out += ("(?:^|[^" + e.curChar + "])")
  for c in e.remainder:
    if len(c) > 1:
      out += "[" + c + "]"
    else:
      out += c
  if i != len(allexprs)-1:
    out += "|"
  out += "\n"
out += ")"

print("Human readable:")
print(out)
print()
print("Single line:")
print(out.replace("\n",""))
Example output:
Human readable:
(?:
(?:^|[^c4])|
(?:^|[^bB])c|
(?:^|[^3])4|
(?:^|[^a])[bB]c|
(?:^|[^2])34|
(?:^|[^1])234
)

Single line:
(?:(?:^|[^c4])|(?:^|[^bB])c|(?:^|[^3])4|(?:^|[^a])[bB]c|(?:^|[^2])34|(?:^|[^1])234)