a bloga bloghttp://sven.stormbind.net/blog/a blogikiwiki2024-02-08T10:12:50ZUse GitHub CLI to List all Repository Secretshttp://sven.stormbind.net/blog/posts/misc_gh_get_all_repo_secrets/2024-02-08T10:12:50Z2024-02-08T10:12:50Z
<p>Write it down before I forget about it again:</p>
<pre><code>for x in $(gh api graphql --paginate -f query='query($endCursor:String) { organization(login:"myorg") {
repositories(first: 100, after: $endCursor, isArchived:false) {
pageInfo {
hasNextPage
endCursor
}
nodes {
name
}
}
}
}' --jq '.data.organization.repositories.nodes[].name'); do
secrets=$(gh secret list --json name --jq '.[].name' -R "myorg/${x}" | tr '\n' ',')
if ! [ -z "${secrets}" ]; then
echo "${x},${secrets}"
fi
done
</code></pre>
<p>Requests a list of all not archived repositories in a GitHub org and queries
repository secrets. If we find some we output the repo name and the
secrets in a comma separated list. Not real CSV, but good enough for further
processing. I've to admit it's kinda beautiful what you can do with the
gh cli by now. Sadly it seems the secrets are not yet available via GraphQL
(or I missed it in the docs), so I just use the gh cli to do the REST calls.</p>
Curing vpnc-scripts Symptomshttp://sven.stormbind.net/blog/posts/deb_vpnc_scripts_butchering/2023-10-25T14:15:03Z2023-10-25T14:05:32Z
<p>I stick to some very archaic workflows, e.g. to connect
to some corp VPN I just run <code>sudo vpnc-connect</code>
and later on <code>sudo vpnc-disconnect</code>. In the past that also
managed to restore my <code>resolv.conf</code>, currently it doesn't.
According to a colleague that's also the case for Ubuntu.</p>
<p>Taking a step back, the sane way would be to use the
NetworkManager vpnc plugin, but that does not work with
this specific case because we use uncool VPN tech which
requires the <code>Enable weak authentication</code> setting for vpnc.
There is a feature request open for that one at
<a href="https://gitlab.gnome.org/GNOME/NetworkManager-vpnc/-/issues/11">https://gitlab.gnome.org/GNOME/NetworkManager-vpnc/-/issues/11</a></p>
<p>Taking another step back I thought that it shouldn't be that
hard to add some checkbox, a boolean and render out another
config flag or line in a config file. Not as intuitive as I
thought this mix of XML and C. So let's quickly look elsewhere.</p>
<p>What happens is that the backup files in <code>/var/run/vpnc/</code> are
created by the vpnc-scripts script called <code>vpnc-script</code>, but not
moved back, because it adds some pid as a suffix and the pid is
not the final pid of the vpnc process. Basically it can not find
the backup when it tries to restore it. So I decided to replace the
pid guessing code with a suffix made up of the gateway IP and the
tun interface name. No idea if that is stable in all circumstance
(someone with a vpn name DNS RR?) or several connections to different
gateways. But good enough for myself, so here is my patch:</p>
<pre><code>vpnc-scripts [master]$ cat debian/patches/replace-pid-detection
Index: vpnc-scripts/vpnc-script
===================================================================
--- vpnc-scripts.orig/vpnc-script
+++ vpnc-scripts/vpnc-script
@@ -91,21 +91,15 @@ OS="`uname -s`"
HOOKS_DIR=/etc/vpnc
-# Use the PID of the controlling process (vpnc or OpenConnect) to
-# uniquely identify this VPN connection. Normally, the parent process
-# is a shell, and the grandparent's PID is the relevant one.
-# OpenConnect v9.0+ provides VPNPID, so we don't need to determine it.
-if [ -z "$VPNPID" ]; then
- VPNPID=$PPID
- PCMD=`ps -c -o cmd= -p $PPID`
- case "$PCMD" in
- *sh) VPNPID=`ps -o ppid= -p $PPID` ;;
- esac
+# This whole script is called twice via vpnc-connect. On the first run
+# the variables are empty. Catch that and move on when they're there.
+if [ -n "$VPNGATEWAY" ]; then
+ BACKUPID="${VPNGATEWAY}_${TUNDEV}"
+ DEFAULT_ROUTE_FILE=/var/run/vpnc/defaultroute.${BACKUPID}
+ DEFAULT_ROUTE_FILE_IPV6=/var/run/vpnc/defaultroute_ipv6.${BACKUPID}
+ RESOLV_CONF_BACKUP=/var/run/vpnc/resolv.conf-backup.${BACKUPID}
fi
-DEFAULT_ROUTE_FILE=/var/run/vpnc/defaultroute.${VPNPID}
-DEFAULT_ROUTE_FILE_IPV6=/var/run/vpnc/defaultroute_ipv6.${VPNPID}
-RESOLV_CONF_BACKUP=/var/run/vpnc/resolv.conf-backup.${VPNPID}
SCRIPTNAME=`basename $0`
# some systems, eg. Darwin & FreeBSD, prune /var/run on boot
</code></pre>
<p>Or rolled into a debian package at <a href="https://sven.stormbind.net/debian/vpnc-scripts/">https://sven.stormbind.net/debian/vpnc-scripts/</a></p>
<p>The colleague decided to stick to NetworkManager, moved the vpnc binary aside and
added a wrapper which invokes vpnc with <code>--enable-weak-authentication</code>. The beauty is,
all of this will break on updates, so at some point someone has to
understand GTK4 to fix the NetworkManager plugin for good. <img src="http://sven.stormbind.net/blog/smileys/smile.png" alt=":)" /></p>
htop on stage in the theatrehttp://sven.stormbind.net/blog/posts/misc_htop_in_the_theatre/2023-06-14T09:29:21Z2023-06-14T09:28:37Z
<p>Always amusing to see some more or less famous
open source tools on stage or in movies. Lately
we watched <a href="https://k-k-t.de/stuecke/the-me">THE ME</a>
(german only) which is mixing live playing of actors and pre
recorded video material. In one of the early video sequences a
fictional console interface is displayed, claiming to be running
on a Macbook, and <a href="https://htop.dev/">htop</a>
is used to look for a suspicious process.</p>
GCP: Private Service Connect Forwarding Rules can not be Updatedhttp://sven.stormbind.net/blog/posts/gcp_psc_forwarding_rule_terraform/2023-05-15T07:54:08Z2023-05-15T07:21:21Z
<p>PSA for those foolish enough to use Google Cloud and try to use private service connect:
If you want to change the serviceAttachment your private service connect forwarding
rule points at, you must delete the forwarding rule and create a new one. Updates
are not supported. I've done that in the past via terraform, but lately encountered
strange errors like this:</p>
<pre><code>Error updating ForwardingRule: googleapi: Error 400: Invalid value for field 'target.target':
'<https://www.googleapis.com/compute/v1/projects/mydumbproject/regions/europe-west1/serviceAttachments/
k8s1-sa-xyz-abc>'. Unexpected resource collection 'serviceAttachments'., invalid
</code></pre>
<p>Worked around that with the help of <code>terrraform_data</code> and <code>lifecycle</code>:</p>
<pre><code>resource "terraform_data" "replacement" {
input = var.gcp_psc_data["target"]
}
resource "google_compute_forwarding_rule" "this" {
count = length(var.gcp_psc_data["target"]) > 0 ? 1 : 0
name = "${var.gcp_psc_name}-psc"
region = var.gcp_region
project = var.gcp_project
target = var.gcp_psc_data["target"]
load_balancing_scheme = "" # need to override EXTERNAL default when target is a service attachment
network = var.gcp_network
ip_address = google_compute_address.this.id
lifecycle {
replace_triggered_by = [
terraform_data.replacement
]
}
}
</code></pre>
<p>See also <a href="https://developer.hashicorp.com/terraform/language/resources/terraform-data#example-usage-data-for-replace_triggered_by">terraform data for replace_triggered_by</a>.</p>
exfat-fuse 1.4 in experimentalhttp://sven.stormbind.net/blog/posts/deb_fuse-exfat-1.4/2023-03-03T16:23:23Z2023-03-03T15:39:48Z
<p>I know a few people hold on to the exFAT fuse implementation due the
support for timezone offsets, so here is a small update for you.
Andrew released 1.4.0, which includes the timezone offset support, which
was so far only part of the git master branch. It also fixes a,
from my point of view very minor, security issue
<a href="https://security-tracker.debian.org/tracker/CVE-2022-29973">CVE-2022-29973</a>.
In addition to that it's the first build with fuse3 support. If you
still use this driver, pick it up in experimental (we're in the bookworm freeze
right now), and give it a try. I'm personally not using it anymore beyond a very
basic "does it mount" test.</p>
CentOS 9, stunnel, an openssl memory leak and a VirtualBox crashhttp://sven.stormbind.net/blog/posts/centos_c9s_stunnel_memleak/2022-11-09T13:55:09Z2022-10-16T18:24:40Z
<p>tl;dr; OpenSSL 3.0.1 leaks memory in <code>ssl3_setup_write_buffer()</code>, seems to be
fixed in <del>3.0.5</del> 3.0.2. The issue manifests at least in stunnel
and keepalived on CentOS 9. In addition I learned the hard way that running a
not so recent VirtualBox version on Debian bullseye let to dh parameter generation
crashing in libcrypto in <code>bn_sqr8x_internal()</code>.</p>
<p>A recent rabbit hole I went down. The actual bug in openssl was nailed down and
documented by
<a href="https://github.com/acassen/keepalived/issues/2199#issuecomment-1277175404">Quentin Armitage on GitHub in keepalived</a>
My bugreport with all back and forth in the RedHat Bugzilla is
<a href="https://bugzilla.redhat.com/show_bug.cgi?id=2128412">#2128412</a>.</p>
<h1>Act I - Hello stunnel, this is the OOMkiller Calling</h1>
<p>We started to use stunnel on Google Cloud compute engine instances running CentOS 9.
The loadbalancer in front of those instances used a TCP health check to validate the
backend availability. A day or so later the stunnel instances got killed by the OOMkiller. Restarting stunnel and looking into <code>/proc/<pid>/smaps</code> showed a heap
segment growing quite quickly.</p>
<h1>Act II - Reproducing the Issue</h1>
<p>While I'm not the biggest fan of VirtualBox and Vagrant I've to admit it's quite
nice to just fire up a VM image, and give other people a chance to recreate that
setup as well. Since VirtualBox is no longer released with Debian/stable I just
recompiled what was available in unstable at the time of the bullseye release, and
used that. That enabled me now to just start a CentOS 9 VM, setup stunnel with a
minimal config, grab netcat and a for loop and watch the memory grow.
E.g. <code>while true; do nc -z localhost 2600; sleep 1; done</code>
To my surprise, in addition to the memory leak, I also observed some crashes but
did not yet care too much about those.</p>
<h1>Act III - Wrong Suspect, a Workaround and Bugreporting</h1>
<p>Of course the first idea was that something must be wrong in stunnel itself. But
I could not find any recent bugreports. My assumption is that there are
still a few people around using CentOS and stunnel, so someone else should probably
have seen it before. Just to be sure I recompiled the latest stunnel package from
Fedora. Didn't change anything. Next I recompiled it without almost all the patches
Fedora/RedHat carries. Nope, no progress.
Next idea: Maybe this is related to the fact that we do not initiate a TLS context
after connecting? So we changed the test case from <code>nc</code> to <code>openssl s_client</code>, and
the loadbalancer healthcheck from TCP to a TLS based one. Tada, a workaround, no
more memory leaking.
In addition I gave Fedora a try (they have Vagrant Virtualbox images in the "Cloud"
Spin, e.g.
<a href="https://ftp.halifax.rwth-aachen.de/fedora/linux/releases/36/Cloud/x86_64/images/">here for Fedora 36</a>)
and my local Debian installation a try. No leaks experienced on both.
Next I reported
<a href="https://bugzilla.redhat.com/show_bug.cgi?id=2128412">#2128412</a>.</p>
<h1>Act IV - Crash in libcrypto and a VirtualBox Bug</h1>
<p>When I moved with the test case from the Google Cloud compute instance to my
local VM I encountered some crashes. That morphed into a real problem when I
started to run stunnel with gdb and valgrind. All crashes happened in libcrypto
<code>bn_sqr8x_internal()</code> when generating new dh parameter (stunnel does that for
you if you do not use static dh parameter). I quickly worked around that by
generating static dh parameter for stunnel.
After some back and forth I suspected VirtualBox as the culprit. Recompiling
the current VirtualBox version (6.1.38-dfsg-3) from unstable on bullseye works
without any changes. Upgrading actually fixed that issue.</p>
<h1>Epilog</h1>
<p>I highly appreciate that RedHat, with all the bashing around the future of
CentOS, still works on community contributed bugreports. My kudos go to
Clemens Lang. <img src="http://sven.stormbind.net/blog/smileys/smile.png" alt=":)" />
Now that the root cause is clear, I guess RedHat will push out a fix for the
openssl 3.0.1 based release they have in RHEL/CentOS 9. Until that is available
at least stunnel and keepalived are known to be affected. If you run stunnel
on something public it's not that pretty, because already a low rate of TCP
connections will result in a DoS condition.</p>
Emulating Raspi2 like hardware with RaspiOS in 2022http://sven.stormbind.net/blog/posts/deb_qemu_rpi2_2022/2022-04-13T19:14:15Z2022-04-12T09:27:36Z
<p>Update of my notes from
<a href="https://sven.stormbind.net/blog/posts/deb_qemu_rpi2_2020/">2020</a>.</p>
<pre><code># Download a binary device tree file and matching kernel a good soul uploaded to github
wget https://github.com/vfdev-5/qemu-rpi2-vexpress/raw/master/kernel-qemu-4.4.1-vexpress
wget https://github.com/vfdev-5/qemu-rpi2-vexpress/raw/master/vexpress-v2p-ca15-tc1.dtb
# Download the official Rasbian image without X
wget https://downloads.raspberrypi.org/raspios_lite_armhf/images/raspios_lite_armhf-2022-04-07/2022-04-04-raspios-bullseye-armhf-lite.img.xz
unxz 2022-04-04-raspios-bullseye-armhf-lite.img.xz
# Convert it from the raw image to a qcow2 image and add some space
qemu-img convert -f raw -O qcow2 2022-04-04-raspios-bullseye-armhf-lite.img rasbian.qcow2
qemu-img resize rasbian.qcow2 4G
# make sure we get a user account setup
echo "me:$(echo 'test123'|openssl passwd -6 -stdin)" > userconf
sudo guestmount -a rasbian.qcow2 -m /dev/sda1 /mnt
sudo mv userconf /mnt
sudo guestunmount /mnt
# start qemu
qemu-system-arm -m 2048M -M vexpress-a15 -cpu cortex-a15 \
-kernel kernel-qemu-4.4.1-vexpress -no-reboot \
-smp 2 -serial stdio \
-dtb vexpress-v2p-ca15-tc1.dtb -sd rasbian.qcow2 \
-append "root=/dev/mmcblk0p2 rw rootfstype=ext4 console=ttyAMA0,15200 loglevel=8" \
-nic user,hostfwd=tcp::5555-:22
# login at the serial console as user me with password test123
sudo -i
# enable ssh
systemctl enable ssh
systemctl start ssh
# resize partition and filesystem
parted /dev/mmcblk0 resizepart 2 100%
resize2fs /dev/mmcblk0p2
</code></pre>
<p>Now I can login via ssh and start to play:</p>
<pre><code>ssh me@localhost -p 5555
</code></pre>
Suntime Calculation with Lua and the Great Gift of Open Sourcehttp://sven.stormbind.net/blog/posts/misc_suntime_calc_lua_and_greatfulness_of_oss/2022-02-04T19:15:11Z2022-02-03T20:11:09Z
<p>tl;dr I ported a part of the python-suntime library to Lua to use it on OpenWRT and
RutOS powered devices. <a href="https://git.sven.stormbind.net/?p=sven/scripts.git;a=blob;f=weblogpro/suntime.lua;hb=HEAD">
suntime.lua</a></p>
<p>There are those unremarkable things which let you pause for
a moment, and realize what a great gift of our time open source software
and open knowledge is. At some point in time someone figured out
how to calculate the sunrise and sunset time on the current date for
your location. Someone else wrote that up and probably again a different
person <a href="https://web.archive.org/web/20161022214335/https://williams.best.vwh.net/sunrise_sunset_algorithm.htm">published it on the internet</a>. The
Internet Archive preserved a copy of it so I can still link to it. Someone
took this algorithm and <a href="https://stackoverflow.com/a/39867990">published a code sample on StackOverflow</a>, which was later on used by the SatAgro guys
to create the
<a href="https://github.com/SatAgro/suntime/blob/master/suntime/suntime.py#L16">
python-suntime</a> library.
Now I could come along, copy the core function of this library, convert it
within a few hours - mostly spent learning a bit of Lua, to a working script
fulfilling my needs.</p>
Running OpenWRT x86 in qemuhttp://sven.stormbind.net/blog/posts/deb_qemu_local_openwrt/2022-01-20T20:20:54Z2022-01-20T20:20:54Z
<p>Sometimes it's nice for testing purpose to have the OpenWRT
userland available locally. Since there is an x86 build
available one can just run it within qemu.</p>
<pre><code>wget https://downloads.openwrt.org/releases/21.02.1/targets/x86/64/openwrt-21.02.1-x86-64-generic-squashfs-combined.img.gz
gunzip openwrt-21.02.1-x86-64-generic-squashfs-combined.img.gz
qemu-img convert -f raw -O qcow2 openwrt-21.02.1-x86-64-generic-squashfs-combined.img openwrt-21.02.1.qcow2
qemu-img resize openwrt-21.02.1.qcow2 200M
qemu-system-x86_64 -M q35 \
-drive file=openwrt-21.02.1.qcow2,id=d0,if=none,bus=0,unit=0 \
-device ide-hd,drive=d0,bus=ide.0 -nic user,hostfwd=tcp::5556-:22
# you've to change the network configuration to retrieve an IP via
# dhcp for the lan bridge br-lan
vi /etc/config/network
- change option proto 'static' to 'dhcp'
- remove IP address and netmask setting
/etc/init.d/network restart
# now you should've an ip out of 10.0.2.0/24
ssh root@localhost -p 5556
# remember ICMP does not work but otherwise you should have
# IP networking available
opkg update
opkg install curl
</code></pre>
ThinkPad P15v Gen1, Xorg and a Samsung QHD Displayhttp://sven.stormbind.net/blog/posts/deb_xorg_samsung_qhd_modelines/2021-10-15T14:19:03Z2021-10-15T14:12:50Z
<p>Wasted quite some hours until I found a working Modeline in this
<a href="https://unix.stackexchange.com/a/547819">stack exchange post</a>
so the ThinkPad works with a HDMI attached Samsung QHD display.</p>
<p>Internal display of the ThinkPad is a FHD display detected as <code>eDP-1</code>,
the external one is <code>DP-3</code> and according to the packaging known by
Samsung as
<a href="https://www.samsung.com/de/business/monitors/high-resolution/qhd-monitor-sa600nwu-ls24a600nwuxen/">S24A600NWU</a>.
The auto deteced EDID modes for QHD - 2560x1440 - did not work at all, the display simply stays
dark. After a lot of back and forth with the i915 driver vs nouveau vs nvidia/nvidia-drm
with and without modesetting, the following Modeline did the magic:</p>
<pre><code>xrandr --newmode 2560x1440_54.97 221.00 2560 2608 2640 2720 1440 1443 1447 1478 +HSync -VSync
xrandr --addmode DP-3 2560x1440_54.97
xrandr --output DP-3 --mode 2560x1440_54.97 --right-of eDP-1 --primary
</code></pre>
<p>Modelines for 50Hz and 60Hz generated with <code>cvt 2560 1440 60</code> did not work, neither did the one
extracted with <code>edid-decode -X</code> from the hex blob found in <code>.local/share/xorg/Xorg.0.log</code>.</p>
<p>From the auto-detected Modelines FHD - 1920x1080 - did work. In case someone struggles
with a similar setup, that might be a starting point. Fun part, if I attach my several years old
Dell E7470 everything is just fine out of the box. But that one just has an Intel GPU and not
the unholy combination I've here:</p>
<pre><code>$ lspci|grep -E "VGA|3D"
00:02.0 VGA compatible controller: Intel Corporation CometLake-H GT2 [UHD Graphics] (rev 05)
01:00.0 3D controller: NVIDIA Corporation GP107GLM [Quadro P620] (rev ff)
</code></pre>