Write it down before I forget about it again:
for x in $(gh api graphql --paginate -f query='query($endCursor:String) { organization(login:"myorg") {
repositories(first: 100, after: $endCursor, isArchived:false) {
pageInfo {
hasNextPage
endCursor
}
nodes {
name
}
}
}
}' --jq '.data.organization.repositories.nodes[].name'); do
secrets=$(gh secret list --json name --jq '.[].name' -R "myorg/${x}" | tr '\n' ',')
if ! [ -z "${secrets}" ]; then
echo "${x},${secrets}"
fi
done
Requests a list of all not archived repositories in a GitHub org and queries repository secrets. If we find some we output the repo name and the secrets in a comma separated list. Not real CSV, but good enough for further processing. I've to admit it's kinda beautiful what you can do with the gh cli by now. Sadly it seems the secrets are not yet available via GraphQL (or I missed it in the docs), so I just use the gh cli to do the REST calls.
I stick to some very archaic workflows, e.g. to connect
to some corp VPN I just run sudo vpnc-connect
and later on sudo vpnc-disconnect
. In the past that also
managed to restore my resolv.conf
, currently it doesn't.
According to a colleague that's also the case for Ubuntu.
Taking a step back, the sane way would be to use the
NetworkManager vpnc plugin, but that does not work with
this specific case because we use uncool VPN tech which
requires the Enable weak authentication
setting for vpnc.
There is a feature request open for that one at
https://gitlab.gnome.org/GNOME/NetworkManager-vpnc/-/issues/11
Taking another step back I thought that it shouldn't be that hard to add some checkbox, a boolean and render out another config flag or line in a config file. Not as intuitive as I thought this mix of XML and C. So let's quickly look elsewhere.
What happens is that the backup files in /var/run/vpnc/
are
created by the vpnc-scripts script called vpnc-script
, but not
moved back, because it adds some pid as a suffix and the pid is
not the final pid of the vpnc process. Basically it can not find
the backup when it tries to restore it. So I decided to replace the
pid guessing code with a suffix made up of the gateway IP and the
tun interface name. No idea if that is stable in all circumstance
(someone with a vpn name DNS RR?) or several connections to different
gateways. But good enough for myself, so here is my patch:
vpnc-scripts [master]$ cat debian/patches/replace-pid-detection
Index: vpnc-scripts/vpnc-script
===================================================================
--- vpnc-scripts.orig/vpnc-script
+++ vpnc-scripts/vpnc-script
@@ -91,21 +91,15 @@ OS="`uname -s`"
HOOKS_DIR=/etc/vpnc
-# Use the PID of the controlling process (vpnc or OpenConnect) to
-# uniquely identify this VPN connection. Normally, the parent process
-# is a shell, and the grandparent's PID is the relevant one.
-# OpenConnect v9.0+ provides VPNPID, so we don't need to determine it.
-if [ -z "$VPNPID" ]; then
- VPNPID=$PPID
- PCMD=`ps -c -o cmd= -p $PPID`
- case "$PCMD" in
- *sh) VPNPID=`ps -o ppid= -p $PPID` ;;
- esac
+# This whole script is called twice via vpnc-connect. On the first run
+# the variables are empty. Catch that and move on when they're there.
+if [ -n "$VPNGATEWAY" ]; then
+ BACKUPID="${VPNGATEWAY}_${TUNDEV}"
+ DEFAULT_ROUTE_FILE=/var/run/vpnc/defaultroute.${BACKUPID}
+ DEFAULT_ROUTE_FILE_IPV6=/var/run/vpnc/defaultroute_ipv6.${BACKUPID}
+ RESOLV_CONF_BACKUP=/var/run/vpnc/resolv.conf-backup.${BACKUPID}
fi
-DEFAULT_ROUTE_FILE=/var/run/vpnc/defaultroute.${VPNPID}
-DEFAULT_ROUTE_FILE_IPV6=/var/run/vpnc/defaultroute_ipv6.${VPNPID}
-RESOLV_CONF_BACKUP=/var/run/vpnc/resolv.conf-backup.${VPNPID}
SCRIPTNAME=`basename $0`
# some systems, eg. Darwin & FreeBSD, prune /var/run on boot
Or rolled into a debian package at https://sven.stormbind.net/debian/vpnc-scripts/
The colleague decided to stick to NetworkManager, moved the vpnc binary aside and
added a wrapper which invokes vpnc with --enable-weak-authentication
. The beauty is,
all of this will break on updates, so at some point someone has to
understand GTK4 to fix the NetworkManager plugin for good.
Always amusing to see some more or less famous open source tools on stage or in movies. Lately we watched THE ME (german only) which is mixing live playing of actors and pre recorded video material. In one of the early video sequences a fictional console interface is displayed, claiming to be running on a Macbook, and htop is used to look for a suspicious process.
PSA for those foolish enough to use Google Cloud and try to use private service connect: If you want to change the serviceAttachment your private service connect forwarding rule points at, you must delete the forwarding rule and create a new one. Updates are not supported. I've done that in the past via terraform, but lately encountered strange errors like this:
Error updating ForwardingRule: googleapi: Error 400: Invalid value for field 'target.target':
'<https://www.googleapis.com/compute/v1/projects/mydumbproject/regions/europe-west1/serviceAttachments/
k8s1-sa-xyz-abc>'. Unexpected resource collection 'serviceAttachments'., invalid
Worked around that with the help of terrraform_data
and lifecycle
:
resource "terraform_data" "replacement" {
input = var.gcp_psc_data["target"]
}
resource "google_compute_forwarding_rule" "this" {
count = length(var.gcp_psc_data["target"]) > 0 ? 1 : 0
name = "${var.gcp_psc_name}-psc"
region = var.gcp_region
project = var.gcp_project
target = var.gcp_psc_data["target"]
load_balancing_scheme = "" # need to override EXTERNAL default when target is a service attachment
network = var.gcp_network
ip_address = google_compute_address.this.id
lifecycle {
replace_triggered_by = [
terraform_data.replacement
]
}
}
See also terraform data for replace_triggered_by.
I know a few people hold on to the exFAT fuse implementation due the support for timezone offsets, so here is a small update for you. Andrew released 1.4.0, which includes the timezone offset support, which was so far only part of the git master branch. It also fixes a, from my point of view very minor, security issue CVE-2022-29973. In addition to that it's the first build with fuse3 support. If you still use this driver, pick it up in experimental (we're in the bookworm freeze right now), and give it a try. I'm personally not using it anymore beyond a very basic "does it mount" test.
tl;dr; OpenSSL 3.0.1 leaks memory in ssl3_setup_write_buffer()
, seems to be
fixed in 3.0.5 3.0.2. The issue manifests at least in stunnel
and keepalived on CentOS 9. In addition I learned the hard way that running a
not so recent VirtualBox version on Debian bullseye let to dh parameter generation
crashing in libcrypto in bn_sqr8x_internal()
.
A recent rabbit hole I went down. The actual bug in openssl was nailed down and documented by Quentin Armitage on GitHub in keepalived My bugreport with all back and forth in the RedHat Bugzilla is #2128412.
Act I - Hello stunnel, this is the OOMkiller Calling
We started to use stunnel on Google Cloud compute engine instances running CentOS 9.
The loadbalancer in front of those instances used a TCP health check to validate the
backend availability. A day or so later the stunnel instances got killed by the OOMkiller. Restarting stunnel and looking into /proc/<pid>/smaps
showed a heap
segment growing quite quickly.
Act II - Reproducing the Issue
While I'm not the biggest fan of VirtualBox and Vagrant I've to admit it's quite
nice to just fire up a VM image, and give other people a chance to recreate that
setup as well. Since VirtualBox is no longer released with Debian/stable I just
recompiled what was available in unstable at the time of the bullseye release, and
used that. That enabled me now to just start a CentOS 9 VM, setup stunnel with a
minimal config, grab netcat and a for loop and watch the memory grow.
E.g. while true; do nc -z localhost 2600; sleep 1; done
To my surprise, in addition to the memory leak, I also observed some crashes but
did not yet care too much about those.
Act III - Wrong Suspect, a Workaround and Bugreporting
Of course the first idea was that something must be wrong in stunnel itself. But
I could not find any recent bugreports. My assumption is that there are
still a few people around using CentOS and stunnel, so someone else should probably
have seen it before. Just to be sure I recompiled the latest stunnel package from
Fedora. Didn't change anything. Next I recompiled it without almost all the patches
Fedora/RedHat carries. Nope, no progress.
Next idea: Maybe this is related to the fact that we do not initiate a TLS context
after connecting? So we changed the test case from nc
to openssl s_client
, and
the loadbalancer healthcheck from TCP to a TLS based one. Tada, a workaround, no
more memory leaking.
In addition I gave Fedora a try (they have Vagrant Virtualbox images in the "Cloud"
Spin, e.g.
here for Fedora 36)
and my local Debian installation a try. No leaks experienced on both.
Next I reported
#2128412.
Act IV - Crash in libcrypto and a VirtualBox Bug
When I moved with the test case from the Google Cloud compute instance to my
local VM I encountered some crashes. That morphed into a real problem when I
started to run stunnel with gdb and valgrind. All crashes happened in libcrypto
bn_sqr8x_internal()
when generating new dh parameter (stunnel does that for
you if you do not use static dh parameter). I quickly worked around that by
generating static dh parameter for stunnel.
After some back and forth I suspected VirtualBox as the culprit. Recompiling
the current VirtualBox version (6.1.38-dfsg-3) from unstable on bullseye works
without any changes. Upgrading actually fixed that issue.
Epilog
I highly appreciate that RedHat, with all the bashing around the future of CentOS, still works on community contributed bugreports. My kudos go to Clemens Lang. Now that the root cause is clear, I guess RedHat will push out a fix for the openssl 3.0.1 based release they have in RHEL/CentOS 9. Until that is available at least stunnel and keepalived are known to be affected. If you run stunnel on something public it's not that pretty, because already a low rate of TCP connections will result in a DoS condition.
Update of my notes from 2020.
# Download a binary device tree file and matching kernel a good soul uploaded to github
wget https://github.com/vfdev-5/qemu-rpi2-vexpress/raw/master/kernel-qemu-4.4.1-vexpress
wget https://github.com/vfdev-5/qemu-rpi2-vexpress/raw/master/vexpress-v2p-ca15-tc1.dtb
# Download the official Rasbian image without X
wget https://downloads.raspberrypi.org/raspios_lite_armhf/images/raspios_lite_armhf-2022-04-07/2022-04-04-raspios-bullseye-armhf-lite.img.xz
unxz 2022-04-04-raspios-bullseye-armhf-lite.img.xz
# Convert it from the raw image to a qcow2 image and add some space
qemu-img convert -f raw -O qcow2 2022-04-04-raspios-bullseye-armhf-lite.img rasbian.qcow2
qemu-img resize rasbian.qcow2 4G
# make sure we get a user account setup
echo "me:$(echo 'test123'|openssl passwd -6 -stdin)" > userconf
sudo guestmount -a rasbian.qcow2 -m /dev/sda1 /mnt
sudo mv userconf /mnt
sudo guestunmount /mnt
# start qemu
qemu-system-arm -m 2048M -M vexpress-a15 -cpu cortex-a15 \
-kernel kernel-qemu-4.4.1-vexpress -no-reboot \
-smp 2 -serial stdio \
-dtb vexpress-v2p-ca15-tc1.dtb -sd rasbian.qcow2 \
-append "root=/dev/mmcblk0p2 rw rootfstype=ext4 console=ttyAMA0,15200 loglevel=8" \
-nic user,hostfwd=tcp::5555-:22
# login at the serial console as user me with password test123
sudo -i
# enable ssh
systemctl enable ssh
systemctl start ssh
# resize partition and filesystem
parted /dev/mmcblk0 resizepart 2 100%
resize2fs /dev/mmcblk0p2
Now I can login via ssh and start to play:
ssh me@localhost -p 5555
tl;dr I ported a part of the python-suntime library to Lua to use it on OpenWRT and RutOS powered devices. suntime.lua
There are those unremarkable things which let you pause for a moment, and realize what a great gift of our time open source software and open knowledge is. At some point in time someone figured out how to calculate the sunrise and sunset time on the current date for your location. Someone else wrote that up and probably again a different person published it on the internet. The Internet Archive preserved a copy of it so I can still link to it. Someone took this algorithm and published a code sample on StackOverflow, which was later on used by the SatAgro guys to create the python-suntime library. Now I could come along, copy the core function of this library, convert it within a few hours - mostly spent learning a bit of Lua, to a working script fulfilling my needs.
Sometimes it's nice for testing purpose to have the OpenWRT userland available locally. Since there is an x86 build available one can just run it within qemu.
wget https://downloads.openwrt.org/releases/21.02.1/targets/x86/64/openwrt-21.02.1-x86-64-generic-squashfs-combined.img.gz
gunzip openwrt-21.02.1-x86-64-generic-squashfs-combined.img.gz
qemu-img convert -f raw -O qcow2 openwrt-21.02.1-x86-64-generic-squashfs-combined.img openwrt-21.02.1.qcow2
qemu-img resize openwrt-21.02.1.qcow2 200M
qemu-system-x86_64 -M q35 \
-drive file=openwrt-21.02.1.qcow2,id=d0,if=none,bus=0,unit=0 \
-device ide-hd,drive=d0,bus=ide.0 -nic user,hostfwd=tcp::5556-:22
# you've to change the network configuration to retrieve an IP via
# dhcp for the lan bridge br-lan
vi /etc/config/network
- change option proto 'static' to 'dhcp'
- remove IP address and netmask setting
/etc/init.d/network restart
# now you should've an ip out of 10.0.2.0/24
ssh root@localhost -p 5556
# remember ICMP does not work but otherwise you should have
# IP networking available
opkg update
opkg install curl
Wasted quite some hours until I found a working Modeline in this stack exchange post so the ThinkPad works with a HDMI attached Samsung QHD display.
Internal display of the ThinkPad is a FHD display detected as eDP-1
,
the external one is DP-3
and according to the packaging known by
Samsung as
S24A600NWU.
The auto deteced EDID modes for QHD - 2560x1440 - did not work at all, the display simply stays
dark. After a lot of back and forth with the i915 driver vs nouveau vs nvidia/nvidia-drm
with and without modesetting, the following Modeline did the magic:
xrandr --newmode 2560x1440_54.97 221.00 2560 2608 2640 2720 1440 1443 1447 1478 +HSync -VSync
xrandr --addmode DP-3 2560x1440_54.97
xrandr --output DP-3 --mode 2560x1440_54.97 --right-of eDP-1 --primary
Modelines for 50Hz and 60Hz generated with cvt 2560 1440 60
did not work, neither did the one
extracted with edid-decode -X
from the hex blob found in .local/share/xorg/Xorg.0.log
.
From the auto-detected Modelines FHD - 1920x1080 - did work. In case someone struggles with a similar setup, that might be a starting point. Fun part, if I attach my several years old Dell E7470 everything is just fine out of the box. But that one just has an Intel GPU and not the unholy combination I've here:
$ lspci|grep -E "VGA|3D"
00:02.0 VGA compatible controller: Intel Corporation CometLake-H GT2 [UHD Graphics] (rev 05)
01:00.0 3D controller: NVIDIA Corporation GP107GLM [Quadro P620] (rev ff)