Public Syndication

Pondering a LED controller

So, a bit over 10 years ago, I made the Hat Lamp.  You can tell how long ago it was as it calls out Dick Smith part numbers for things like resistors.  (How long ago did they give that up?)

Anyway, the original still works, although my wiring is less than perfect.  I’ve thought about modernising it.  Back in 2007, addressable LEDs didn’t exist.  The project got by with nothing more than a 74HC14.  It used one gate as a pierce oscillator, a second to generate a 180° out-of-phase signal, and a third to perform “automatic” control based on the light that fell on a LDR mounted on the top of the hat.

I’ve thought about whether I modernise it.  I have access to 3D printing facilities at HSBNE, so destroying a hard hat is no longer of concern: the design I could come up with could be made to fit a hard hat without modification, meaning it would retain its safety standard qualifications.  LED technology has marched on, in 2007 a 1W LED was considered bright.  The Ay-up headlights I use when cycling are many times brighter than that.  These headlights do come with a headband accessory, which I have, but I find they’re a bit cumbersome.  They however work great on a hard hat or helmet.

That said, for WICEN activities, they’re often too bright, even on their lowest power setting.

MCUs are also cheaper today than they used to be.  And we have addressable LEDs.  Meaning that I could have the old alternating red “alien-abduction head first” pattern the old one, or any number of patterns to suit the occasion.

That said, I’m a little concerned about APA Electronic throwing their weight around.  I actually was considering the APA102s, as they use a SPI style interface which is less timing-sensitive than World Semi’s WS2812s, but really, I hadn’t made a firm choice.  Then, Pimoroni got that letter.  I have no idea whether that letter was (1) a hoax (as in, not actually sent from APA Electronic), (2) the matter settled or (3) the matter still in progress.

In any case, the patent referenced talks about synchronous interfaces.  One common gripe with the WS2812s is that the interface relies on strict timing, which is harder to do with higher-level MCUs and CPUs.  Arduinos can work them fine, but as it’s a somewhat custom serial link, you’ve got to be able to bit-bang it via GPIOs and not all systems are good at that.  Using SPI avoids that problem at the cost of an extra wire.  I wondered if there was another way.

This is what I came up with as a concept.  UARTs idle “high” when not transmitting, so the TX line can serve as a pull-up resistance when the master is not sending anything.  Some MCUs can also re-map their pins (e.g. NXP LPC81x, TI CC2538, Nordic NRF52840), others natively support half-duplex UART Rx/Tx on a single pin (e.g. Microchip ATTiny202).

That allows us to have a shared “bus” with 3 wires between each module: VDD, Data and VSS… the same as the WS2812s.  Unlike the WS2812s, this bus would be built on the UART standard, thus less sensitive to timing jitter.

The problem then is, how do you address each LED?  The WS2812 and APA102s solve this by making the whole bus function as one big shift register.  This makes the electronics simple, but has the cost that you can only communicate one way, from MCU to LED controller.  Thus, you have to maintain a framebuffer, and if you want to just change the colour of one LED, you’ve got to shift the entire framebuffer out.

Why can’t the LEDs have more brains?  How hard is it to DIY a LED controller?

The above arrangement uses the concept of “upstream” and “downstream” ports.  A bi-lateral switch is used to disconnect the “downstream” port under the control of the LED firmware.  If each slave on power up waited for some initialisation signal from the master, all could then disconnect their downstream ports, which then means the next command would only be received by the first LED module.

On telling it its assigned address, it could connect its downstream neighbour and you’d then be able to talk to those two LEDs, one of which would be at some unknown address, and the other, assigned.

This would repeat until you got to the end of the chain.  Given that the downstream LED can “hear” its upstream neighbour when it is connected, it’s not hard for the downstream LED to assume its address is one after its upstream neighbour.

Disconnection would be achieved via some sort of tri-state bi-directional buffer such as a bilateral switch.  I’ve used 4066s in the diagram, but in reality, I’d be using a single-unit version like a SN74LVC1G66.

It’s also common to consider these as matrices.  It’d be really neat, if you say had an array of say 320×240 pixels that the LEDs should use 4-byte addressing, where the lowest 9 bits was the X co-ordinate and the upper bits the Y co-ordinate.  Thus the addressing would count from 0x0000000 to 0x0000013f, then skip to 0x0000200.  That’s a simple arithmetic operation.  By connecting the next neighbour then announcing the address, this could trigger the neighbour to do the same automatically.  The master would then hear each and every pixel announce its address as it comes online.  When the messages stop, initialisation is complete.

A major problem with asynchronous communications is figuring out the baud rate.  Luckily, LIN has solved that problem.  No, I won’t actually use the LIN protocols, I’ll just use its sync frame, support for which is built into many MCU UART modules.  LIN uses a header which includes a BREAK followed by the 0x55 byte which helps the slave figure out the correct baud rate being used.  If I use that same sequence, I get autobauding for free.

So putting this together, how would this work?  Let’s assume everything has just been reset.  Each protocol frame would be bounded by a header and a trailer, the header being based on the LIN standard.  Not sure what the trailer will look like at this point, maybe a CRC.

  1. On power-on, the slaves all link their downstream ports.  (Thus if a controller crashes, you lose just that one pixel.  It also allows all slaves to receive the initial configuration commands.)
  2. The master then tells the slaves to commence a roll-call.  The instruction would be made up of:
    1. The “commence roll-call” op-code (1 byte)
    2. The length in bytes of the addresses to be used (1 byte; call its value L)
    3. The number of bits in the address representing a single row (1 byte; call this D)
    4. The number of pixels per row (L bytes, call this M)
  3. The slaves immediately disconnect their downstream ports then respond back with
    1. The “OK” op-code
  4. Since the head of the line would have disconnected every other slave, the master only hears one “OK” response.  The master performs the following computation:
    • Address = (2^(8L)) – 1
  5. The master starts the roll-call off by sending the following on the bus.
    1. The “Address announcement” op-code (1 byte)
    2. The address it calculated (L bytes)
  6. Since just the first slave is connected, it hears this.  It connects its downstream neighbour, then with the received address, it performs the following algorithm:
    • Address = UpstreamAddress + 1
    • If (Address & ((2^D)-1) > M:
      • Address = ((Address >> D) + 1) << D
  7. With its new address, and the immediate neighbour connected, it sends
    1. The “Address announcement” op-code (1 byte)
    2. The address it calculated (L bytes)

Ad infinite um, until the last in the chain announces its address.

The master of course hears all, including its own traffic.  As an example, if we considered a 320×200 pixel panel with 32-bit addressing; thus L=4, D=9 and M=320, it would hear this:

  • HEADER OP_ROLL_CALL L=4 D=9 M=320 TRAILER: Begin roll-call
  • HEADER RES_OK TRAILER: Slaves ready for roll-call
  • HEADER OP_ADDR_ANN ADDR=0xffffffff TRAILER: Master “my address is 0xffffffff”
  • HEADER OP_ADDR_ANN ADDR=0x00000000 TRAILER: First pixel “my address is 0x00000000”
  • HEADER OP_ADDR_ANN ADDR=0x00000001 TRAILER: Second pixel “my address is 0x00000001”
  • etc
  • HEADER OP_ADDR_ANN ADDR=0x0000013f TRAILER: 320th pixel “my address is 0x0000013f”
  • HEADER OP_ADDR_ANN ADDR=0x00000200 TRAILER: 321st pixel “my address is 0x00000200”
  • etc

At the end, everybody knows their address, including the master (which is derived from the address length; its address is “all ones”), and because each slave connected its neighbour, everyone can communicate together.

Operation codes could be implemented that allow a pixel to be set to a given colour, or to report its present colour.  Since they all know how to interpret the address to form co-ordinates, it’s possible for the master to send a command that says “fill rectangle (X1,Y1)-(X2,Y2) with colour C”, all pixels hear this simultaneously and the action is performed.

Or better yet, “pixels in area (X1,Y1)-(X2,Y2), copy the colour from the neighbour to your right”, which would allow for scrolling text displays.  The pixels would know immediately who to ask, and could have an “agreed upon” order in which to perform operations.  Thus (X1,Y1) would know to ask (X1+1,Y1) for its colour, copy that to its own output, then tell (X1,Y1+1) to perform its step.  (X1,Y2) would know that once it copied its colour from (X1+1,Y2), it needs to poke (X1+1,Y1).  Finally (X2,Y2) would know to tell the master that the operation is complete.

Blitting can also be done.  You know the operation, you know how to obtain the input data, the rest can be done independent of the master MCU.

The microcontrollers don’t need a lot of brains to do this, nor does the master for that matter, it’s distributed brains that get the job done.  The part I’m thinking of for this is the Microchip ATTiny202, which can be bought for under 60c a piece and features hardware UART and up to 4 PWM channels.

For sure, add in the bilateral switch, some passives, a PCB and a RGB LED and you’ve blown more money than the competition, but in this case, you’ve got a fully programmable LED controller with open-source firmware, that’s not patent encumbered.

It might be a little while before this MCU is available, Mouser reckon they’ll have them late October, which is fine I can wait.  Until then, plenty of time to research the problem.

Solar Cluster: Battery monitor PC now running Gentoo

So, after some argument, and a bit of sitting on a concrete floor with the netbook, I managed to get Gentoo loaded onto the TS-7670.  Right now it’s running off the MicroSD card, I’ll get things right, then shift it across to eMMC.

ts7670 ~ # emerge --info
Portage 2.3.40 (python 3.5.5-final-0, default/linux/musl/arm/armv7a, gcc-6.4.0, musl-1.1.19, 4.14.15-vrt-ts7670-00031-g1a006273f907-dirty armv5tejl)
=================================================================
System uname: Linux-4.14.15-vrt-ts7670-00031-g1a006273f907-dirty-armv5tejl-ARM926EJ-S_rev_5_-v5l-with-gentoo-2.4.1
KiB Mem:      111532 total,     13136 free
KiB Swap:    4194300 total,   4191228 free
Timestamp of repository gentoo: Fri, 17 Aug 2018 16:45:01 +0000
Head commit of repository gentoo: 563622899f514c21f5b7808cb50f6e88dbd7d7de
sh bash 4.4_p12
ld GNU ld (Gentoo 2.30 p2) 2.30.0
app-shells/bash:          4.4_p12::gentoo
dev-lang/perl:            5.24.3-r1::gentoo
dev-lang/python:          2.7.14-r1::gentoo, 3.5.5::gentoo
dev-util/pkgconfig:       0.29.2::gentoo
sys-apps/baselayout:      2.4.1-r2::gentoo
sys-apps/openrc:          0.34.11::gentoo
sys-apps/sandbox:         2.13::musl
sys-devel/autoconf:       2.69-r4::gentoo
sys-devel/automake:       1.15.1-r2::gentoo
sys-devel/binutils:       2.30-r2::gentoo
sys-devel/gcc:            6.4.0-r1::musl
sys-devel/gcc-config:     1.8-r1::gentoo
sys-devel/libtool:        2.4.6-r3::gentoo
sys-devel/make:           4.2.1::gentoo
sys-kernel/linux-headers: 4.13::musl (virtual/os-headers)
sys-libs/musl:            1.1.19::gentoo
Repositories:

gentoo
    location: /usr/portage
    sync-type: rsync
    sync-uri: rsync://virtatomos.longlandclan.id.au/gentoo-portage
    priority: -1000
    sync-rsync-verify-jobs: 1
    sync-rsync-extra-opts: 
    sync-rsync-verify-metamanifest: yes
    sync-rsync-verify-max-age: 24

ACCEPT_KEYWORDS="arm"
ACCEPT_LICENSE="* -@EULA"
CBUILD="arm-unknown-linux-musleabi"
CFLAGS="-Os -pipe -march=armv5te -mtune=arm926ej-s -mfloat-abi=soft"
CHOST="arm-unknown-linux-musleabi"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-Os -pipe -march=armv5te -mtune=arm926ej-s -mfloat-abi=soft"
DISTDIR="/home/portage/distfiles"
ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"
FCFLAGS="-O2 -pipe -march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=hard"
FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync multilib-strict news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe -march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=hard"
GENTOO_MIRRORS=" http://virtatomos.longlandclan.id.au/portage http://mirror.internode.on.net/pub/gentoo http://ftp.swin.edu.au/gentoo http://mirror.aarnet.edu.au/pub/gentoo"
INSTALL_MASK="charset.alias"
LANG="en_AU.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
USE="arm bindist cli crypt cxx dri fortran iconv ipv6 modules ncurses nls nptl openmp pam pcre readline seccomp ssl tcpd unicode xattr zlib" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon plan sheets stage words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="musl" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6 php7-0" POSTGRES_TARGETS="postgres9_5 postgres10" PYTHON_SINGLE_TARGET="python3_6" PYTHON_TARGETS="python2_7 python3_6" RUBY_TARGETS="ruby23" USERLAND="GNU" VIDEO_CARDS="dummy fbdev v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, LC_ALL, LINGUAS, MAKEOPTS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

I still have to update the kernel.  I actually did get kernel 4.18 to boot, but I forgot to add in support for the watchdog, so U-Boot tickled it, then the watchdog got hungry and kicked the reset half way through the boot sequence.

Rolling back to my older 4.14 kernel works.  I’ll try again with 4.18.5 in a moment.  Failing that, I have also brought the 4.14 patches up to 4.14.69 which is the latest LTS release of the kernel.

I’ve started looking at the power supply sysfs device class, with a view to exposing the supply voltage via sysfs.  The thinking here is that collectd supports reading this via the “battery” module (and realistically, it is a battery that is being measured: two 105Ah AGMs).

Worst case is I do something a little proprietary and deal with it in user space.  I’ll have to dig up the Linux kernel tree I did for Jacques Electronics all those years ago, as that had some examples of interfacing sysfs to a Cypress PSOC device that was acting as an I²C slave.  Rather than using an off-the-shelf solution, they programmed up a MCU that did power management, touchscreen sensing, keypad sensing, RGB LED control and others, all in one chip.  (Fun to try and interface that to the Linux kernel.)

Technologic Systems appear to have done something similar.  The device ID 0x78 implies a 10-bit device, but I think they’re just squatting on that 7-bit address.  They hail 0x78 then read out 4 bytes, which the last two bytes are the supply voltage ADC readings.  They do their own byte swapping before scaling the value to get mV.

Solar Cluster: Gentoo Linux MUSL for ARMv5 finally built

It’s taken several months and had a few false starts, but at long last I have some stage tarballs for Gentoo Linux MUSL for ARMv5 processors.  I’m not the only one wanting such a port, even looking for my earlier thread on the matter, I stumbled on this post.  (Google translate is hopeless with Russian, but I can get the gist of what’s being said.)

This was natively built on the TS-7670 using an external hard drive connected over USB 2.0 as both swap and the chroot.  It took two passes to clean everything up and get something that’s in a “release-able” state.

I think my next steps now will be:

  • Build an updated kernel … I might see if I can expose that I²C register via a sysfs file or something that collectd can pick up whilst I’m at it.  I have the kernel sources and bootloader sources.
  • Prepare the 32GB MicroSD card I bought a few weeks back with the needed partitions and load Gentoo onto that.
  • Install the MicroSD card and boot off it.
  • Back up the eMMC
  • Re-format the eMMC and copy the MicroSD card to it.

It’s supposed to be wet this weekend, so it sounds like a good project for indoors.

Progress reporting in Gentoo

I have a bad habit where it comes to updating systems, I tend to do it less frequently than I should, and that can sometimes snowball like it has for my mail server.  Even if it’s a fresh install, sometimes there’s a large number of packages that need installing.

Now Portage does report where it’s up to, but often that has long scrolled past the buffer on your terminal.  You can look at /var/log/emerge.log for this information, but sometimes it’s nice to just see a percentage progress and a pseudo graphical representation.

With this in mind, I cooked up a little script which just tails /var/log/emerge.log and displays a progress bar along with the last message reported. The script is quite short:

#!/bin/bash

shopt -s checkwinsize

stdbuf -o L tail -n 0 -F /var/log/emerge.log | while read line; do
	changed=0
	eval $( echo ${line} | \
		sed -ne '/[0-9]\+ of [0-9]\+/ { s:^.*(\([0-9]\+\) of \([0-9]\+\)).*$:done=\1 total=\2 changed=1:; p; }' )

	if [ "${changed}" = 1 ]; then
		case "${line}" in
			*"::: completed emerge"*)
				;;
			*)
				done=$(( ${done} - 1 ))
				;;
		esac

		percent=$(( ( ${done}*100 ) / ${total} ))
		width=$(( ${COLUMNS:-80} - 8 ))
		progress=$(( ( ${done}*${width} ) / ${total} ))
		remain=$(( ${width} - ${progress} ))

		progressbar="$( for n in $( seq 1 ${progress} ); do echo -n '#'; done )"
		remainbar="$( for n in $( seq 1 ${remain} ); do echo -n ':'; done )"

		printf '\033[2A\033[2K%s\n\033[2K\033[1G[\033[1m%s\033[0m%s] \033[1m%3d%%\033[0m\n' \
			"${line:0:${COLUMNS:-80}}" "$progressbar" "$remainbar" "$percent"
	else
		printf '\033[2A\033[2K%s\n\n' "${line:0:${COLUMNS:-80}}"
	fi

	if echo "${line}" | grep -q '*** terminating.'; then
		exit
	fi
done

What’s it look like?

It works well with GNU Screen as seen above.

Solar Cluster: Measuring the battery voltage

So, I was just updating the project details for this project, and I happened to see this blog post about reading the DC voltage input on the TS-7670v2.

I haven’t yet gotten around to finishing the power meters that I was building which would otherwise be reading these values directly, but they were basically going to connect via Modbus to the TS-7670v2 anyway.  One of its roles, aside from routing between the physical management network (IPMI and switch console access), was to monitor the battery.

I will have to explore this.  Collectd doesn’t have a general-purpose I²C module, but it does have one for barometer modules, so with a bit of work, I could make one to measure the voltage input which would tell me what the battery is doing.

Docker CE on RHEL 7.3

Lately, I’ve been doing a lot of development work on Tridium Niagara kit.  The Tridium platform is fundamentally built on Sun^WOracle’s Java environment, and is very popular in the building management industry.  There’s an estimate of over 600000 JACE devices (building management controllers) deployed worldwide, so I can fully understand why my workplace is chasing them.

That means coming to grips with their environment, and getting it to talk to ours.  Officially, VRT is a Debian/Ubuntu shop.  They used to dabble with Red Hat years ago, back when VRT and Red Hat were next-door neighbours (in Gardner Close, Milton) but VRT switched to Ubuntu around 2008 after a brief flirt with Gentoo.  Thus, must of our tooling assumes a Debian-based system.

Docker CE on Debian and Ubuntu is a snap.  However, Tridium it would seem, are Red Hat fans, and only support their development environment on Microsoft Windows (yes shudder) or Red Hat Enterprise Linux.  Thus, we have a RHEL 7.3 VM we pass around when we’re doing VM development.  I figured since we’re trying to link Niagara to WideSky, it would be nice to be able to deploy WideSky on RHEL.

WideSky uses Docker as the basis for its deployment, so this sounded simple enough.  Install Docker and docker-compose, throw a bog-standard deployment in there, docker-compose up -d, off we go.

Not so fast.

While there’s Docker EE for RHEL, budget is tight and we really don’t need the support as this isn’t a “production” instance as such.  If the VM gets sick we just roll it back to a known good version and continue from there.  It doesn’t make sense to spend money on purchasing Docker EE.  There’s a CentOS version of Docker CE, and even unofficial instructions on how to shoehorn this into RHEL.  I dutifully followed these, but then hit a road-block with container-selinux: the repository no longer has that version.

Rather than looking for what version they have now, or play Russian Roulette hunting for a random RPM from some mirror site (been there, done that many moons ago before I knew better)… a better plan was to grab the sources and sic rpmbuild onto them so we get a RHEL-native binary.

Building container-selinux on RHEL

  1. Begin by installing dependencies:
    # yum install -y selinux-policy selinux-policy-devel rpm-build rpm-devel git
  2. Download the sources for the RPM:
    $ git clone https://git.centos.org/r/rpms/container-selinux.git
    $ git checkout c7-alt
    $ cd SPECS
  3. Have a look at the .spec file to see where it expects to source the sources from, up the top of the file I downloaded, I saw:
    %global git0 https://github.com/projectatomic/%{name}
    %global commit0 dfb449b771ca4977bb7d5fb6cd7be3cfc14d6fca
  4. Fetch the sources, then check out that commit:
    $ git clone https://github.com/projectatomic/container-selinux
    $ git checkout dfb449b771ca4977bb7d5fb6cd7be3cfc14d6fca
  5. Rename the check-out directory as container-selinux-${GIT_COMMIT_ID}
    $ cd ..
    $ mv container-selinux container-selinux-dfb449b771ca4977bb7d5fb6cd7be3cfc14d6fca
  6. Package it up into a tarball, excluding the .git directory and plop that file in ~/rpmbuild/SOURCES
    $ tar --exclude container-selinux-dfb449b771ca4977bb7d5fb6cd7be3cfc14d6fca/.git \
    -czvf ~/rpmbuild/sources/container-selinux-dfb449b.tar.gz \
    container-selinux-dfb449b771ca4977bb7d5fb6cd7be3cfc14d6fca
  7. Build!
    $ rpmbuild -ba container-selinux.spec

All going to plan, you should have a shiny new RPM file in ~/rpmbuild/RPMS.  Install that, then you can proceed with installing the CentOS version of Docker CE.  If you’re doing this for a production environment, and absolutely must use Docker CE, then I’d advise that perhaps taking the source RPMs for Docker CE and building those on RHEL would be advisable over using raw CentOS binaries, but each to your own.

# docker info
Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 0
Server Version: 18.06.0-ce
Storage Driver: overlay2
 Backing Filesystem: xfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: d64c661f1d51c48782c9cec8fda7604785f93587
runc version: 69663f0bd4b60df09991c08812a60108003fa340
init version: fec3683
Security Options:
 seccomp
  Profile: default
Kernel Version: 3.10.0-693.11.1.el7.x86_64
Operating System: Red Hat Enterprise Linux
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 3.702GiB
Name: localhost.localdomain
ID: YVHJ:UXQV:TBAS:E5MH:B4GL:VT2H:A2BW:MQMF:3AGA:FBBX:MINO:24Z6
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

Playing with PostGIS

So, on the bike, I use a portable GPS to keep track of my speed and to track the mileage done on the bike so I know when to next put it in for service. Originally I just relied on the trip counter in the GPS, but then found that this could develop quite an error if left to tick over for a few months.

Thus, I wrote a simple CGI application in Perl and SQLite3 that would track the odometer readings. Plain, simple, and it’s worked quite well, but remembering to punch in the current odometer reading is a chore, and my stats are only as granular as I submit: if I want to see what distance I did on a particular day, I either have to have had the foresight to store readings at the start and end of that day, or I’m stuffed.

I also keep the GPX tracklogs. While the Garmin 650 is not great at handling lots of tracklogs (and for some moronic reason, they name the files “Day DD-MMM-YY HH.MM.SS.gpx”, not something sensible like “YYYY-MM-DDTHH-MM-SS.gpx”), it’s good enough that I can periodically siphon off the track logs for storage on my laptop. I then have a record of where I’ve been.

Theoretically, this also has the distance travelled, I could make a service that just consumes the GPX files, and tallies up the distances that way. Maybe even visualise heat-map style, where I go most. (No prizes for guessing “work” … but where else?)

The existing system uses SQLite, and specifically, its views, as poor man’s stored procedures. It’s hacky, inefficient, and sooner or later I’ll have performance problems. PostGIS is an extension onto PostgreSQL which supports a large number of spatial operations, including finding the length of a series of points, which is exactly the problem I’m trying to solve right now. The catch is, how do you import the data?

Enter GDAL

GDAL is a library of geographic functions for answering these kinds of questions. It ships with a utility ogr2ogr, which can take geographic information in a variety of formats, and convert to a variety of output formats. Crucially, this tool supports consuming GPX files and writing to a PostGIS database.

Loading one file, is easy enough:

$ ogr2ogr -oo GPX_ELE_AS_25D=YES \
  -dim 3 \
  -gt 65536 \
  -lco GEOM_TYPE=geography \
  -preserve_fid \
  -f PostgreSQL \
  "PG:dbname=yourdb" yourfile.gpx \
  tracks track_points

The arguments here were found by trial-and-error.  Specifically, -oo GPX_ELE_AS_25D=YES and -dim 3 tell ogr2ogr to preserve the elevation in the point information (as well as keeping a copy of it in the ele column). -lco GEOM_TYPE=geography tells ogr2ogr to use the geography data type in PostGIS.

Look in the database, and you’ll see two tables, tracks and track_points. Sadly, you don’t get to choose the names of these (not easily anyway, there is -nln, but it then will create one table with the given name, put the tracks in it, then blow it away and replace it with a table of the same name containing points), and there’s no foreign keys between the two.

The fun starts when you try to import a second GPX file. Run that command again, and because of -preserve_fid, you’ll get a primary key clash. Take that away, and the track_fid column in track_points becomes meaningless.

If you drop -preserve_fid, then track_fid gets set to 0 for all points.  Useless.

Importing many GPX files

Out of the box, this just wasn’t going to fly, so we needed to do things a little different.  Firstly, I duplicated the schema that GDAL creates, creating my own tables which will ultimately store the data.  I then used a wrapper shell script that calls psql before and after ogr2ogr so I can re-map the primary keys to maintain relationships.

Schema SQL

CREATE SEQUENCE public.gpx_points_ogc_fid_seq
    INCREMENT 1
    START 1
    MINVALUE 1
    MAXVALUE 2147483647
    CACHE 1;

CREATE SEQUENCE public.gpx_tracks_ogc_fid_seq
    INCREMENT 1
    START 1
    MINVALUE 1
    MAXVALUE 2147483647
    CACHE 1;

CREATE TABLE public.gpx_tracks
(
    ogc_fid integer NOT NULL,
    name character varying COLLATE pg_catalog."default",
    cmt character varying COLLATE pg_catalog."default",
    "desc" character varying COLLATE pg_catalog."default",
    src character varying COLLATE pg_catalog."default",
    link1_href character varying COLLATE pg_catalog."default",
    link1_text character varying COLLATE pg_catalog."default",
    link1_type character varying COLLATE pg_catalog."default",
    link2_href character varying COLLATE pg_catalog."default",
    link2_text character varying COLLATE pg_catalog."default",
    link2_type character varying COLLATE pg_catalog."default",
    "number" integer,
    type character varying COLLATE pg_catalog."default",
    gpxx_trackextension character varying COLLATE pg_catalog."default",
    the_geog geography(MultiLineStringZ,4326),
    CONSTRAINT gpx_tracks_pkey PRIMARY KEY (ogc_fid)
)
WITH (
    OIDS = FALSE
)
TABLESPACE pg_default;

CREATE TABLE public.gpx_points
(
    ogc_fid integer NOT NULL,
    track_fid integer,
    track_seg_id integer,
    track_seg_point_id integer,
    ele double precision,
    "time" timestamp with time zone,
    magvar double precision,
    geoidheight double precision,
    name character varying COLLATE pg_catalog."default",
    cmt character varying COLLATE pg_catalog."default",
    "desc" character varying COLLATE pg_catalog."default",
    src character varying COLLATE pg_catalog."default",
    link1_href character varying COLLATE pg_catalog."default",
    link1_text character varying COLLATE pg_catalog."default",
    link1_type character varying COLLATE pg_catalog."default",
    link2_href character varying COLLATE pg_catalog."default",
    link2_text character varying COLLATE pg_catalog."default",
    link2_type character varying COLLATE pg_catalog."default",
    sym character varying COLLATE pg_catalog."default",
    type character varying COLLATE pg_catalog."default",
    fix character varying COLLATE pg_catalog."default",
    sat integer,
    hdop double precision,
    vdop double precision,
    pdop double precision,
    ageofdgpsdata double precision,
    dgpsid integer,
    the_geog geography(PointZ,4326),
    CONSTRAINT gpx_points_pkey PRIMARY KEY (ogc_fid),
    CONSTRAINT gpx_points_track_fid_fkey FOREIGN KEY (track_fid)
        REFERENCES public.gpx_tracks (ogc_fid) MATCH SIMPLE
        ON UPDATE RESTRICT
        ON DELETE RESTRICT
)
WITH (
    OIDS = FALSE
)
TABLESPACE pg_default;

The wrapper script

 1 #!/bin/sh
 2 
 3 DB=tracklog
 4 
 5 for f in "$@"; do
 6         psql tracklog <<EOF
 7 DROP TABLE IF EXISTS tracks;
 8 DROP TABLE IF EXISTS track_points;
 9 EOF
10         ogr2ogr -oo GPX_ELE_AS_25D=YES \
11                 -dim 3 \
12                 -gt 65536 \
13                 -lco SPATIAL_INDEX=FALSE \
14                 -lco GEOM_TYPE=geography \
15                 -overwrite \
16                 -preserve_fid \
17                 -f PostgreSQL \
18                 "PG:dbname=${DB}" "$f" \
19                 tracks track_points
20 
21         # Re-map FIDs then insert into real tables.
22         psql tracklog <<EOF
23         CREATE TEMPORARY TABLE track_fids AS
24         SELECT  ogc_fid AS orig_fid,
25                 nextval('gpx_tracks_ogc_fid_seq') AS ogc_fid
26         FROM    tracks;
27 
28         CREATE TEMPORARY TABLE point_fids AS
29         SELECT  ogc_fid AS orig_fid,
30                 nextval('gpx_points_ogc_fid_seq') AS ogc_fid
31         FROM    track_points;
32 
33         INSERT INTO gpx_tracks
34         SELECT  track_fids.ogc_fid AS ogc_fid,
35                 tracks.name as name,
36                 tracks.cmt as cmt,
37                 tracks."desc" as "desc",
38                 tracks.src as src,
39                 tracks.link1_href as link1_href,
40                 tracks.link1_text as link1_text,
41                 tracks.link1_type as link1_type,
42                 tracks.link2_href as link2_href,
43                 tracks.link2_text as link2_text,
44                 tracks.link2_type as link2_type,
45                 tracks."number" as "number",
46                 tracks.type as type,
47                 tracks.gpxx_trackextension as gpxx_trackextension,
48                 tracks.the_geog as the_geog
49         FROM    track_fids, tracks
50         WHERE   track_fids.orig_fid=tracks.ogc_fid;
51 
52         INSERT INTO gpx_points
53         SELECT  point_fids.ogc_fid AS ogc_fid,
54                 track_fids.ogc_fid AS track_fid,
55                 track_points.track_seg_id AS track_seg_id,
56                 track_points.track_seg_point_id AS track_seg_point_id,
57                 track_points.ele AS ele,
58                 track_points."time" AS "time",
59                 track_points.magvar AS magvar,
60                 track_points.geoidheight AS geoidheight,
61                 track_points.name AS name,
62                 track_points.cmt AS cmt,
63                 track_points."desc" AS "desc",
64                 track_points.src AS src,
65                 track_points.link1_href AS link1_href,
66                 track_points.link1_text AS link1_text,
67                 track_points.link1_type AS link1_type,
68                 track_points.link2_href AS link2_href,
69                 track_points.link2_text AS link2_text,
70                 track_points.link2_type AS link2_type,
71                 track_points.sym AS sym,
72                 track_points.type AS type,
73                 track_points.fix AS fix,
74                 track_points.sat AS sat,
75                 track_points.hdop AS hdop,
76                 track_points.vdop AS vdop,
77                 track_points.pdop AS pdop,
78                 track_points.ageofdgpsdata AS ageofdgpsdata,
79                 track_points.dgpsid AS dgpsid,
80                 track_points.the_geog AS the_geog
81         FROM    track_points, track_fids, point_fids
82         WHERE   point_fids.orig_fid=track_points.ogc_fid
83         AND     track_fids.orig_fid=track_points.track_fid;
84 
85         DROP TABLE tracks;
86         DROP TABLE track_points;
87         DROP TABLE track_fids;
88         DROP TABLE point_fids;
89 EOF
90 done

Getting the length of a track

Having imported all the data, we can do something like this:

SELECT ogc_fid, name,
  ST_Length(the_geog, false)/1000 as dist_in_km
FROM gpx_tracks order by ogc_fid desc limit 10;

and get this:

1754 Day 20-JUL-18 18:09:02′ 9.83689686312541′
1753 Day 15-JUL-18 09:36:16′ 5.75919119415676′
1752 Day 14-JUL-18 17:12:24′ 0.071734341651265′
1751 Day 14-JUL-18 17:12:23′ 0.0729574875289383′
1750 Day 13-JUL-18 08:13:32′ 9.88420745610283′
1749 Day 06-JUL-18 09:00:32′ 9.81221316219109′
1748 Day 30-JUN-18 01:11:26′ 9.77607205972035′
1747 Day 23-JUN-18 05:02:04′ 19.6368592034475′
1746 Day 22-JUN-18 18:03:37′ 9.91964760346248′
1745 Day 12-JUN-18 21:22:26′ 0.0884092391531763′

Visualisation with QGIS

Turns out, this is straightforward…

  1. In your workspace, there’s a tree with the different layer types you can add, including PostGIS… right-click on this and select New Connection… fill in the details for your PostgreSQL database.
  2. Below that is XYZ Tiles…, right click again, select New Connection for OpenStreetMap, and use the URL https://a.tile.openstreetmap.org/{z}/{x}/{y}.png (also, see their policy).
  3. Drag the OpenStreetMap connection to your layers
  4. Expand the PostGIS connection you just made, and look for the gpx_tracks table, drag this on top of your OpenStreetMap layer.

Below is everywhere I’ve been with the GPS tracklog running.  Much of what you see is the big loop a few of us did in 2012, including my trip to Ballarat for the 2012 LCA.

If I zoom in on Brisbane, unsurprisingly, some areas show up very clearly as being common haunts for me:

A bit of SQL voodoo, and I come up with this:

In orange is the territory covered on the Boulder (minus what was covered before I got the GPS), in blue the territory covered on the Talon 29 ER 0, and in red, on my current commuter (Toughroad SLR2).

Thoughts on the Mini Musk Sub

So, the local media here (can’t comment for other parts of the world) have been quite busy reporting on the fate of The Wild Boars soccer team and their coach, stuck in a flooded cave in Thailand.  With the great work of many, the group is now free of the cave, and getting the medical attention they need.

Pats on the back all around.  It could have very well been a dozen funerals that needed to be organised instead of servings of various meals.

Overshadowing this somewhat, has been the somewhat childish spat between Vern Unsworth and Elon Musk over the miniature submarine that was proposed as a vehicle for transporting the children through the cave system.

Now, I’ll admit right up front, what I know is what I’ve heard from the media here.  In amongst the reports, it was commented that the gaps though which people had to squeeze through, were as small as 38cm in places.

That does not leave you much room.  That’s bloody confined in the extreme.  A submarine that could fit a child and squeeze thorough such a gap?  It’d be positively claustrophobic!

Now, Mr Unsworth did label this as a PR stunt.  Maybe it was … maybe the design was just naïve.  I think the goal was a noble one, and Elon Musk’s team did a great job in giving it a go, even if they did overlook a few critical details.

However, I think I’ll take Mr Unsworth’s advice over Mr Musk’s regarding whether the device was practical, as he was actually there.  If the device got stuck, the results could have been fatal.  The team was already in a dangerous situation and had lost one member of their team already, they really weren’t in a position to experiment.  I think responding with “stick it where it hurts” is being overly harsh, but otherwise I think the criticism was entirely valid.

You do not, however, call someone a “pedo”, without very good grounds for doing so.  That is slanderous.  And what exactly is “sus” about living in Thailand?  Tesla’s been suffering some quite bad press lately, I really do not think this juvenile behaviour helps anyone.

One is free to believe that ego is not a dirty word, but that does not mean one’s humility should be locked under the stairs!


Update 2018-07-17: Hmm, I was saying…? Tesla sheds almost $US2b after Elon Musk’s ‘pedo’ attack on British diver.

Telstra: another mob that didn’t get the RFC5233 memo

So this evening, I got a bit of marketing from Telstra. This was to an email address I had used to register the SIM card that I’m trying out in the Kite. I naturally followed the same approach I have with other such suppliers as an anti-phishing tactic.

The email is not unsolicited, but it is a commercial email nonetheless. I figured I’d just quietly opt-out, no need to make a fuss. The email itself was legitimate, so no concern about boobytrapped unsubscribe links. Naturally, I copied the address from their email and paste it into the form on their webpage. I get told this:

Errm, excuse me? That is the email address that I wish to unsubscribe, and if it were invalid, I would not be trying to unsubscribe because I would not have gotten the email in the first place!

Okay, so I’ll need to go through a human to get this resolved, what joy. Navigate the labyrinth that is the Telstra support site (they really don’t want you to be able to make complaints), and I get to a complaints form. First thing I note, they forgot to close an <a> tag (end of line 154)…

<p>If you require immediate assistance with a complaint, <b>Consumer customers</b> can call us anytime on 132200 and say "complaint".<br><br>
If you are a <b>Business customer</b> and require immediate assistance with a complaint, you can call us anytime on 132000 and say "complaint".</p>
<b>Enterprise and Government customers:</b> please go to your specialised contact page <a href="https://www.telstra.com.au/business-enterprise/contact-us/make-a-complaint" target="_self">here</a>.
&nbsp;
<p>Further information on how we handle complaints can be found in our <a href="https://www.telstra.com.au/content/dam/tcom/personal/help/pdf/telstra-complaint-handling-process.pdf">complaints handling process document (PDF).</p></pre>
</div>
<div id="surveyMainDiv" class="main-background">
<div class="place-holder-div" id="surveyMainDivBannerDiv"></div>
<div id="surveyContentDiv" class="content-background">

As a result, Firefox thinks everything to the end of the form, is part of the link! They also close a tag that isn’t open: <pre>.

UPDATE 2018-07-07: This has now been fixed.

Right, so there’s two things. I persevere with the form, resorting to keyboard shortcuts since clicking on any form element brings up that PDF.

Happy that I’ve covered what I wanted to say, I hit the submit. Only to find out the same person who designed the last form, must have designed this one too.

Great, so that’s now three things to complain about.

What really saddens me with Telstra is that today their management tell us they “aspire to be a technology company”. The fact that years ago, Telecom Australia was very much a respected member of the ITU meant it pretty much was a technology company… and the fact they can’t get something as basic as email address validation or a simple web form right, really does show how far they have fallen.

I fully expect this will go back-and-forth while they ask for my browser details (irrelevant, this is broken HTML at their end), my OS (again irrelevant), and then the claim that: “Ohh, we don’t support that!” Which will hold about as much water as a tissue paper G-string.


So, an update. I had a reply back, basically they stated a few things:

  1. they claim to not have seen any marketing emails for the past two months sent to me. (how hard did they look?)
  2. they claim to have taken my name off the list (we’ll see)

They make no comment about fixing the forms. The complaints form now has its closing </a> tag back, so clicking on form elements no longer causes it to pop up with a PDF download. Great, 1 problem of 3 fixed.

I finally had a moment to reply, and did so. In their email, they give an address to send the reply to (because we’re to cool to set the Reply-To header or use the correct From address):

I got back an immediate response:

Delivery has failed to these recipients or distribution lists:

ComplaintResolutionCentre@team.telstra.com
The recipient’s e-mail address was not found in the recipient’s e-mail system. Microsoft Exchange will not try to redeliver this message for you. Please check the e-mail address and try resending this message, or provide the following diagnostic text to your system administrator.


Sent by Microsoft Exchange Server 2007

Diagnostic information for administrators:

Generating server: srv.dir.telstra.com

ComplaintResolutionCentre@team.telstra.com
#550 5.1.1 RESOLVER.ADR.RecipNotFound; not found ##

Original message headers:

Received: from ipani.tcif.telstra.com.au (10.97.216.198) by
 ties-smtp.in.telstra.com.au (172.49.40.197) with Microsoft SMTP Server id
 8.3.485.1; Sat, 7 Jul 2018 17:58:02 +1000
Received: from ipocni.tcif.telstra.com.au ([10.97.216.53])  by
 ipbani.tcif.telstra.com.au with ESMTP; 07 Jul 2018 17:58:02 +1000
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: =?us-ascii?q?A0GkBACJcUBb/+KwZZaFN5wRlRWBaTKBT?=
 =?us-ascii?q?YYSBgMCAgKGSwtCJwE8FYEggwqqCQUOgmyEHYUAgStDAWaJaIMgSYRqCAUFAQs?=
 =?us-ascii?q?IB1eCWYo0hF4Pg1eBKA6YUIQOgmt2imKIYIUYPYIxoUUCDRsDggU?=
X-IPAS-Result: =?us-ascii?q?A0GkBACJcUBb/+KwZZaFN5wRlRWBaTKBTYYSBgMCAgKGSwt?=
 =?us-ascii?q?CJwE8FYEggwqqCQUOgmyEHYUAgStDAWaJaIMgSYRqCAUFAQsIB1eCWYo0hF4Pg?=
 =?us-ascii?q?1eBKA6YUIQOgmt2imKIYIUYPYIxoUUCDRsDggU?=
X-IronPort-AV: E=Sophos;i="5.51,320,1526306400"; 
   d="png'150?scan'150,208,217,150";a="119258049"
X-Amp-Result: UNKNOWN
X-Amp-Original-Verdict: FILE UNKNOWN
X-Amp-File-Uploaded: False
X-SBRS: None
Received: from eth2015.qld.adsl.internode.on.net (HELO
 mail.longlandclan.id.au) ([150.101.176.226])  by ipxcno.tcif.telstra.com.au
 with ESMTP; 07 Jul 2018 17:57:59 +1000
Received: from [IPv6:2001:44b8:21ac:7053:a64e:31ff:fe53:99cc] (unknown
 [IPv6:2001:44b8:21ac:7053:a64e:31ff:fe53:99cc])	by mail.longlandclan.id.au
 (Postfix) with ESMTPSA id C159B51F720	for
 <ComplaintResolutionCentre@team.telstra.com>; Sat,  7 Jul 2018 17:57:56 +1000
 (EST)
Subject: [SR 1-1580842703975] Re: Follow Up-Your complaint with Telstra
References: <1e3d0bcc-a187-42cb-ac52-1e1ef0f4673b@wsmsg3704.srv.dir.telstra.com>
To: <ComplaintResolutionCentre@team.telstra.com>
From: Stuart Longland <me@mydomain.org>
Openpgp: id=77102FB21549FFDE5E13B83A0C7F53F4F359B8EF;
 url=https://stuartl.longlandclan.id.au/key.asc
Message-ID: <b5da1c9c-bc3d-8b2f-0f56-55361dc16503@longlandclan.id.au>
Date: Sat, 7 Jul 2018 17:57:51 +1000
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.7.0
MIME-Version: 1.0
In-Reply-To: <1e3d0bcc-a187-42cb-ac52-1e1ef0f4673b@wsmsg3704.srv.dir.telstra.com>
Content-Type: multipart/mixed;
	boundary="------------37DC9E91B74192D682B54693"
Content-Language: en-GB
Return-Path: me@mydomain.org
Reporting-MTA: dns;srv.dir.telstra.com
Received-From-MTA: dns;ipani.tcif.telstra.com.au
Arrival-Date: Sat, 7 Jul 2018 07:58:02 +0000

Final-Recipient: rfc822;ComplaintResolutionCentre@team.telstra.com
Action: failed
Status: 5.1.1
Diagnostic-Code: smtp;550 5.1.1 RESOLVER.ADR.RecipNotFound; not found

Oops… so there’s another complaint:

I note there’s another address (with an ‘s’ on the end) in the footer of the email, and so I have sent them the following:

Hi,
It's taken a little while to get back to you on this as I've been flat
out, but here goes.

On 07/07/18 17:20, Telstra_Notifications wrote:
> Your complaint with Telstra
>
> Reference no: SR x-xxxxxxxxxxxxx
>
> Dear Mr Longland,
>
> Thank you for getting in touch with us on 28 June 2018 about a
> complaint relating to your Telstra account number xxxx xxxxx xxxx.
>
> I’m sorry that you’ve experienced an issue with your service, but
> I'm pleased to offer you the following resolution.

To be clear, the issue is not with the mobile service itself, that's
been fine for the purpose I've used it. The issue is in the marketing
that came with it, that was unwanted.

> You were concerned that:
>
> * You’d like to be removed from Telstra’s marketing list

Yes, this is correct. It might be polite to ask people when they sign
up whether they want to be on this marketing list or not.

In my case, the service is temporary: I have the loan of a prototype
mobile phone: iSquare Mobility Kite v1.

http://www.kiteboard.io/ is the device being trialled.

The manufacturer has loaned it so that I can trial the device on the
Australian mobile networks, and see how it performs in weak-signal
conditions. I have loan of it possibly for another month or so at most.

(So far, it performs *MUCH* better than the ZTE T83 I use, and holds its
own against the ZTE T84 which uses the same chipset as the Kite.)

I'd have used my own SIM card, but my card is too big to fit in this
phone (mine is a miniature SIM, this phone requires a micro-SIM), and
given its temporary custody, it made no sense to get my existing Telstra
service moved to a new SIM.

Thus for this purpose, I just activated a pre-paid service to be able to
try the device out. I also have a service activated with Optus as it's
a dual-SIM device.

Once iSquare Mobility ask for the return of the device, naturally I'll
have little use for the two pre-paid SIM cards that are presently in it,
and won't have any interest of any offers from Telstra (or Optus).

I have an old 3G phone I can possibly use up the remaining credit of the
Telstra SIM in, otherwise I'll just use my current phone service which
I've had since 2001.

> * Telstra should fix broken complaints form
>
> I've confirmed that:
>
> * We have checked your account and found no marketing emails sent to
> you for the past two months

Allow me to present exhibit A; sent Thu, 28 Jun 2018 00:39:53 -0700.
This is attached.

I'm a little surprised your list management software had trouble finding
it, unless of course, you didn't read the complaint message carefully to
see the address my account was *actually* registered under.

I see you don't mention the issues with the form. One issue makes the
form damn-near unusable for anyone due to malformed HTML causing the
entire form to act as a hyperlink to the complaints information PDF.

The other, prevented me from self-unsubscribing and was the reason for
the complaint in the first place.

Don't worry, the world already knows:
https://vk4msl.com/2018/06/28/telstra-another-mob-that-didnt-get-the-rfc5233-memo/

I see the missed tag on the complaint form has now been corrected.

The original issue that started this, so far has not been corrected.
I've attached screenshots for your reference.

> We know you've been put out by this matter so we'd like to fix things
> by:
>
> * Confirming the medium of marketing (SMS, Email, phone call, MMS,
> face to face marketing, etc) and date you received it

This is email marketing. There have not been any other forms of marketing.

> * Removing your name and details from Telstra’s marketing list.
> Please be advised that this is only applicable for Telstra marketing
> calls.

Yep, I understand this. This is a silent number, and a temporary one at
that. By Christmas time, this service will be no-more, as it will be
surplus to requirements.

> If you’d like to talk more about this or accept this offer, please
> contact me on 1800 241 787* PIN 5172 or email
> ComplaintResolutionCentre@team.telstra.com quoting your Telstra
> reference SR x-xxxxxxxxxxxxx number. I'm available Tuesday-Saturday,
> 9am-5pm (AEST).

For reference, ComplaintResolutionCentre@team.telstra.com bounces. I've
attached the bounce message I received, and have also submitted it as SR
x-xxxxxxxxxxxxx just in case this email doesn't get through.

So that's now 4 issues in total, with 1 resolved so far.

If you could fix up the broken email validation on the opt-out form and
complaints form, and fix the broken email address in your reply messages
then that will resolve the remaining issues.

Thanks in advance.
Regards,
--
Stuart Longland (aka Redhatter, VK4MSL)

I haven't lost my mind...
...it's backed up on a tape somewhere.