September 2024

lvmcache fun and games

This blog has never been on what I’d call, a high-performance server. In fact, things are a little on the slow side. I try to be frugal with my system resource allocation, with the assumption that my little site does not get a lot of traffic (much less since it’s no longer syndicated on Gentoo Planet). However, I think I managed to get the performance up a notch…

The site runs on my solar powered server cluster, with a couple of Ceph RBDs, one for the root OS and one for the data (MariaDB / www root / /home). The VM runs AlpineLinux. The VM host was over-provisioned with a larger SSD than required, allowing me to dedicate some space for local cache.

I had thought I could set something up that would organise the cache on the VM host, and abstract it from the VM, but so far, I’ve not gotten around to doing that. (I did have something sort-of working in OpenNebula with flashcache at work, but it was flaky.)

In libvirt, I provisioned a new RBD to serve as the backing store (thus keeping a pristine copy to roll back to should things go pear shaped), and a new LVM volume for the cache. For the time being, I moved the existing volume to be the last device. So I had:

  • /dev/vda: OS
  • /dev/vdb: Data volume
  • /dev/vdc: Cache volume
  • /dev/vdd: temporary Old /dev/vdb for data migration

Failed approaches

Firstly, what didn’t work for me, was bcachefs and bcache.

bcachefs

bcachefs wanted to fight me every step of the way, making formatting the volumes difficult with sketchy documentation (especially as I wanted a write-through cache to facilitate VM migration).

bcachefs format gives some very cryptic error messages, and has a somewhat quirky argument syntax for formatting. The command I figured out through trial-and-error was this:

bcachefs format \
    --replicas=1 \
    --durability=0 /dev/vdc1 \
    --durability=1 /dev/vdb1 \
    --foreground_target /dev/vdc1 \
    --promote_target /dev/vdc1 \
    --background_target /dev/vdb1 \
    --metadata_target /dev/vdc1

The problem was convincing mount to actually mount it. I was supposed to specify every device, but each time it flatly refused, no matter what order I used, it told me “no such device”.

bcache

This is the underlying caching logic that bcachefs was built on, so I figured I’d try that. This worked better, but I found AlpineLinux had no real knowledge of bcache, and thus did not provide any means for me to bring up /dev/bcache0 before localmount mounted it.

I could have written a OpenRC init script to do this, but I wasn’t certain about this path, so decided to put the idea aside.

Winning approach: lvmcache

Luckily lvm2 has a built-in method: lvmcache. After installing the lvm2 package in AlpineLinux, I blatted the partition tables on my two virtual disks, formatted them as LVM physical volumes, and added them to a volume group.

~ # pvcreate /dev/vdb /dev/vdc
  Physical volume "/dev/vdb" successfully created.
  Physical volume "/dev/vdc" successfully created.
~ # vgcreate data /dev/vdb
  Volume group "data" successfully created
~ # vgextend data /dev/vdc 
  Volume group "data" successfully extended

Now to create the logical volumes, first… I created the volumes themselves. This wound up being a little tricky because I wanted to use all the available space on each volume… I had tried specifying -L ${SZ}G but this ignored the fact that LVM uses a bit of header space on each physical volume. It complained, but in doing so, told me the size in extents that was available, so I was able to use -l ${SZ} to specify that number of extents:

~ # lvcreate --size 8G --name datavol data /dev/vdb
 Insufficient free space: 2048 extents needed, but only 2047 available
~ # lvcreate -l 2047 --name datavol data /dev/vdb  
 Logical volume "datavol" created.
~ # lvcreate -n cachevol -l 4095 data /dev/vdc
 Volume group "data" has insufficient free space (1023 extents): 4095 required.
~ # lvcreate -n cachevol -l 1023 data /dev/vdc
 Logical volume "cachevol" created.

Now I had two separate LVM volumes, one on each physical device. Now to link them:

~ # lvconvert --type cache --cachevol cachevol data/datavol
Erase all existing data on data/cachevol? [y/n]: y
 Logical volume data/datavol is now cached.

Great, except I forgot to specify the write mode. Turns out, this is a lvchange away:

~ # lvchange --cachemode writethrough /dev/data/datavol  
 Logical volume data/datavol changed.

I could now format /dev/data/datavol with a filesystem, and migrate the data across. rsync here we come. An update to /etc/fstab and we were in business.

So far, things seem to be more snappy, so we’ll keep an eye on things. It’s survived a couple of reboots, the question is what happens when I boost a post on Mastodon, does all the ActivityPub instances out there cause problems? Guess I’ll find out in a moment.

Station hopping once again

So, a few years ago, we said goodbye to the Brisbane institution that was Classic Hits 4KQ… with the decision of some Sydney bright sparks at Here, There & Everywhere, owners of Australian Radio Network, acquiring rival network Grant Broadcasting and merging the two together. This put ARN over their quota for the number of stations they were allowed to operate in the Brisbane region, and so they sold off their oldest to the Sports Entertainment Network.

That pretty much ended my all-day radio listening right there. The presenters from 4KQ’s morning show wound up on 4BC, but that’s about as much as was left of the old station. Well of course no good thing can last forever, and so Laurel, Gary & Mark broadcast their final show this morning. I will admit there was a bit of sadness as Greenday’s “Time Of Your Life” played (in place of Laurel’s Last Word)… then cron on my desktop PC muted qt-dab at 0900… it was time for me to get to work myself.

Peter Fegan takes over that shift on the 30th… and I for one, won’t be tuning in. 4BC now has nothing to offer me.

I had a quick tune around of the DAB+ and FM stations… to see what was on offer.

B105 is not worth bothering with, and haven’t been for the better part of 30 years now, such is the sad state of top-40 music these days. Sister station Triple M was a station I was listening to a lot at the early part of the century. Southern Cross Austereo runs both, as well as a stack of DAB+ only stations, however, the DAB+ stations are all at 32kbps. 48kbps is just tolerable for audio quality on the stereo here… and through headphones I can barely notice some compression distortion. 32kbps is okay for voice, but sounds terrible for music, ringing left-right and centre. So if I go to one of those stations, it’ll be via FM not DAB+.

In my hopping around, I came across one station that was broadcasting at 24kbps… through the speaker on my little portable DAB+ set I could hear the ringing artefacts. Not as bad as Coles TAS (at 16kbps!), but still bloody terrible! I didn’t realise there was a step between 16kbps and 32kbps… but I stand by my comment that 32kbps is the bare minimum for music… with 48kbps and above highly recommended.

4BC’s sister station 4BH actually adopted a lot of 4KQ’s old format… so for now I’ve set qt-dab up there. The main reason why I went to 4BC is the morning show, I had a choice to make: the people or the format… and so I decided to stick with “the people” for the morning, figuring I can fill the rest of my day with my own music. That’s what I’ve been doing the past couple of years. With “the people” gone, I’m now left with “the format” as the deciding factor. As for audio quality, they broadcast on MW at 1116kHz or DAB+ multiplex 9B at 96kbps — one of the highest bitrate commercial DAB+ streams.

Best case scenario I guess is the intrepid trio from 4KQ pop up here… seems this is the most likely place they’d appear. That said, this is quite likely it for a radio show that’s been going for 30+ years. End of the road. While Laurel and Mark are nowhere near retirement age, I would not be surprised at all if Gary decided to hang his headphones up and officially retire, being the oldest of the three.

I’ll admit I’ve learned a lot about the music I listen to through this show. The trio were enlightening and entertaining in equal measure. I guess time will tell as to whether this really is the end, or just a change of venue.

As for me with channel hopping… I’ve flirted with the idea of starting up a station of my own, but really this is “pipe dream” material. I’d need to team up with someone who knew the media business and could serve to run the organisation, whilst I’d be focussing on the technical matters of getting things on-air. Even there, I think I’d be serving beneath someone more senior — I’d be the apprentice broadcast engineer basically. Given the deep pool of people a prospective station general manager would have to choose from, I think there’d be a lot of competition. This is a job where you measure your success by the number of knives in your back!

Worst case… I have my music, I might tune in to a news broadcast occasionally, but otherwise my radio listening days may have finally ended. We shall see.

Elections and voting

So, it’s political season again, and here in Queensland we made the rather foolhardy decision to run our State election roughly a month or so away from the US Federal election.

Foolhardy because the media is too busy guffawing over the allegation of cats and dogs being “eaten” in a debate for an election that people like myself have no say over (and should not have any say over), to properly cover the election that is actually mandatory for people like myself to participate in.

But I digress… in amongst all the comments, there was a post made just recently by Taylor Swift. Now, I know she’s a very successful singer, not that I can name any of her songs (my tastes are for older fare). But, in a recent post, she made a very valid point. Here’s the post (transcribed from a screenshot) in full:

Like many of you, I watched the debate tonight. If you haven’t already, now is a great time to do your research on the issues at hand and the stances these candidates take on the topics that matter to you the most. As a voter, I make sure to watch and read everything I can about their proposed policies and plans for this country.

Recently I was made aware that AI of ‘me’ falsely endorsing Donald Trump’s presidential run was posted to his site. It really conjured up my fears around AI, and the dangers of spreading misinformation. It brought me to the conclusion that I need to be very transparent about my actual plans for this election as a voter. The simplest way to combat misinformation is with the truth.

I will be casting my vote for Kamala Harris and Tim Walz in the 2024 Presidential Election. I’m voting for @kamalaharris because she fights for the rights and causes I believe need a warrior to champion them. I think she is a steady-handed, gifted leader and I believe we can accomplish so much more in this country if we are lead by calm and not chaos. I was so heartened and impressed by her selection of running mate @timwalz, who has been standing up for LGBTQ+ rights, IVF, and a woman’s right to her own body for decades.

I’ve done my research, and I’ve made my choice. Your research is all yours to do, and the choice is yours to make. I also want to say, especially for first time voters: Remember that in order to vote, you have to be registered! I also find it’s much easier to vote early. I’ll link where to register and find early voting dates and info in my story.

With love and hope,

Taylor Swift
Childless Cat Lady

Now, I’ll set aside her endorsement for the US Democrats party. As she points out, she personally looked into the policies of the parties, and came to that conclusion. It’s the approach here that I want to highlight, and it’s as valid in the US with its esoteric election system as it is here in Australia with our (admittedly partially flawed, but generally highly regarded) preferential system.

I won’t be sharing who I’ll be voting for in the Queensland state election, as to be perfectly honest, I haven’t actually done my homework on that matter yet. Same goes for next year’s federal election.

The take-away I observe is the remark: “If you haven’t already, now is a great time to do your research on the issues at hand and the stances these candidates take on the topics that matter to you the most.

She’s not saying “vote blue because I did”, she’s saying to go do your homework, and figure out how you will vote. “Your research is all yours to do, and the choice is yours to make.

Here in Queensland, where I am falls under the seat of “Cooper” (formerly “Ashgrove”). We currently have a Labor candidate, Jonty Bush. From what I understand, she’s done a reasonable job and I don’t have a problem with her being voted back in, but I really need to figure out where she’ll sit in my ballot paper. Probably not the top, I usually like to reserve the top spaces for smaller parties, but as I say, I haven’t researched the matter much at all right now, so this is all subject to change, and nothing actually requires me to publish what I’m going to do in any case.

Labor recently started a trial of 50c public transport fares… with a view of encouraging their use. I note that the Greens have a policy that goes further: abolishing fares altogether long before the trial began, so I think we see where Labor’s policy came from. Polling seems to suggest Labor may be on the way out anyway. (And I remember the mess that the LNP made last time! We wound up with Campbell Newman as our local member.)

I’ll have to dig around and see what other policies the parties have, but this is an example of just one issue that one might consider. I strongly urge people to consider more than one issue! You might not care about public transport, but care greatly about mining activities: maybe you’re a voter who has lots of shares in mines, LNP seems to be cosying up with the miners too. That’s up to you.

My approach has been to start with all parties with a score of 0; and awarding points or deducting points, depending on how I feel about each policy they publish. I then use those “scores” to figure out my preferences. Try to ignore “who” made the policy, and just consider the policy’s content directly. (It’s a pity they don’t offer the policy docs in just plain text instead of a PDF or office suite file.) You can decide for yourself if you do this, or something completely different.

Crucially, there might be some hot-button issue where none of the parties agree on your position. The Israel-Palestine war is a good example, where I’ve heard people in the US say (effectively): “I’m going to vote Republican because the Democrats won’t stop arming Israel!” I’ve got bad news for you, the other side isn’t about to stop the flow of weapons either (quite the opposite in fact), it may be prudent to put that issue aside and focus on everything else for now. Get the “least worst” candidate in, then work with them on the issues that you had to set aside. Some will listen, some won’t.

Anyway, this is just my thoughts on the matter. No matter what part of the world you’re in, the next few years are going to be “interesting” to say the least. We’re already seeing what happens when someone can summon up a supercomputing data-centre to conjure up synthetic photos from text prompts, and I’d be living in a fantasy land if I were to try and make out all users of such systems were benevolent. Thus it’s incumbent on us to seek the source for policy research, go to the party’s website and look there. Don’t believe everything you see on social media, cats and dogs are not being served up on Ohio dinner tables, and not every “policy” published there will be authentic.

For us in Queensland, the Electoral Commission Queensland website would be a good starting point. The ABC will also publish details on their Elections page closer to the date.