computing

Solar Cluster: First power module built

So, I got most of the bits together to build a first power module for the cluster. This is after a mix-up with some of the parts, namely:

  • the 2 and 3-pin KK connectors I ordered just came as housings, no pins.
  • the supplier mixed up my order and sent rubber feet instead of 3-pin KK connector housings.

So I’ve got some 2-pin housings, but no pins … no problem, this was my fault for not checking and I think I’ve found the right pins for the task. I’ve placed a second order for these (along with some other bits to play with).

The rubber feet will be put on one side: I have no immediate use for them, and the supplier has dispatched the 3-pin KK housings already. A mix-up at their end hopefully rectified. So I’ll probably have those early next week, and the pins should arrive Thursday or Friday.

In the meantime though, I rummaged around the junk box and found a 3-pin KK that was snipped off a socket-370 CPU fan. For what it’s worth, the plug it’s connecting to is de-soldered from an old motherboard too.

The purpose of this module is to switch on or off a +12V charging supply… the intent is that to the board, it looks like a MOSFET, with the same control signals one would see. It is intended to be hot-pluggable (in more ways than one).

The red/black Anderson connector is the DC input, which can in theory be up to 16V DC. It passes in through the white Anderson, through both MOSFETs then out the yellow Anderson to the flying lead that will be connected to the +12V bus bar.

A second flying lead allows connection of the 0V supply to the 0V bus bar. I’m deficient in the black wire department, so a scrap length of heavy gauge speaker wire will do here.

I’ll have to work on how to mount those connectors, at the moment they’re just hanging loose which is not good long-term. I’ll probably glue these to a small piece of perspex, drill/tap some mounting holes and screw it down to stop it flapping in the breeze.

Giving the Raspberry Pi the finger

So, recently, the North West Digital Radio group generously donated a UDRC II radio control board in thanks for my initial work on an audio driver for the Texas Instruments TLV320AIC3204 (yes, a mouthful).

This board looks like it might support the older Pi model B I had, but I thought I’d play it safe and buy the later revision, so I bought version 3 of the Pi and the associated 7″ touch screen.  Thus, an order went to RS for a whole pile of parts, including one Raspberry Pi3 computer, a blank 8GB MicroSD card, a power supply, the touch screen kit and a case.

Fitting the UDRC

To fit the UDRC, the case will need some of the plastic cut away,  rectangular section out of the main body and a similarly sized portion out of the back cover.

Modifications to the case

Modifications to the case

When assembled, the cut-away section will allow the DB15-HD and Mini-DIN6 connectors to protrude out slightly.

Case assembled with modifications

The UDRC needs some minor modifications too for the touch screen.  Probe around, and you’ll find a source of 5V on one of the unpopulated headers.  You’ll want to solder a two-pin header to here and hook that to the LCD control board using the supplied jumper leads.  If you’ve got one, use a right-angled header, otherwise just bend a regular one like I did.

5V supply for the LCD on the UDRC

5V supply for the LCD on the UDRC

You’ll note I’ve made a note on the DB15-HD, a monitor does NOT plug in here.

From here, you should be ready to load up a SD card.  NWDR recommend the use of Compass Linux, which is a Raspbian fork configured for use with the UDRC.  I used the lite version, since it was smaller and I’m comfortable with command lines.

Configuring screen rotation

If you try to boot your freshly prepared SD card, the first thing you’ll notice is that the screen is up-side-down.  Clearly a few people didn’t communicate with each-other about which way was up on this thing.

Before you pull the SD card out, it is worth mounting the first partition on the SD card and editing config.txt on the root directory of that partition. If doing this on a Windows computer ensure your text editor respects Unix line endings! (Blame Microsoft. If you’re doing this on a Mac, Linux, BSD or other Unix-ish computer, you have nothing to worry about.)

Add the following to the end of the file (or anywhere really):

# Rotate the screen the "right way up"
lcd_rotate=2

Now save the file, unmount the SD card, and put it in the Pi before assembling the case proper.

Setting up your environment

Now, if you chose the lite option like I did, there’ll be no GUI, and the touch aspect of the touchscreen is useless.  You’ll need a USB keyboard.

Log in as pi (password raspberry), run passwd to change your password, then run sudo -s to gain a root shell.

You might choose like I did to run passwd again here to set root‘s password too.

After that, you’ll want to install some software.  Your choice of desktop environment is entirely up to you, I prefer something lightweight, and have been using FVWM for years, but there are plenty of choices in Debian as well as the usual suspects (KDE, Gnome, XFCE…).

For the display manager, I’ll choose lightdm. We also need an on-screen keyboard. I tried a couple, including matchbox-keyboard and the rather ancient xvkbd. Despite its age, I found xvkbd to be the most usable.

Once you’ve decided what you want, run apt-get install with your list of packages, making sure to include xvkbd and lightdm in your list.  Other applications I included here were network-manager-gnome, qasmixer, pasystray, stalonetray and gkrellm.

Enabling the on-screen keyboard in lightdm

Having installed lightdm and xvkbd, you can now configure lightdm to enable the accessibility options.

Open up /etc/lightdm/lightdm-gtk-greeter.conf, look for the line show-indicators and tack ;~a11y on the end.

Now down further, look for the commented out keyboard setting and change that to keyboard=xvkbd. Save and close the file, then run /etc/init.d/lightdm restart.

You should find yourself staring at the log-in screen, and lo and behold, there should be a new icon up the top-right. Tapping it should bring up a 3 line menu, the bottom of which is the on-screen keyboard.

On-screen keyboard in lightdm

On-screen keyboard in lightdm

The button marked Focus is what you hit to tell the keyboard which application is to receive the keyboard events.  Tap that, then the application you want.  To log in, tap Focus then the password field.  You should be able to tap your password in followed by either the Return button on the virtual keyboard or the Log In button on the form.

Making FVWM touch-friendly

I have a pretty old configuration that has evolved over the last 10 years using FVWM that was built around keyboard-centric operation and screen real-estate preservation.  This configuration mainly needed two changes:

  • Menus and title bar text enlarged to make the corresponding UI elements finger-friendly
  • Adjusting the size of the FVWM BarButtons to suit the 800×480 display

Rather than showing how to do it from scratch, I’ll just link to the configuration tarball which you are welcome to play with.  It uses xcalendar which isn’t in the Debian repositories any more, but is available on Gentoo mirrors and can be built from source (you’ll want to install xutils-dev for xmake), stalonetray and gkrellm are both in the standard Debian repositories.

FVWM on the Raspberry Pi

FVWM on the Raspberry Pi

Enabling the right-click

This took a bit of hunting to figure out.  There is a method that works with Debian Wheezy which allows right-clicks by way of long presses, but this broke in Jessie, and the 2016-05-23 release of Compass Linux is built on the latter.  So another solution is needed.

Philipp Merkel however, wrote a little daemon called twofing.  Once installed, doing a right click is simply a two-fingered tap on the screen, there’s support for other two-fingered gestures such as pinching and rotation as well.  It is available on Github, and I have forked this, adding some udev rules and scripts to integrate it into the Raspberry Pi.

The resulting Debian package is here.  Download the .deb, run dpkg -i on it, and then re-start the Raspberry Pi (or you can try running udevadm trigger and re-starting X).  The udev rules should create a /dev/twofingtouch symbolic link and the installed Xsession.d/Xreset.d scripts should take care of starting it with X and shutting it down afterwards.

Having done this, when you log in you should find that twofing is running, and that right clicks can be performed using a two-fingered prod.

Finishing up

Having done the configuration, you should now have a usable workhorse for numerous applications.  The UDRC shows up as a second sound card and is accessible via ALSA.  I haven’t tried it out yet, but it at least shows up in the mixer application, so the signs are there.  I’ll be looking to add LinBPQ and FreeDV into the mix yet, to round the software stack off to make this a general purpose voice/data radio station for emergency communications.

Dual-stack OpenVPN port sharing with HTTPS

Sometimes, it is desirable to have a TLS-based VPN tunnel for those times when you’re stuck behind an oppressive firewall and need to have secure communications to the outside world.  Maybe you’re visiting China, maybe you’re making an IoT device and don’t want to open your customers’ networks to world+dog by making your device easy to compromise (or have it pick on Brian Krebs).

OpenVPN is able to share a port with a non OpenVPN server.  When a tunnel is established, it looks almost identical to HTTPS traffic because both use TLS.  The only dead giveaway would be the OpenVPN session lasts longer, but then again, in this day of websockets and long polling, who knows how valid that assumption will be?

The lines needed to pull this magic off?  Here, we have sniproxy listening on port 65443. You can use nginx, Apache, or any other HTTPS web server here.  It need only be listening on the IPv4 loopback interface (127.0.0.1) since all connections will be from OpenVPN.

port 443
port-share localhost 65443

There’s one downside.  OpenVPN will not listen on both IPv4 and IPv6.  In fact, it takes a ritual sacrifice to get it to listen to an IPv6 socket at all.  On UDP, it’s somewhat understandable, and yes, they’re working on it.  On TCP, it’s inexcusable, the problems that plague dual-stack sockets on UDP mostly aren’t a problem on TCP.

It’s also impossible to selectively monitor ports.  There’s a workaround however.  Two, in fact.  Both involve deploying a “proxy” to re-direct the traffic.  So to start with, change that “port 443” to another port number, say 65444, and whilst you’re there, you might as well bind OpenVPN to loopback:

local 127.0.0.1
port 65444
port-share localhost 65443

Port 443 is now unbound and you can now set up your proxy.

Workaround 1: redirect using xinetd

The venerable xinetd superserver has a rather handy port redirection feature.  This has the bonus that the endpoint need not be on the same machine, or be dual-stack.


service https_port_forward
{
flags = IPv6               # Use AF_INET6 as the protocol family
disable = no               # Enable this service
type = UNLISTED            # Not listed in standard system file
socket_type = stream       # Use "stream" socket (aka TCP)
protocol = tcp             # Protocol used by the service
user = nobody              # Run proxy as user 'nobody'
wait = no                  # Do not wait for close, spawn a thread instead
redirect = 127.0.0.1 65444 # Where OpenVPN is listening
only_from = ::/0 0.0.0.0/0 # Allow world + dog
port = 443                 # Listen on port 443
}

Workaround 2: socat and supervisord

socat is a Swiss Army knife of networking, able to tunnel just about anything to anything else.  I was actually going to deploy that route, but whilst I was waiting for socat and supervisord to install, I decided to explore xinetd‘s capabilities.  Both will do the job however.

There is a catch though, socat does not daemonise. So you need something that will start it automatically and re-start it if it fails. You might be able to achieve this with systemd, here I’ll use supervisord to do that task.

The command to run is:
socat TCP6-LISTEN:443,fork TCP4:127.0.0.1:65444

and in supervisord you configure this accordingly:

[program:httpsredirect]
directory=/var/run
command=socat TCP6-LISTEN:443,fork TCP4:127.0.0.1:65444"
redirect_stderr=true
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
autostart=true
autorestart=true

Solar Cluster: First power module, partially built

So, I’ve still got to think about how to do the connectors, but so far, this is what the module looks like.

I’ve spaced the MOSFETs apart in case I decide to use physically bigger parts. The schematic looks like this.

Experimenting with the orientation of the PowerPole connectors, yes, you can mount a pair with one rotated 90°, but then it is impossible to configure the mating pair to match that 90° orientation, the rotated connector will always be 180° out.

So there goes that idea. However, I can use a 2D plywood “funnel” to make it impossible to misalign the connectors. The following not-to-scale diagram shows the gist of what I’m thinking of.

Effectively, I have 3 connectors: White represents Source, Green represents the Gate and Yellow represents the Drain of a typical MOSFET. There’s a pull-up to ensure the MOSFET stays OFF unless explicitly pulled low to turn it ON. The three pins wire directly up to where Q2 and Q4 are located.

In the event of magic smoke escape, I can grab a pair of oven mitts (the heat-sink will be hot, right?), grab the heat-sink and pull upwards to release it and disconnect it from the circuit. The process should be idiot proof enough that I should be able to leave instructions for anyone to follow.

Re-installing a module is the reverse process, just align the module and pull down. It’ll click into place and should start operating straight away.

I’m toying with the idea of including a small thermal fuse on the heat-sink to remove the MOSFETs from circuit should they get too hot. Reason being, someone might not be around if the magic smoke does escape, although provided the heat-sink does the job in the load test next weekend, this should be unlikely.

These look like they’ll do what they want. Okay, means I have to settle for 25A instead of 30A… but the charger is only 20A right now anyway. The 30A max rating was chosen based on the capability of the connectors. 25A is “good enough”.

Solar Cluster: Separating concerns

So, my last attempt at a fully integrated power controller was a smouldering failure. Q7 decided it wasn’t happy about where things were going, and let the world know by the only way it knew: smoke signals!

Curiously, only one of the MOSFETs in use seems to be damaged. When we look at its mate on the other side, Q2, sure it’s discoloured, which could be indication that it has been stressed, or maybe it just got burned by the other MOSFET.

It goes without saying that the pair in the background are fine: no current flowed through them during this test.

This got me thinking, did it get too hot, or did something else go wrong? This is the schematic and PCB layout of that part of the board.

Now looking at it, one thing strikes me. Seems I might have the source pins connected back-to-back, not the drain pins. Could that be it? I think I intended it the other way but didn’t pay enough attention to the schematic symbol. This could be a factor, or maybe the other MOSFET might’ve blown instead.

One thing is certain, I cannot join the two tabs together to the same heat-sink like I was intending with this schematic. Mia culpa!

Thinking about the design, the idea of putting the MOSFETs might be a tad naïve, as by far they are going to be the most likely component to fail on the board. The concept of a separate power module would work better since it’s easy then to just rip one out and put another in its place. Designed right, this could even be hot-pluggable, and can incorporate a heat-sink, and can make use of larger or smaller MOSFETs for different applications.

Luckily, the existing board layout will accommodate this just fine. We put 3-pin KK connectors in place of Q2 and Q4, and we can jumper across Q7 and Q8. This means the feed to the battery only needs to be a light-gauge wire, sufficient to power the controller and measure the battery voltage.

The pin-out isn’t ideal for this: it would be better to have a pin connecting to 0V instead of the battery +12V, but it’s workable. The gate pin becomes an open-collector output, and can theoretically drive (low-current) relays or MOSFETs.

As for what to build this power module on? Well, without going and buying a heat-sink, I’m spoiled for choice:

All of these dwarf the TO-220 package transistor I’m using… okay the one shown is an IRF-9540, but it’s still a TO-220 put there for scale reference. Most of these are for Intel CPUs that are long obsolete, and the top right has the CPUs in question still firmly attached.

The Pentium CPU heat-sink/fan would be the closest in the size, I was hoping I might’ve had a 486 heat-sink laying around, I’m of the opinion that if the power module needs a fan on any of these heat sinks, I’m doing it wrong. This might not be the case if I wanted the full 70A capability, but I’m pushing for 30 which is less than 50% capacity.

The only passive ones I have, and happen to have multiple of, are the ones on the lower right, which were extracted from dead Netgear (Bay Networks) switches. The BGA package still stuck to one of them is a Broadcom BCM5308A2KTB Ethernet switch SoC… it talked to a couple of SRAM packages (duly harvested) and a number of Ethernet PHYs.

The thought is that two MOSFETs could be fixed to the underside with a small PCB. (Well okay, there’s room for all four, but then I’ve got to somehow electrically insulate the two pairs.)

A connector of some sort, either a PCB edge connector, or perhaps a specially keyed Andersen Power Pole connector pairs (which can be rotated 90°) could connect power and control in one secure mounting. Two 30A connectors and a 15A would serve this job well, and they come in a range of colours for the housings. Thus I can avoid the red/black typical colouring to avoid confusion.

Solar Cluster: Full load test: FAIL

So, I drag the cluster, battery and 20A charger out to the deck to do a full load test. This is the first time I’ve fired this newly built controller on a full load. Uncharted territory so far.

Here’s the set up.

The charge controller that I built earlier is in a nice shiny re-purposed case that I had laying around. Just the right size too. These were USB extender devices that my father’s workplace had used in a project: they wanted the innards, so we got the empty boxes. I just mounted the PCB on a small piece of plastic to insulate it from the case, and routed my DC leads out through a hole in the case originally intended for an RJ-45 jack.

At this point, everything is humming along fine. Our battery charger is on stand-by.

The meter is showing a sedate 12.4V and the controller is happy with this. That said, I’ll have to work on the visibility of those LEDs. The two on the power MOSFET control lines are off at this stage.

So I give the system a bit of curry. I transfer a copy of a Linux kernel git repository to each, tell them to update their working copies from that, and build a version of kernel v4.8.5. This made the current jump up to about 10A. So far so good, the battery is holding.

Then, about 30 seconds in, the controller decides it’s a bit too low, so it kicks the charger on. There’s a few false starts, as the charger delays its start-up a bit. Eventually though they get into sync and start charging. At this point, the charger is taking the load of the battery and the cluster.

Great. So it continues for a minute, then decides it wants to shut down, which it does, followed by a moment of oscillation. It seems the controller is too impatient for the charger, waiting for the power to come on…but then… what’s that smell???

Oops, guess that MOSFET got just a little too hot. I was hoping to avoid the need for heatsinks by over-dimensioning the MOSFETs. These are supposedly able to take 70A, I realise that’s with a heatsink, but I thought that at 20A, they would be able to handle it.

One somewhat roasted MOSFET says otherwise. Interestingly, only one has visible marks, its mate looks okay, but likely isn’t.

The MOSFETs don’t have to be mounted directly on the PCB, we can re-locate them to where we can squeeze a heatsink in if I can’t get one in between the two already. The thought was each pair have a heatsink in between them. More pondering to do it seems.

Solar Cluster: Light testing of the charge controller

So, late yesterday afternoon, I devised a light test of the controller to see how it would perform.

For this I disconnected all but one of the nodes, and hooked up one of my old 10Ah LiFePO₄ packs and my 3A charger hooked to mains. The LM2576-based charger is just able to hold this load and provide 1A charging current.

The first thing I noticed is that the fan seemed to turn on and off a lot… this could be a difference in the temperature sensors between the DIP version of the ATTiny24A that the prototype used and the SOIC version which the new controller used.

The test ran overnight. The node basically was idling, as were the two Ethernet switches. But, it served the purpose. I now know the logic is sound, although I might want to adjust my set-points a little.

That’s the output data from a small digital power meter that was hooked up in circuit. This device is unable to display negative current, so the points at which the battery was charging is shown as 0A. Left axis is voltage, right is current. You can see that the charger gets brought in when the battery dips below 12V and clicks off just before 13.2V.

I can probably go a little higher than that, maybe about 13.6V. I may also need to re-visit the fixed resistor settings on the linear regs inside the nodes to knock them down a few more pegs to prevent the BMCs whining about the high voltage.

Next weekend, I might consider hooking up the 20A mains charger and giving it a full load test.

Solar Cluster: First charge controller board built

So, I ordered the parts last weekend and over the course of the last week, they arrived, in three dispatches.

Might’ve been cheaper for RS to wait for the lot to arrive, I’d have been happy for them to do that had they asked, but never mind, they all turned up. So today, in clouds of solder smoke out on the back deck, I set to work soldering up the first of these boards.

At first, I just put all the SMD parts on, the programming header, the 5V reg, and the DC input lead. This proved a little fun. When I ordered the parts, I had initially put a lot of resistors, all 0805 size, to fill some of my stock. Then I took some off the list because I was over budget a little. Murphy dictates the ones that you take off are the ones you need.

I had thought I had spares anyway… turns out those were 1206s. And, in a pinch, one can use 1206 size resistors on an 0805 size footprint, or at least the “hand-soldering” variants that Kicad produces. The 10k pull-up resistors used on the MOSFETs and the 47k pull-up on the reset line are all 1206s, and they fit okay.

Had I remembered I had 1206s not 0805s, I’d have used 1206s. They’re a little friendlier to work with.

Lessons:

  • always check the list before hitting the order button!
  • In a pinch, you might be able to squeeze a 1206 onto an 0805 footprint, but don’t bet on it!

The other thing I could have done better would be to nudge the footprint for the 7805 along a little so that the switch-mode alternative part (ROHM BP5293-50) would fit. Better yet would be to not be so darn lazy and actually make a footprint for it. It’s a little cosy, but it snuggles up nicely against the capacitor.

Similarly, slightly bigger holes for the power connectors would help too, although the pins on the MOSFETs aren’t exactly big and they’re supposed to handle 70A. The lead length is short though, so not much resistance, I’ll probably get away with it.

The LEDs also proved a challenge, namely figuring out which way around they went. Most were all Osram CHIPLED parts, had a milky-white lens, and no markings visible on the top (one nondescript marking on the bottom), so it was difficult to tell anode from cathode. I wound up using my electronics kit to supply 4.5V through a 1k resistor out to two wires (a twisted pair from some CAT5e) which I could touch to each side to know which was which.

The one exception was the batch of yellow LEDs I bought, Osram didn’t seem to do the yellow ones so wound up buying Kingbright ones, which featured two green dots showing the cathode (the “bar” end of the schematic symbol). When I run out of the existing stock of red, orange and green LEDs, I’ll get more like these I think as it makes life much easier.

This was shortly after programming, the code running on the board for the first time.

Seeing the LEDs blinking, I disconnected the board and proceeded to fit the remaining through-hole components. The resulting board seems to be making the right noises, turning the fan on and off in response to the internal temperature sensor, and running from a 9V source.

Now I guess I’ll wire up the remaining leads, hook it to DC power and see what happens.

Solar Cluster: Boards Arrived

So, after a couple of email enquiries, the truth is unveiled. As it turns out, Swiss Post send parcels via one or more transit countries which due to a quirk in their tracking UI, may appear as the destination.

Asendia were in touch last night informing me that: “Please kindly note that sometimes tracking website will show the wrong destination. Canada is only the transit country.” Ahh okay, no problem then. Confusing, but that’s fine. 🙂

I know now for later not to panic if it says Canada.

This morning, there was a surprise parcel arrived at work, containing 6 PCBs. Bonus! Guess I had better get the other parts on order. I won’t have them ready for this week end, but I predict much solder smoke in my future next weekend.

Solar Cluster: Getting PCBs made

So a few weeks ago, I gave the charge controller a test, seeing if it in fact reacted in the manner I expected before deciding whether to proceed with the existing prototype or whether I should iterate the design.

In the end, I decided I’d tweak the design and get new boards built. By using SMD parts and a 4-layer board, I was able to shrink my design down to a 5×5cm square, which is relatively inexpensive to have fabricated.

I’ll be getting a few boards which means I can have some spares in case something goes bang or if I want to scale out my battery bank.

The updated design is published in the files section. This also incorporates @K.C. Lee‘s advice regarding back-to-back MOSFETs.

After some fun and games, one PCB fab house telling me to “check my passwords match” (when I know for certain that they did match), and another seemingly ignoring the inner two layers, I settled on a PCB manufacturer (thanks to PCBShopper) and got the boards ordered.

I put down my home address for the billing address and my work address as the delivery address. Both given as being in “Queensland, Australia”.

This is a learning experience for me, I’m used to just drawing my circuit out with a dalo pen, but unfortunately my skills aren’t up to producing a board for SOICs.

They reported that they shipped the boards on the 21st, and had previously estimated about 2-3 weeks for delivery. No problem there.

Just one niggling concern…

Not familiar with Swiss Post procedures, I’d have expected it to show Hong Kong → Australia, but maybe that’s how they do things. I do hope someone didn’t get Queensland, Australia mixed up with Quebec, Canada!

Update: Just been in touch, no the manufacturer didn’t get it mixed up, and it’s the right tracking number. They’re chasing it up with Swiss Post.