battery-charging

Solar Cluster: Solar back on…

So, this weekend I did plan to run from solar full time to see how it’d go.

Mother nature did not co-operate.  I think there was about 2 hours of sunlight!  This is what the 24 hour rain map looks like from the local weather radar (image credit: Bureau of Meteorology):

In the end, I opted to crimp SB50 connectors onto the old Redarc BCDC1225 and hook it up between the battery harness and the 40A power supply. It’s happily keeping the batteries sitting at about 13.2V, which is fine. The cluster ran for months off this very same power supply without issue: it’s when I introduced the solar panels that the problems started. With a separate controller doing the solar that has over-discharge protection to boot, we should be fine.

I also have mostly built-up some monitoring boards based on the TI INA219Bs hooked up to NXP LPC810s. I have not powered these up yet, plan is to try them out with a 1ohm resistor as the stand-in for the shunt and a 3V rail… develop the firmware for reporting voltage/current… then try 9V and check nothing smokes.

If all is well, then I’ll package them up and move them to the cluster. Not sure of protocols just yet. Modbus/RTU is tempting and is a protocol I’m familiar with at work and would work well for this application, given I just need to represent voltage and current… both of which can be scaled to fit 16-bit registers easy (voltage in mV, current in mA would be fine).

I just need some connectors to interface the boards to the outside world and testing will begin. I’ve ordered these and they’ll probably turn up some time this week.

Solar Cluster: Remember kids, over-discharge protection matters!

So, yesterday I had this idea of building an IC storage unit to solve a problem I was facing with the storage of IC tubes, and to solve an identical problem faced at HSBNE.

It turned out to be a longish project, and by 11:30PM, I had gotten far, but still had a bit of work to do.  Rather than slog it out overnight, I thought I’d head home and resume it the next day.  Instead of carting the lot home, and back again, I decided to leave my bicycle trailer with all the project gear and my laptop, stashed at HSBNE’s wood shop.

By the time I had closed up the shop and gotten going, it was after midnight.  That said, the hour of day was a blessing: there was practically no traffic, so I road on the road most of the way, including the notorious Kingsford-Smith Drive.  I made it home in record time: 1 hour, 20 minutes.  A record that stood until later this morning coming the other way, doing the run in 1:10.

I was exhausted, and was thinking about bed, but wheeling the bicycle up the drive way and opening the garage door, I caught a whiff.  What’s that smell?  Sulphur??

Remember last post I had battery trouble, so isolated the crook battery and left the “good” one connected?

The charger was going flat chat, and the battery case was hot!  I took no chances, I switched the charger off at the wall and yanked the connection to the battery wiring harness.  I grabbed some chemical handling gloves and heaved the battery case out.  Yep, that battery was steaming!  Literally!

This was the last thing I wanted to deal with at nearly 2AM on a Sunday morning.  I did have two new batteries, but hadn’t yet installed them.  I swapped the one I had pulled out last fortnight, and put in one of the new ones.  I wanted to give them a maintenance charge before letting them loose on the cluster.

The other dud battery posed a risk though, with the battery so hot and under high pressure, there was a good chance that it could rupture if it hadn’t already.  A shower of sulphuric acid was not something I wanted.

I decided there was nothing running on the cluster that I needed until I got up later today, so left the whole kit off, figuring I’d wait for that battery to cool down.

5AM, I woke up, checked the battery, still warm.  Playing it safe, I dusted off the 40A switchmode PSU I had originally used to power the Redarc controller, and plugged it directly into the cluster, bypassing the batteries and controller.  That would at least get the system up.

This evening, I get home (getting a lift), and sure enough, the battery has cooled down, so I swap it out with another of the new batteries.  One of the new batteries is charging from the mains now, and I’ll do the second tomorrow.

See if you can pick out which one is which…

Solar Cluster: WTF

So… with the new controller we’re able to see how much current we’re getting from the solar.  I note they omit the solar voltage, and I suspect the current is how much is coming out of the MPPT stage, but still, it’s more information than we had before.

With this, we noticed that on a good day, we were getting… 7A.

That’s about what we’d expect for one panel.  What’s going on?  Must be a wiring fault!

I’ll admit when I made the mounting for the solar controller, I didn’t account for the bend radius in the 6gauge wire I was using, and found it was difficult to feed it into the controller properly.  No worries, this morning at 4AM I powered everything off, took the solar controller off, drilled 6 new holes a bit lower down, fed the wires through and screwed them back in.

Whilst it was all off, I decided I’d individually charge the batteries.  So, right-hand battery came first, I hook the mains charger directly up and let ‘er rip.  Less than 30 minutes later, it was done.

So, disconnect that, hook up the left hand battery.  45 minutes later the charger’s still grinding away.  WTF?

Feel the battery… it is hot!  Double WTF?

It would appear that this particular battery is stuffed.  I’ve got one good one though, so for now I pull the dud out and run with just the one.

I hook everything up,  do some final checks, then power the lot back up.

Things seem to go well… I do my usual post-blackout dance of connecting my laptop up to the virtual instance management VLAN, waiting for the OpenNebula VM to fire up, then log into its interface (because we’re too kewl to have a command line tool to re-start an instance), see my router and gitea instances are “powered off”, and instruct the system to boot them.

They come up… I’m composing an email, hit send… “Could not resolve hostname”… WTF?  Wander downstairs, I note the LED on the main switch flashing furiously (as it does on power-up) and a chorus of POST beeps tells me the cluster got hard-power-cycled.  But why?  Okay, it’s up now, back up stairs, connect to the VLAN, re-start everything again.

About to send that email again… boompa!  Same error.  Sure enough, my router is down.  Wander downstairs, and as I get near, I hear the POST beeps again.  Battery voltage is good, about 13.2V.  WTF?

So, about to re-start everything, then I lose contact with my OpenNebula front-end.  Okay, something is definitely up.  Wander downstairs, and the hosts are booting again.  On a hunch I flick the off-switch to the mains charger.  Klunk, the whole lot goes off.  There’s no connection to the battery, and so when the charger drops its power to check the battery voltage, it brings the whole lot down.

WTF once more?  I jiggle some wires… no dice.  Unplug, plug back in, power blinks on then off again.  What is going on?

Finally, I pull right-hand battery out (the left-hand one is already out and cooling off, still very warm at this point), 13.2V between the negative terminal and positive on the battery, good… 13.2V between negative and the battery side of the isolator switch… unscrew the fuse holder… 13.2V between fuse holder terminal and the negative side…  but 0V between negative side on battery and the positive terminal on the SB50 connector.

No apparent loose connections, so I grab one of my spares, swap it with the existing fuse.  Screw the holder back together, plug the battery back in, and away it all goes.

This is the offending culprit.  It’s a 40A 5AG fuse.  Bought for its current carrying capacity, not for the “bling factor” (gold conductors).

If I put my multimeter in continuance test mode and hold a probe on each end cap, without moving the probes, I hear it go open-circuit, closed-circuit, open-circuit, closed-circuit.  Fuses don’t normally do that.

I have a few spares of these thankfully, but I will be buying a couple more to replace the one that’s now dead.  Ohh, and it looks like I’m up for another pair of batteries, and we will have a working spare 105Ah once I get the new ones in.

On the RAM front… the firm I bought the last lot through did get back to me, with some DDR3L ECC SO-DIMMs, again made by Kingston.  Sounded close enough, they were 20c a piece more (AU$855 for 6 vs $AU864.50).

Given that it was likely this would be an increasing problem, I thought I’d at least buy enough to ensure every node had two matched sticks in, so I told them to increase the quantity to 9 and to let me know what I owe them.

At first they sent me the updated invoice with the total amount (AU$1293.20).  No problems there.  It took a bit of back-and-forth before I finally confirmed they had the previous amount I sent them.  Great, so into the bank I trundle on Thursday morning with the updated invoice, and I pay the remainder (AU$428.70).

Friday, I get the email to say that product was no longer available.  They instead, suggested some Crucial modules which were $60 a piece cheaper.  Well, when entering a gold mine, one must prepare themselves for the shaft.

Checking the link, I found it: these were non-ECC.  1Gbit×64, not 1Gbit×72 like I had ordered.  In any case I was over it, I fired back an email telling them to cancel the order and return the money.  I was in no mood for Internet shopper Russian Roulette.

It turns out I can buy the original sticks through other suppliers, just not in the quantities I’m after.  So I might be able to buy one or two from a supplier, I can’t buy 9.  Kingston have stopped making them and so what’s left is whatever companies have in stock.

So I’ll have to move to something else.  It’d be worth buying one stick of the original type so I can pair it with one of the others, but no more than that.  I’m in no mood to do this in a few years time when parts are likely to be even harder to source… so I think I’ll bite the bullet and go 16GB modules.  Due to the limits on my debit card though, I’ll have to buy them two at a time (~$900AUD each go).  The plan is:

  1. Order in two 16GB modules and an 8GB module… take existing 8GB module out of one of the compute nodes and install the 16GB modules into that node.  Install the brand new 8GB module and the recovered 8GB module into two of the storage nodes.  One compute node now has 32GB RAM, and two storage nodes are now upgraded to 16GB each.  Remaining compute node and storage node each have 8GB.
  2. Order in two more 16GB modules… pull the existing 8GB module out of the other compute node, install the two 16GB modules.  Then install the old 8GB module into the remaining storage node.  All three storage nodes now have 16GB each, both compute nodes have 32GB each.
  3. Order two more 16GB modules, install into one compute node, it now has 64GB.
  4. Order in last two 16GB modules, install into the other compute node.

Yes, expensive, but sod it.  Once I’ve done this, the two nodes doing all the work will be at their maximum capacity.  The storage nodes are doing just fine with 8GB, so 16GB should mean there’s plenty of RAM for caching.

As for virtual machine management… I’m pretty much over OpenNebula.  Dealing with libvirt directly is no fun, but at least once configured, it works!  OpenNebula has a habit of not differentiating between a VM being powered off (as in, me logging into the guest and issuing a shutdown), and a VM being forcefully turned off by the host’s power getting yanked!

With one, there should be some event fired off by libvirt to tell OpenNebula that the VM has indeed turned itself off.  With the latter, it should observe that one moment the VM is there, and next it isn’t… the inference being that it should still be there, and that perhaps that VM should be re-started.

This could be a libvirt limitation too.  I’ll have to research that.  If it is, then the answer is clear: we ditch libvirt and DIY.  I’ll have to research how I can establish a quorum and schedule where VMs get put, but it should be doable without the hassle that OpenNebula has been so far, and without going to the utter tedium that is OpenStack.

Solar Cluster: Solar controller replaced, upgrading RAM and VM management

So, this morning I decided to shut the whole lot down and switch to the new solar controller.  There’s some clean-up work to be done, but for now, it’ll do.  The new controller is a Powertech MP3735.  Supposedly this one can deliver 30A, and has programmable float and bulk charge voltages.  A nice feature is that it’ll disconnect the load when it drops below 11V, so finding the batteries at 6V should be a thing of the past!  We’ll see how it goes.

I also put in two current shunts, one on the feed into/out of the battery, and one to the load.  Nothing is connected to monitor these as yet, but some research suggested that while in theory it is just an op-amp needed, that op-amp has to deal with microvolt differences and noise.

There are instrumentation amplifiers designed for that, and a handy little package is TI’s INA219B.  This incorporates aforementioned amplifier, but also adds to that an ADC with an I²C interface.  Downside is that I’ll need an MCU to poll it, upside is that by placing the ADC and instrumentation amp in one package, it should cut down noise, further reduced if I mount the chip on a board bolted to the current shunt concerned.  The ADC measures bus voltage and temperature as well.  Getting this to work shouldn’t be hard.  (Yes, famous last words I know.)

A few days ago, I also placed an order for some more RAM for the two compute nodes.  I had thought 8GB would be enough, and in a way it is, except I’ve found some software really doesn’t work properly unless it has 2GB RAM available (Gitea being one, although it is otherwise a fantastic Git repository manager).  By bumping both these nodes to 32GB each (4×8GB) I can be less frugal about memory allocations.

I can in theory go to 16GB modules in these boxes, but those were hideously expensive last time I looked, and had to be imported.  My debit card maxes out at $AU999.99, and there’s GST payable on anything higher anyway, so there goes that idea.  64GB would be nice, but 32GB should be enough.

The fun bit though, Kingston no longer make DDR3 ECC SO-DIMMs.  The mob I bought the last lot though informed me that the product is no longer available, after I had sent them the B-Pay payment.  Ahh well, I’ve tossed the question back asking what do they have available that is compatible.

Searching for ECC SODIMMs is fun, because the search engines will see ECC and find ECC DIMMs (i.e. full-size).  When looking at one of these ECC SODIMM unicorns, they’ll even suggest the full-size version as similar.  I’d love to see the salespeople try to fit the suggested full-size DIMM into the SODIMM socket and make it work!

The other thing that happens is the search engine sees ECC and see that that’s a sub-string of non-ECC.  Errm, yeah, if I meant non-ECC, I’d have said so, and I wouldn’t have put ECC there.

Crucial and Micron both make it though, here’s hoping mixing and matching RAM from different suppliers in the same bank won’t cause grief, otherwise the other option is I pull the Kingston sticks out and completely replace them.

The other thing I’m looking at is an alternative to OpenNebula.  Something that isn’t a pain in the arse to deploy (like OpenStack is, been there, done that), that is decentralised, and will handle KVM with a Ceph back-end.

A nice bonus would be being able to handle cross-architecture QEMU VMs, in particular for ARM and MIPS targets.  This is something that libvirt-based solutions do not do well.

I’m starting to think about ways I can DIY that solution.  Blockchain was briefly looked at, and ruled out on the basis that while it’d be good for an audit log, there’s no easy way to index it: reading current values would mean a full-scan of the blockchain, so not a solution on its own.

CephFS is stable now, but I’m not sure how file locking works on it.  Then there’s object storage itself, librados.  I’m not sure if there’s a database engine that can interface to that, or maybe to Amazon S3 storage (radosgw can emulate that), that’ll be the next step.  Lots to think about.

Solar Cluster: BCDC1225 switching between solar and mains not reliable

So yeah, it seems history repeats itself.  The Redarc BCDC1225 is not reliable in switching between solar inputs and 12V input derived from the mains.

At least this morning’s wake-up call was a little later in the morning:

From: ipmi@hydrogen.ipmi.lan
To: stuartl@longlandclan.id.au
Subject: IPMI hydrogen.ipmi.lan
Message-Id: <20171023194305.72ECB200C625@atomos.longlandclan.id.au>
Date: Tue, 24 Oct 2017 05:43:05 +1000 (EST)

Incoming alert
IP : xxx.xxx.xxx.xxx
Hostname: hydrogen.ipmi.lan
SEL_TIME:"1970/01/27 02:03:00" 
SENSOR_NUMBER:"30"
SENSOR_TYPE:"Voltage          "
SENSOR_ID:"12V             " 
EVENT_DESCRIPTION:"Lower Critical going low                                         "
EVENT_DIRECTION:"Assertion  "
EVENT SEVERITY:"non-critical"

We’re now rigging up the Xantrex charger that I was using in early testing and will probably use that for mains. I have a box wired up with a mains SSR for switching power to it.  I think that’ll be the long-term plan and the Redarc charger will be retired from service, perhaps we might use it in some non-critical portable station.

Solar Cluster: Solar Testing

So I’ve now had the solar panels up for a month now… and so far, we’ve had a run of very overcast or wet days.

Figures… and we thought this was the “sunshine state”?

I still haven’t done the automatic switching, so right now the mains power supply powers the relay that switches solar to mains.  Thus the only time my cluster runs from solar is when either I switch off the mains power supply manually, or if there’s a power interruption.

The latter has not yet happened… mains electricity supply here is pretty good in this part of Brisbane, the only time I recall losing it for an extended period of time was back in 2008, and that was pretty exceptional circumstances that caused it.

That said, the political football of energy costs is being kicked around, and you can bet they’ll screw something up, even if for now we are better off this side of the Tweed river.

A few weeks back, with predictions of a sunny day, I tried switching off the mains PSU in the early morning and letting the system run off the solar.  I don’t have any battery voltage logging or current logging as yet, but the system went fine during the day.  That evening, I turned the mains back on… but the charger, a Redarc BCDC1225, seemingly didn’t get that memo.  It merrily let both batteries drain out completely.

The IPMI BMCs complained bitterly about the sinking 12V rail at about 2AM when I was sound asleep.  Luckily, I was due to get up at 4AM that day.  When I tried checking a few things on the Internet, I first noticed I didn’t have a link to the Internet.  Look up at the switch in my room and saw the link LED for the cluster was out.

At that point, some choice words were quietly muttered, and I wandered downstairs with multimeter in hand to investigate.  The batteries had been drained to 4.5V!!!

I immediately performed some load-shedding (ripped out all the nodes’ power leads) and power-cycled the mains PSU.  That woke the charger up from its slumber, and after about 30 seconds, there was enough power to bring the two Ethernet switches in the rack online.  I let the voltage rise a little more, then gradually started re-connecting power to the nodes, each one coming up as it was plugged in.

The virtual machine instances I had running outside OpenNebula came up just fine without any interaction from me, but  it seems OpenNebula didn’t see it fit to re-start the VMs it was responsible for.  Not sure if that is a misconfiguration, or if I need to look at an alternate solution.

Truth be told, I’m not a fan of libvirt either… overly complicated for starting QEMU VMs.  I might DIY a solution here as there’s lots of things that QEMU can do which libvirt ignores or makes more difficult than it should be.

Anyway… since that fateful night, I have on two occasions run the cluster from solar without incident.  On the off-chance though, I have an alternate charger which I might install at some point.  The downside is it doesn’t boost the 12V input like the other one, so I’d be back to using that Xantrex charger to charge from mains power.

Already, I’m thinking about the criteria for selecting a power source.  It would appear there are a few approaches I can take, I can either purely look at the voltages seen at the solar input and on the battery, or I can look at current flow.

Voltage wise, I tried measuring the solar panel output whilst running the cluster today.  In broad daylight, I get 19V off the panels, and at dusk it’s about 16V.

Judging from that, having the solar “turn on” at 18V and “turn off” at 15V seems logical.  Using the comparator approach, I’d need to set a reference of 16.5V and tweak the hysteresis to give me a ±3V swing.

However, this ignores how much energy is actually being produced from solar in relation to how much is being consumed.  It is possible for a day to start off sunny, then for the weather to cloud over.  Solar voltage in that case might be sitting at the 16V mentioned.

If the current is too low though, the cluster will drain more power out than is going in, and this will result in the exact conditions I had a few weeks ago: a flat battery bank.  Thus I’m thinking of incorporating current shunts both on the “input” to the battery bank, and to the “output”.  If output is greater than input, we need mains power.

There’s plenty of literature about interfacing to current shunts.  I’ll have to do some research, but immediately I’m thinking an op-amp running from the battery configured as a non-inverting DC gain block with the inputs going to either side of the current shunt.

Combining the approaches is attractive.  So turn on when solar exceeds 18V, turn off when battery output current exceeds battery input current.  A dual op-amp, a dual comparator, two current shunts, a R-S flip-flop and a P-MOSFET for switching the relay, and no hysteresis calculations needed.

Solar Cluster: Power distribution harnesses

So, having got the rack mostly together, it is time to figure out how to connect everything.

I was originally going to have just one battery and upgrade later… but when it was discovered that the battery chosen was rather sick, the decision was made that I’d purchase two new batteries. So rather than deferring the management of multiple batteries, I’d have to deal with it up-front.

Rule #1 with paralleling batteries: don’t do it unless you have to. In a perfect world, you can do it just fine, but reality doesn’t work that way. There’s always going to be an imbalance that upsets things. My saving grace is that my installation is fixed, not mobile.

I did look at alternatives, including diodes (too much forward voltage drop), MOSFET switching (complexity), relay switching (complexity again, plus contact wear), and DIY uniselectors. Since I’m on a tight deadline, I decided, stuff it, I’ll parallel them.

That brings me to rule #2 about paralleling batteries: keep everything as close to matched as possible. Both batteries were bought in the same order, and hopefully are from the same batch. Thus, characteristics should be very close. The key thing here, I want to keep cable lengths between the batteries, load and charger, all equal so that the resistances all balance out. That, and using short runs of thick cables to minimise resistance.

I came up with the following connection scheme:

You’ll have to forgive the poor image quality here. On reflection, photographing a whiteboard has always been challenging.

Both batteries are set up in an identical fashion: 40A fuse on the positive side, cable from the negative side, going to an Andersen SB50/10. (Or I might put the fuse on the negative side … haven’t decided fully yet, it’ll depend on how much of each colour wire I have.) The batteries themselves are Giant Power 105Ah 12V AGM batteries. These are about as heavy as I can safely manage, weighing about 30kg each.

The central harness is what I built this afternoon, as I don’t yet have the fuse holders for the two battery harnesses.

The idea being that the resistance between the charger and each battery should be about the same. Likewise, the resistance between the load and each battery should be about the same

The load uses a distribution box and a bus bar. You’ve seen it before, but here’s how it’s wired up… pretty standard:

You might be able to make out the host names there too (periodic table naming scheme, why, because they’re Intel Atoms) … the 5 nodes are on the left and the two switches to the right of the distribution box. I have 3 spare positions.

In heavy black is the 0V bus bar.

This is what I’ve been spending much of my pondering, doing. Part of this harness is already done as it was installed that way in the car, the bit that’s missing is the circuit to the left of the relay that actually drives it. Redarc intended that the ignition key switch would drive the relay, I’ll be exploiting this feature.

Some time this week, I hope to make up the wiring harnesses for the two batteries, and get some charge into them as they’ve sat around for the past two months in their boxes steadily discharging, so I’d be better to get a charger onto them sooner rather than later.

The switch-over circuit can wait for now: just hard-wire it to the mains DC feed for now since there’s no solar yet. The principle of operation is that the comparator (an LM311) compares the solar voltage to a reference (derived from a 5V regulator) and kicks in when the voltage is high enough. (How high? No idea, maybe ~18V?). When that happens, it outputs a logic high signal that turns off the MOSFET. When too low, it pulls the MOSFET gate low, turning it on.

The MOSFET (a P-channel) provides the “ignition key switch” signal to the BCDC1225, fooling it into thinking it is connected to vehicle power, and the charger will boost as needed. The key being that the BCDC1225 makes the decision as to whether the battery needs charging, and how much charge.

By bolting together off-the-shelf parts, we should have something that I can source replacements for should the smoke escape, and there’s no high voltages to deal with.

Solar Cluster: Selecting batteries and sources

I’ve been doing quite a bit of thinking on this. Solid-state works but suffers from voltage drop. Relays work but either require the coil to be energised constantly (~1W load) unless you look for latching relays, for which 30A units are hard to come by.

These look promising though. A latching relay is nice since I only need to pulse the coil, not hold it on indefinitely.

That got me thinking what else can I use to switch power? The ideal for me is something that has practically no voltage drop and remembers its state without power. A latching relay fits this requirement. So does a uniselector or stepping switch. Those were commonplace in telephone exchanges years ago, but have since gone the way of the dodo as semiconductor technology replaced it.

The nice thing about a uniselector though for my application is you can switch between N points, instead of just two like a regular relay. So if I buy a third battery, I can wire it up to the uniselector, and have it switch the compute load between the batteries. Likewise, I can connect a charger to the battery most in need of a charge. MCU measures battery voltages, picks the battery with highest voltage to run the load, and the lowest voltage to get a charge. Easy.

That got me thinking… can I make a uniselector? Well of course I can! I basically need to make a rotary switch that can revolve around indefinitely. The shaft of the switch would then be turned by a DC motor.

The stator of a N-way switch would have N+1 pads, one which is the “common”, and the other N would be to each selection. The common pad would be a 180° arc, the others would be 180°/N.

The rotor would feature two brushes 180° apart with a wire connecting them. It is free to move vertically, but must rotate with the shaft, a spring between a nut on the end of the shaft and the rotor applies tension to keep the rotor pressed firmly against the stator.

The interface between rotor and stator features some triangular grooves, so that when the rotor is turned, it pushes it away from the stator, breaking contact. When the rotor passes a critical point, the spring pressing the rotor against these grooves makes the rotor “want” to continue turning until it hits the bottom of the groove, at which point it “sinks” down towards the stator and eventually makes contact again.

Visually, it looks like this:

A small microswitch mounted on the stator could tell us when touch-down takes place, if we use the normally-closed contact to power the motor it will automatically stop the motor when the next position is reached. We then just need to override that open switch by applying a pulse to get things moving.

Power is only needed when we want to change the selector switch. This should be simple enough to fabricate here out of plywood. I don’t have a 3D printer, but you could do it with one of those very easily.

The nature of this switch makes it a break-before-make switch, which has a downside when using it to select which battery to use: there’s a momentary break in power.

I can use diodes to carry the current temporarily. If I run a high-current diode from each battery to the output via a current sensor. If the current sensor measures current flowing through the diodes whilst a battery is selected, then we know that battery is lower than the others by at least the diode voltage drop, and we should consider switching.

Solar Cluster: A no-microcontroller automatic battery selector

Another approach to the selecting a source is to avoid microcontrollers altogether and just rely on non-programmable logic. This is inspired a bit by the Saturday Clock, or rather, my thinking of how it could be done without an MCU.

In selecting a source, we really only care about one thing: is the battery voltage high enough? If no, we need to hunt for one that is.

This question can be answered by a simple analogue comparator such as the LM311, a shift register, and a few other logic gates.

Here, we have such a circuit. Up the top, is our shift register, set up as a ring counter. The buffers there are stand-ins for diodes, if any of them is a 1, the output is a 1 and the NOT gate on the input outputs a 0.

The outputs of the shift register are used to select a battery, which has its own comparator and select logic. The comparator is represented here by the D flip-flop at the extreme left: in essence I’m using this as a switch, Logisim doesn’t provide one, only a momentary button. We need a signal that is high when the battery is above acceptable voltage. We also need its inverse.

The select line from the shift register controls the gate on two tri-state buffers, allowing us to inhibit the comparator’s output. The buffered “good” signal is used to SET the “enable” D-flip-flop that drives the switch turning the battery on. This same (buffered “good”) signal also passes thorough a diode-OR arrangement that indicates whether a source is “available”.

To emulate make-before-break, inverted “select” signal and the “source available” signal pass through an AND gate and into the RESET of the “enable” flip flop, so it gets turned off when another source is turned on.

Finally, the buffered “bad” signal from all modules is fed back on one shared line, inhibiting the clock until a battery drops below the minimum level.

A glitch here is if multiple batteries are initially turned on with none above the minimum voltage, this will cause multiple sources to be selected. This is not too hard to manage in software, and the solution might in fact be to implement this on an ATTiny24A as mentioned in the previous post; this logic circuit can be implemented quite easily in C, with comparators in hardware or using the ADC as a software comparator as I’m doing in the charge controller.

Solar Cluster: Charge controller testing, battery bank dimensioning and other thoughts

So, further progress on the charge controller.

Thinking about the problem … I realised that I really do not want to be testing for VBNVH when entering the CHARGE_CHECK state, as it’ll prematurely terminate the charge when the battery is being bulk-charged.

Better to wait until the charger decides to stop … which we’ll see due to the battery voltage ceasing to increase. We do want to check we’re not critically high however, so we can swap out VH for VCH.

Next, when we find that VBNVBL, meaning the battery is not charging, there we can check for VBNVH and stop charging at that point.

That change, worked pretty well, but it was still flapping between sources. A little state management helped this. If we declare another state variable, charger_warning, we can flip this to 1 upon first detecting that VBNVBL, then wait a little longer. If after a time-out this is still the case, then we can take action. Thus we define a new timer tCWARN, which delays acting on the not-charging case.

A bit of threshold tweaking, and things are behaving themselves. I’m using an el-cheapo 3-way camping fridge as the stand-in for the cluster. This is an Aldi special bought some years ago that draws about 5-6A… and when running on 12V power, features no thermostat.

We’re finding that its cooling capacity is no match for Brisbane’s early autumn weather anyway, so it’s pretty much would be a constant load even if the thermostat worked on 12V.

The battery we’re using is an old 105Ah AGM battery… which is one of two batteries from the caravan we have here. They were the original batteries, and this battery’s mate had failed when both were replaced. We’ve noted this battery getting warm whilst charging, so we think it might now be on the way out too.

What to replace it with? LiFePO₄ is AU$1000 for 100Ah, so too expensive. AGM is still the better bet. I can get 300Ah AGM batteries, but they weigh nearly as much as I do. I can just manage the 105Ah, so we’ll stick with those. I will need more than one long-term.

That brings the thorny issue of connecting them. I am not keen to hook batteries in parallel for various reasons. At least not permanently.

Now, the charger I’m using for mains is a 3-channel charger. I can make additional charge controllers (with the caveat that I need heatsinks for the MOSFETs…sigh!) and I can look for 3-channel solar chargers, or just get multiple chargers for the extra batteries. This can be done.

The load is the elephant in the room. I’d ideally like to manage it as a single load, although conceivably, I could put the switch and one storage node on one battery, a second storage node and a compute node on a second, and the final storage and compute nodes on the third. If the switch goes however, my cluster is toast.

I can put batteries in parallel, but this really does need to be done with care, using carefully matched batteries. So the better solution is to have a controller that chooses the battery with the highest voltage.

There might be an analogue means of implementing this, but a microcontroller is a single-chip solution. The ATTiny24As have up to 8 ADC/GPIO pins and three non-ADC GPIO pins. It’s what I’m already using for the charge controller, so is an easy choice.

The cluster will not tolerate a break-before-make switch-over. I thought about using a capacitor bank to keep the cluster alive during a brief (~1sec max) switch-over. Back-of-the-envelope calculations suggested I would need a 10F capacitor bank. I can get a 16V 470mF capacitor for AU$70 each… and would need 20 of these. Ouch!

A small battery is another option, maybe a 7Ah, but that has its own maintenance issues, and represents a single point of failure.

I can get Schottky diodes capable of 40A, they still present a 0.6V voltage drop. At 30A, that represents 16W! For comparison, a relay with a 225ohm coil resistance will draw ~60mA when the battery is at the maximum of 15V, representing a load of 1W.

Or I can use more MOSFETs like the ones I’m already using, which draw even less power; poor man’s solid-state relays. Latching relays also exist, but they can be rather expensive, more so than a solid-state relay.

I can probably get away with temporary parallel connections, so a make-before-break would let me switch sources. Or, I could place my switch across a Schottky, meaning I put up with that 16W load for a brief moment while I switch sources.

So more to think about, but we are getting close. I can defer this decision until I get a second battery, but I am getting close to the point where the cluster will be running full-time.