Power failure recovery

SUMMARY

One of the most complex areas of HCA for users to understand is the capabilities and limits of power failure recovery.  We receive a lot of questions on what is saved, and how to set things up for power failure recovery to be automatic.  This technical note describes in some detail how power failure recovery works.

Introduction

There is a general principal that guides all the following discussion.  While simple to state, the full details of how to accomplish it require much more explanation.   But first the general principal:

The goal of power failure recovery is to restore the state of each device, program and group to what they would have been had the power not failed.

To start to explain, it is best to look at an example.  Suppose that the power failed at 7pm and was restored at 7:15pm.  This means for devices (and groups):

1. Devices that were on at 7pm and not scheduled to go off between 7pm and 7:15pm should be on when power is restored.
2. Devices that were on at 7pm and should have been turned off sometime between 7pm and 7:15pm should be off when power is restored.
3. Devices that were not on at 7pm and should have been turned on sometime between 7pm and 7:15pm should be on when power is restored.

And for programs:

1. Programs that were running when the power failed should be restarted when the power is restored if they would not have completed by 7:15pm.  For example a program could have started a 30 minute delay at 6:55pm.  If the power had not gone off that delay would have been completed at 7:25pm.  The program is restarted at the delay element but the 15 minutes lost due to the power outage is taken into account and the delay still completes at 7:25pm.
2. Like devices, programs that were scheduled to start between 7pm and 7:15pm should be started when power returns and whatever actions they would have taken between their start time and 7:15pm should happen.  Like the previous case this includes taking into account delay elements that may have completed by 7:15pm and those that would have continued past 7:15pm.

These are the general principals for devices, programs, and groups.  There are specific properties for devices, programs, and groups to allow you to fine tune the power failure restoration for each.  For example, for each device it can be selected that the device be ON or OFF after a power failure, or follow the schedule catch-up described above.  Similar options exist for programs.

So how does all this work?

To understand power failure recovery there are three important questions:

1. When the power returns how does HCA know the state each device was in before the power failed?
2. How does HCA know that the power has failed?
3. How does HCA know it is recovering from a power failure?

Answering question one depends upon whether the computer is connected to a uninterruptible power supply (UPS).  If it is, then the computer remains on during the power outage and HCA keeps running.  HCA maintains the state of each device in memory as it always does.  If computer is not connected to a UPS, when the power fails the computer goes down.  In this case, HCA writes a file once a minute with all the state of the HCA design.

Here is what is in that file:

Now that you know about this state file, there is a fourth question: When does HCA need to write this state file?

The answers to these four questions depend on whether a UPS is being used, and also which X10 interface is being used.

The first point to determine is if the interface reports a power outage or just goes dead when the power goes out.  Remember that it doesn't make any difference if it does or not unless there is a UPS involved.

Here is why this table has these answers:

The LynX-10 and LynX-PORT interfaces are powered separately from the X10 powerline interface (TW523 / PSC05).  The interface power transformer can be plugged into the UPS and the powerline interface into a wall outlet.  So it can stay alive during a power outage.

The CM11 is all in one so it must be plug into a wall outlet.  When the power is out it goes dead.

The SmartHome HouseLinc uses a separate power transformer and an  X10 interface (TW523 / PSC05) but regardless of how powered just doesn't report power failures.  That's just the way it's designed.

The LynX-10PLC must be plugged into a wall outlet and not the UPS.  But because of it's design it manages to get a power out message to the computer even though it has lost power. It's fast enough and has a big enough power storage device (capacitor) to do this.

The SmartHome USB PowerLinc is powered from the computer USB port so it remains alive during a power outage if the computer does also.  The serial PowerLinc is like the CM11 and goes dead when the power goes out.

And finally the PowerLinc Controllers are powered from the house power.  They go dead when the power goes out.  But if your computer is connected to a UPS then Windows sends HCA a USB disconnect message which it uses as an indication of a power outage.

So back to the question, When does HCA need to write the state file?

These are the four possibilities::

1. The computer is not connected to a UPS and uses an interface that doesn't report power outages.
2. The computer is not connected to a UPS and uses an interface that reports power outages.
3. The computer is connected to a UPS and uses an interface that doesn't report power outages.
4. The computer is connected to a UPS and uses an interface that does report power outages.

No UPS and any interface

Without a UPS when the power fails, the computer goes down and HCA terminates.   There is no subtlety here - everything comes to a halt quickly.  In this case, here are the answers to the four questions:

When the power returns how does HCA know the state each device was in before the power failed?
It reads the state file and that supplies all the needed information.

How does HCA know that the power has failed?
It doesn't need to know.  All HCA knows is that the power could fail at any moment and to always be ready.

When does HCA need to write this state file?
Every minute.  The power could fail at any moment so the state file must always be written.

How does HCA know it is recovering from a power failure?
When HCA starts up, the first action it takes is to look for the state file.   This file always has the same name and is always in the same folder.  If the file is found and contains information that tells HCA that it is not from a normal shutdown, then HCA starts power failure recovery. 

UPS and an interface that doesn't report power outages

While a UPS may be powering the computer, the interface to the power line can't also be plugged into the UPS (powerline signals and UPSs don't mix).  When the power fails, HCA continues to run but the interface is dead.  Commands sent to it will not be transmitted and requests for status will not be answered.  Again, the four important questions:

When the power returns how does HCA know the state each device was in before the power failed?
The state of all devices is held in memory as normal.  During the time the power was out programs continued to start and run, the current schedule is watched for times to take action.  In general everything just happens as normal.

How does HCA know that the power has failed?
Periodically the interface is polled to see if it is alive.  If it does not respond HCA assumes that the power has failed.  There are sufficient time safeguards built-in to allow plenty of time for the interface to respond even if it is busy.

When does HCA need to write this state file?
Normally the state file does not need to be written. But UPSs do run out of battery power after an extended power outage.  When HCA determines the power is off it starts writing the state file every ten seconds.

How does HCA know it is recovering from a power failure?
When the power returns the interface starts responding.   This "sign of life" tells HCA that the power is on.

When the power returns, for each device in the home design HCA sends an ON, OFF, or DIM command to set that device to the desired state.

If the UPS runs out of battery power the computer shuts down.  This is exactly like the non-UPS case.  The state file is ready on the disk for power failure restoration to find it when the computer restarts.  This happens in the same manner as the non-UPS case described above.

UPS and an interface that reports power outages

Some of these interfaces use the TW523 / PSC05 to connect  to the power line. In this case these interfaces are powered from an AC transformer (except the LynX-10 PC Addin card which is powered by the computer).  If the computer is using a UPS, then the interface should be powered by the UPS also.  That is, if an AC transformer is used it should also be plugged into the UPS.

Having the interface plugged into the UPS means that it will remain active when the power is off.  The TW523 / PSC05  however must not be plugged into the UPS.  When the power goes off, the interface tells HCA that the TW523 / PSC05 is dead.

If the interface doesn't use the TW523 / PSC05, (the Marrick LynX-10PLC, CM11, PowerLincs) it should be plugged directly into a wall outlet.  That is, not into the UPS.

Again, the big four questions:

When the power returns how does HCA know the state each device was in before the power failed?
The state of all devices is held in memory as normal.  During the time the power was out programs continued to start and run; the current schedule was watched for times to take action.  In general everything just happens as normal.

How does HCA know that the power has failed?
The interface tells us it has.

When does HCA need to write this state file?
Normally the state file does not need to be written. But UPSs do run out of battery power after an extended power outage.  When HCA determines the power is off it writes the state file every ten seconds.

How does HCA know it is recovering from a power failure?
Depending upon the interface HCA may be told when power fails it and not when it is restored, or not when the power fails but when it is restored.   HCA polls the interface to see if it is alive, and if used that the TW523 is alive.

When the power returns, for each device in the home design HCA sends an ON, OFF, or DIM command to set that device to the desired state.

If the UPS runs out of battery power the computer shuts down.  This is exactly like the non-UPS case.  The state file is ready on the disk for power failure restoration to find it when the computer restarts.  This happens in the same manner as the non-UPS case described above.

Making all this happen

In order to make all this happen various things must be set up.  It is important to do all these things even if you are using a UPS since extended power outages may exhaust the UPS batteries.

First, you should have these options checked in HCA Properties (Startup tab):

The UPS option should be checked as appropriate.

The first of these options tells HCA to reload your design automatically when it is started.  The second option tells HCA to enter the Active state after power failure restoration happens. This allows the current schedule to continue.  The third option makes everything possible.

But that is only part of the story.  Your computer must be setup to do two important things:

Unless you do all these things, power failure recovery will not be automatic.

Fine points

There are several small points about power failure detection and recovery that are descried here.

With some UPSs there may be special programs that run on the computer and communicate with the UPS.  When the UPS is about out of battery power this program can be instructed to shut down any running Windows programs (it sends a File-Exit command to each running program).  If such a program is used, HCA needs to do one special action when it is terminated.  Normally when HCA is terminated the state file is marked to show a normal shutdown.  This is important so that when HCA is started how knows how it shutdown.  If a normal shutdown, HCA need not do power failure restoration. If this UPS program shuts down HCA using File-Exit, HCA may appear to be doing a normal shutdown.  However, if HCA is shutdown when the power is out (must be a UPS in use or this would be impossible), the state file is not marked as a normal shutdown.  Remember we will know that the power is still out because of the methods appropriate to each interface.  We don't actually check the interface at this point.  HCA does know that as it is shutting down and information internal to HCA shows the power is out.  This information was setup earlier when the power first went out.

You should take great care when you use the power failure recovery options for devices.   Remember that a power failure can happen at any time and HCA puts things back as they were had the power not failed.  For example, suppose you turned a heater on using a control panel someplace in your home.  HCA also receives that signal and updates its state to show the heater as ON.  If the power fails before you turn the heater off, when power is restored HCA will again turn the heater on.  You should take great care with controlling anything in your home that could be dangerous if turned on when unattended.

It is possible to designate one schedule to be the current schedule when HCA restarts after a power failure.  Normally the schedule that was the current schedule when the power failed becomes the current schedule upon power failure restoration.  If you use this option and designate a schedule, that schedule is made the current schedule.

Using the program advanced properties you can designate a program that is started when HCA restarts after a power failure.

Try it out

Its hard to simulate a power failure.  Many users wanting to watch power failure restoration try various things and get concerned that its not working.  Here are some guidelines to creating a power failure condition:

With a UPS and an interface that doesn't report power outages:  Kill the power to the interface.  Wait about one minute and HCA should report a power failure.  Re-power the interface and HCA should report the power has been restored.

With a UPS and an interface that reports power failures.  If you are using the LynX-10PLC, pull it out of the wall socket.  If using a TW523 / PSC05 interface, leave the interface powered on and pull the TW523 / PSC05 out of the wall.  Wait a minute and HCA should report a power failure.  Plug the interface, or TW523 back in the wall and HCA should report the power has been restored.

Without a UPS.  Its generally not a good idea to just pull the computer's plug out of the wall or hit the power button.  You can achieve the same result by using the Windows Task Manager (what pops up when you press the three key combination ctrl-alt-del) and terminating HCA.  After that shutdown Windows as normal and then restart the computer.  Once the computer starts and Windows loads, HCA should restart and detect that it is restarting from a power failure.

If you try any other method, you may be creating an impossible condition - one that just can't occur.  For example, suppose you are using a CM11 and you don't have the UPS option checked and you just pull it out of the wall.  For example, HCA will not detect that the CM11 is dead since it does not normally poll the CM11 when a UPS is not in use.  There are a lot of these impossible conditions so be careful in your testing.

Conclusion

HCA Power failure recovery is a very complex system.  It requires some actions outside of its control, like Windows automatic logon and having Windows   automatically start HCA.  Once you get everything set up just right, power failure recovery should be automatic and robust.