New High-Availability Capability for TimelyBill

TimelyBill is excited to announce the addition of a new high-availability capability!

TimelyBill’s primary data center infrastructure is already N+1 to N+5 on all key hardware, power supplies, and bandwidth.  This new functionality is related to database availability.  Each TimelyBill customer has both a production and stage database instance.  These instances are unique to a customer and not shared.

Additionally, we operate the database files, log files, operating systems, and backups on different disks.  This allows us to maintain a database or recover it to an exact point in time locally, even with a loss of an entire disk LUN.  From a database backup perspective, we continue to operate in Full Recovery mode.  We currently maintain 90 days of full backups off-site and five days of transaction logs on-site. Some of our customers have taken advantage of the transaction log backups, allowing us to restore customer data to a specific time.  

It’s important to know that TimelyBill’s virtual machines are already running in a high-availability cluster, and we maintain at least one full-performance standby server in that cluster. Any “crash” of a physical server in our environment will automatically cause its virtual machines to be moved and brought online with another server.

So what’s new?

TimelyBill is adding a third database instance per customer!

This instance will operate in a separate virtual machine, always on a separate physical machine in the cluster, using disk storage from a separate physical disk array, and be a “hot” synchronous copy of your primary database instance.  With this new capability, there could be a catastrophic loss of an entire virtual machine or even the disk array (not the entire disk array!), and we would not need to engage in any backup/restore procedures. Customers wouldn’t even be aware of the loss because the secondary instance is always online and up to date.  

Outside of being another great enhancement to our High Availability and Disaster Recovery plans, this will also provide a noticeable improvement during monthly Windows/security patches.  Microsoft patches have become very significant, almost always require a restart, and sometimes take quite a few minutes to complete during the restart.  With this new capability, database requests will continue to be served, even during Windows updates and virtual machine restarts, without human intervention.

This new addition was driven by our desire to constantly improve the resilience of the TimelyBill environment.  Please note that we have NOT had any incident or event with our current operation.

Don't hesitate to ask if you have any questions on this or any other questions on the TimelyBill infrastructure.