Planning for High Availability

Key Strategies in High-Availability Application Design

HA infrastructure is not the only way to increase application availability. Application servers typically have HA features (J2EE servers, in particular), but a technically demanding application requires supplemental design strategies. HA application design is concerned with maintaining application operation in the midst of application failures, infrastructure failures and real-time maintenance. The strategies below can be used individually or in combination:

Redundancy: Each element of an HA application must have a backup that can take over if the primary fails. Load balancing features share the load during normal operation and by shifting the load when a node fails. Alternatively, one or more hot standbys might take over if a primary fails, and the design must account for transactions that were in-flight when the failure occurred.

Recoverable state design: An application's handling of in-flight transactions is largely determined by its approach to state management. "Stateless execution" is often put forth as an HA design principle, but while it is true that an individual element is "more HA" if stateless, the application as a whole typically cannot be viewed as stateless -- users make a series of requests and later requests build on earlier ones. Thus, it is necessary to store state between exchanges, replicate the state (so that it is not subject to a single point of failure) and then re-establish state after recovery.

Failure detection: To initiate recovery of state, and for any failure scenario not handled transparently to the application, there must be "detect and retry" logic within the application. The server side of the application may be able to do this transparently (preferred), but the client side may have to do it. The application may have to "fail gracefully" by saving transaction information, notifying a user or administrator and performing cleanup upon application restart.

Watchers and heartbeats: An HA application must be watched in real time to ensure it is still running. Two key design strategies are process watchers, which monitor execution of application processes on the watcher's machine, and heartbeats, where a network-based element responds to periodic "Are you still there?" messages.