There are many important reasons to restart a Jive instance. While rolling restart uses Jive clustering architecture to restart the individual servers with minimal interruption to customers, a full restart implies that the site will be unavailable as long as the servers are starting up, making access to the instance impossible while it last.
Read more about Jive Clustering Architecture.
Restarting Jive runs everything from scratch. This way, your Jive instance gets rid of the temporary files that might affect directly the performance of the instance, delivering thus a better performance after the restart.
Some of the general known issues that are proven to be corrected with rolling or full restarts are lack of sync between the instance and its services or within the nodes of a whole instance, the corruption of data that is held in hot memory (in the application server), or lack of response in specific nodes.
How Jive Restarts work from an Architectural Point of View
As you might know, there are two types of restarts in Jive: Rolling restarts and Full restarts.
- Rolling restarts
When you run a rolling restart to a Jive instance, the nodes are restarted one by one until they are fully restarted. That is the reason rolling restarts are "less invasive" and have no downtime.
The ETA for the rolling restart to complete depends on how many nodes the instance has and how busy the nodes are.
- Full restarts
Regarding the behavior of the nodes when there are full restarts when a restart happens, all nodes are restarted at the same time, which implies downtimes of 40 minutes to 60 minutes on average. While the server is down, the maintenance page is displayed since all nodes are stopped concurrently and then subsequently started.
There are some limitations to what can be done with a rolling restart and what requires a full restart. For example, while many plugins can be installed via a rolling restart, those that need to install their own database schema cannot.
- Regarding any restart
As with any restart, the node will be sluggish at first after the node starts. This happens while the node's fast memory caches are being populated from the disk. The node should be back to full responsiveness within minutes of normal use.
Relevant training documentation
- More information on performing restarts can be found in the training article JIVE-211 Full Restarts and Rolling Restarts.
Relationship with other concepts and topics
Explanation of the relationships
- Jive Restart - Performance
Restarts might be the action that solves issues related to slowness, latency, or poor performance of a Jive instance, If an instance is running slow, a step to try after assessing the situation will be to restart the instance (with permission from the customer).
- Jive Restart - Plugins
Several plugins which need to install their database schema require a restart to complete the installation.
- Jive Restart - Integration
Regardless of the fact that Integrations in Jive might include Jive for Office or SSO integrations, usually Jive on its documentation refers to Integration as Storage Integration or Authentication Integrations.
- Jive Restart - Outages
Usually, when a production instance is down it can often be solved by a restart. Another symptom that might be solved with restarts is the error message: "Production down" or "(Jive Version) - Production Down".
- Jive Restart - SSO/SAML
When you plan to make some changes in SAML authentication settings (usually on a Production Instance), when there are changes in the IDP Metadata or changes in log URL in SSO/SAML, restarts must be applied
- Jive Restart - Authentication
When it comes to Authentication, the operations that you perform at the basic level (And when we say "basic" we mean if you change the certificate, IDP Metadata, etc.) will need a restart, since you are changing the way users authenticate into the system.
- Jive Restart - Licenses
A customer’s instance needs to be rolling restarted when the license expires and after a license renewal.
- Jive Restart - Cache Clearing
There are situations where cache gets corrupted or generates errors on Jive instances. When you find yourself needing to issue a restart of the cache server, a full restart must be executed.
More information about this can be found in the JIVE-402 Backend Caches KB Article.
Most Common Issues
The most common Jive Restart issues that you can find while using Jive are issues related to Jive's performance, Cache issues, or Outages, to mention some of them. In general terms, cases that need rolling restarts are more common than cases that need full restarts.
Solved customer tickets
In the following list, you will find some tickets that were already solved by Support L2 Agents regarding this topic:
Important elements to keep in mind
- Restarts should be done only as a last resort, in cases in which you suspect that something is off-sync, and some in-memory structure was corrupted in the application server. This means that you must've tried already some troubleshooting steps like validating if the data remains in the DB, that all nodes and CDN are responding in the correct way, that is not client-dependent (or geographic-zone-dependent), and that is not a bug.
- Logs are strongly recommended to be collected and saved before any type of restart (for troubleshooting purposes). There have been some cases where log files get overwritten after the rolling restart.
- When you conclude that a Restart must be executed, you must consider the following statement: "When both types of restarts can fix the issue, the recommended solution should be a fix that least disrupts the end-user services". In most cases, rolling restarts can solve most of the issues, except for issues related to authentication and integration services which require full restarts.
If you suspect that there is a database stale lock or deadlock, a restart could solve the issue.
Possible sources of confusion
- Cloud instances, hosted instances, and on-premise
It is important to take into consideration that different types of instances in Jive require different procedures to restart. That being said, the restart procedure for Jive cloud instances, will be different than the restart procedure for hosted instances or on-premise ones. Make sure you understand the difference between cloud instances, hosted instances, and on-premise instances.
- Restart Order
One thing that might be confusing about rolling restarts is the order in which the nodes must be restarted. As some cases state one "version" of the order that you must stop/start an instance, some other tickets state a different version. The truth is that web app nodes and cache nodes are the nodes that must be restarted mandatory in a specific order:
- The web app node(s) must be stopped first and the cache node(s) must be stopped last.
- Once stopped, the cache node(s) must be started first, and the web app node(s) must be started last).
- The rest of the node(s) (for example, document converter) can be restarted in no specific order.