Warning: You are viewing an older version of this documentation. Most recent is here: 40.0.1

Troubleshooting¶

Even with rigorous testing and development procedures, a software without bugs (error, flaw, failure or fault) doesn’t exists. So, what to do when you encounter such defects?

Online Health Checks¶

Stamus Central Server provides inline health checks to ensure all services and connections are running properly on both SCS and Stamus Probes.

To run the Stamus Central Server health check, go to Global Appliance Settings from the Stamus Logo dropdown menu. In the side left menu, select Troubleshoot under the section Action.

To run a health check on a Stamus Network Probe, go under Appliances, select the desired Network Probe and click on Troubleshoot in the left side menu under More Info.

Troubleshooting Report¶

A troubleshooting report is an archive (tarball) that collects the most important configuration and logs files for our support team.

Note

If the error is only localized on SCS, generate a report for SCS. If the error is on the probe, generate a report on both SCS and the Probe.

Stamus Central Server¶

To generate and send a troubleshooting report to our support:

from the drop down menu in the upper left corner (Stamus icon) choose Global Appliance Settings
on the left hand side panel under the Action tab click on Troubleshoot
on the left hand side panel under the Advanced tab click on Generate a troubleshoot report
the generation of the report could take a few minutes
once the report is generated it will be downloaded in your browsers default download folder

Stamus Network Probe¶

To generate and send a troubleshooting report to our support:

click on the Appliances main tab
select the desired Stamus probe
on the left hand side panel under the More info tab click on Troubleshoot
on the left hand side panel under the Advanced tab click on Generate a troubleshoot report
the generation of the report could take a few minutes
once the report is generated it will be downloaded in your browsers default download folder

Common Questions¶

TCP reassembly gaps¶

Under the tab “Problem Indicators”, there is a graph called “TCP reassembly gaps”.

If this graph is filling up it may indicate missing packets in streams which may results, or be caused by, packet loss, bad checksums or an engine running out of memory.

If this is a persistent problem this may lead to missing detection in some streams.

To narrow down the cause of this problem such as a specific protocol or a specific host or set of hosts, deploy the following rule:

alert tcp any any -> any any (
   msg:"SURICATA STREAM reassembly sequence GAP -- missing packet(s)";
   stream-event:reassembly_seq_gap;
   classtype:protocol-command-decode;
   sid:2210048;
   rev:2;
   )

To deploy the rule, add it as a file as source and add it to the currently used ruleset. Then, push/update the ruleset on the desired probe.

Leave the rule for a little while such as 1h or 2h and deactivate it because this rule can be quite verbose.

Then, in the Hunting interface, alerts should appears for this specific rule. One way of searching for it is using a filter on Message.

External ELK cluster upgrade¶

If you are using an external ELK cluster, there are a couple of things to know before proceeding with an upgrade of that ELK cluster.

First, external ELK cluster aren’t part of the components we do support. However, in order to avoid troubles, make sure to check with support@stamus-networks.com if have experience with some specific ELK versions as we regularly update ELK for our own needs.

Second, when upgrading an ELK cluster, do not upgrade the ELK embedded on Stamus Central Server otherwise we don’t be able to support it anymore. Only the external cluster can be upgraded as you wish.

Finally, once you have upgraded your ELK cluster, make sure to Apply Changes in Stamus Central Server. This is required to ensure Stamus Central Server will use and generate the appropriate templates for your cluster version.

To apply changes, go under Probe Management > Appliances > Apply changes (in the left action menu). Only SCS needs to be “upgraded” so there is no need to Apply Changes on the Stamus Network Probes.