Background
Dispel supports syslog collection and forwarding to a customer's SIEM of choice. Through standard rsyslog forwarding, Dispel can provide this service in a number of formats, across customer specified ports and protocols. Log collection can be required at specific nodes, or enforced across all deployed systems, depending on your organization's data transfer limitations.
Standard SIEM tools within our Integration Portfolio include:
Splunk (Dispel is a Splunk+ Partner)
IBM QRadar
Rapid7
Azure Sentinel
Google Chronicle
ELK (Elastic Search, Logstash, Kibana)
To see a full list of our integrations, please visit: https://dispel.com/resources/integrations#siem
While standard syslog forwarding is an out of the box integration that does not require any additional integration fees, customers may choose to work with Dispel to build out custom formatting and variable capture to further enrich the data we are sending to you.
This article will cover the ways in which standard rsyslog forwarding is accomplished to provide customers with confidence in how this will work in their environments.
Note
This documentation is provided to help customers understand how Dispel performs a syslog forwarding integration with your SIEM tool of choice. These actions are undertaken by Dispel as a service, unless this is being provided by a partner, or a separate agreement has been reached with the customer.
Rsyslog & Dispel
Rsyslog is a built in Linux utility for forwarding logs to log aggregators over a single TCP or UDP port. Dispel uses rsyslog in 2 main scenarios.
Forwarding remote access traffic logs from the Dispel MTD network to your log aggregator of choice.
Forwarding of wicket system information to your log aggregator of choice.
Both of these scenarios can have significant variations in the path that logs are taking to arrive at their destination. Most commonly, logs are forwarded to a local SIEM (Splunk, QRadar, Rapid7, etc) through the wicket, but can also be forwarded to log aggregators over the internet, to cloud-native tools like Azure Sentinel, or to multiple locations.
Scenario 1 -- Forwarding Remote Access Traffic Logs
Configuring rsyslog
You will need to configure a rsyslog config file so that your Linux host knows where the SIEM (Splunk, QRadar, Rapid7, etc) appliance is. This configuration is, in standard practice, under the purview of Dispel's Orchestration Engine to ensure global conformity.
Open a terminal session on the Linux Host (Remote Access Network or Wicket) you wish to forward logs from.
We need to create the remote syslog file that resides in the /etc/rsyslog.d directory. To do so, type the command below.
[nano/vi/vim/text editor of choice] vi /etc/rsyslog.d/dispel_logging.conf
Hit the “insert” key on the keyboard, or, you can simply hit the “i” key. Paste the following configuration into the configuration file.
Note that the 10.8 source can vary depending on deployment, but is the default configuration in any Dispel deployment. The correct source can be found as the first 2 octets of any user IP under Regions -> Groups and Members tab of a given region.
The IP & Port will be the IP & Port of SIEM (QRadar, Splunk, Rapid7, Sentinel, etc) collector/processor appliance.
UDP is denotated by a single "@" and TCP by 2 "@@". The default configuration uses UDP.
:msg,contains,"SRC=10.8" @<ip>:<port>
:msg,contains,"SRC=10.8" /var/log/dispel_syslog.log
& stop
Enforcing TLS Encryption on Forwarded Logs
Note
The default configuration is unencrypted as the traffic is generally local traffic or passing through encrypted tunnels. For Dispel's standard deployment, we strongly recommend customers enforce TLS encryption, which can be enabled with the following additions to the configuration file.
Enforce TLS 1.2+ encryption on your forwarded traffic.
:msg,contains,"SRC=10.8"
action(type="omfwd"
protocol="<tcp/udp>"
target="<ip>"
port="<port>"
StreamDriver="gtls"
StreamDriverMode="1"
StreamDriverAuthMode="anon"
gnutlsPriorityString="MinProtocol=TLSv1.2"
template="json-template")
Save (Escape, :wq) the file then create the log file, and give syslog permission to access it.
touch /var/log/dispel_syslog.log
chown syslog:adm /var/log/dispel_syslog.log
Then restart the rsyslog service by typing the command:
service rsyslog restart
Configuring log rotation
Next we will configure the log file to rotate appropriately.
Create the logrotate.d configuration file by entering the following command:
[nano/vi/vim/text editor of choice] vi /etc/rsyslog.d/dispel_logging.conf
Hit the “insert” key on the keyboard. Or, you can simply hit the “i” key and paste in the following configuration. By default we keep 7 weeks of historic logs.
/var/log/dispel_syslog.log {
weekly
missingok
rotate 7
compress
notifempty
}Save (Escape, :wq) the file then restart the rsyslog service by typing the command:
service rsyslog restart
Scenario 2 -- Forwarding Wicket Syslog Information
Configuring the rsyslog file
You will need to edit your rsyslog.conf file so that your Wicket knows where the SIEM (Splunk, QRadar, Rapid7, etc) appliance is. This configuration is, in standard practice, under the purview of Dispel's Orchestration Engine to ensure global conformity.
Open a terminal session on the Wicket you wish to forward logs from.
We need to edit the remote syslog file that resides in the /etc directory. To do so, type the command below.
[nano/vi/vim/text editor of choice] vi /etc/rsyslog.conf
The file will open and will look something like this.
Scroll to the very bottom of the file or go straight there by using the command (vi specific)(shift) g.
Place the cursor at the end of the last line and hit the “insert” key on the keyboard. Or, you can simply hit the “i” key.
At the beginning of the newly inserted line, type:
UDP is denotated by a single "@" and TCP by 2 "@@". The default configuration and the configuration in the example uses UDP.
*.* @ [IP Address of Syslog Aggregator]:[Port of Syslog, default 514]
where <IP Address> is the IP of your SIEM (QRadar, Splunk, Rapid7, Sentinel, etc) collector/processor appliance. In the example below we use IP 10.10.10.50 with port 514.
Save (Escape, :wq) the file then restart the service by typing the command:
service rsyslog restart
Configuring auditd
Auditd is a Linux utility that is responsible for writing audit records to the disk, which can then be forwarded using rsyslog.
Next, please edit the auditd file to add the required rules.
As a best practice, always create a backup before performing configuration changes. To create a backup of the existing auditd rules configuration file, use the following command
cp /etc/audit/rules.d/audit.rules /etc/audit/rules.d/audit.rules.bkup
Note: If you receive permissions errors during this process, you may need to log in as root or elevate your permissions by using sudo if front of the commands
Edit the audit rules by using your standard text editor.
vi /etc/audit/rules.d/audit.rules
Use the insert key to enter Insert Mode and move cursor to the end of the file.
Copy ALL the content below and paste to where the cursor is. ALL the lines below should now be included in the file.
# Program called -a exit,always -F arch=b64 -S execve -a exit,always
-F arch=b32 -S execve
#It is possible to specify single commands to reduce the load with
-F <path_to_binary> (see auditd documentation)
# Process spawns child
-a exit,always -F arch=b64 -S fork -S vfork -S clone -a exit,always -F arch=b32 -S fork -S vfork -S clone
# File monitoring for edition and attributes modification
-w /boot -p wa -w /etc/pam.d -p wa -w /etc/shadow -p wa -w /etc/passwd -p wa -w /etc/rsyslog -p wa -w /etc/openldap -p wa -w /etc/sysconfig/syslog -p wa -w /etc/syslog.conf -p wa -w /etc/sysconfig/network-scripts -p wa -w /etc/default/ufw -p wa -w /etc/sudoers -p wa -w /etc/sudoers.d/ -p wa
Note: You may wish to tune the above list and correlation rules with files or directories that you wish to monitor. These parameters are highly configurable and Dispel is happy to work with your team to establish best practices for your organization.
Next, Update the audit syslog plugin configuration.
If you are using RHEL v7 (not applicable for Dispel Traffic/Wickets), open the /etc/audisp/plugins.d/syslog.conf file and verify that the parameters match the following values:
active = yes
direction = out
path = builtin_syslog
type = builtin
args = LOG_LOCAL6
format = string
If you are using RHEL v8 (not applicable for Dispel Traffic/Wickets), open the /etc/audit/plugins.d/syslog.conf file and verify that the parameters match the following values:
active = yes
direction = out
path = builtin_syslog
type = builtin
args = LOG_LOCAL6
format = string
Update the auditd.conf if installed auditd is v2.6 or later.
sudo vi /etc/audit/auditd.conf
Find log_format = RAW and modify the value to match the below
log_format = ENRICHED
For other distributions the process should be similiar.
Finally, restart the auditd service by typing the following command.
service auditd restart
or as applicable,
systemctl restart auditd
Logging bash history
Bashrc is a Linux configuration file that is executed when a user logs into a Linux session. We are using it here to log bash history for any user that logs into the Wicket.
Edit the global bashrc file, the location may vary between distributions/bash versions.
sudo vi /etc/bashrc or sudo vi /etc/bash.bashrc
Append the following the end of the file
export PROMPT_COMMAND='RETRN_VAL=$?;logger -p local6.debug "[BASH_HIST] [PID=$$]: USR=$(whoami) DIR=$(pwd) CMD=$(history 1 | sed "s/^[ ]*[0-9]\+[ ]*//" )"'
Extra applications
Applications running on your server may not use the built in logger and syslog falicities to generate their logs, and therefore will not be forwarded automatically.
For example, Apache and Nginx will often log directly to disk.
Similairly applications may use log4j (in the case of java apps) or similiar libaries to log direct to disk by default, in these cases you will need to implenment specific log fowarding profiles in syslod/ngsyslog/rsyslog or make changes in the logger’s configuration (log4j config for example) to forward the relevant logs to your SIEM collector.
Such application logs will likely also need fine tuning within your SIEM, we are happy to work with your team throughout the process.
A brief overview of the Reserve, Connect, Log, Destroy workflow within Dispel.
*Some documentation augmented with screenshots taken from online guides, including: https://wiki.secure-iss.com/en/Public/SOC/LinuxLogForwarding