What is Splunk's Universal Forwarder?
Author: Becca Lambert
Release Date: 21/04/2023
The Splunk Universal Forwarder is a streamlined iteration of the Splunk Enterprise software, tailored to facilitate the forwarding of data. Splunk itself serves as a platform, specialising in the exploration, monitoring, and examination of machine-generated data. This encompasses diverse data forms, including log files, events, and various outputs originating from software, applications, and system processes.
Splunk Enterprise is undoubtedly an invaluable tool when it comes to understanding the masses of data generated every day by every device and every endpoint within a network, but in order for Splunk to work its magic we need to be able to collect and consolidate data from the devices within the network. This is where forwarders come in; they help stream the collected data into the Splunk environment where this can be indexed accordingly.
Types of forwarders used in Splunk:
The Splunk Universal Forwarder: This is considered as the most basic, but also the best tool for sending the data into the indexers. It contains only the components required for forwarding the data. No excess features which may not be needed or suitable for the task, which can put unneeded strain on resources and increase. It has a sole purpose and performs it well and is the primary method of forwarding data in Splunk Enterprise and Splunk Cloud.
A Heavy Forwarder: This is a full Splunk Enterprise instance, capable of indexing, searching, changing and of course forwarding data. Due to the fact that a heavy forwarder is a fully fledged instance of Splunk it does require its own forwarder licence, however this means it is able to parse data before it gets to the indexes, it’s also able to apply filters and hashes to the data and all the other configurations required for removing unwanted data, filtering or routing. It’s also able to index locally or send to another Splunk instance if required and can sometimes be used as an intermediary between the universal forwarders and the indexers. If the user prefers, there is the option of disabling some of these features in order to reduce resource use on the system.
The Light Forwarder: This is now a practically obsolete forwarder, as there hasn’t been any need for is as from the launch of Splunk 6.0, when the universal forwarder was able to take over the majority of its purposes, but this was very similar to a heavy forwarder in that is was a fully working Splunk Enterprise instance. However, all of the features such as searching, indexing and almost everything else was disabled, making it to all intents and purpose a universal forwarder, so this is no longer commonly used.
What is the Splunk Universal Forwarder?
The Splunk Universal Forwarder is a reliable and secure means to stream collected data from your machine or any remote network endpoint to your data receiver. This receiver is typically your Splunk index, where all the Splunk data is stored and consolidated. But the universal forwarder does much more than just forward on the data from one point to another, it ensures that the data sent into Splunk is correctly formatted including adding metadata tagging, meaning metadata such that the data source, source type and host can be correctly identified. There is also the option for you to manipulate your data before it reaches the indexes or manually add the data in.
Benefits of using the Splunk Universal Forwarder
It can’t be denied that the Splunk Universal Forwarder has many benefits associated with it. It’s highly scalable so is the perfect fit for any size environment; they can scale to tens of thousands of remote systems collecting multiple terabytes of data. There is no user interface associated with the Universal Forwarder helping to minimise its resource usage meaning as it also uses significantly less hardware resources than other Splunk products. Due to the reduced resource usage, you can install thousands of them without impacting the network performance and most importantly without increasing cost.
Here is just some of the capabilities of the universal forwarder:
- Metadata tagging, including source, source type and host
- Configurable buffering
- Data compression
- SSL security
- Use of any available network ports
- The ability to be managed via a deployment server
Drawbacks of using the universal forwarder
Although the universal forwarder is widely considered to be the best way to forward your data into Splunk it is not without its drawbacks. The first of these is that the universal forwarder is not compatible with python. Secondly the universal forwarder is only able to forward unparsed data, meaning that in order to send event based data, the universal forwarder would not be compatible and the heavy forwarder would be the better option.
In Conclusion:
The Splunk universal forwarder is a secure and reliable method of forwarding your data from your endpoints into Splunk. It has all the capabilities needed to forward data, without any potentially unnecessary features putting a strain on the environment or upping cost. It's the perfect solution for any day to day forwarding. However when needing to forward data that's not unparsed or needing to forward data compatible with python, then in this case a heavy forwarder is the better option.
More Resources like this one:
Splunk Security Operations Suite (SOS) Demo:
Monitor, Detect & Mitigate
w/ Splunk ES, UBA/UEBA & SOAR
Splunk Cloud provides functionality and features of Splunk Enterprise but without the costs and time requirements.
