Tips to improve your SIEM
One of the most common questions I get are “We’ve got a SIEM setup, what steps can we take to improve it”. This differs greatly depending on how mature your SIEM is, which platform you are using; and your business. However here are a number of generic steps you can take to improve any SIEM you are involved in running or administrating.
These steps are not everything you can do with a SIEM, or in any specific order. I do hope however that at least one of these steps can help improve your setup; and make monitoring a little less painful.
Integrate your Asset Data
First up, lets start taking about integration of extra data into SIEM to supplement logs. One of the best types of data to integrate is your asset profiles from your CMDB. Most modern SIEMs allow this to be done easily via either one time imports or API’s (If you would like help in doing this, or any other points here, send me a DM on Twitter and I will do my best to help). Some platforms will also learn about your assets over time using network flows. However I would recommend doing an import if possible to get accurate data.
Once you have the data integrated into your SIEM, the benefits are fantastic. Most notably is the fact that this data will help you to quickly prioritise incidents. Instead of wondering “is this source IP being targeted even important” you can instead quickly tell it is an important server. Likewise, if you have a large number of incidents at once you can known which incidents to prioritise based on the criticality of assets.
Oh, Integrate your Vulnerability Data too!
This links in well with the first point and I recommend doing both together. Similar to above using one time imports or APIs; most SIEMs allow you to integrate your vulnerability data with your SIEM. This method will depend on your vulnerability manager and your SIEM; however it is usually pretty simple.
Once integrated, vulnerability data is also great for both dealing with and prioritising incidents. Rather than an analyst having to check separate platforms, or wait for replies from your VM team; they can instead tell straight away whether a device is vulnerable when an alert triggers. This can help security incidents get raised and dealt with quicker, which can make the difference between a breach occurring or not.
Empower your Analysts
This isn’t a change to your SIEM as such, but rather something which can impact it heavily – the training and morale of the analysts using it. A lot of companies spend a lot on SIEM implementations but then neglect its ongoing upkeep – including training the analysts using it. Don’t be one of these companies – ensure you train the analysts how to use the SIEM, and also give them ongoing training.
There are a number of ways you can achieve this. Firstly, most vendors will include free training and certifications when you buy their product; so keep sure that you ask for this and get training for as many analysts as possible. Training and certifications for any SIEM platform are always available too, or analysts can learn in their own time using resources like those I wrote about here. Alternative to this, and my personal favorite; is giving analysts a test platform to learn on with a similar setup to your own SIEM.
Test SIEM Platform
A ‘Test’ SIEM platform has a large list of benefits to any company who have live SIEM deployments. First off, a test platform is a great, safe platform to use to let analysts learn how to search logs and perform investigations without impacting live services and data. You can learn how to set up a test SIEM environment using this article I wrote here. Once it is set up, you can then ingest sample data which analysts can search and test rules with, such as https://vizsec.org/data/ and https://github.com/redcanaryco/atomic-red-team.
Testing Use Cases
Apart from training new analysts, there is another benefit to setting up a test SIEM environment – for the purpose of testing and validating new use cases. A lot of the time people think of a new use case idea, then they just implement it in a live SIEM platform and think their work is done – please don’t do this – PLEASE. To start with, a lot of the time rules implemented in live platforms are never tested to see if they actually work. If the rules do work, then they are usually prone to false positives and cause analysts a lot of work.
So instead, start testing your rules in a test environment. Ensure that they actually work and that they won’t trigger countless false positives. Doing this – your rules will now work and you will make life a lot easier for analysts monitoring the live SIEM. For test data, you can use the links in the above section.
Use case creation process
When you are creating use cases, I recommend that you come up with a properly defined process to do so. Too many use cases for SIEM’s are created Adhoc with no naming convention, purpose or real plan. This leads to a lot of issues, namely far too many pointless use cases existing. With these, nobody really understands what they exist for or how to deal with them. Let’s not let that happen to you.
So what can we do about it? Well, a good start is having a template to follow when writing use cases. Use the below to give you some ideas of how a template can look.
- Name of the use case. (I recommend mapping them to the MITRE killchain. and including the tactic in the name, for example EXEC for execution or LM for lateral movement.)
- Purpose – Why is the use case being created?
- Problem Statement – What business problem is the use case solving?
- Requirement Statement – What is required to make the use case work?
- Design Specifications – More in depth specification of exactly what needs to be done (Auditing / Logging / Use case design) Who is going to take these actions to makes this a reality?.
- SOC Actions (Playbook) – Which actions are required by the SOC if your use case triggers? How should it be investigated and which information needs reported? Clearly define which team/s the alert should be escalated to and the expected remediation steps.
- Use case logic – Document clearly the exact logic being used for the use case and all variables (References, Log Sources, Asset Data etc)
- Limitations – Are there any known limitations to the use case? Can attackers bypass our detection using other methods? Is the log source we are receiving events from stable?
- Alternative Solutions – Is there any alternative options to creating the use case (Reporting, mitigation’s etc)
- Change notes. If any changes are made to the use case, note them here along with who change the case, why and a date.
- Approvals. It is always worth while having your work double or even triple checked. Before a use case goes into live, get a colleague or two to check your logic and playbooks before signing it off.
Your use cases will now no longer have any issues and your SIEM will be a glowing beacon that detects everything. Jokes, people will still find things that need fixed; that’s why you need…. dun dun dunnnn…. a proper tuning process!
Use case tuning process
Over time, issues will be found with use cases in the SIEM. This could either be a false positive, an expected behaviour that needs tuned out; or the detection missed something. Anyways, a properly defined process needs to exist so that analysts actually have a way to report these issues; or so that your Senior / Engineering teams can discover the issues themselves.
There are three ways I have found to tackle the issue of tuning use cases, I like combining them all. Firstly, you can set up an email template which your monitoring analysts can use to report anything they find to Senior or Engineering teams. This does not need to be anything complicated. Just the name of the use case, the time it triggered and a description of why they think it needs improved; along with recommendations of how to do so, if they think they know exactly what needs fixed. The Senior / Engineering team can then look into the report, perform the fix if required and give feedback to the analyst.
Along with this, I personally run a weekly report automatically which sends me every time an alert has been marked as false positive, along with closure notes. Any time this is used, an analyst should be raising why they believe it is a false positive to Senior / Engineering. If they have, great; we will already know about it and it will either have been, or be getting; dealt with. However, I will also come across cases where this has been used without report being raised. In this case, I instead investigate the alert myself and then discuss with the analyst responsible.
Finally, run a monthly report to review all alerts which have caused false positive triggers throughout the month. Is there a certain alert or type of alert causing this? Dig into why they are happening and fix them! Following these steps will improve the quality of your use cases and lower the amount of false positives you see.
Log Sources (Yes, they are important)
Do you check your log sources regularly? Are they working? Are they auditing the correct events? Do you have events that are giving you no value? Are you getting value from the log sources? These are all questions that you should be asking yourself.
Log sources are often forgotten, however I believe everyone should have a proper process to onboard them and then check their health over time.
- Log Source gets onboarded.
- Why are we onboarding this log source – what does it give us?
- Ensure auditing is turned on correctly for any log sources you onboard using best practices.
- Do you need ALL the logs though? Probably not. For example, rather than collecting all drop data from the proxy on the Firewall, collect it from the Proxy itself; and not both! Look at your logs, and drop the garbage, it will save you a lot of storage and will also make investigations easier.
- Once a log source is onboarded and you are happy it provides value, ensure you are getting as much value as you can. Have you set up all the critical use cases for this data type, do we have reports set up which show us if there are spikes or drops on this?
Ok, now the initial setup is done, but don’t stop there. We want to ensure the log source stays healthy, there is a few things we can do.
- Set up use cases to tell you when a log source hasn’t received an event for X amount of time (This will differ on log source type – for example Endpoint Security vs a Firewall)
- Regularly manually check the status of all log sources (This should be a daily task for your Senior / Engineering team.
That’s it. Don’t neglect your log sources please, after all; if they don’t work; nothing else does.
Use your SIEM for Threat Hunting
SIEM shouldn’t just be used for monitoring and compliance. You have all those logs sitting there, you might as well do something useful with them right? Correct!
Detecting things in real time is great, but sadly a detection doesn’t exist for every single know attack vector, or we might just not want to use them due to false positives. That’s where threat hunting comes in. Most SIEM’s have logs there for 3 months – 1 year +, and we can look back through these logs for signs of compromise.
But what do I start looking for in my logs? Well, here are a few helpful resources to get you started –
A post from a while back to find badness in Windows Security Event Logs – https://blueteamblog.com/threat-hunting-with-windows-security-event-logs
A post from a while back to find badness in your Proxy logs – https://blueteamblog.com/various-threat-hunting-methods-using-web-proxy-logs
Awesome threat hunting Github list – https://github.com/threat-hunting/awesome_Threat-Hunting
Another Github list – https://github.com/A3sal0n/CyberThreatHunting
Threat Hunting using SIEM is great. It gives you a chance to find behaviour you initially missed. You can also use the behaviour you do find to create future use cases – so a win win. Finally, it is also a good thing to do if there are just currently no alerts or incidents to deal with in your SIEM. Got a spare 20 mins and want to mix up your day? Open one of the links above and look for these indicators in your SIEM; you will likely be surprised what you find!
Well, I hope you enjoyed my
ramblings tips on how you can improve your SIEM and found some of this useful. These are by no means all the tips, or the best tips; they are just the tip really of what you can do to start improving your SIEM. Awful pun I know, forgive me. If you do have any questions around anything I mentioned above, DM me at https://twitter.com/blueteamblog where I am always happy to help.