What is Sysmon?
System Monitor (Sysmon) is a Windows system service and device driver that, once installed on a system, remains resident across system reboots to monitor and log system activity to the Windows event log. It provides detailed information about process creations, network connections, and changes to file creation time.
I use Sysmon on a daily basis as part of my job as a blue teamer. It is a wonderful tool and, depending on the configuration it’s fed, extremely powerful. If you’re looking to get started on crafting a configuration for your organization then I would highly recommend you use SwiftOnSecurity’s Sysmon config as a starting point. When combined with an event forwarder and a log stack such as ELK you can quickly and, more importantly, freely spin up an incident response and detection capability.
This post will serve as a quick and overarching view of how Sysmon operates. In (possible) other future posts I will dig into each portion in finer detail. I also plan on writing some posts about using Sysmon to catch attackers. For now, I will simply explain how I’ve mapped Sysmon features to their specific sources of data. I feel it is important for defenders to know their tools and how they work in order to understand their limitations and weaknesses.
Before we begin you should to have a rough understanding of what kind of data Sysmon provides.
|Event ID||Event Name|
|2||Process Changed a File Creation Time|
|4||Sysmon Service State Changed|
|8||Create Remote Thread|
|9||Raw Access Read|
|12||Registry Event (Object Create and Delete)|
|13||Registry Event (Value Set)|
|14||Registry Event (Key and Value Rename)|
|15||File Create Stream Hash|
|16||Sysmon Configuration Change|
|17||Named Pipe Created|
|18||Named Pipe Connected|
Make sure to keep these in mind as we will return to them later. Next, I want to go over my test environment and how you can set up Sysmon to follow along.
I’m going to be using a virtual machine with Windows 7 x64 installed in VMware Workstation 12. I’d recommend you create a snapshot before you begin since we will be using tools that load drivers for analysis and it’s always better to err on the side of safety.
Once you have your Windows box loaded and set you’re going to need some things.
1. Sysinternals Suite
2. PC Hunter
3. SwiftOnSecurity’s Sysmon config
Before we install Sysmon we’re going to make some changes to the configuration that you just downloaded. We want to be sure that all of the events are turned on so we can get a complete analysis of Sysmon’s functionality. To do this just make the changes on the following lines:
Line 251: Add
Line 273: Add
Line 279: Add
Line 534: Add
Once you have made the modifications to the config file go ahead and save it as
Sysmon consists of a Windows service as well as a device driver. It’s a single binary that covers both 32-bit and 64-bit versions of Windows. When you install Sysmon it will extract binaries into your system root, register an event log manifest, and load the configuration you gave it. If you do not specify a configuration file then it will default to a small configuration with a couple of event codes turned on.
In order to install Sysmon open an elevated command prompt,
cd to the sysinternals directory and issue the following command:
sysmon -accepteula -i sysmon.xml
You should see the following:
To view the current Sysmon configuration:
To update the configuration, you can pass it a configuration xml file after the
To uninstall Sysmon:
Now that Sysmon is up and running let’s go over the topic of kernel callbacks.
Antivirus and security products used to often provide monitoring and detection capability by utilizing various kernel patching techniques. One example is System Service Descriptor Table (SSDT) hooking. Nowadays this method seems to be rarely used, and not possible on x64, due to a couple of reasons. First, any unhandled exception caused would lead to a BSoD. Second, starting with Windows Vista x64 Microsoft released PatchGuard, implementing Kernel Patch Protection. PatchGuard checks for integrity of the SSDT, among other things, and will throw a bugcheck (BSoD) if any changes are detected. Although PatchGuard bypasses exist, they could be patched by Microsoft at any moment and thus cannot be relied upon by a security product vendor.
When introducing PatchGuard Microsoft was aware that it would prevent some security software from functioning as intended. Because of this they added alternative methods security vendors could use called kernel callbacks. These callbacks do not perform any modification to the Windows kernel and are safe to use. A driver can register a callback for a supported action and will receive a notification either before (pre) or after (post) the action has occurred. Receiving the notification before the action has finished allows the driver an opportunity to prevent that action from occurring.
There are kernel callbacks available for many different actions. Here is a brief description of some and the API call to register that callback:
|PsSetLoadImageNotifyRoutine||Image (DLL/Driver) Loading|
|FltRegisterFilter||Minifilter Registration (I/O)|
One important thing to note is that the registration of a file system minifilter driver allows the same sort of pre-operation and post-operation notifications as the ones for process, threads, etc but for file system operations instead.
Event Tracing for Windows
Event Tracing for Windows (ETW) is a tracing technology available since Windows 2K that is commonly used for debugging purposes. It consists of Providers which emit certain events and have a schema that describe those events, as well as Consumers which create sessions to consume providers and create callbacks to handle these events.
Event Tracing for Windows provides visibility into most core Windows subsystems and, in Windows 10, there are more than 1000 different types of providers. They range from network activity to PowerShell method invocation and even WinINet requests and response headers.
These providers can provide security software additional information, such as network events, that they wouldn’t be able to get from kernel callbacks or unless they created a Windows Filtering Platform driver.
Event Tracing for Windows is a big enough topic for multiple blog posts, however I just want you to be familiar with what it is before we see how Sysmon uses it.
Sysmon in Action - Kernel Callbacks
By this point you’ve probably figured out that Sysmon uses all of the above methods to perform logging, so now we’re going to use PC Hunter and some built in Windows utilities to view them.
Let’s start with PC Hunter. One of the first things you can do when you open it to begin exploring is to look at a list of running processes.
In my example you can see that the Sysmon process is running with process ID 1072, parent process ID 508 – services.exe, the location of the image on disk, the location of the process’ EPROCESS structure, and some metadata.
This is the user mode portion of Sysmon that I mentioned earlier.
Now view the Kernel Module (Drivers) tab.
You can see the name and full path of the driver, it’s Image base location and size, any corresponding service name, the load order, as well as the driver “File Corporation.”
Identifying the driver that Sysmon loads is important because it’s used to locate what kernel callbacks it has registered as well as any minifilter drivers.
Next, take a look at the Kernel -> Notify Routine tab.
This lists all of the registered kernel callbacks (notify routines) along with the driver responsible for registering it and the entry address of the routine. As you can see, Sysmon has registered
CmpCallback kernel callbacks. These directly translate into some of the event codes that I outlined above.
In the Kernel -> File System tab look at the MiniFilter file system item.
This tab shows registered MiniFilter objects and which IRP Major Function Codes it handles. This allows the driver to perform an action, in Sysmon’s case logging, when it receives an I/O Request Packet. This provides the data for Sysmon to log things like file creation, time stomping, and named pipe creation.
Moving on to the Kernel Hooks -> Object Type tab.
Here you can see the callbacks that were registered through the
ObRegisterCallback API. The Sysmon driver registers both a pre-operation and post-operation handle callback on processes. This allows Sysmon to log all handle operations that involve processes and is extremely helpful when watching for things like credential dumping since you’ll be able to see a process request a handle to lsass.exe.
I’m not quite sure why Sysmon registers a pre-operation callback if it’s not going to prevent or strip the handle rights. It’s possible it registers it so that it can log the attempt at opening a handle in case another security product prevents the handle from being created and it never gets to the post-operation bit. If you know why it does this feel free to get in contact and I’ll update this post.
Sysmon in Action - ETW
Now that we know what types of kernel callbacks Sysmon registers we can take a look at some of it’s ETW related activity. To do this we won’t need any special tools since everything we need comes with Windows by default. One of the first things we want to look at is any currently running event tracing sessions. This will allow us to locate traces that Sysmon may be consuming. For this we can use the logman utility in an elevated command prompt.
These are all ETW consumers that are currently running. As you can see one of those is a Sysmon consumer that feeds the Windows Event Log. Once we have the name of the consumer we can query it specifically.
logman EventLog-Microsoft-Windows-Sysmon-Operational -ets
This allows you to view a lot more information about the consumer and which provider it is using. You can see things like it’s current status, the root path, file mode, and buffer information. In the provider view you can see it’s GUID, keywords, and level. Level will control the level of detail such as Error, Informational, or Verbose and keywords define sub-components.
Once you have the name of the provider you can query it specifically using logman.
logman query providers “Microsoft-Windows-Sysmon”
This will show much of what was seen earlier, but it allows you to view the levels and keywords in plain English. In this case Sysmon is logging Error and Information levels to the Windows Event Log.
You may be wondering at this point where Sysmon is actually collecting the network events from since it’s not being consumed in the
Microsoft-Windows-Sysmon-Operational trace. That’s a great question. Let’s take a closer look at the
NT Kernel Logger consumer that we saw earlier when we listed all of the consumers.
logman query “NT Kernel Logger” -ets
You can see that this consumer is currently running and is currently consuming events from the
Windows Kernel Trace provider. More importantly, you can see from the
KeywordsAny field that
net keyword is enabled. To see the description of that keyword just query the provider with logman.
logman query providers “Windows Kernel Trace”
net keyword being enabled means that the
Windows Kernel Trace provider is currently logging Network TCP/IP events. This is how Sysmon is using ETW in order to log network events.
At this point we’ve enumerated all of the sources Sysmon is using to log all of the event IDs that it provides. Just to wrap up here is a table of event IDs and their corresponding sources of data.
|Event ID||Event Name||Data Source|
|2||Process Changed a File Creation Time||Minifilter|
|3||Network Connection||Windows Kernel Trace - net|
|4||Sysmon Service State Changed||Sysmon Service|
|8||Create Remote Thread||PsSetCreateThreadNotifyRoutine|
|9||Raw Access Read||Minifilter|
|12||Registry Event (Object Create and Delete)||CmpCallback|
|13||Registry Event (Value Set)||CmpCallback|
|14||Registry Event (Key and Value Rename)||CmpCallback|
|15||File Create Stream Hash||Minifilter|
|16||Sysmon Configuration Change||Sysmon Service|
|17||Named Pipe Created||Minifilter|
|18||Named Pipe Connected||Minifilter|
I hope this post has helped you to better understand how Sysmon works overall. If you spot any mistakes please don’t hesitate to contact me and I’ll correct it.
Stay tuned for future, and more detailed, posts!
- Tracking Hackers on Your Network with Sysinternals Sysmon
- Hidden Treasure: Intrusion Detection with ETW (Part 1)
- Hidden Treasure: Intrusion Detection with ETW (Part 2)