TZWorks LLC
System Analysis and Programming
www.tzworks.com
(Version 0.30)
TZWorks LLC software and related documentation ("Software") is governed by separate licenses issued from TZWorks LLC. The User Agreement, Disclaimer, and/or Software may change from time to time. By continuing to use the Software after those changes become effective, you agree to be bound by all such changes. Permission to use the Software is granted provided that (1) use of such Software is in accordance with the license issued to you and (2) the Software is not resold, transferred or distributed to any other person or entity. Refer to your specific EULA issued to for your specific the terms and conditions. There are 3 types of licenses available: (i) for educational purposes, (ii) for demonstration and testing purposes and (iii) business and/or commercial purposes. Contact TZWorks LLC (info@tzworks.com) for more information regarding licensing and/or to obtain a license. To redistribute the Software, prior approval in writing is required from TZWorks LLC. The terms in your specific EULA do not give the user any rights in intellectual property or technology, but only a limited right to use the Software in accordance with the license issued to you. TZWorks LLC retains all rights to ownership of this Software.
The Software is subject to U.S. export control laws, including the U.S. Export Administration Act and its associated regulations. The Export Control Classification Number (ECCN) for the Software is 5D002, subparagraph C.1. The user shall not, directly or indirectly, export, re-export or release the Software to, or make the Software accessible from, any jurisdiction or country to which export, re-export or release is prohibited by law, rule or regulation. The user shall comply with all applicable U.S. federal laws, regulations and rules, and complete all required undertakings (including obtaining any necessary export license or other governmental approval), prior to exporting, re-exporting, releasing, or otherwise making the Software available outside the U.S.
The user agrees that this Software made available by TZWorks LLC is experimental in nature and use of the Software is at user's sole risk. The Software could include technical inaccuracies or errors. Changes are periodically added to the information herein, and TZWorks LLC may make improvements and/or changes to Software and related documentation at any time. TZWorks LLC makes no representations about the accuracy or usability of the Software for any purpose.
ALL SOFTWARE ARE PROVIDED "AS IS" AND "WHERE IS" WITHOUT WARRANTY OF ANY KIND INCLUDING ALL IMPLIED WARRANTIES AND CONDITIONS OF MERCHANTABILITY, FITNESS FOR ANY PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT SHALL TZWORKS LLC BE LIABLE FOR ANY KIND OF DAMAGE RESULTING FROM ANY CAUSE OR REASON, ARISING OUT OF IT IN CONNECTION WITH THE USE OR PERFORMANCE OF INFORMATION AVAILABLE FROM THIS SOFTWARE, INCLUDING BUT NOT LIMITED TO ANY DAMAGES FROM ANY INACCURACIES, ERRORS, OR VIRUSES, FROM OR DURING THE USE OF THE SOFTWARE.
The Software are the original works of TZWorks LLC. However, to be in compliance with the Digital Millennium Copyright Act of 1998 ("DMCA") we agree to investigate and disable any material for infringement of copyright. Contact TZWorks LLC at email address: info@tzworks.com, regarding any DMCA concerns.
Event Tracing for Windows or ETW, is a built-in, logging and diagnostic framework available to all. It can be dynamically enabled and the data it generates can be logged to a file or consumed in real time by another application. It can be used for performance analysis, general debugging, or in our case, for forensics purposes.
ETW was first introduced in Windows 2000. It started as a modest set of providers and was expanded with each new version of Windows. With Windows 10, the number of providers grew to over 1000. Starting with Win10, TraceLogging was added to ETW to allow for event tracing for user-mode applications and kernel-mode drivers.
Event trace logs can be found in many places. Below are some of the locations observed on a sample system volume.
%ProgramData%\Microsoft\Windows\WER\ReportQueue\*\ [{guid}-WER-MMDDYYYY-####] %ProgramData%\Microsoft\Windows\wfp\ [wfpdiag] %ProgramData%\USOShared\Logs\ [NotificationUxBroker, UpdateSessionOrchestration] %LocalAppData%\Packages\Microsoft.CommsPhone_..\LocalCache\ [CallsAppLog, CallsBackgroundTaskLog] %LocalAppData%\Packages\Microsoft.Messaging_...\LocalCache\ [MessagingBackgroundTaskLog] %LocalAppData%\Packages\Microsoft.Windows.Photos_...\LocalState\ [PhotosAppTracing] %LocalAppData%\Diagnostics\*\*\ [{guid}.Diagnose, {guid}.Repair, {guid}.Verify] %LocalAppData%\ElevatedDiagnostics\*\*\ [{guid}.Diagnose.Admin, {guid}.Repair.Admin, {guid}.Verify.Admin] %LocalAppData%\Microsoft\Office\ [OTeleData] %LocalAppData%\Microsoft\OneDrive\logs\Personal\ [TraceCurrent, TraceArchive] %LocalAppData%\Microsoft\Windows\Explorer\ [ExplorerStartupLog] %LocalAppData%\Temp [{guid}.Repair, {guid}.Diagnose, {guid}.Verify] %UserProfile%\Tracing\WPPMedia\ [Skype_MediaStack] %systemroot%\Logs\dosvc\ [dosvc.YYYYMMDD_xxxxx] %systemroot%\Logs\NetSetup\ [service.x] %systemroot%\Logs\SIH\ [SIH.YYYYMMDD.xxxxx] %systemroot%\Logs\SystemRestore\ [Restore.UI] %systemroot%\Logs\WindowsBackup\ [WBEngine.x] %systemroot%\Logs\WindowsUpdate\ [WindowsUpdate.YYYYMMDD.xxxxx] %systemroot%\Panther\ [setup] %systemroot%\Performance\WinSAT\DataStore\ [winsat] %systemroot%\security\logs\ [SceSetupLog] %systemroot%\SoftwareDistribution\Download\*\*\ [WindowsUpdate.YYYYMMDD.xxxx, SceSetupLog, LwtNetLog, Wifi, EtwRTDiagLog, EtwRTEventLog-Application, EtwRTEventLog-System, ShutdownCKCL, BootCKCL, SecondaryLogonCKCL, wfpdiag, KnobsCsp, ...] %systemroot%\System32\LogFiles\WMI\ [Wifi, FamilySafetyAOT, LwtNetLog, ..] %systemroot%\System32\LogFiles\WMI\RtBackup\ [EtwRTDiagLog, EtwRTEventLog-System, EtwRTEventLog-Application, EtwRTUBPM, EtwRTEventlog-Security, EtwRTDefenderAuditLogger, EtwRTWFP-IPsec Diagnostics, EtwRTDiagtrack-Listener, EtwRTDefenderApiLogger, ...] %systemroot%\System32\NDF\ [eventlog] %systemroot%\System32\SleepStudy\ [sleepstudy-trace-yyyy-mm-dd-hh-mm-ss, SleepStudyTraceSession] %systemroot%\System32\WDI\{guid}\{guid}\ [snapshot] %systemroot%\System32\WDI\LogFiles\ [SecondaryLogonCKCL, ShutdownCKCL, BootCKCL] %systemroot%\System32\winevt\Logs\ [AirSpaceChannel]
From a forensic standpoint, Event Traces logs (and the underlying ETW framework) have a lot of information that could be useful to the DFIR analyst; from timestamps, to user SID, thread and process ID, formatted messages, etc. In general, these logs can be useful to collect information to assist in the auditing process. Unfortunately, these logs can also be used by attackers to extract information by targeting certain applications and extracting metadata using the same ETW framework.
While there are various tools from Microsoft to analyze these log files, the purpose of tela is to provide a Windows API agnostic tool that can parse these logs quickly. The other objective was to be able to parse many of these logs in a batch manner, while rendering their content in a CSV format for review in a spreadsheet tool or easy ingestion into a separate database.
tela is a command line tool that parses Windows Event Trace Log (ETL) files. The tool does not make use of Windows libraries critical to parsing ETL data, so there are binaries for Linux and OSX, as well, to allow parsing of this artifact offline on one of the other platforms.
There are other options to assist the user in analyzing ETL type data, including: (a) statistics of the ETL, (b) create a subset ETL file, (d) enumeration of the providers, and (d) extraction of the binary XML (BXML) template data stored in portable executable (PE) files which is used for packing of the ETL data.
One can display the menu options by typing in the executable's name with no parameters. A screen shot of the menu is shown below:
Usage: tela -log "log1 | log2 | .." = pull data from extracted logs dir c:\somedir\*.etl /b /s | tela -pipe tela -enumdir <folder> -num_subdirs <#> [options] Basic options -csvl2t = log2timeline output -pipe = pipe files into evtwalk for processing -quiet = don't display status during run -filter <*partial*|*.ext> = filters stdin data from -pipe option -dateformat mm/dd/yyyy = "yyyy-mm-dd" is the default -timeformat hh:mm:ss = "hh:mm:ss.xxxxxxxxx" is the default -no_whitespace = remove whitespace between csv delimiter -csv_separator "|" = use a pipe char for csv separator --- the following options are for debugging purposes only --- Other options -stats = list the stats in the ETL file(s) -templates <file> = file with template instructions -guids = list the providers in the ETL file(s) -pullguid <guid> -out <etl> = create subset ETL file with just GUID records Standalone utility options -extract_bxml <pefile> = pull out tela encoded bxml for event ids -providers <sw hive> = pull out providers from software hive
There are 2 basic options for parsing ETL files. The first is to parse an individual ETL, via the -log option. Below is the syntax. In the example below, the output is redirected to and output file called results.txt. The data formatted in this file will have its fields pipe delimited.
tela64 -log E:\testcase\etl\LwtNetLog.etl > results.txt
The second option is to process one or more ETL files via the -pipe option. This allows one to have many ETL files in a folder and process them all in one sessions.
dir c:\dump\ETL_data /b /s /a | tela64 -pipe > results.txt
During the analysis and design tela it was necessary to determine which providers and event ID's were present in a specific ETL file. This was helpful in going after metadata that was being reversed and that would only be present for certain combinations of provider/event ID's. For this use-case, the -stats option was added to the design. Using this option will cause tela to histogram the events by their respective provider.
Below is a sample output using this option. Various sections are truncated to not take up too much space in the document, but enough to get the point across as to what the output looks like. If the provider name for the provider guid is known by tela, it be outputted as shown below. For each provider, a set of event ID's are displayed along with the number of occurrences.
file: E:\testcase\etl\LwtNetLog.etl 6a1f2b00-6a90-4c38-95a5-5cab3b056778 : Microsoft-Windows-DHCPv6-Client event id | occurances 51005 | 3 51062 | 2 cdead503-17f5-4a3e-b7ae-df8cc2902eb9 : Microsoft-Windows-NDIS event id | occurances 10032 | 17 10048 | 10 10058 | 1 10104 | 3 10105 | 11 10303 | 39 10304 | 37 ... truncated .. 67d07935-283a-4791-8f8d-fa9117f3e6f2 : Microsoft-Windows-Wcmsvc event id | occurances 1003 | 3 1005 | 4 1006 | 3 ... truncated .. df271536-4298-45e1-b0f2-e88f78619c5d : Microsoft-Windows-Ndu event id | occurances 2014 | 4 ... truncated ..
Without this option, one would need to wade through many lines (typically 1000's) of output to identify whether a specific pair of provider to event ID was present. So, even though it was added as an aid in the development of tela, the option was left in since it can also be useful to the analyst, as a quick look report.
One can list the providers in the form of guids (Globally Unique Identifiers) that are contained in a particular ETL file via the option -guids. Below is an example of this:
tela64 -log E:\testcase\etl\LwtNetLog.etl -guids
file: E:\testcase\etl\LwtNetLog.etl
6a1f2b00-6a90-4c38-95a5-5cab3b056778 : Microsoft-Windows-DHCPv6-Client
cdead503-17f5-4a3e-b7ae-df8cc2902eb9 : Microsoft-Windows-NDIS
67d07935-283a-4791-8f8d-fa9117f3e6f2 : Microsoft-Windows-Wcmsvc
df271536-4298-45e1-b0f2-e88f78619c5d : Microsoft-Windows-Ndu
fbcfac3f-8459-419f-8e48-1f0b49cdb85e : Microsoft-Windows-NetworkProfile
0c478c5b-0351-41b1-8c58-4a6737da32e3 : Microsoft-Windows-WFP
43d1a55c-76d6-4f7e-995c-64c711e5cafe : Microsoft-Windows-WinINet
e6835967-e0d2-41fb-bcec-58387404e25a : Microsoft-Windows-BrokerInfrastructure
... truncated...
Primarily used for a debugging option, is to extract certain records from a large ETL file and create a subset ETL file with only one type of provider. In this case, if the original ETL file contained many providers, and we are having issues parsing a few of the providers' data, we can extract those that we are evaluating and create an entirely new ETL file with just those records of interest. This syntax for this would be:
tela64 -log E:\testcase\etl\LwtNetLog.etl -pullguid "cdead503-17f5-4a3e-b7ae-df8cc2902eb9" -out ndis.etl
What the above command will do is parse the LwtNetLog.etl file copying only the records that are from the provider associated with the guid "cdead503-17f5-4a3e-b7ae-df8cc2902eb9", and store them into a new ETL file called ndis.etl. The resulting ETL file should be parsable with any of the Microsoft tools as well as with tela.
An internal option is exposed that allows for users to point tela to an external template type file, via the syntax -template <file>. The purpose of this option was to allow the tool have parsing extensibility as new providers become of interest.
Although this option is functional, the generation of these templates was meant to be done internally until this functionality matures out of the beta phase. This is because, generation of these template files requires one to have good knowledge of the WEVT_TEMPLATE data from the provider. Further, this requires one to: (a) use a tool to parse the resource section of the PE file, such as the TZWorks tool pescan, (b) understand the output of the raw WEVT_TEMPLATE data, and (c) how to apply that data to parse ETL structures.
To assist us in generating these custom template files, one should submit the following data to TZWorks: (a) The GUID and name of the provider, which should be able to to be pulled via the -providers <sw hive> option; and from that data, use the MessageFilePath data to locate the PE file containing the resource data. (b) From the path/name of the PE file, use the -extract_bxml <pefile> option to pull the needed WEVT_TEMPLATE data. Sending both outputs to TZWorks will allow us to reconstruct the proper template file that tela can use to parse the data.
Below is an example of the data one would see going through the process above. The first command pulls the providers and the associated data known about that provider. The subkey would be the provider GUIDs and the (unnamed) registry value name is the associated provider name. The MessageFileName value name is the resource containing the WEVT_TEMPLATE needed by tela to parse the ETL data. The second command goes after a specific provider resource file and extracts the BXML data from the WEVT_TEMPLATE.
tela64 -providers e:\testcase\hives\software\software.bin subkey | (unnamed) | MessageFileName ... {fbcfac3f-8459-419f-8e48-1f0b49cdb85e} | Microsoft-Windows-NetworkProfile | %windir%\system32\netprofm.dll ... {f997cd11-0fc9-4ab4-acba-bc742a4c0dd3} | Microsoft-Windows-RPC-FirewallManager | %SystemRoot%\system32\RpcEpMap.dll ... {f82fb576-e941-4956-a2c7-a0cf83f6450a} | Microsoft-Windows-PerfOS | %SystemRoot%\system32\perfos.dll ... {f6da35ce-d312-41c8-9828-5a2e173c91b6} | Microsoft-Windows-Dhcp-Nap-Enforcement-Client | %Systemroot%\system32\dhcpqec.dll ... truncated... tela64 -extract_bxml "%windir%\system32\netprofm.dll" provider guid event id BXML in quasi-json format fbcfac3f-8459-419f-8e48-1f0b49cdb85e | 4001 | bxml{"InterfaceGuid":"15";"CurrentOrNextState":"4"} ... fbcfac3f-8459-419f-8e48-1f0b49cdb85e | 10002 | bxml{"Name":"1";"Description":"1";"Guid":"15";"Type":"8";"State":"8";"Category":"8"} fbcfac3f-8459-419f-8e48-1f0b49cdb85e | 10003 | bxml{"NetworkProfileEventState":"8";"Guid":"15"} ... truncated...
Option | Description |
---|---|
-log | Identify which event log(s) to operate on. The syntax is: -log <ETL to analyze>. To operate one more than one at a time, use: -log "<ETL1> | <ETL2> | ..." |
-csvl2t | Outputs the data fields in accordance with the log2timeline format. |
-pipe | This option allows one to parse in multiple tracelogs from standard input in one session. |
-enumdir | Experimental. Used to process files within a folder and/or subfolders. Each file is parsed in sequence. The syntax is -enumdir <"folder"> -num_subdirs <#>. |
-filter | Filters data passed in via stdin via the -pipe option. The syntax is -filter <"*.ext | *partialname* | ...">. The wildcard character '*' is restricted to either before the name or after the name. |
-quiet | This option is tells the app not to display progress status during a run. |
-no_whitespace | This option will remove all white space between the field value and the delimiter separator. |
-csv_separator | This option is to modify the existing CSV delimiter to some other delimiter. Format is: -change_csv_separator "|". |
-dateformat | Output the date using the specified format. Default behavior is -dateformat "yyyy-mm-dd". Using this option allows one to adjust the format to mm/dd/yy, dd/mm/yy, etc. The restriction with this option is the forward slash (/) or dash (-) symbol needs to separate month, day and year and the month is in digit (1-12) form versus abbreviated name form. |
-timeformat | Output the time using the specified format. Default behavior is -timeformat "hh:mm:ss.xxxxxxxxx" One can adjust the format to microseconds, via "hh:mm:ss.xxxxxx" or milliseconds, via "hh:mm:ss.xxx", or no fractional seconds, via "hh:mm:ss". The restrictions with this option is that a colon (:) symbol needs to separate hours, minutes and seconds, a period (.) symbol needs to separate the seconds and fractional seconds, and the repeating symbol 'x' is used to represent number of fractional seconds. |
-templates | Use the specified file that stores BXML templates to parse ETL file. The syntax is: -templates <file>. |
-pullguid | Create a subset ETL file with just the specified GUID (provider) records. The syntax is: -pullguid <guid> -out <subset ETL file>. |
-guids | List all the providers (guids) used in the ETL file(s) |
-stats | List all the providers (guids) and the Event IDs used in the ETL file(s). The Event IDs will be listing in terms of occurrence (like a histogram). |
-extract_bxml | Extract the BXML Templates from the portable executable (PE) file. The syntax is: -extract_bxml <pefile>. This option relies on pescan ver: 0.39 or later to be in the same directory as tela. |
-providers | Extract the providers and their associate guid and resource file from the specified Software hive. The syntax is: -providers <sw hive>. This option relies on cafae to be in the same directory as tela. |
-utf8_bom | All output is in Unicode UTF-8 format. If desired, one can prefix an UTF-8 byte order mark to the CSV output using this option. |
This tool has authentication built into the binary. The primary authentication mechanism is the digital X509 code signing certificate embedded into the binary (Windows and macOS).
The other mechanism is the runtime authentication, which applies to all the versions of the tools (Windows, Linux and macOS). The runtime authentication ensures that the tool has a valid license. The license needs to be in the same directory of the tool for it to authenticate. Furthermore, any modification to the license, either to its name or contents, will invalidate the license.
The tools from TZWorks will output header information about the tool's version and whether it is running in limited, demo or full mode. This is directly related to what version of a license the tool authenticates with. The limited and demo keywords indicates some functionality of the tool is not available, and the full keyword indicates all the functionality is available. The lacking functionality in the limited or demo versions may mean one or all of the following: (a) certain options may not be available, (b) certain data may not be outputted in the parsed results, and (c) the license has a finite lifetime before expiring.