TZWorks LLC
System Analysis and Programming
www.tzworks.com
(Version 0.60)
TZWorks LLC software and related documentation ("Software") is governed by separate licenses issued from TZWorks LLC. The User Agreement, Disclaimer, and/or Software may change from time to time. By continuing to use the Software after those changes become effective, you agree to be bound by all such changes. Permission to use the Software is granted provided that (1) use of such Software is in accordance with the license issued to you and (2) the Software is not resold, transferred or distributed to any other person or entity. Refer to your specific EULA issued to for your specific the terms and conditions. There are 3 types of licenses available: (i) for educational purposes, (ii) for demonstration and testing purposes and (iii) business and/or commercial purposes. Contact TZWorks LLC (info@tzworks.com) for more information regarding licensing and/or to obtain a license. To redistribute the Software, prior approval in writing is required from TZWorks LLC. The terms in your specific EULA do not give the user any rights in intellectual property or technology, but only a limited right to use the Software in accordance with the license issued to you. TZWorks LLC retains all rights to ownership of this Software.
The Software is subject to U.S. export control laws, including the U.S. Export Administration Act and its associated regulations. The Export Control Classification Number (ECCN) for the Software is 5D002, subparagraph C.1. The user shall not, directly or indirectly, export, re-export or release the Software to, or make the Software accessible from, any jurisdiction or country to which export, re-export or release is prohibited by law, rule or regulation. The user shall comply with all applicable U.S. federal laws, regulations and rules, and complete all required undertakings (including obtaining any necessary export license or other governmental approval), prior to exporting, re-exporting, releasing, or otherwise making the Software available outside the U.S.
The user agrees that this Software made available by TZWorks LLC is experimental in nature and use of the Software is at user's sole risk. The Software could include technical inaccuracies or errors. Changes are periodically added to the information herein, and TZWorks LLC may make improvements and/or changes to Software and related documentation at any time. TZWorks LLC makes no representations about the accuracy or usability of the Software for any purpose.
ALL SOFTWARE ARE PROVIDED "AS IS" AND "WHERE IS" WITHOUT WARRANTY OF ANY KIND INCLUDING ALL IMPLIED WARRANTIES AND CONDITIONS OF MERCHANTABILITY, FITNESS FOR ANY PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT SHALL TZWORKS LLC BE LIABLE FOR ANY KIND OF DAMAGE RESULTING FROM ANY CAUSE OR REASON, ARISING OUT OF IT IN CONNECTION WITH THE USE OR PERFORMANCE OF INFORMATION AVAILABLE FROM THIS SOFTWARE, INCLUDING BUT NOT LIMITED TO ANY DAMAGES FROM ANY INACCURACIES, ERRORS, OR VIRUSES, FROM OR DURING THE USE OF THE SOFTWARE.
The Software are the original works of TZWorks LLC. However, to be in compliance with the Digital Millennium Copyright Act of 1998 ("DMCA") we agree to investigate and disable any material for infringement of copyright. Contact TZWorks LLC at email address: info@tzworks.com, regarding any DMCA concerns.
The forensic community has been providing consistent feedback on the TZWorks® toolset in identifying which tools and capabilities are important to them. This feedback is valuable, allowing us at TZWorks LLC to focus on areas useful to the community and is one of the reasons we continue to develop new tools. One reocurring request is to provide a GUI (Graphical User Interface) front end to some of the command line tools. While we internally prefer command line tools for automated processing, we do have a handful of GUI based tools that we develop for internal use only, primarily for reversing and analyzing new artifacts. So we decided to take one of our internal tools, take out some of the more arcane options, and merge the backend processing with ntfswalk and some other tools to come up with gena.
The name gena is short for Graphical Engine for NTFS Analysis. Along the way, we made some significant additions to ntfswalk, as well, to allow gena to be used as a flexible tool for data extraction. We also incorporated some capabilities from ntfscopy ntfsdir, and wisp. So while this document is a readme for gena, there are references to some of these other tools throughout.
When developing a GUI app we continue to use the FOX (Free Objects for X) C++ toolkit, since it is cross platform, light weight, fast in performance and compiles into a static library that is relatively small when compared to the other GUI toolkits. FOX, however, has some drawbacks in that it may not have the look and feel that a normal Windows application would have. The GUI front end for gena uses the FOX toolkit.
Similar to the other TZWorks® tools that were mentioned, gena is designed to work with live (mounted) NTFS volumes. There is also functionality for traversing either NTFS images (a) created with the dd utility or (b) from a monolithic volume consisting of VMWare VMDK files. Whether gena is being used for live incident response collection or to process an image in an off-line manner, there are options to filter by file extensions, a timestamp range, various binary signatures, partial filenames and/or directory contents. For targeted files found, one can list the summary metadata, extract the header bytes of the file data, or extract the entire file contents into a designated directory. Since the GUI front end and backend parsing engine are both Windows API agnostic, there are compiled versions for Windows, Linux and macOS.
To use this tool, an authentication file is required to be in the same directory as the binary in order for the tool to run.
The gena GUI is divided into two main window panes. The first pane is on the left; it is a quasi-Windows Explorer view of the volume under analysis. The right pane is governed by a series of tabs that reflect differing functionality. The number of tabs is dependent on the MFT record (or inode) that is selected on the left tree-view pane. This is the area where one can invoke the ntfswalk tool or peer into the details of an MFT record. The first and default tab is for the ntfswalk dialog. It exposes various options unique to ntfswalk.
When one loads a volume for analysis, a series of additional tabs automatically become visible. This includes: a tab for MFT record metadata (timestamp, cluster runs, etc.), a tab for the hexadecimal dump of the file record under analysis, and a separate tab for each NTFS attribute containing data. The term 'data' as used here, can include: (a) the unnamed data stream, (b) any alternate named data streams, (c) the directory's INDX data, and (d) a Logged Stream data, among others. To keep things simple, each unique data set is selectable with a separate data tab.
Currently there are five types of source data gena can analyze, including: (a) NTFS drive/volume stored as a ‘dd’ type image, (b) NTFS mounted drive/volume, (c) an exported $MFT file, (d) a VMWare volume, and (e) a Volume Shadow Copy. To analyze any of the sources one uses the 'File' menu and selects the desired data source.
There are a few data format options, which are grouped by the following categories: (a) Final Output, (b) Date Format, and (c) Time Resolution.
The first category, Final Output, tells gena how to format the results ntfswalk produces. This can either be: (a) default output, (b) CSV format, (c) Log2Timeline format, (d) BodyFile format or (e) a Hashfile.
The default option uses one line to specify the attributes per inode and uses the pipe character '|' as a separator. The CSV option is similar to the default option, but forces all output to be in comma separated value format. Thus, any data that had comma in its name is changed to space character to protect the integrity for the CSV fields.
The Log2Timeline option forces all output to be in CSV format and tries to conform to the log2timeline format specified by the website http://log2timeline.net/. This output needs to be independently verified for correctness.
The Bodyfile option outputs the data fields in accordance with the body-file version3 specified in the Sleuthkit. The date and timestamp outputted to the body-file is in terms of UTC. So if using the body-file in conjunction with the mactime.pl utility, one needs to set the environment variable TZ=UTC. This output also needs to be independently verified for correctness.
The last option is the Hashfile format. This option will generate both a MD5 and SHA1 entry as well as other metadata per inode that was processed.
Note: If extracting data (ie. files) during the run, the first two options are the only options available. If not extracting file content data, then all five options are available.
The date format can be selected based on desired convention. The Date Format menu offers three common format options: mm/dd/yyyy, dd/mm/yyyy, and yyyy/mm/dd. The default is set to the USA conventional format of: mm/dd/yyyy.
The time format has three resolution options: (a) seconds, (b) milliseconds and (c) microseconds. The default is set to milliseconds.
gena was originally designed to be a front end to the ntfswalk command line tool. The additional capabilities added to version 0.45 of ntfswalk, coupled with gena, make it easy to target specific files in a scripted manner, whether for incident response or normal forensic data collection. gena puts many (but not all) of the available ntfswalk command line options at easy access to the user and graphically arranges them by functionality. Therefore, this is the tab to use, if you want to extract many files or analyze an entire volume.
If using gena just to setup a script for ntfswalk to run later on another target box, one can use gena to select the desired options without loading an image, and select the 'Generate Script' button. This action will package all the options selected and generate a script that is compatible to be invoked directly by ntfswalk (in a command line mode). To handle this script, a new -script command line option has been added to ntfswalk. See the ntfswalk documentation for more details.
On the other hand, if one wants to perform a data collection or volume analysis on a live system (or off-line image), one can point gena at any NTFS volume (or image) and it will operate on that target. Selecting the 'Spawn ntfswalk' button will do just that: spawn an instance of ntfswalk as a separate process passing to it the options the user has selected. It is transparent to gena and ntfswalk whether the target volume is a live mounted image or a ‘dd’ image. Both are processed the same way. As an aside, since gena will spawn ntfswalk, both tools should be in the same directory along with their authenticating licenses for this mode to work. Using this latter technique, gena can spawn as many instances of ntfswalk that is desired while they run in parallel, assuming the host machine can handle the processing and memory load.
Since there is this close relationship between gena and ntfswalk, only version 0.45 and higher of ntfswalk will work with gena.
The filtering options allows one to hone exactly what type of files are desired for analysis or extraction. This turns out to be the primary way to speed up any target collection. To understand the filter options, some background on ntfswalk is useful. The newer versions of ntfswalk have been enhanced in a number of ways from the prior versions. While still evolving, filtering is one of the bigger enhancements and was geared to help out in incident response collection.
The filtering architecture has been beefed up to allow for more combinations of options, some of which use OR logic and others which use AND logic. Consequently, as the number of command line options increased, the complexity (and confusion factor) increased as well. Fortunately, the ntfswalk dialog tab in gena keeps the OR and the AND logic straight between the various options by the specific grouping of options.
One can filter on any number of extensions, partial names, directories, and signatures. This means they will all use OR logic, which means that if any of the inode's analyzed passes any one filter mentioned above it will go to the next step. The AND logic comes into play when looking at (a) filtering by date range and (b) filtering on deleted files. Once an inode passes the filtering tests, it then proceeds to be outputted to the results file, and if instructed, copied to the specified directory.
The most basic (and fastest) filtering one can perform is by filtering on the NTFS file record (or external) metadata. This includes things as: (a) file name, (b) file extension, (c) time stamp, (d) parent directory, (e) or inode. The other mode of filtering is on internal, file content data. This mode of filtering is discussed in the next section.
The more thorough (and slower) filtering one can perform is by filtering on the file content data. The ntfswalk engine will use deterministic signature analysis to determine whether a file meets one of the available categories. Only a few of the more common signatures were added to gena. These include: (a) LNK files, (b) Event logs, (c) Registry hives, (d) SQLite3 databases, and (e) executable/library files.
This category of options is broken up into four use-cases. The first set is whether one just wishes to scan (a) all records in the $MFT data or (b) just the deleted records in the $MFT data. The second set is whether one wants to scan (c) all the clusters of a volume or (d) just the deleted clusters of a volume. A fast option is to only scan the $MFT data (eg. data in inode 0), while the more complete option is to scan 'All clusters/All records'. The default option is the former (all records in the $MFT data section).
First, one must select a date range. If at least one date (of the 4 MACB standard information dates) for an inode is within this range, it will pass the test and be processed. Note all dates need to be expressed in UTC format and the date string needs to conform to: mm/dd/yyyy hh:mm:ss.
Each run will produce a results file. In addition to the normal data that is provided, other data can be inserted as well. In addition to changing the default hexadecimal output to base10 discussed previously, one can record the volume offset of the file record as well as record the cluster run.
If desiring to extract files to an archive directory, one can select the 'Copy Target Files' checkbox. If one is worried about copying malicious data from a target box, one can change the extension of the file extracted by appending *.bin to the file, so it is not accidently run should the file contain some malicious executable content. To do this, select the 'Append .bin as extension' checkbox.
Other options include copying the file as 'raw, which means copy the file as the cluster data is represented and don't translate the data (this applies to the files that use the native NTFS compression). Also, the 'raw' option will copy all allocated clusters associated with the file and not just the 'used' clusters. Therefore, the 'raw' option will yield the slack that is available. The last option is whether one wishes to include any sparse clusters as part of the file copy. The sparse clusters, while present on some files, are not backed by physical clusters and therefore ntfswalk will copy zeros in place of these sparse clusters, if this option is selected.
The extraction directory, for both the results file and any files extracted, is specified by the text entry box "Results Directory". The default directory is 'GENA_Results' under the current working directory. Whatever directory is inputted here will be used as the parent directory for archiving data. ntfswalk will create subdirectories under this parent directory to categorize different runs as well as categorize data within a run.
If you selected files to be extracted, then for each file extracted, the file name will consist of 3 items: (a) the timestamp of the last modify time, (b) the computed MD5 hash of the file and (c) the original filename.
The metadata in this tab includes an 'interpreted' version of the NTFS file record. This means that if gena sees a standard information attribute, it will show the properties and timestamps associated with this sort of NTFS attribute. Likewise, if gena sees a filename attribute, it will show the name, parent inode, and timestamps associated with this attribute. For data attributes, gena shows the size data (allocated vice file size) as well as any cluster runs that house the data. For other attributes where parsing is not done, but data is available, gena will show these attributes in a separate tab.
This tab shows a hexadecimal view of the volume under analysis. When synced with the selected inode from the directory tree, it will display the raw file record data
With this tab, one has the freedom to look at any volume offset that is desired. To do this, just select the 'Manually inputted' radio button and the offset field will be activated as well as the display button. One has the option of traversing the volume either by volume offset address or by volume logical cluster number.
When selecting any directory from the tree view, a wisp (Windows INDX Slack Parser) tab is displayed. Since gena can parse the INDX attribute to find the children associated to a directory, we incorporated a lite version of the TZWorks® wisp engine. This will allow gena not only to parse normal children, but also slack entries (or children that have been deleted). This is very useful in identifying deleted files after they have been wiped from the NTFS volume, since the file artifact could still be in the parent directory's INDX attribute. Each entry will have a name, file size, and a set of timestamps.
gena is very useful for displaying file data. Not only does it show the normal data (unnamed data stream) and the alternate (named) data stream, but it also shows the Bitmap data, Logged Stream data, and various types of INDX data. A good example is looking at the $Secure inode (#9). It has a number of attributes that contain data. Without going into too much detail, it contains: (a) an $SDS alternate data stream, (b) $SII index root and allocation attributes, (c) $SII bitmap attribute, (d) $SDH index root and allocation attributes, and (d) an $SDH bitmap attribute.
While some of the attributes might only have a little bit of data, or the data may reside as resident data within the file record itself, gena will display a separate tab for each data category.
Within each of these data tabs are options to go to a specific offset, find a string or pattern, or export the data to a separate file to analyze with another forensics tool. In some cases, it may be useful to represent the data as ASCII or Unicode, instead of hexadecimal, so those options are there as well.
The Utilities menu was meant to be a catch-all location for helpful utilities. In this case, there was only one we had time to add to this and it includes the option to scan the mounted drives on a live system.
When run, it will enumerate all the drives on a system and return a dialog box showing the drive, drive#, drive size, disk signature, partition information (offset, size and type), etc. This is helpful when using gena to analyze a drive, or image of a drive, with multiple partitions where one wishes to find the volume offset.
For this tool to work, the X Window System libraries are required for both Linux and macOS (they are not required for Windows). These libraries use the X11 protocol and graphics primitives to render the graphical user interface components. These libraries are common on Unix-like OS's.
If one is unfamiliar with X Windows or the libraries associated with it, one can download an installer package from XQuartz.org, which is an open-source effort to develop a version of the X Windows System that runs on Linux and macOS.
After the X11 libraries are installed, one needs to ensure they are running prior to running this tool.
This tool has authentication built into the binary. The primary authentication mechanism is the digital X509 code signing certificate embedded into the binary (Windows and macOS).
The other mechanism is the runtime authentication, which applies to all the versions of the tools (Windows, Linux and macOS). The runtime authentication ensures that the tool has a valid license. The license needs to be in the same directory of the tool for it to authenticate. Furthermore, any modification to the license, either to its name or contents, will invalidate the license.
The tools from TZWorks will output header information about the tool's version and whether it is running in limited, demo or full mode. This is directly related to what version of a license the tool authenticates with. The limited and demo keywords indicates some functionality of the tool is not available, and the full keyword indicates all the functionality is available. The lacking functionality in the limited or demo versions may mean one or all of the following: (a) certain options may not be available, (b) certain data may not be outputted in the parsed results, and (c) the license has a finite lifetime before expiring.