TZWorks LLC
System Analysis and Programming
www.tzworks.com


TZWorks®
Portable Executable Scanner - pescan

(Version 0.65)



Information about our End User's License Agreements (EULAs)
for software on TZWorks, LLC Website www.tzworks.com

User Agreement

TZWorks LLC software and related documentation ("Software") is governed by separate licenses issued from TZWorks LLC. The User Agreement, Disclaimer, and/or Software may change from time to time. By continuing to use the Software after those changes become effective, you agree to be bound by all such changes. Permission to use the Software is granted provided that (1) use of such Software is in accordance with the license issued to you and (2) the Software is not resold, transferred or distributed to any other person or entity. Refer to your specific EULA issued to for your specific the terms and conditions. There are 3 types of licenses available: (i) for educational purposes, (ii) for demonstration and testing purposes and (iii) business and/or commercial purposes. Contact TZWorks LLC (info@tzworks.com) for more information regarding licensing and/or to obtain a license. To redistribute the Software, prior approval in writing is required from TZWorks LLC. The terms in your specific EULA do not give the user any rights in intellectual property or technology, but only a limited right to use the Software in accordance with the license issued to you. TZWorks LLC retains all rights to ownership of this Software.

Export Regulation

The Software is subject to U.S. export control laws, including the U.S. Export Administration Act and its associated regulations. The Export Control Classification Number (ECCN) for the Software is 5D002, subparagraph C.1. The user shall not, directly or indirectly, export, re-export or release the Software to, or make the Software accessible from, any jurisdiction or country to which export, re-export or release is prohibited by law, rule or regulation. The user shall comply with all applicable U.S. federal laws, regulations and rules, and complete all required undertakings (including obtaining any necessary export license or other governmental approval), prior to exporting, re-exporting, releasing, or otherwise making the Software available outside the U.S.

Disclaimer

The user agrees that this Software made available by TZWorks LLC is experimental in nature and use of the Software is at user's sole risk. The Software could include technical inaccuracies or errors. Changes are periodically added to the information herein, and TZWorks LLC may make improvements and/or changes to Software and related documentation at any time. TZWorks LLC makes no representations about the accuracy or usability of the Software for any purpose.

ALL SOFTWARE ARE PROVIDED "AS IS" AND "WHERE IS" WITHOUT WARRANTY OF ANY KIND INCLUDING ALL IMPLIED WARRANTIES AND CONDITIONS OF MERCHANTABILITY, FITNESS FOR ANY PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT SHALL TZWORKS LLC BE LIABLE FOR ANY KIND OF DAMAGE RESULTING FROM ANY CAUSE OR REASON, ARISING OUT OF IT IN CONNECTION WITH THE USE OR PERFORMANCE OF INFORMATION AVAILABLE FROM THIS SOFTWARE, INCLUDING BUT NOT LIMITED TO ANY DAMAGES FROM ANY INACCURACIES, ERRORS, OR VIRUSES, FROM OR DURING THE USE OF THE SOFTWARE.

Removal

The Software are the original works of TZWorks LLC. However, to be in compliance with the Digital Millennium Copyright Act of 1998 ("DMCA") we agree to investigate and disable any material for infringement of copyright. Contact TZWorks LLC at email address: info@tzworks.com, regarding any DMCA concerns.


About the pescan Tool (top)

pescan is a command line tool to scan portable executable (PE) files to identify how they were constructed. Various metadata is displayed, identifying items such as:

pescan also has the ability to consume any standard PEiD file containing signatures and use it during the scanning process. If a PEiD file is supplied, additional statistics will be displayed based on the PEiD signatures found.

Internally, pescan uses the same PE libraries that are used in pe_view to analyze the PE internals. This allows pescan to be cross platform, and hence, there are compiled versions for Windows, Linux and OS-X, both in 32 and 64 bit binaries.

To use this tool, an authentication file is required to be in the same directory as the binary in order for the tool to run.


How to use this Tool (top)

One can display the menu options by typing in the executable name without parameters. Below is the menu with the various options. Details of each option can be found here.

    Usage

      pescan <pefile> [options]
      pescan <pefile> -peid <peid file>  > results.txt
      pescan <pefile> -anomalies > results.txt
      dir *.exe /b /s | pescan -pipe -csv > results.csv
      pescan -enumdir <folder> -num_subdirs <#> [options]

     Basic options
      -csv                      = output in comma separated value format
      -out <results file>       = send output to a file
      -peid <peid file>         = use this PEiD file during scan (see readme)

     Additional options
      -pipe                     = use stdin to pipe in files to parse
      -hostname <name>          = display hostname in output
      -base10                   = output in base10 vice hex [only with -csv]
      -string_resource          = extract string resources [not avail w/ -csv]
      -msg_table                = extract any msg tables [not avail w/ -csv]
      -wevt_temp                = extract eventlog templates [not avail w/ -csv]
      -dump_entrypt <# bytes>   = dump the bytes at entrypoint [up 0x50 bytes]
      -md5                      = output MD5 hash of binary
      -sha1                     = output SHA1 hash of binary
      -appid                    = compute AppID [one of several possible ones]
      -dateformat mm/dd/yyyy    = "yyyy-mm-dd" is the default
      -timeformat hh:mm:ss      = "hh:mm:ss" is the default
      -pair_datetime            = combine date/time into 1 field for csv
      -no_whitespace            = remove whitespace between csv delimiter
      -quiet                    = don't display status during run
      -csv_separator "|"        = use a pipe char for csv separator
      -anomalies                = enum PE attributes differing from norm
      -rating <min rating>      = display results for ratings at or above
      -filter <*partial*|*.ext> = filters stdin data from -pipe option</pre>

The command line options include:

1. To dump stats of an individual PE file, use:

pescan <pefile>     

2. To incorporate output from the PEiD signature type file, use the following option:

pescan <pefile> -peid <peid file>

3. To process many PE files, one can pass target files via standard input into pescan. To use this option, one needs to specify the '-pipe' option. Also, there are other various options available when piping in files via the std input. This includes: (a) incorporating a PEiD signature file, (b) outputting the results in comma separated value (csv) format, and (c) specifying an output file.

<std input> | pescan -pipe [-peid <peid file>] [-csv] [-out <results file>] [-quiet]

Below is an example of piping in a directory of files into pescan. The output will yield a csv file that can be opened in excel or other tool to view csv files.

    For windows:
dir c:\windows\system32\* /b /s | pescan -pipe -csv -out results.csv
    
    For Linux or Mac
find /home -name *.exe -type f | ./pe_view -pipe -csv -out ./results.csv

The output shows 2 types of data. (1) The first type of output consists of the basic stats of the PE file being analyzed. This includes: type of PE file, compile time, entrypoint rva, imagebase, company name (if present), linker version, minimum version of windows required to run, etc. (2) The second type of output tries to infer something about the PE internals. For example, if one uses PEiD signatures, it will try to identify how the PE was compiled or packed. Independently, it will also try to determine if the construction of the PE file looks like something that would warrant offline analysis. For example, if pescan determined that a file is packed and/or has an embedded executable file inserted into one of the sections, or if there is a section with high entropy, it will state that. Finally, pescan will come up with a numeric metric to determine how abnormal the PE internals are. Since this metric is subjective and will undoubtedly change as more heuristics are drawn upon in future versions, it should not be used to infer anything negative about the PE file analyzed. It only reflects how it compares to most PE files distributed as part of the Windows operating system.


Handling Volume Shadow Copies (top)

For starters, to access Volume Shadow copies, one needs to be running with administrator privileges. Also, it is important to note that, Volume Shadow copies as is discussed here, only applies to Windows Vista, Win7, Win8, and beyond. It does not apply to Windows XP.

To make it easier with the syntax, we've built in a shortcut syntax in order to more easily access a specific file in a specified Volume Shadow copy, via the %vss% keyword. This internally gets expanded into \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy. Thus, to access index 1 of the volume shadow copy, one would combine the keyword and index, index, (eg. %vss%1) in front of the normal path of the file. For example, to access notepad located in the Windows\System32 directory from the HarddiskVolumeShadowCopy4, the following syntax can be used:

    pescan64 %vss%4\windows\system32\notepad.exe

    File selected:  \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy4\windows\system32\notepad.exe
    Company name:   Microsoft Corporation
    Compile date:   07/13/2009 23:56:35 [UTC]
    Create date:      [UTC]
    Access date:      [UTC]
    Modify date:      [UTC]
    File size:      0x00193536 [193536]
    PE type:        64 bit - exe
    Linker version: 9.0
    Min OS version: Win7 or Win08/R2
    Entrypoint VA:  0x00003570 [13680] - FileAddr: 0x00002b70 [11120]
    Imagebase:      0x0000000100000000 [4294967296]
    Checksum:       Validated

To determine which indexes are available from the various Volume Shadows, one can use the Windows built-in utility vssadmin, as follows:

    vssadmin list shadows

    -- or to filter out extraneous detail --
    
    vssadmin list shadows | find /i "volume"

While the amount of data can be voluminous from that above command, the keywords one needs to look for are names that look like this:


    Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy1
    Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy2
    ...

From the example above, notice the number after the word HarddiskvolumeShadowCopy. This would be the number that is appended to the %vss% keyword.


PEiD Signatures (top)

The PEiD reference below, has some links to locations where one can obtain a PEiD signature file or get more information about its format. Below are the rules for how pescan parses PEiD signature files, should you wish to generate your own signatures.

  1. Signature file is a text document
  2. Each line is parsed using the following rules.
    1. Each line is parsed using the following rules.
    2. General Rules.
      ';'
      '[' ']'
      'signature ='
      'ep_only ='
      '??'
    3. The name of the signature is encompassed in square brackets []. The opening square bracket should start at the beginning of the line.
      1. Each line is parsed separately
      2. When a line that starts with a semi-colon ';', then the entire line is ignored and used for comments
      3. A blank line is ignored
    4. The signature is preceded by the keyword 'signature ='. The signature is represented by hexadecimal bytes where each byte is separated by spaces. Wildcard bytes are represented by '??'
    5. If the signature is meant to be scanned at the PE entrypoint, it will be designated as 'ep_only = true'; otherwise if the signature is to be search throughout the file contents, it will be designated as 'ep_only = false;
    6. For version this version of pe_view, only the signatures that are specified as 'ep_only = true' are used. Later versions of pe_view may incorporate entire PE scanning.
  3. Below is an example of a signature:
    [Microsoft Visual C++ 8]
    signature = E8 ?? ?? 00 00 E9 ?? ?? FF FF
    ep_only = true
        

List of options (top)

Option Description
-csv Outputs the data fields delimited by commas. Since filenames can have commas, to ensure the fields are uniquely separated, any commas in the filenames get converted to spaces.
-out Send the output to the specified file. The syntax is: -out <results file>.
-peid Use this option to pass a specified PEiD file during the scan. The syntax is: -peid <peid file> [-section | -full]. The default option is to scan only the PE entry point for the PEiD patterns. The -section sub-option allows non-entry point signatures identified in the PEiD file to be scanned throughout the PE section containing the entry point. Finally, the -full sub-option allows the non-entry point signatures to be scanned anywhere in the PE file. The default option is the fastest, followed by the -section sub-option. The -full sub-option is the slowest.
-hostname Option is used to populate the output records with a specified hostname. The syntax is -hostname <name to use>.
-pipe Used to pipe files into the tool via STDIN (standard input). Each file passed in is parsed in sequence.
-enumdir Experimental. Used to process files within a folder and/or subfolders. Each file is parsed in sequence. The syntax is -enumdir <"folder"> -num_subdirs <#>.
-filter Filters data passed in via stdin via the -pipe option. The syntax is -filter <"*.ext | *partialname* | ...">. The wildcard character '*' is restricted to either before the name or after the name. An example of using the filter to screen out just PE files woudl the following: dir C:\Windows\System32 /b /s | pescan -pipe -filter "*.exe|*.dll|*.sys"
-msg_table Extract any message tables that are embedded into the PE file. This option is not available with the -csv option.
-wevt_temp Extract any eventlog templates that are embedded into the PE file. This option is not available with the -csv option.
-base10 Ensure all size/address outputs are displayed in base-10 format versus hexadecimal format. Default is hexadecimal format. This option only works with the CSV option.
-dump_entrypt Dump the specified number of bytes starting at the PE's entry point. This dump is limited to the first 0x50 bytes. The syntax is: -dump_entrypt <# bytes>
-md5 Output the MD5 hash of the PE file.
-sha1 Output the SHA1 hash of the PE file
-appid Compute the Application Identifier (AppID) that can be used in Jump Lists. (note) AppID's can be manually set by the application itself to be any arbitrary value. This is just an implementation of the algorithm used to compute the AppID using the application's path.
-anomalies Look for anomalous PE construction and packing. Anomalous here is defined to be those items that are differing from the normal. High scores outputted here warrant deeper analysis of the PE file.
-rating Similar to the -anomalies option in functionality, but only output those records with a score at or above the specified rating. The syntax is: -rating <min rating>.
-no_whitespace Used in conjunction with -csv option to remove any whitespace between the field value and the CSV separator.
-csv_separator Used in conjunction with the -csv option to change the CSV separator from the default comma to something else. Syntax is -csv_separator "|" to change the CSV separator to the pipe character. To use the tab as a separator, one can use the -csv_separator "tab" OR -csv_separator "\t" options.
-dateformat Output the date using the specified format. Default behavior is -dateformat "yyyy-mm-dd". Using this option allows one to adjust the format to mm/dd/yy, dd/mm/yy, etc. The restriction with this option is the forward slash (/) or dash (-) symbol needs to separate month, day and year and the month is in digit (1-12) form versus abbreviated name form.
-timeformat Output the time using the specified format. Default behavior is -timeformat "hh:mm:ss.xxx" One can adjust the format to microseconds, via "hh:mm:ss.xxxxxx" or nanoseconds, via "hh:mm:ss.xxxxxxxxx", or no fractional seconds, via "hh:mm:ss". The restrictions with this option is that a colon (:) symbol needs to separate hours, minutes and seconds, a period (.) symbol needs to separate the seconds and fractional seconds, and the repeating symbol 'x' is used to represent number of fractional seconds. (Note: the fractional seconds applies only to those time formats that have the appropriate precision available. The Windows internal filetime has, for example, 100 nsec unit precision available. The DOS time format and the UNIX 'time_t' format, however, have no fractional seconds). Some of the times represented by this tool may use a time format without fractional seconds and therefore will not show a greater precision beyond seconds when using this option.
-pair_datetime Output the date/time as 1 field vice 2 for csv option
-quiet This option suppresses status output as each file is processed.
-utf8_bom All output is in Unicode UTF-8 format. If desired, one can prefix an UTF-8 byte order mark to the CSV output using this option.

CSV field definitions (top)

Field Definition
compiled Embedded date PE was compiled
time-UTC (after compiled) Embedded time PE was compiled
created Target PE file create date (based on filesystem reporting). Note: for non Windows builds (eg Linux and Mac OS-X), this date is the Metadata change date of the file.
time-UTC (after created) Target PE file create time (based on filesystem reporting). Note: for non Windows builds (eg Linux and Mac OS-X), this time is the Metadata change time of the file.
access Target PE file access date (based on filesystem reporting)
time-UTC (after access) Target PE file access time (based on filesystem reporting)
modify Target PE file modify date (based on filesystem reporting)
time-UTC (after modify) Target PE file modify time (based on filesystem reporting)
size Size of PE file
type Type of PE file: exe, sys, dll
cpu Target CPU (32 or 64 bit)
linker Linker used
min OS Specified minimum OS
entry rva Specified relative virutal address of entry point
entryfaddr Computed File address of entry point
imagebase Specified image base
cert Does PE file have a digital certificated embedded
company Name of company embedded into file
chksum Does PE file have a checksum embedded.
rating Empirical rating based on how normal the PE file is. Low numbers are more normal. High numbers have attributes that are not usually found in a default install of a Windows box.
notes Explanation of the ratings

Known Issues (top)

  1. If running on a Windows operating system post XP and running under admin permissions, any network shares established prior as a regular (non-admin) user, will be isolated from other accounts (including the admin account). This problem occurs because User Account Control (UAC) treats members of the Administrators group as standard users. Therefore, network shares that are mapped by logon scripts are shared with the standard user access token instead of with the full administrator access token.

Authentication and License File (top)

This tool has authentication built into the binary. The primary authentication mechanism is the digital X509 code signing certificate embedded into the binary (Windows and macOS).

The other mechanism is the runtime authentication, which applies to all the versions of the tools (Windows, Linux and macOS). The runtime authentication ensures that the tool has a valid license. The license needs to be in the same directory of the tool for it to authenticate. Furthermore, any modification to the license, either to its name or contents, will invalidate the license.

Limited versus Demo versus Full in the tool's output banner

The tools from TZWorks will output header information about the tool's version and whether it is running in limited, demo or full mode. This is directly related to what version of a license the tool authenticates with. The limited and demo keywords indicates some functionality of the tool is not available, and the full keyword indicates all the functionality is available. The lacking functionality in the limited or demo versions may mean one or all of the following: (a) certain options may not be available, (b) certain data may not be outputted in the parsed results, and (c) the license has a finite lifetime before expiring.


Version history (top)


References (top)

  1. Microsoft Portable Executable and Common Object File Format Specification.
  2. An In-Depth Look into the Win32 Portable Executable File Format, by Matt Pietrik, MSDN Magazine.
  3. Wikipedia, the free encyclopedia. PE format
  4. FOX-toolkit version 1.6.43 [5] PEiD references: PEiD, PEiD Forum, Example userdb.txt