TZWorks LLC
System Analysis and Programming
www.tzworks.com
(Version 0.64)
TZWorks LLC software and related documentation ("Software") is governed by separate licenses issued from TZWorks LLC. The User Agreement, Disclaimer, and/or Software may change from time to time. By continuing to use the Software after those changes become effective, you agree to be bound by all such changes. Permission to use the Software is granted provided that (1) use of such Software is in accordance with the license issued to you and (2) the Software is not resold, transferred or distributed to any other person or entity. Refer to your specific EULA issued to for your specific the terms and conditions. There are 3 types of licenses available: (i) for educational purposes, (ii) for demonstration and testing purposes and (iii) business and/or commercial purposes. Contact TZWorks LLC (info@tzworks.com) for more information regarding licensing and/or to obtain a license. To redistribute the Software, prior approval in writing is required from TZWorks LLC. The terms in your specific EULA do not give the user any rights in intellectual property or technology, but only a limited right to use the Software in accordance with the license issued to you. TZWorks LLC retains all rights to ownership of this Software.
The Software is subject to U.S. export control laws, including the U.S. Export Administration Act and its associated regulations. The Export Control Classification Number (ECCN) for the Software is 5D002, subparagraph C.1. The user shall not, directly or indirectly, export, re-export or release the Software to, or make the Software accessible from, any jurisdiction or country to which export, re-export or release is prohibited by law, rule or regulation. The user shall comply with all applicable U.S. federal laws, regulations and rules, and complete all required undertakings (including obtaining any necessary export license or other governmental approval), prior to exporting, re-exporting, releasing, or otherwise making the Software available outside the U.S.
The user agrees that this Software made available by TZWorks LLC is experimental in nature and use of the Software is at user's sole risk. The Software could include technical inaccuracies or errors. Changes are periodically added to the information herein, and TZWorks LLC may make improvements and/or changes to Software and related documentation at any time. TZWorks LLC makes no representations about the accuracy or usability of the Software for any purpose.
ALL SOFTWARE ARE PROVIDED "AS IS" AND "WHERE IS" WITHOUT WARRANTY OF ANY KIND INCLUDING ALL IMPLIED WARRANTIES AND CONDITIONS OF MERCHANTABILITY, FITNESS FOR ANY PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT SHALL TZWORKS LLC BE LIABLE FOR ANY KIND OF DAMAGE RESULTING FROM ANY CAUSE OR REASON, ARISING OUT OF IT IN CONNECTION WITH THE USE OR PERFORMANCE OF INFORMATION AVAILABLE FROM THIS SOFTWARE, INCLUDING BUT NOT LIMITED TO ANY DAMAGES FROM ANY INACCURACIES, ERRORS, OR VIRUSES, FROM OR DURING THE USE OF THE SOFTWARE.
The Software are the original works of TZWorks LLC. However, to be in compliance with the Digital Millennium Copyright Act of 1998 ("DMCA") we agree to investigate and disable any material for infringement of copyright. Contact TZWorks LLC at email address: info@tzworks.com, regarding any DMCA concerns.
Normally, when one is debugging a Windows application with WinDbg (or other debugger), the debugger will automatically pull the appropriate metadata from desired the PE file and then contact Microsoft's symbol server via the Internet to fetch the public debug symbols for that file. To do this, one normally points the debugger to the Microsoft symbol server website, and assuming the debugger has an Internet connection, the process of downloading symbols is automatic.
For the cases where the machine running the debugger has no Internet connection (eg. for a closed network), one needs to either setup an internal symbol server with the requisite symbols or find a way to separately pull the appropriate Microsoft symbols from the Internet and sneaker-net them over to your closed network. This tool is geared towards helping one perform the latter option.
Since this is a manual process, it is broken into two steps. The first step requires one to extract some necessary metadata from each PE file that is desired to be analyzed in the debugger. For the newer Windows operating systems, this includes reading the debug section of the PE file and extracting the unique GUID and age value. sf has an option to extract this data. The second step is to take the results of the metadata extraction (in the form of a results file) and transport it over to a machine with an Internet connection. This metadata file created in the first step is then used as an input parameter to sf to query the Microsoft symbol server to retrieve the appropriated PDB files. sf will package the PDB files received into a single cabinet (.cab) file. This allows one to easily transport the symbols back to the host machine running the debugger.
While there are a number of formats available for debug symbols, sf only supports the later Microsoft format that uses a GUID and age value for the PDB signature.
To use this tool, an authentication file is required to be in the same directory as the binary in order for the tool to run.
One can display the menu options by typing in the executable name without parameters. Below is the menu with the various options. Details of each option can be found here.
sf - full ver: 0.63; Copyright (c) TZWorks LLC Usage Options to extract metadata from pe files sf -guid <pe file> = extract from a single PE file sf -guid -min = extract from common PE files Options to retrieve PDB files from Microsoft's symbol server sf -pullpdb_min <output cab file> = get PDB syms for common system files sf -pullpdbx <input metadata file> <output cab file> sf -pullpdb <path /pefile> = get PDB syms for specified PE file Option to pipe in path/filename and extract metadata dir c:\windows\system32\*.exe /b /s | sf -guid -pipe -filter <*partial*|*.ext> = filters stdin data from -pipe option sf -enumdir <folder> -num_subdirs <#> [options] Output options -outdir <path to store pdb files> -out <name to store a single pdb file> Experimental options sf -pullpdb_ex -name <name> -guid <guid> -age <age>
Option A. To extract individual PE file metadata, use the following option and redirect the output to a results file:
sf -guid <path/filename> > results.txt
Option B. To extract the GUID and age info from minimum set of common files, use the following option and redirect the output to a results file:
sf -guid -min > results.txt
Option C. To extract all the executable PE files from a specified directory, use this option:
dir C:\windows\system32\*.exe /b /s | sf -pipe -guid > results.txt
Option D. If you wish to customize what files to extract the metadata on, then generate a list of files to extract metadata on and pipe the contents of the file into sf. See below:
i. Generate a text file that will serve as the input file to sf. This file just lists the PE file you wish to get PDB files on. Below is a sample input.txt file contents.
C:\Windows\system32\ntoskrnl.exe C:\Windows\system32\ntkrnlpa.exe C:\Windows\system32\hal.dll c:\windows\system32\ntdll.dll C:\Windows\system32\user32.dll C:\Windows\system32\shell32.dll C:\Windows\system32\comctl32.dll C:\Windows\system32\gdi32.dll
ii. After generating the above file, pipe the file contents into sf with the -guid option and capture the output into a results file, eg.
type input.txt | sf -pipe -guid > results.txt
a. Extract the metadata into a results file using any of the options outlined in step 1 above.
b. Sneaker net the metadata file over to a box that has an Internet connection.
c. Use the sf tool to retrieve the symbols specified in the metadata file. The resulting PDB files will be stored into a cabinet file (cab). To do this, use the following:
sf -pullpdbx <input_file_from_step_1> <output_file.cab>
d. After the cabinet file is created with the appropriate PDB files sneaker net it over to the host machine that is running the debugger, and extract the cabinet file into your private symbols directory.
a. Retrieving PDB files for a machine that is already connected to the Internet. The resulting PDB file will be stored in the current working directory.
sf -pullpdb <path/pefile>
b. Retrieving the common PDB files for a machine that has Internet connectivity. The output file is a cabinet file containing the PDBs.
sf -pullpdb_min <output_file.cab>
Option | Description |
---|---|
-guid | Extract the debug GUID and associated metadata, so that one can identify the proper public symbols to use when pulling PDB files down from a symbol server. The syntax is: -guid -min to pull metadata from a canned set of common Windows system files, or -guid <pe file> to pull metadata from a specific PE file. |
-pullpdb_min | Extracts a canned set of common Windows system files, contacts the Microsoft public symbol server and builds a CAB file consisting of the proper public debug (PDB) files. The syntax is: -pullpdb_min <output cab file>. |
-pullpdbx | Given an input metadata file (obtained by using the -guid option above and redirecting the output to a file), contact the Microsoft symbol server and build a CAB file consisting of the proper public debug (PDB) files. The computer needs to have an active Internet connection, so this app can contact the symbol server. The syntax is: -pullpdbx <input metadata file> <output cab file>. |
-pullpdb | Extract the specified PE file's public debug symbols (PDB) from the Microsoft symbol server. The computer needs to have an active Internet connection, so this app can contact the symbol server. The syntax is: -pullpdb <PE path/file>. |
-pullpdb_ex | Extract the specified PE file's public debug symbols (PDB) from the Microsoft symbol server. Instead of passing in the path/file of the PE file, one passes in the name of the PDB file (-name <pdb name>), the GUID (-guid <guid>) and the age (-age <age>). All parameters must be provided and match that of the underlying PE file. The syntax is: -pullpdb_ex -name <pdb name> -guid <guid> -age <age>. |
-pipe | Used to pipe in desired PE files into the tool via STDIN (standard input). Each path/PE file passed in is parsed in sequence. This option is used in conjunction with the -guid option. |
-enumdir | Experimental. Used to process files within a folder and/or subfolders. Each file is parsed in sequence. The syntax is -enumdir <"folder"> -num_subdirs <#>. |
-filter | Filters data passed in via stdin via the -pipe option. The syntax is -filter <"*.ext | *partialname* | ...">. The wildcard character '*' is restricted to either before the name or after the name. |
-outdir | Folder to output the retrieved final PDB file. |
-out | Location to output the retrieved final PDB file. |
Starting with version 0.46, the sf tool can make use of the OpenSSL libraries to handle SSL/TLS communications. This was added since the Microsoft symbol server started requiring symbol downloads to be performed in a secure manner.
Four libaries (or DLLs) were created. Two for 32 bit and two for 64 bit. The appropriate set of libraries (32 bit) or (64 bit) need to reside in the same directory of the sf tool. Without these files, the sf tool will only work in a non-SSL/TLS manner.
// 32 bit versions (provided for the sf 32 bit version of the tool) libcrypto-1_1_win32.dll libssl-1_1_win32.dll // 64 bit versions (provided for the sf 64 bit version of the tool) libcrypto-1_1_win64.dl libssl-1_1_win64.dll
The above libraries are compiled versions of the source code downloaded from the OpenSSL organization's website. The build specifics include Visual Studio's v100 platform toolset where the the Microsoft's runtime library is statically linked in.The OpenSSL organization and their contributors retain all copyrights assoicated with the source code associated with the above libraries.
For those interested in reviewing the source code, it can be downloaded at the website [https://www.openssl.org]. In accordance with the permissions from the OpenSSL organization to use this set of libraries, a licensing statement is also distributed with the binaries and can be reviewed from the file: openssl.license.txt.
This tool has authentication built into the binary. The primary authentication mechanism is the digital X509 code signing certificate embedded into the binary.
The other mechanism is the runtime authentication, which ensures that the tool has a valid license. The license needs to be in the same directory of the tool for it to authenticate. Furthermore, any modification to the license, either to its name or contents, will invalidate the license.
The tools from TZWorks will output header information about the tool's version and whether it is running in limited, demo or full mode. This is directly related to what version of a license the tool authenticates with. The limited and demo keywords indicates some functionality of the tool is not available, and the full keyword indicates all the functionality is available. The lacking functionality in the limited or demo versions may mean one or all of the following: (a) certain options may not be available, (b) certain data may not be outputted in the parsed results, and (c) the license has a finite lifetime before expiring.