TZWorks LLC
System Analysis and Programming
www.tzworks.com


TZWorks®
Windows Push Notification Parser - wpn

(Version 0.29)



Information about our End User's License Agreements (EULAs)
for software on TZWorks, LLC Website www.tzworks.com

User Agreement

TZWorks LLC software and related documentation ("Software") is governed by separate licenses issued from TZWorks LLC. The User Agreement, Disclaimer, and/or Software may change from time to time. By continuing to use the Software after those changes become effective, you agree to be bound by all such changes. Permission to use the Software is granted provided that (1) use of such Software is in accordance with the license issued to you and (2) the Software is not resold, transferred or distributed to any other person or entity. Refer to your specific EULA issued to for your specific the terms and conditions. There are 3 types of licenses available: (i) for educational purposes, (ii) for demonstration and testing purposes and (iii) business and/or commercial purposes. Contact TZWorks LLC (info@tzworks.com) for more information regarding licensing and/or to obtain a license. To redistribute the Software, prior approval in writing is required from TZWorks LLC. The terms in your specific EULA do not give the user any rights in intellectual property or technology, but only a limited right to use the Software in accordance with the license issued to you. TZWorks LLC retains all rights to ownership of this Software.

Export Regulation

The Software is subject to U.S. export control laws, including the U.S. Export Administration Act and its associated regulations. The Export Control Classification Number (ECCN) for the Software is 5D002, subparagraph C.1. The user shall not, directly or indirectly, export, re-export or release the Software to, or make the Software accessible from, any jurisdiction or country to which export, re-export or release is prohibited by law, rule or regulation. The user shall comply with all applicable U.S. federal laws, regulations and rules, and complete all required undertakings (including obtaining any necessary export license or other governmental approval), prior to exporting, re-exporting, releasing, or otherwise making the Software available outside the U.S.

Disclaimer

The user agrees that this Software made available by TZWorks LLC is experimental in nature and use of the Software is at user's sole risk. The Software could include technical inaccuracies or errors. Changes are periodically added to the information herein, and TZWorks LLC may make improvements and/or changes to Software and related documentation at any time. TZWorks LLC makes no representations about the accuracy or usability of the Software for any purpose.

ALL SOFTWARE ARE PROVIDED "AS IS" AND "WHERE IS" WITHOUT WARRANTY OF ANY KIND INCLUDING ALL IMPLIED WARRANTIES AND CONDITIONS OF MERCHANTABILITY, FITNESS FOR ANY PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT SHALL TZWORKS LLC BE LIABLE FOR ANY KIND OF DAMAGE RESULTING FROM ANY CAUSE OR REASON, ARISING OUT OF IT IN CONNECTION WITH THE USE OR PERFORMANCE OF INFORMATION AVAILABLE FROM THIS SOFTWARE, INCLUDING BUT NOT LIMITED TO ANY DAMAGES FROM ANY INACCURACIES, ERRORS, OR VIRUSES, FROM OR DURING THE USE OF THE SOFTWARE.

Removal

The Software are the original works of TZWorks LLC. However, to be in compliance with the Digital Millennium Copyright Act of 1998 ("DMCA") we agree to investigate and disable any material for infringement of copyright. Contact TZWorks LLC at email address: info@tzworks.com, regarding any DMCA concerns.


About the wpn Tool (top)

Starting in Windows 8, Microsoft created the Windows Push Notification Services (WNS) to allow applications to send tile, toast, badge and raw updates. The tiles are updates on application tiles such as weather updates, stock updates, etc. The toasts are another word for popups that occur, for example, when a new device is plugged into the computer requesting what action to take place. The badges are small overlays on the tiles on the desktop used to show the status or act as an active counter. When looking at the internals of the notification data, one can see the time the notification was sent out, the expiry date, and the message that was displayed to the user.

Later versions of Windows 10 made changes to the internal store format of the notification records. The newer updated format makes use of the SQLite architecture to store the data. Similar to the older style database, each user account has its own database instance to record the users notifications; for newer Windows 10, it can be found in this location: C:\Users\<useracct>\AppData\Local\Microsoft\Windows\Notifications\wpndatabase.db.

The wpn tool only targets the newer format of the notifications in Windows 10 and not the older format that was started in Win8. The wpndatabase.db has a number of tables. From these tables, wpn looks at most of them, but primary targets two of them for the bulk of the information; the Notification and NotificationHandler tables. Collectively, they contain the requisite information to determine the application that initiated the notification, content of the notification, and when it was posted.


How to use this Tool (top)

Below is the menu with the various options available. Details of each option can be found here.

    Usage

      wpn -db <wpndatabase.db> [options]
      dir "C:\Users\*wpndatabase.db" /b /s | wpn -pipe [options]
      wpn -enumdir <folder> -num_subdirs <#> [options]

     Basic options
      -csv                            = output in CSV format
      -csvl2t                         = log2timeline output
      -bodyfile                       = sleuthkit output

     Additional options
      -all_fields                     = show all fields for -csv output
      -username <name>                = for -csvl2t output
      -hostname <name>                = for -csvl2t output
      -csv_separator "|"              = use a pipe char for csv separator
      -dateformat mm/dd/yyyy          = "yyyy-mm-dd" is the default
      -timeformat hh:mm:ss            = "hh:mm:ss" is the default
      -no_whitespace                  = remove whitespace between csv delimiter
      -pipe                           = pipe db's into tool for processing
      -quiet                          = no progress shown

     Experimental options
      -incl_slack                     = analyze slack space
      -carve                          = bypass SQLite lib during parse
      -parse_chunk                    = requires at least 1 db page

     Usage Examples
      wpn -db <db file>               = use only SQLite lib for parse
      wpn -db <db file> -incl_slack   = use SQLite and TZWorks libs for parse
      wpn -db <db file> -carve        = use only TZWorks libs for parse
      wpn -db <db page> -parse_chunk  = use only TZWorks libs for parse

The semantics to run wpn just requires one to use the -db option and pass in the path/file of the wpndatabase.db to parse. The parsed output will dump to the screen, unless one redirects the output to a file, as shown below:

    wpn64 -db  CollectionFolder\wpndatabase.db  > results.txt

The above command will only parse the more commonly used fields from the Notification and NotificationHandler tables. To dump all the fields, one can include the -all_fields option in the command above.

Processing Multiple Databases

Like many of the other TZWorks tools, one can process many database files in one pass, by using the -pipe option. If going this route, one does not use the -db option, but relies on standard input to identify which database files to parse. Below is an example:

    dir c:\users\*wpndatabase.db /b /s | wpn64 -csv > results.csv

Recovering Discarded Records (top)

The wpndatabase.db is a SQLite database. As such, the SQLite architecture uses a transactional system to ensure all the database commits are done in an atomic fashion. SQLite implements this via the use of temporary files to allow the system to rollback to a previous state should the database abruptly quit in the middle of a transaction due to an unexpected power failure or a crash of the application controlling the SQLite database. The supplemental files used for this database are: wpndatabase.db-wal and wpndatabase.db-shm.

The first supplemental file is the 'write-ahead' log (WAL) and the second is the 'shared memory' file (SHM). The WAL file acts as a buffer for new records or changes to existing records to be recorded prior to flushing them to the database. This is the supplemental file of interest of the two from a forensics standpoint. The WAL file, by its nature, can grow very large since it can record many transactions prior to flushing them to the database.

From a forensics standpoint, this offers the investigator additional data to analyze. Not only is the current record present, but potentially the previous version of the record as well. This occurs when the WAL contains the latest record prior to the commit (flush) while the database contains the older record.

In addition to the above behavior, it is common for the WAL file to save more than one transaction prior to the overall commit (flush to the database). Unfortunately, there is not a simple SQLite query to pull out invalid older records or examination of slack space. To build up this history, consisting of previous records, one needs to reconstruct them. This requires traversing the SQLite file internals to locate and extract invalid records, data in slack space, etc, and then try to match the data to the proper table schema available.

Fortunately, wpn does a best effort attempt to do this with the experimental -incl_slack option. This option will look at invalid records and slack space within valid pages, in both the database and the WAL file to try to recover any records it finds. It goes without saying, that anytime a parser tries to reconstruct records from corrupted data, it should be cause for concern. That is why this option is still experimental. Consequently, using this option could cause the parser to crash if it tries to reconstruct a record that causes an out-of-bound condition. More datasets are required with differing boundary conditions for this option in the tool to come out of the experimental category.


Bypassing the Embedded SQLite library (top)

The wpn tool has the SQLite library embedded into the binary. More information about this is discussed in the section "Use of the SQLite Library". Sometimes, however, one may not wish to use the SQLite library for analyzing the records, so an option was added to bypass the SQLite library and use the TZWorks internal SQLite algorithms to parse the database. This functionality can be invoked one of two ways: (a) with the -carve option or (b) the -parse_chunk option. Out of the two options, one should opt for the first, -carve option. This option will try to traverse the database (even corrupted ones) and pull out all the same information as if using the normal SQLite library. The difference here is the -carve option is more immune to database corruption than the SQLite library is.

The purpose for the second option -parse_chunk, is to go a step further and operate on only a subset of the database. More specifically, if at least a page of the database is available, this option will try to pull out any Notification, NotificationHandler and WNSPushChannel records it finds. The limitation of this option is the data in the record must be in the page (and data that goes across to other pages will be truncated).


Use of the SQLite library (top)

The wpndatabase.db is a SQLite database. For the purposes of the wpn tool we statically link in the SQLite library to ensure the tool has minimal dependencies. The source code for the SQLite library is an amalgamation of the SQLite 'C' source files, version 3.32.3. More information about SQLite, the documentation and the source code can be seen at the official SQLite website [http://www.sqlite.org].

Normally when we build a tool to parse a raw artifact, we prefer not to use outside libraries, however, in this case, the SQLite library has an option to open a SQLite database in 'read-only' mode. From the testing done and from the documentation, it appears this is acceptable for this release. As an experimental option, we incorporated our internal library to assist parsing the slack and free space of a SQLite database. This gives the wpn tool the ability to recover additional records that have been deleted or overwritten. More testing in this area needs to be done to ensure the record recovery is robust to malformed records that have been overwritten.


List of options (top)

Option Description
-db Specifies which database file to act on. The format is: -db <activitiescache db to parse>.
-csv Outputs the data fields delimited by commas. Since filenames can have commas, to ensure the fields are uniquely separated, any commas in the filenames get converted to spaces.
-csvl2t Outputs the data fields in accordance with the log2timeline format.
-bodyfile Outputs the data fields in accordance with the 'body-file' version3 specified in the SleuthKit. The date/timestamp outputted to the body-file is in terms of UTC. So if using the body-file in conjunction with the mactime.pl utility, one needs to set the environment variable TZ=UTC.
-base10 Ensure all size/address outputs are displayed in base-10 format versus hexadecimal format. Default is hexadecimal format.
-all_fields Show all fields available for CSV output.
-username Option is used to populate the output records with a specified username. The syntax is -username <name to use> .
-hostname Option is used to populate the output records with a specified hostname. The syntax is -hostname <name to use>.
-pipe Used to pipe files into the tool via STDIN (standard input). Each file passed in is parsed in sequence.
-enumdir Experimental. Used to process files within a folder and/or subfolders. Each file is parsed in sequence. The syntax is -enumdir <"folder"> -num_subdirs <#>.
-filter Filters data passed in via STDIN via the -pipe option. The syntax is -filter <"*.ext | *partialname* | ...">. The wildcard character '*' is restricted to either before the name or after the name.
-no_whitespace Used in conjunction with -csv option to remove any whitespace between the field value and the CSV separator.
-csv_separator Used in conjunction with the -csv option to change the CSV separator from the default comma to something else. Syntax is -csv_separator "|" to change the CSV separator to the pipe character. To use the tab as a separator, one can use the -csv_separator "tab" OR -csv_separator "\t" options.
-dateformat Output the date using the specified format. Default behavior is -dateformat "yyyy-mm-dd". Using this option allows one to adjust the format to mm/dd/yy, dd/mm/yy, etc. The restriction with this option is the forward slash (/) or dash (-) symbol needs to separate month, day and year and the month is in digit (1-12) form versus abbreviated name form.
-quiet Show no progress during the parsing operation
-incl_slack Experimental option to look at slack space free blocks to see if any records are present.
-carve Experimental option. Bypass the SQLite embedded library and parse using TZWorks internal algorithms. This is for the situation where the database to be parsed is corrupted and the SQLite library has trouble parsing it.
-parse_chunk Experimental option. Given a portion of the database (at least 1 page), this option will examine the data to see if any records exist and parse out the contents.
-utf8_bom All output is in Unicode UTF-8 format. If desired, one can prefix an UTF-8 byte order mark to the CSV output using this option.

Known Issues (top)

For csv (comma separated values) output, there are restrictions in the characters that are outputted. Since commas are used as a separator, any data that had a comma in its name is changed to a semicolon. For the default (non-csv) output no changes are made to the data.

(Windows only) When processing filenames with characters that are not ASCII, one option is to change the code page of the command window from the default code page to UTF-8. This can be done via the command:

    chcp 65001

If running on a Windows operating system post XP and running under admin permissions, any network shares established prior as a regular (non-admin) user, will be isolated from other accounts (including the admin account). This problem occurs because User Account Control (UAC) treats members of the Administrators group as standard users. Therefore, network shares that are mapped by logon scripts are shared with the standard user access token instead of with the full administrator access token.


Authentication and License File (top)

This tool has authentication built into the binary. The primary authentication mechanism is the digital X509 code signing certificate embedded into the binary (Windows and macOS).

The other mechanism is the runtime authentication, which applies to all the versions of the tools (Windows, Linux and macOS). The runtime authentication validates that the tool has a valid license. The license needs to be in the same directory of the tool for it to authenticate. Furthermore, any modification to the license, either to its name or contents, will invalidate the license.

Limited versus Demo versus Full in the tool's output banner

The tools from TZWorks will output header information about the tool's version and whether it is running in limited, demo or full mode. This is directly related to what version of a license the tool authenticates with. The limited and demo keywords indicates some functionality of the tool is not available, and the full keyword indicates all the functionality is available. The lacking functionality in the limited or demo versions may mean one or all of the following: (a) certain options may not be available, (b) certain data may not be outputted in the parsed results, and (c) the license has a finite lifetime before expiring.


Version history (top)


References (top)

  1. Windows Push Notification Services (WNVS) Overview; https://docs.microsoft.com/en-us/windows/uwp/design/shell/tiles-and-notifications/windows-push-notification-services--wns--overview
  2. Badge Notifications for UWP apps; https://docs.microsoft.com/en-us/windows/uwp/design/shell/tiles-and-notifications/badges
  3. Toast Content; https://docs.microsoft.com/en-us/windows/uwp/design/shell/tiles-and-notifications/adaptive-interactive-toasts
  4. Raw Notification Overview; https://docs.microsoft.com/en-us/windows/uwp/design/shell/tiles-and-notifications/raw-notification-overview
  5. SQLite library statically linked into tool [Amalgamation of many separate C source files from SQLite version 3.32.3].
  6. SQLite documentation [http://www.sqlite.org]