NTFS

NTFS uses several system files to maintain the file system. NTFS reserves the first 24 records for specific use, and the first 12 are used by these system files.

Rec No.

Filename

Description

$MFT

Master file table - database that tracks every file in the volume

$MFTMIRR

a backup copy of the first four records

$LogFile

Transaction logging file

$AttrDef

NTFS attribute definitions

"."

Root directory of the disk

$Bitmap

tracks the allocation (in-use vs free) of each cluster in volume

$Boot

boot record of the volume

$BadClus

tracks allocation of each cluster

$Secure

Tracks security information for files within the volume

$UpCase

Table of Unicode upper case chars used for file sorting

$Extend

directory containing $ObjId, $Quota, $Reparse and $UsnJrnl

$MFT

Master File Table: MFT is the main element of New Technology File System (NTFS) disk partitions.

The MFT contains entries for all existing files written on the partition. Deleted files that were written or temporarily are stores as a file record. MFT entries are sequential so can be used to identify when a file was added to a device.

A MFT record can be broken down into the following:

Each entry has a size of 1024 bytes (can be 4096 in rare circumstances). Each valid entry will begin with a signature of "FILE" or "0x46 0x49 0x4c 0x45". If there's an error in the entry, the signature will be "BAAD" or "0x42 0x41 0x044".

Offset 0x08 - $LogFile Sequence Number (LSN) used to determine whether the filesystem is consistent or needs to have certain actions redone or undone

Offset 0x10 - Sequence Number - counter that tracks the number of times an MFT record has been reused. when allocated for the first time, the SN will be set to 1, when delete it becomes unallocated. SN is incremented again etc

Offset 0x12 - number of $FILE_NAME attributes set - multiple can track both long and short names

0ffsec 0x16 - can have the following values: 0x00 0000 Not in Use 0x01 0001 File in Use 0x02 0010 Not in Use 0x03 0011 Not in Use

Offset 0x18 & 0x1C - current size of the bytes being used by the FILE record and total number of bytes allocated to the record.

Offset 0x20 - File Reference to Base Record - only used for extended records

Offset 0x28 - Attribute ID - the number a new attribute will be assigned when added to the current FILE record.

Offset 0x30 - FixUp information - first two bytes are the fixup code, made up of Update Sequence Number (USN) and Update Sequence Array (USA).

common attribute types::names for FILES

Common Attribute types::names for DIRECTORIES

0x10::$STANDARD_INFORMATION

0x30::$FILE_NAME (Long)

0x30::$FILE_NAME (Short - sometimes)

0x80::$DATA

0x90::$INDEX_ROOT

0x80::$DATA (alternate data stream -sometimes)

0xA0::$INDEX_ALLOCATION (sometimes)

$STANDARD_INFORMATION

this MFT attribute contains important information such as the file's flags, security information, $UsnJrnl sequence Number and MACB.

identified by a signature value of 0x10.

Offset 0x18 - find a set of four timestamps, Created, Modified, t Entry modified and Last Accessed.

Offset 0x38 - set of flags that track the file attributes, such as hidden, read-only etc

the rest of the properties provide tracking numbers for the file used by the $Secure, $Quota, and $UsnJrnl system files.

$FILE_NAME

holds important information about the objects name, parent directory and propeties such as Archive,Hidden, ReadOnly etc. Also another set of timestamps. If a file name has a small name, e.g. 'file.txt' it only needs a single $FILE_NAME attribute, if it has a long name two $FILE_NAME attributes are created (one long name, one DOS compatible) e.g. 'this is a long name.txt' and 'THISIS~1.txt'.

Attribute is identified by the signature value of 0x30

offset 0x18 - Parent directory reference, consists of tqo parts, the first 6 bytes are MFT entry number or parent directory, following are sequence number for that record.

offset 0x20 - beginning of another set of MACB timestampts. updated differently that $STANDARD_INFORMATION

Offset 0x50 - same as $STANDARD_INFORMATION

Offset 0x58 - single byte tells us the number of 2-byte Unicode chararcters that make upo the name. if you double the value, you can determine the total number of bytes to read. as the max you can store in a byte is 0xFF or 255 characters, this is also a direct correlation to how long a name can be set to!

Offset 0x59 - Namespace type that tells us what kind of name to expect, possible values include: 0x00 - POSIX - case preserved, different case = different name 0x01 - Win32 - case-sensitive long name 0x02 - DOS - DOS-Compatible short name 0x03 - Win32/DOS - Name is short enough to need only one $FILE_NAME

$DATA

identified by a signature value of 0x80. content of the attribute will contain either actual content or information needed to figure out which cluster to read and obtain data.

Virtual Cluster Numbers (VCN) track the contiguous, they start at zero and end with a count of the number of clusters consumed by the content of the file

the allocated size is the size of the clusters consumed by the file. The true size is the actual size of the content of the file. The initialized size is the size of the clusters reserved for this file to grow into.

Alternative Data Streams (ADS)

ADS's are the presence of a second $DATA attribute. the primary attribute is never named, deriving its name from the $FILE_NAME attribute. Any subsequent $DATA attributes must be named to be able to address them. This is done by a Unicode string just before the content of the ADS for resident attributed

Most tools can't report on the existence of ADS, let alone size or content

due to this a lot of malware takes advantage ^

To look for ADS:

dir /r
Get-Item * -Stream *

$I30

Directories are essentially file, data they store is metadata information about their contents. With NTFS, directories store metadata in the $I30 index. When entries get deleted from this index, they are not overwritten. They are marked unused (slack) entries.

Instead of a $DATA attribute; directories store $I30.

The index is made up of $INDEX_ROOT and $INDEX_ALLOCATION.

if there are only a few entries, they will be listed in the $INDEX_ROOT attribute, which always resides inside the MFT. If there are more than a few entries, a secondary, $INDEX_ALLOCATION attribute will be used to store the additional entries. The $INDEX_ALLOCATION will always be non-residential and is structured almost exactly the same as $DATA attributes.

the $I30 attribute begins with the signature value of "INDX"

the allocated size of entries tells us how much space is available to store entries. By subtracting size of entries from allocated size of entries.

The index entry also has a MFT record entry number to point back to the OS. Timestamps stored in the $STANDARD_INFORMATION attribute not in the $FILE_NAME

Can be parsed using Indx2csv, #indxor velociraptor

Zone.Identifier ADS

When a file is downloaded; an identifying ADS name "Zone.Identifier". It can be a quick way to find downloaded files, and potentially even where they came from!

When files are downloaded an Alternative Data Stream (ADS) is added to the file, dubbed "Mark Of the Web" (MoTW). which is provided via the Windows API Function IAttachmentExecute

InternetExplorer only tags certain types of files, that it might deem harmful, PowerShell and FTP unlikely to tag at all!

The Zone.Identifier could include, download location, but definitely includes ZoneID: NoZone = -1, MyComputer = 0, Intranet = 1, Trusted = 2, Internet = 3, Untrusted = 4

Untrusted is added by SmartScreen (defender)

$MFT stores:

filename
file size
unique Security ID in the $STANDARD_INFORMATION attribute
File creation, last modified, last access and last changes SI timestamps in $STANDARD_INFORMATION attribute
If the File Record is in use - if file is deleted the MFT file record is set to no longer in use, is not directly deleted during the file deletion. Metadata information and content for MFT resident files, can be retrieved for recently deleted files (until it gets overwritten by a new MFT entry).

$MFTMIRR

contains a backup of primary $MFT in case the primary record cannot be read due to physical damage of the disk. Information in record 0 that they system needs to find to read in the rest of the $MFT file.Default cluster size if 4k and records are 1k, this usually works out to be the first four records

$LogFile

Part of the journaling feature of NTFS; maintains a low-level record of changes made to the NTFS volume. Every disk operation is journalised prior to being committed. In case of a crash, the $LogFile is used to revert the disk operations.

While typically it only contains historical data for last few hours (unless the system has been fairly idle) the $LogFile is the file system resiliency - constantly tracking changes to the following NTFS features:

$MFT File
$I30index
$Extend\$UsnJrnl
$Bitmap(that tracks cluster allocation)

Although fairly verbose, it's very efficient, it only logs the data that is being changed.

$LogFile size defaults to 64Mb (can be adjusted)

NTFS can use the $LogFile to complete that file system transactions when power is restored

$Volume

File contains the friendly name of the volume for display in My Computer and other locations, as well as the NTFS version number and set of flags that tell the system if the volume was unmounted cleanly

$AttrDef

Defines the NTFS attributed for the version of NTFS in use on this volume.

"."

Record 5 is the root directory. No different than any other directory except that it is always record number 5 and always a single dot.

$Bitmap

the Bitmap file tracks the allocation status of the clusters of the volume. Each cluster is associated with a bit, set to 0x1 if the cluster is in use.

$Boot

Partition Boot Sector: metadata file which starts at sector 0 and can be up to 16 sectors long, describes ht basic NTFS volume info and indicates the location of the $MFT file.

$BadClus

provides the filesystem a way of marking, and not using, clusters where there is physical damage. it's a sparse file that has a file size equal to the volume size and is initially filled with all zeros. Clusters that are all zeros don't actually get written to the disk. If a cluster is determined to be 'bad' data will be written in this file at the offset that corresponds to the location of that bad cluster. existence in this file causes the $Bitmapfile to mark that cluster as in use. thus, no other file will try to use the cluster in the future.

$Secure

The Secure file contains security descriptors for all files and folders on the NTFS volume. Security descriptors are stored within the $SDS names data stream of the $secure file. the $secure file also defines two other named streams - $SDH and $SII

Each file in the system is referenced in the $Secure file with the Security ID and Security Descriptor (which is referenced in the $MFT in the $STANDARD_INFORMATION attribute).

While no metadata is stored, the Security ID can be used to map the file info and data from the MFT to its Security Descriptor.

The Security Descriptor references:

owner of the file (pointed to a SID)
Access rights Discretionary Access Control List (DACL)
how access is audited in the System Access Control List (SACL)

$UpCase

contains a table of upper and lowercase Unicode letters for each Unicode code page in use for the filenames within the system. used in sorting the files by name, so that 'A' and 'a' are next to each other when sorting alphabetically.

$Extend

there's a total of 24 MFT records, rather than place the new system files in those records, a directory entry was placed in record 11, to hold the new system files. the files below are written by the format command before user files are written, they almost always are located in the first four records that are not reserved (number 24-28). They aren't static like the first 12:

$Extend\$objId

contains an index of all the object IDs in use within the volume. allow for a file to be tracked even if the file gets moved, renamed or changes.

$Extend\$Quota

contains information about how much allocated space each user is allowed to use and is currently using

$Extend\$Reparse

contains an index of all the re-parse points on the volume. Re-parse points have a multitude of uses, by most commonly used for symbolic links and also used for mounting other volumes as a directory

$Extend\$UsnJrnl

Maintains the update sequence number (USN) change journal (also known as the change journal) the $UsnJrnl keeps track of changes to files and directories, along with a code that denotes what type of change occurred. e.g. the windows backup application uses the $UsnJrnl to track the files that have had changes to decide what to backup! Antivirus, and windows search index also use the $UsnJrnl.

The $UsnJrnl is very concise, and much easier to interpret as a responder. Also normally stores data between several days to weeks! The records are stored in an Alternative Data Streams (ADS) named $J

Windows caps the $UsnJrnl to 32mb, but some Servers are set to 512MB!

Has an ADS called $Max that holds metadata about the size

$J

Used as an alternative data stream (ADS) for $Extend\$UsnJrnl. Individual records aren't numbered, instead tracked based on their offset into the data stream. Due to this, to analyse the $Extend\$UsnJrnl you actually need to export the $J attribute. Normally a Sparse file (filled with blank space) but when forensically extracting will take the full file size - but does compress very well (as mainly empty space)

$J often spans around +3GB in size

Common activity patterns:

Action

$LogFile codes

$UsnJrnl Codes

File / Directory creation

AddIndexEntryAllocation AddIndexEntryRoot InitializeFileRecordSegment

FileCreate

File/ Directory deletion

DeleteIndexEntryAllocation DeleteIndexEntryRoot DeallocateFileRecordSegment

FileDelete

File / Directory Rename or Move

DeleteIndexEntryAllocation AddIndexEntryAllocation

RenameOldName RenameNewName

ADS Creation

CreateAttribute with name ending in ":ADS"

StreamChange NamedDataExtend

File Data Modification

*Op codes for $LogFile often are not efficient to determine file modification

DataOverwrite | DataExtend |DataTruncation

Extracting Data:

It's good to target directories when looking at parsing $LogFile and $Extend\$UsnJrnl. Also check for: .exe, .dll, .sys, .pyd, .ps1, .vbs, .bat, .rar, .7z, .cab

Parent Directories to Filter

C:\Windows & C:\Windows\system32

Directories coveted by attackers

C:\Windows\Prefetch

Attackers often delete prefect files

Attackers Working Dir

Discover unknown attacker tools and exfil

Temp directories

Focus on executables

C:\Users\*\Downloads

Find recently downloaded files

C:\Users\*\Appdata\Roaming\Microsoft\Windows\Recent

Find recently downloaded files

C:\$Recycle.Bin\<SID>

Check for deleted file prior to Recycle bin empty

MFTECmd

MFTECmd can parse and extract info from $MFT, $UsnJrnl, $J, $Secure:$SDS data stream and $LogFile

# A $MFT file on a mounted partition should be specified.
# For instance, to extract $MFT data from a forensics image, the image should first be mounted and the $MFT specified as <DRIVER_LETTER:\$MFT to MFTECmd.exe.
-f # filename 
--json # json format
--csv "<dir>" # dir to save csv 
--csvf <name> # name of csv
--body "<dir>" #dir to save csv
-- bodyf <name> # filename to save csv 
--bdl <name> # drive letter (C, D, etc) to use with bodyfile
--blf # when true use Lf vs CRLF for newlines

MFTECmd.exe -f '<$MFT_FILE>' --csv <OUTPUTDIR_PATH> "G:\timeline" --csvf mft.csv
MFTECmd.exe -f '<DIR\C\$Extend\$J>' -m '<DIR\C\$MFT>' --csv <OUTPUTDIR_PATH> "G:\timeline" --csvf usnjrnl.csv
MFTECmd.exe -f '<$Extend\$UsnJrnl:$J>' -m '<DIR\C\$MFT>' --vss --csv <OUTPUTDIR_PATH> "G:\timeline" --csvf vss-Usn.csv

can use mactime to convert body file into readable timeline

LogFileParser

The primary output gives an overview of the data contained in the $LogFile, (logfile.csv) but includes summary information about each event in the $LogFile but if more info is available will produce supplementary files

LogFileParser.exe /LogFileFile:c:\temp\$LogFile /TimeZone:2.00 /MftRecordSize:4096 /ExtractDataUpdates:1 /ExtractDataUpdatesSize:8 /SectorsPerCluster:64 /TSFormat:1 /TSPrecision:NanoSec /Unicode:1
LogFileParser.exe /LogFileFile:c:\temp\$LogFile /TimeZone:-5.00 /MftRecordSize:1024 /SectorsPerCluster:1 /TSFormat:1 /TSPrecision:MilliSec /Unicode:0
LogFileParser.exe /LogFileFile:c:\temp\$LogFile /TSFormat:3 /ReconstructDataruns:1
LogFileParser.exe /LogFileFile:c:\temp\$LogFile /TimeZone:7.00 /MftRecordSize:1024 /TSFormat:2 /TSPrecision:None /SourceIs32bit:1
LogFileParser.exe /LogFileFile:c:\temp\$LogFile /TimeZone:7.00 /MftRecordSize:1024 /TSFormat:2 /TSPrecision:None /SourceIs32bit:1 /SkipSqlite3:1
LogFileParser.exe /LogFileFile:c:\temp\$LogFile /OutputPath:E:\LFP-Output
LogFileParser.exe /LogFileFile:c:\temp\$LogFile /MftCsvFile:c:\temp\$MFT
LogFileParser.exe /LogFileFragmentFile:c:\temp\fragment.bin /OutputPath:E:\LFP-Output
LogFileParser.exe /LogFileFragmentFile:c:\temp\fragment.bin /OutputPath:E:\LFP-Output /VerifyFragment:1
LogFileParser.exe /LogFileFragmentFile:c:\temp\fragment.bin /OutputPath:E:\LFP-Output /VerifyFragment:1 /OutFragmentName:FragmentsRCRDCollection.bin
LogFileParser.exe /LogFileFile:E:\LFP-Output\FragmentsRCRDCollection.bin /OutputPath:E:\LFP-Output /SkipFixups:1 /BrokenLogFile:1

istat

Sleuth kits istat - designed to parse metadata information from file systems, including NTFS, FAT, and ExFAT. Can be used against lots of image files, e.g. raw, E01, VMDK, VHD

-z <zone> # time zone of original machine 
-s <seconds> # time skew of original machine 
istat <options> <image> <inode>

Icat

Sleuth kits icat - designed to go to metadata and extract out file or attribute content good for extraction of $DATA content

-r # recover deleted files
-s # display slack space at end of file
icat <options> <image> <inode> > extracted_data.txt

tsk_recover

tsk_recover is a sleuth kit Metadata file carving tool with several options available, by default it identifies unallocated inodes ( $MFT entries) and extract them all to a specified output directory

works to extract all deleted files, instead of singular with Icat

tsk_recover <image> <output-directory>

PhotoRec

file Carving tool - runs on Unix, and Windows. Knows 480 file extensions. and support NTFS, FAT, Ext2/3/4. Once it parses files, it will search metadata to try and find original file metadata including size and original name. can run directly on E01

photorec

Indx2csv

Indx2CSV parses both slack and active entries of $I30, $Extend\$objId, $Extend\$Reparse. Reads INDX records exported with forensics tools.

Examples:Indx2Csv.exe /IndxFile:c:\temp\chunk.wfixups.INDX
Indx2Csv.exe /IndxFile:c:\temp\chunk.wfixups.INDX /OutputPath:e:\temp
Indx2Csv.exe /IndxFile:c:\temp\chunk.wfixups.INDX /OutputPath:e:\temp /OutputFormat:l2t
Indx2Csv.exe /IndxFile:c:\temp\chunk.wfixups.INDX /OutputPath:e:\temp /Unicode:1 /StrictNameCheck:1
Indx2Csv.exe /IndxFile:c:\temp\chunk.wfixups.INDX /TimeZone:2.00 /TSFormat:1 /TSPrecision:NanoSec /Unicode:1
Indx2Csv.exe /IndxFile:c:\temp\chunk.wofixups.INDX /Fixups:0 /TimeZone:-5.00 /TSFormat:1 /TSPrecision:MilliSec
Indx2Csv.exe /IndxFile:c:\temp\chunk.wofixups.INDX /Fixups:0 /TSFormat:1 /TSPrecision:MilliSec /Slack:1 /Unicode:0
Indx2Csv.exe /IndxFile:C:\temp\fragment.bin /ScanMode:10 /VerifyFragment:1 /OutputPath:e:\I30Output /OutFragmentName:FragmentCollection.bin /CleanUp:1
Indx2Csv.exe /IndxFile:e:\I30Output\FragmentCollection.bin /OutputPath:e:\I30Output /ScanMode:10 /Fixups:0

IndxParse.py

-h, --help  # show this help message and exit
-c          # Output CSV
-b          # Output Bodyfile
-d          # Find entries in slack space

usage: INDXParse.py [-h] [-c | -b] [-d] filename

PreviousSRUM NextEVTX

Last updated 1 year ago