파일 시스템

Using EFS

User can invoke EFS features through Windows Explorer or by using a command-line utility called cipher.exe. To use Windows Explorer to encrypt file, open File property window by right clicking on file name. Click Advanced... button - Advanced Attributes dialog will be opened allowing you to mark file as encrypted.

Before saving new settings Windows will prompt user to encrypt file only or the whole folder. It address very important issue - while the file itself could be perfectly protected, the application which opens the file may create a temporary copies of the file while working with the document. The example is Microsoft Word. When user opens encrypted document, EFS decrypts it transparently for Word. Then during the work, Word creates temporary hidden file where it automatically saves the document in the process of editing and deletes it on the exit. This hidden file presents a real breach in security because it contains user data in plain (not encrypted) form. Encrypting the whole folder instead of file only solves this problem.

Partition Boot Sector

Figure 5-1 Formatted NTFS Volume

Table 5-1 describes the boot sector of a volume formatted with NTFS. When you format an NTFS volume, the format program allocates the first 16 sectors for the $Boot metadata file. First sector, in fact, is a boot sector with a "bootstrap" code and the following 15 sectors are the boot sector's IPL (initial program loader). To increase file system reliability the very last sector an NTFS partition contains a spare copy of the boot sector.

Table 5-1 NTFS Boot Sector

On NTFS volumes, the data fields that follow the BPB form an extended BPB. The data in these fields enables Ntldr (NT loader program) to find the master file table (MFT) during startup. On NTFS volumes, the MFT is not located in a predefined sector, as on FAT16 and FAT32 volumes. For this reason, the MFT can be moved if there is a bad sector in its normal location. However, if the data is corrupted, the MFT cannot be located, and Windows NT/2000 assumes that the volume has not been formatted.

The following example illustrates the boot sector of an NTFS volume formatted while running Windows 2000. The printout is formatted in three sections:

• Bytes 0x00- 0x0A are the jump instruction and the OEM ID (shown in bold print).
• Bytes 0x0B-0x53 are the BPB and the extended BPB.
• The remaining code is the bootstrap code and the end of sector marker (shown in bold print).

Physical Sector:Cyl 0, Side 1, Sector 1 
      00000000:EB 52 90 4E 54 46 53 20 -20 20 20 00 02 08 00 00 .R.NTFS ........ 
      00000010:00 00 00 00 00 F8 00 00 -3F 00 FF 00 3F 00 00 00 ........?...?... 
      00000020:00 00 00 00 80 00 80 00 -4A F5 7F 00 00 00 00 00 ........J....... 
      00000030:04 00 00 00 00 00 00 00 -54 FF 07 00 00 00 00 00 ........T....... 
      00000040:F6 00 00 00 01 00 00 00 -14 A5 1B 74 C9 1B 74 1C ...........t..t. 
      00000050:00 00 00 00 FA 33 C0 8E -D0 BC 00 7C FB B8 C0 07 .....3.....|.... 
      00000060:8E D8 E8 16 00 B8 00 0D -8E C0 33 DB C6 06 0E 00 ..........3..... 
      00000070:10 E8 53 00 68 00 0D 68 -6A 02 CB 8A 16 24 00 B4 ..S.h..hj....$.. 
      00000080:08 CD 13 73 05 B9 FF FF -8A F1 66 0F B6 C6 40 66 ...s......f...@f 
      00000090:0F B6 D1 80 E2 3F F7 E2 -86 CD C0 ED 06 41 66 0F .....?.......Af. 
      000000A0:B7 C9 66 F7 E1 66 A3 20 -00 C3 B4 41 BB AA 55 8A ..f..f....A..U. 
      000000B0:16 24 00 CD 13 72 0F 81 -FB 55 AA 75 09 F6 C1 01 .$...r...U.u.... 
      000000C0:74 04 FE 06 14 00 C3 66 -60 1E 06 66 A1 10 00 66 t......f`..f...f 
      000000D0:03 06 1C 00 66 3B 06 20 -00 0F 82 3A 00 1E 66 6A ....f;....:..fj 
      000000E0:00 66 50 06 53 66 68 10 -00 01 00 80 3E 14 00 00 .fP.Sfh.....>... 
      000000F0:0F 85 0C 00 E8 B3 FF 80 -3E 14 00 00 0F 84 61 00 ........>.....a. 
      00000100:B4 42 8A 16 24 00 16 1F -8B F4 CD 13 66 58 5B 07 .B..$......fX [.. 
      00000110:66 58 66 58 1F EB 2D 66 -33 D2 66 0F B7 0E 18 00 fXfX.-f3.f...... 
      00000120:66 F7 F1 FE C2 8A CA 66 -8B D0 66 C1 EA 10 F7 36 f......f..f....6
      00000130:1A 00 86 D6 8A 16 24 00 -8A E8 C0 E4 06 0A CC B8 ......$......... 
      00000140:01 02 CD 13 0F 82 19 00 -8C C0 05 20 00 8E C0 66 ..............f 
      00000150:FF 06 10 00 FF 0E 0E 00 -0F 85 6F FF 07 1F 66 61 ..........o...fa 
      00000160:C3 A0 F8 01 E8 09 00 A0 -FB 01 E8 03 00 FB EB FE ................ 
      00000170:B4 01 8B F0 AC 3C 00 74 -09 B4 0E BB 07 00 CD 10 .....<.t........ 
      00000180:EB F2 C3 0D 0A 41 20 64 -69 73 6B 20 72 65 61 64 .....A disk read 
      00000190:20 65 72 72 6F 72 20 6F -63 63 75 72 72 65 64 00 error occurred. 
      000001A0:0D 0A 4E 54 4C 44 52 20 -69 73 20 6D 69 73 73 69 ..NTLDR is missi 
      000001B0:6E 67 00 0D 0A 4E 54 4C -44 52 20 69 73 20 63 6F ng...NTLDR is co 
      000001C0:6D 70 72 65 73 73 65 64 -00 0D 0A 50 72 65 73 73 mpressed...Press 
      000001D0:20 43 74 72 6C 2B 41 6C -74 2B 44 65 6C 20 74 6F Ctrl+Alt+Del to 
      000001E0:20 72 65 73 74 61 72 74 -0D 0A 00 00 00 00 00 00 restart........
      000001F0:00 00 00 00 00 00 00 00 -83 A0 B3 C9 00 00 55 AA ..............U. 

The following table describes the fields in the BPB and the extended BPB on NTFS volumes. The fields starting at 0x0B, 0x0D, 0x15, 0x18, 0x1A, and 0x1C match those on FAT16 and FAT32 volumes. The sample values correspond to the data in this example.

Protecting the Boot Sector

Because a normally functioning system relies on the boot sector to access a volume, it is highly recommended that you run disk scanning tools such as Chkdsk regularly, as well as back up all of your data files to protect against data loss if you lose access to a volume.

NTFS System Files

NTFS includes several system files, all of which are hidden from view on the NTFS volume. A system file is one used by the file system to store its metadata and to implement the file system. System files are placed on the volume by the Format utility.

Table 5-4 Metadata Stored in the Master File Table

NTFS 파일시스템은 FAT파일시스템에 비해 보다 나은 퍼포먼스, 신뢰성, 및 호환성을 제공합니다.

NTFS 파일시스템은 빠른 검색, 읽기, 쓰기을 제공하기 위해 설계된 파일시스템입니다. 또한 대용량 하드디스크에 대한 파일시세틈 복구기능을 제공합니다.

NTFS 파일시스템으로 포맷하는 경우 여러 메타데이타파일(MFT - Master File Table, $Bitmap, $LogFile )들이 생성됩니다. 이러한 메타데이타파일은 NTFS 볼륨의 파일이나 폴더들에 대한 정보가 저장됩니다.

The first information on an NTFS 볼륨의 첫번째 정보는 파티션 부트섹터 ($Boot metadata file) 입니다.

파티션 부트섹터는 sector 0 에 위치하며 16 sectors 의 크기로 확장될수 있습니다.

부트섹터는 기본 NTFS 볼륨정보 및 $MFT의 위치 정보가 저장됩니다.

다음은 디스크를 NTFS 볼륨으로 포맷했을 때 레이아웃입니다.

※ 포맷된 NTFS 볼륨

NTFS vs FAT

NTFS Sparse Files (NTFS5 only)

A sparse file has an attribute that causes the I/O subsystem to allocate only meaningful (nonzero) data. Nonzero data is allocated on disk, and non-meaningful data (large strings of data composed of zeros) is not. When a sparse file is read, allocated data is returned as it was stored; non-allocated data is returned, by default, as zeros.

NTFS deallocates sparse data streams and only maintains other data as allocated. When a program accesses a sparse file, the file system yields allocated data as actual data and deallocated data as zeros.

NTFS includes full sparse file support for both compressed and uncompressed files. NTFS handles read operations on sparse files by returning allocated data and sparse data. It is possible to read a sparse file as allocated data and a range of data without retrieving the entire data set, although NTFS returns the entire data set by default.

With the sparse file attribute set, the file system can deallocate data from anywhere in the file and, when an application calls, yield the zero data by range instead of storing and returning the actual data. File system application programming interfaces (APIs) allow for the file to be copied or backed as actual bits and sparse stream ranges. The net result is efficient file system storage and access. Next figure shows how data is stored with and without the sparse file attribute set.

Figure 5-4 Windows 2000 Data Storage

Important
If you copy or move a sparse file to a FAT or a non-NTFS volume, the file is built to its originally specified size. If the required space is not available, the operation does not complete.

NTFS Optimization

If you investigate your storage needs, you can tune some of global NTFS parameters to achieve significant increase of disk performance. Other techniques like disk defragmentation could help you either.

There are several factors (we do not mention here drive type, rpm ...) that affect the NTFS Performance: Cluster Size, location and fragmentation of Master File Table (MFT) and paging file, NTFS Volume compression, NTFS Volume Source (created or converted from existing FAT volume).

Define Cluster Size Properly

Cluster is an allocation unit. If you create file lets say 1 byte in size, at least one cluster should be allocated on FAT file system. On NTFS if file is small enough, it can be stored in MFT record itself without using additional clusters. When file grows beyond the cluster boundary, another cluster is allocated. It means that the bigger the cluster size, the more disk space is wasted, however, the performance is better.

The following table shows the default values that Windows NT/2000/XP uses for NTFS formatting:

Drive size                   
   (logical volume)             Cluster size          Sectors
   ----------------------------------------------------------
     512 MB or less               512 bytes           1
     513 MB - 1,024 MB (1 GB)   1,024 bytes (1 KB)    2
   1,025 MB - 2,048 MB (2 GB)   2,048 bytes (2 KB)    4
   2,049 MB and larger          4,096 bytes (4 KB)    8

However, when you format the partition manually, you can specify cluster size 512 bytes, 1 KB, 2 KB, 4 KB, 8 KB, 16 KB, 32 KB, 64 KB in the format dialog box or as a parameter to the command line FORMAT utility.

What it gives us? Determine average file size and format the partition accordingly. How to determine? The simplest (but rough) way is to divide number of files on the drive by total disk usage in kilobytes. Another idea is to estimate the approximate data size in advance before formatting the hard drive. If you are going to store multimedia stuff that is usually huge in size, make cluster bigger to increase a performance. If you plan to store small web pages or text documents, make cluster size smaller not to lose a lot of disk space. Think!

Note: On Volumes, having cluster size more than 4 KB compression is not supported

MFT Reservation and Fragmentation

MFT contains frequently used system files and indexes, so performance of MFT affects a lot to the entire volume performance.

By default NTFS reserves zone, 12.5% of volume size for MFT and does not allow writing there any user's data, which lets MFT to grow. However, when, for example, a lot of files are placed to the drive, MFT can grow beyond the reserved zone and becomes fragmented. Another reason is when you delete file, NTFS does not always use its space in MFT to store new one, it just marks MFT entry as deleted and allocates new entry  for the new file. It provides some performance and recovery benefits, however it forces MFT  to be fragmented.

The more MFT fragmentation, the more the HDD heads movements to access the data, the less overall performance of file system.

Starting from Windows NT 4.0 SP4 you can define MFT Zone Reservation value through the Registry.
Key: HKEY_LOCAL_MACHINESYSTEMCurrentControlSetControlFileSystem
Value NtfsMftZoneReservation of DWORD type (1 to 4) allows you to specify MFT Zone for the newly created/formatted volumes(12.5 percent, 25 percent, 37.5 percent, 50 percent of NTFS volume accordingly)

File and Directory Fragmentation

Not only MFT but also other files and directories can be fragmented when drive becomes full and you perform lots of copying and deleting actions with files. It also slows down system performance, so using standard Defragmentation Tools regularly is advisable when you perform a lot of copy/move/delete operations on the volume. Starting from Windows 2000, Disk Defragmenter is a part of Operating System and you can find it in Computer Management console.

If you do not have such tools, you can try doing it manually. Just copy files and folders to another partition, leaving original one nearly empty, and then copy them back. This solution is less effective than using standard Defragmentation Toools, however, it could greatly improve the volume performance in case if your partition was heavily fragmented.
Note: It's not an acceptable solution if you have security/permissions setup on these files. While copying between partitions you'll loose this information.

To prevent directory fragmentation, just perform complete drive defragmentation before installing new applications or copying lots of files onto the volume.

Conversion volume from FAT to NTFS

If you did not setup Windows on the newly created NTFS Volume, but converted volume from FAT to NTFS, usually it causes MFT fragmentation so generally converted partitions are much slower than the ones being originally created as NTFS. Defragmentation tools are in general unable to defragment MFT, however, you can try to backup the whole system, re-format the volume with proper cluster size, and restore it back.

NTFS automatically updates the last access time and date stamp on folders and files when NTFS traverses its B-tree folder structure. To enhance NTFS performance you can disable this behavior and reduce NTFS's operational overhead without significantly impairing functionality. In the HKEY_LOCAL_MACHINESYSTEMCurrentControlSetControlFileSystem Registry key, change the NtfsDisableLastAccessUpdate value of type REG_DWORD from the default value 0 (enabled) to 1 (disabled). This Registry value doesn't exist by default, so you need to enter it manually.

You can increase NTFS performance if you disable DOS 8.3 name creation. In the HKEY_LOCAL_MACHINESYSTEMCurrentControlSetControlFileSystem Registry key, change the NtfsDisable8dot3NameCreation value of type REG_DWORD from the defaoult value 0 (enabled) to 1 (disabled).

Recommended Reading:

HOW TO: Defragment Your Disk Drive Volumes in Windows XP (Q314848):
http://support.microsoft.com/kb/314848

Description of the New Command Line Defrag.exe Included with Windows XP:
http://support.microsoft.com/kb/283080

The Default Cluster Size for the NTFS and FAT File Systems (Q314878) :
http://support.microsoft.com/default.aspx?scid=kb;EN-US;q314878

How NTFS Reserves Space for its Master File Table (MFT) (Q174619)
http://support.microsoft.com/default.aspx?scid=kb;en-us;Q174619

Optimizing NTFS
http://technet.microsoft.com/en-us/library/cc767961.aspx

How to Disable the 8.3 Name Creation on NTFS Partitions
http://support.microsoft.com/kb/121007

NTFS Multiple Data Streams

NTFS supports multiple data streams, where the stream name identifies a new data attribute on the file. A handle can be opened to each data stream. A data stream, then, is a unique set of file attributes. Streams have separate opportunistic locks, file locks, and sizes, but common permissions.

This feature enables you to manage data as a single unit. The following is an example of an alternate stream:

myfile.dat:stream2	

A library of files might exist where the files are defined as alternate streams, as in the following example:

library:file1

:file2

:file3

A file can be associated with more than one application at a time, such as Microsoft® Word and Microsoft® WordPad. For instance, a file structure like the following illustrates file association, but not multiple files:

program:source_file
           :doc_file
           :object_file
           :executable_file

To create an alternate data stream, at the command prompt, you can type commands such as:

echo text>program:source_file
    more<program:source_file 

Important
When you copy an NTFS file to a FAT volume, such as a floppy disk, data streams and other attributes not supported by FAT are lost.

NTFS Master File Table (MFT)

Each file on an NTFS volume is represented by a record in a special file called the master file table (MFT). NTFS reserves the first 16 records of the table for special information. The first record of this table describes the master file table itself, followed by a MFT mirror record. If the first MFT record is corrupted, NTFS reads the second record to find the MFT mirror file, whose first record is identical to the first record of the MFT. The locations of the data segments for both the MFT and MFT mirror file are recorded in the boot sector.

Full list of metadata files are presented in the "System Files" chapter.

Figure provides a simplified illustration of the MFT structure:

Figure 5-2 MFT Structure

The master file table allocates a certain amount of space for each file record. The attributes of a file are written to the allocated space in the MFT. Small files and directories (typically 512 bytes or smaller), such as the file illustrated in next figure, can entirely be contained within the master file table record.

Figure 5-2 MFT Record for a Small File or Directory:

This design makes file access very fast. Consider, for example, the FAT file system, which uses a file allocation table to list the names and addresses of each file. FAT directory entries contain an index into the file allocation table. When you want to view a file, FAT first reads the file allocation table and assures that it exists. Then FAT retrieves the file by searching the chain of allocation units assigned to the file. With NTFS, as soon as you look up the file, it's there for you to use.

Directory records are housed within the master file table just like file records. Instead of data, directories contain index information. Small directory records reside entirely within the MFT structure. Large directories are organized into B-trees, having records with pointers to external clusters containing directory entries that could not be contained within the MFT structure.

NTFS Compressed Files

Windows NT/2000 supports compression on individual files, folders, and entire NTFS volumes. Files compressed on an NTFS volume can be read and written by any Windows-based application without first being decompressed by another program.

Decompression occurs automatically when the file is read. The file is compressed again when it is closed or saved. Compressed files and folders have an attribute of C when viewed in Windows Explorer.

Only NTFS can read the compressed form of the data. When an application such as Microsoft® Word or an operating system command such as copy requests access to the file, the compression filter driver decompresses the file before making it available. For example, if you copy a compressed file from another Windows NT/2000-based computer to a compressed folder on your hard disk, the file is decompressed when read, copied, and then recompressed when saved.

NTFS allows for the compression of an entire volume, of one or more folders within a volume, or even one or more files within a folder of an NTFS volume.

The compression algorithms in NTFS are designed to support cluster sizes of up to 4 KB. When the cluster size is greater than 4 KB on an NTFS volume, none of the NTFS compression functions are available.

Each NTFS data stream contains information that indicates whether any part of the stream is compressed.

NTFS provides real-time access to a compressed file, decompressing the file when it is opened and compressing it when it is closed. When writing a compressed file, the system reserves disk space for the uncompressed size. The system gets back unused space as each individual compression buffer is compressed.

NTFS File Attributes

The NTFS file system views each file (or folder) as a set of file attributes. Elements such as the file's name, its security information, and even its data, are all file attributes. Each attribute is identified by an attribute type code and, optionally, an attribute name.

When a file's attributes can fit within the MFT file record, they are called resident attributes. For example, information such as filename and time stamp are always included in the MFT file record. When all of the information for a file is too large to fit in the MFT file record, some of its attributes are nonresident. The nonresident attributes are allocated one or more clusters of disk space elsewhere in the volume. If all attributes can not fit into one MFT record NTFS creates additional MFST records and puts the Attribute List attribute to the first file's MFT record to describe the location of all of the attribute records.

Table 5-3 lists all of the file attributes currently defined by the NTFS file system. This list is extensible, meaning that other file attributes can be defined in the future.

Table 5-3 File Attributes Defined by NTFS

Issues with EFS

Temporary file is not erased. When EFS encrypts file, it copies its contents into temporary hidden file named Efs0.tmp in the same folder, as encrypting file. Then, it encrypts plain text by blocks and writes encrypted data into original file. After the process is done, temporary file is deleted. The problem is that EFS simply marks it as deleted without actually erasing its contents, which makes possible easy access to unprotected data by low-level data recovery software like Active@ Undelete. Solution - to wipe free disk space. Usually, even if plain text overwritten ones, small magnetic traces remain detectible, thus giving a chance to read erased data with proper equipment. To minimize this possibility, use commercially available software providing sophisticated data erasing algorithms like Active@ Eraser or ZDelete.NET or wipe unused volume and MFT space with Active@ KillDisk.

File names in encrypted folder are not protected. Actually, encrypting folder contents means automatically applying encryption to all files in the folder, not encrypting directory data itself. Since the file name itself could contain sensitive information, it could be a breach in security. One of the solutions would be using encrypted .zip archives instead of folders, which are treated by Windows XP almost like folders. Thus, only one file is needed to be encrypted and archived data themselves are harder to crack.

EFS security relies on public/private key pair which is stored on local computer. Windows protects all private keys by encrypting them through Protected Storage service. Protected Storage encrypts all private keys with Session Key, derived from 512 bit Master Key, and stores them in %User Profile%Application DataMicrosoftCryptoRSAUser SID. The Master Key is encrypted by Master Key Encryption Key, which is derived from user password by using a Password Based Key Derivation Function and stored in %User Profile%Application DataMicrosoftProtectUser SID. Despite the efforts Windows takes to protect keys, the fact, that all information is stored on local computer, gives an attacker, who's got an access to hard drive, a chance to figure out keys and use them to decrypt protected data. The overall security could be significantly enhanced by encrypting private keys with System Key. The syskey.exe utility can be used to store System Key on a floppy disk and remove it from computer. In this case user must insert a diskette with System Key when computer boots up. Nevertheless, this method should be taken with precautions since if key diskette is lost, there's no way to get access to computer.

HFS Plus

저용량 저장매체의 경우 일반적으로 FAT 파일시스템을 적용하는 경우가 많습니다.

FAT 파일시스템의 구조를 살펴보면 아래그림과 같습니다.

 MBR - Master boot record
- Boot - boot record
- FAT16 table
- FAT32 table
- Dir (for FAT16 and FAT32)