HFS Plus or HFS+ is a file system developed by Apple Inc. to replace their Hierarchical File System (HFS) as the primary file system used in Macintosh computers (or other systems running Mac OS). It is also one of the formats used by the iPod digital music player. HFS Plus is also referred to as Mac OS Extended (or, erroneously, “HFS Extended”), where its predecessor, HFS, is also referred to as Mac OS Standard (or, erroneously, as “HFS Standard”). During development, Apple referred to this filesystem with the codename Sequoia.^[3]

HFS Plus is an improved version of HFS, supporting much larger files (block addresses are 32-bit length instead of 16-bit) and using Unicode (instead of Mac OS Roman or any of several other character sets) for naming the items (files, folders) – names which are also character encoded in UTF-16^{[verification needed]} and normalized to a form very nearly the same as Unicode Normalization Form D (NFD)^[4] (which means that precomposed characters like are decomposed in the HFS+ filename and therefore count as two characters^[5] and UTF-16 implies that characters from outside the Basic Multilingual Plane — often seldom used and characters from ancient writing systems — also count as two characters in an HFS+ filename). HFS Plus permits filenames up to 255 UTF-16 characters in length, and n-forked files similar to NTFS, though until recently, almost no software takes advantage of forks other than the data fork and resource fork. HFS Plus also uses a full 32-bit allocation mapping table, rather than HFS’s 16 bits. This was a serious limitation of HFS, meaning that no disk could support more than 65,536 allocation blocks under HFS. When disks were small, this was of little consequence, but as larger-capacity drives became available, it meant that the smallest amount of space that any file could occupy (a single allocation block) became excessively large, wasting significant amounts of space. For example, on a 1 GB disk, the allocation block size under HFS is 16 KB, so even a 1 byte file would take up 16 KB of disk space. Unlike most other file systems HFS Plus supports hard links to directories.

Like HFS, HFS Plus uses B-trees to store most volume metadata.

History

HFS+ was introduced with the January 19, 1998 release of Mac OS 8.1.^[1] However its first appearance, as a beta filesystem, was in the never-released Copland OS betas.

With the release of the 10.2.2 update on November 11, 2002, Apple added optional journaling features to HFS Plus for improved data reliability. These features were easily accessible in Mac OS X Server, but only accessible through the command line in the standard desktop client.^[6] With Mac OS X v10.3, all HFS Plus volumes on all Macs are set to be journaled by default. Within the system, an HFS Plus volume with a journal is identified as HFSJ.

10.3 also introduced another version of HFS Plus called HFSX. HFSX volumes are almost identical to HFS Plus volumes, except that they are never surrounded by the HFS Wrapper that is typical of HFS Plus volumes and they optionally support case sensitivity for file and folder names. HFSX volumes can be recognized by two entries in the Volume Header, a value of HX in the signature field and 5 in the version field.^[1]

Additionally, Mac OS X 10.3 marked Apple's adoption of Unicode 3.2 decomposition, superseding the Unicode 2.1 decomposition used previously. This change has caused problems for developers writing software for Mac OS X.^[7]

With 10.4, Apple added support for Inline Attribute Data records, something that had been a part of the Mac OS X implementation of HFS Plus since at least 10.0, but always marked as "reserved for future use".^[8] Until the release of Mac OS X Server 10.4, HFS Plus supported only the standard UNIX file system permissions, however 10.4 introduced support for access control list-based file security, which provides a richer mechanism to define file permissions and is also designed to be fully compatible with the file permission models on other platforms such as Microsoft Windows XP and Windows Server 2003.^[9]

Design

HFS Plus volumes are divided into sectors (called logical blocks in HFS), that are usually 512 bytes in size. These sectors are then grouped together into allocation blocks which can contain one or more sectors; the number of allocation blocks depends on the total size of the volume. HFS Plus uses a larger value to address allocation blocks than HFS, 32 bits rather than 16 bits; this means it can access 4,294,967,296 (= 2³²) allocation blocks rather than the 65,536 (= 2¹⁶) allocation blocks available to HFS.^[1]

Formerly, HFS Plus volumes were embedded inside an HFS standard filesystem. This was phased out by the Tiger transition to Intel Macs, where the HFS Plus filesystem was not embedded inside a wrapper. The wrapper was designed for two purposes; it allowed Macintosh computers without HFS Plus support in their ROM to boot HFS Plus volumes and it also was designed to help users transition to HFS Plus by including a minimal HFS volume with a read-only file called Where_have_all_my_files_gone?, explaining to users with versions of Mac OS 8.0 and earlier without HFS Plus, that the volume requires a system with HFS Plus support. The original HFS volume contains a signature and an offset to the embedded HFS Plus volume within its volume header. All allocation blocks in the HFS volume which contain the embedded volume are mapped out of the HFS allocation file as bad blocks.^[1]

There are nine structures that make up a typical HFS Plus volume:^[1]

1. Sectors 0 and 1 of the volume are HFS boot blocks. These are identical to the boot blocks in an HFS volume. They are part of the HFS wrapper.
2. Sector 2 contains the Volume Header equivalent to the Master Directory Block in an HFS volume. The Volume Header stores a wide variety of data about the volume itself, for example the size of allocation blocks, a timestamp that indicates when the volume was created or the location of other volume structures such as the Catalog File or Extent Overflow File. The Volume Header is always located in the same place.
3. The Allocation File which keeps track of which allocation blocks are free and which are in use. It is similar to the Volume Bitmap in HFS, in which each allocation block is represented by one bit. A zero means the block is free and a one means the block is in use. The main difference with the HFS Volume Bitmap, is that the Allocation File is stored as a regular file, it does not occupy a special reserved space near the beginning of the volume. The Allocation File can also change size and does not have to be stored contiguously within a volume.
4. The Catalog File is a B-tree that contains records for all the files and directories stored in the volume. The HFS Plus Catalog File is very similar to the HFS Catalog File, the main differences being records are larger to allow more fields and to allow for those fields to be larger (for example to allow the longer 255-character unicode file names in HFS Plus). A record in the HFS Catalog File is 512 bytes in size, a record in the HFS Plus Catalog File is 4 KB in Mac OS and 8 KB in Mac OS X. Fields in HFS are of fixed size, in HFS Plus the size can vary depending on the actual size of the data they store.
5. The Extents Overflow File is another B-tree that records the allocation blocks that are allocated to each file as extents. Each file record in the Catalog File is capable of recording eight extents for each fork of a file; once those are used extents are recorded in the Extents Overflow File. Bad blocks are also recorded as extents in the Extents Overflow File. The default size of an extent record in Mac OS is 1 KB and 4 KB in Mac OS X.
6. The Attributes File is a new B-tree in HFS Plus that does not have a corresponding structure in HFS. The Attributes File can store three different types of 4 KB records: Inline Data Attribute records, Fork Data Attribute records and Extension Attribute records. Inline Data Attribute records store small attributes that can fit within the record itself. Fork Data Attribute records contain references to a maximum of eight extents that can hold larger attributes. Extension Attributes are used to extend a Fork Data Attribute record when its eight extent records are already used.
7. The Startup File is designed for non-Mac OS systems that don't have HFS or HFS Plus support. It is similar to the Boot Blocks of an HFS volume.
8. The second to last sector contains the Alternate Volume Header equivalent to the Alternate Master Directory Block of HFS.
9. The last sector in the volume is reserved for use by Apple. It is used during the computer manufacturing process.

Other operating systems

Linux

The Linux kernel includes the hfsplus module^[10] for mounting HFS+ filesystems. HFS+ fsck and mkfs have been ported to Linux and are part of the hfsprogs package.^[11] These drivers currently have issues with corruption of HFS+ drives with a capacity greater than 2 TB. As such Linux distributions such as Ubuntu do not allow mounting of HFS+ drives or partitions greater than 2 TB.^[12]

The Linux HFS+ kernel driver has support to read and write to HFS+ non-journaled drives/parititions but only has read support of journaled HFS+. Journaling ability was added to HFSplus when Mac OS X came out and is by default on for Mac OS X installations. Journaling is a redundant behavior of a filesystem that helps protect data loss. If planning to write to an HFS+ partition then drive journaling must be turned off in OSX.^[13]

Windows

On Windows, a fairly complete filesystem driver for HFS+ exists as a commercial software package called MacDrive.^[14] This package allows Windows users to read and write HFS+ formatted drives, and read Mac-format optical disks.

Another solution is provided by Paragon, with their HFS+ for Windows driver; this supports both read and write on HFS+ partitions.^[15]

Apple has released read-only HFS+ drivers for Windows XP, Windows Vista, and Windows 7 in Boot Camp in Mac OS X 10.6. Microsoft has created a HFS+ driver for the XBox 360 mainly for the purpose of reading HFS+ formatted iPods.

A free (GPL) alternative to MacDrive is HFSExplorer written by Erik Larsson.^[16] HFSExplorer is an application for viewing and extracting files from an HFS+ volume (Mac OS Extended) or an HFSX volume (Mac OS Extended, Case-sensitive) located either on a physical disk, on a .dmg disk image, or in a raw file system dump. HFSExplorer is less complete than MacDrive in the sense that it can read, but not write to HFS formatted volumes.

See also

• Comparison of file systems

References

External links

• Apple Technote 1150 - HFS Plus Volume Format
• Apple Technote 1189 - The Monster Disk Driver Technote
• hfsdebug - A debugger for HFS Plus volumes by Amit Singh
• hfsprogs - Userspace support tools for HFS+ filesystems under Linux systems (adapted from Apple's native tools).
• iBored - A disk editor and viewer supporting HFS Plus

• ext2에서 자료 삭제 및 손실 없이 ext3으로 변경할 수 있다(자료를 백업할 필요가 없음).
• 저널링
• 온라인 파일 시스템 증대
• 큰 규모의 디렉터리를 위한 Htree(btree의 고급판)

The ext2 or second extended filesystem is a file system for the Linux kernel. It was initially designed by Rémy Card as a replacement for the extended file system (ext).

The canonical implementation of ext2 is the ext2fs filesystem driver in the Linux kernel. Other implementations (of varying quality and completeness) exist in GNU Hurd, MINIX 3, Mac OS X (third-party), Darwin (same third-party as Mac OS X but untested), some BSD kernels, in Atari MiNT, and as third-party Microsoft Windows drivers.

ext2 was the default filesystem in several Linux distributions, including Debian and Red Hat Linux, until supplanted more recently by ext3, which is almost completely compatible with ext2 and is a journaling file system. ext2 is still the filesystem of choice for flash-based storage media (such as SD cards, and USB flash drives) since its lack of a journal minimizes the number of writes and flash devices have only a limited number of write cycles. Recent kernels, however, support a journal-less mode of ext4, which would offer the same benefit along with a number of ext4-specific benefits.

History

The early development of the Linux kernel was made as a cross-development under the Minix operating system. Naturally, it was obvious that the Minix file system would be used as Linux's first file system. The Minix file system was mostly free of bugs, but used 16-bit offsets internally and thus only had a maximum size limit of 64 megabytes. There was also a filename length limit of 14 characters. Because of these limitations, work began on a replacement native file system for Linux.

To ease the addition of new file systems and provide a generic file API, VFS, a virtual file system layer was added to the Linux kernel. The extended file system (ext), was released in April 1992 as the first file system using the VFS API and was included in Linux version 0.96c. The ext file system solved the two major problems in the Minix file system (maximum partition size and filename length limitation to 14 characters), and allowed 2 gigabytes of data and filenames of up to 255 characters. But it still had problems: there was no support for separate access, inode modification and data modification timestamps.

As a solution for these problems, two new filesystems were developed in January 1993: xiafs and the second extended file system (ext2), which was an overhaul of the extended file system incorporating many ideas from the Berkeley Fast File System. ext2 was also designed with extensibility in mind, with space left in many of its on-disk data structures for use by future versions.

Since then, ext2 has been a testbed for many of the new extensions to the VFS API. Features such as POSIX ACLs and extended attributes were generally implemented first on ext2 because it was relatively simple to extend and its internals were well-understood.

On Linux kernels prior to 2.6.17,^[1] restrictions in the block driver mean that ext2 filesystems have a maximum file size of 2TB.

ext2 is still recommended over journaling file systems on bootable USB flash drives and other solid-state drives. ext2 performs fewer writes than ext3 since it does not need to write to the journal. As the major aging factor of a flash chip is the number of erase cycles, and as those happen frequently on writes, this increases the life span of the solid-state device.^[2] Another good practice for filesystems on flash devices is the use of the noatime mount option, for the same reason.

ext2 data structures

The space in ext2 is split up into blocks. These blocks are divided into block groups, analogous to cylinder groups in the Unix File System. There are typically thousands of blocks on a large file system. Data for any given file is typically contained within a single block group where possible. This is done to reduce external fragmentation and minimize the number of disk seeks when reading a large amount of consecutive data.

Each block group contains a copy of the superblock and block group descriptor table, and all block groups contain a block bitmap, an inode bitmap, an inode table and finally the actual data blocks.

The superblock contains important information that is crucial to the booting of the operating system, thus backup copies are made in multiple block groups in the file system. However, typically only the first copy of it, which is found at the first block of the file system, is used in the booting.

The group descriptor stores the location of the block bitmap, inode bitmap and the start of the inode table for every block group and these, in turn are stored in a group descriptor table.

Inodes

Every file or directory is represented by an inode. The inode includes data about the size, permission, ownership, and location on disk of the file or directory.

Example of ext2 inode structure:

Quote from the linux kernel documentation for ext2:

"There are pointers to the first 12 blocks which contain the file's data in the inode. There is a pointer to an indirect block (which contains pointers to the next set of blocks), a pointer to a doubly-indirect block (which contains pointers to indirect blocks) and a pointer to a trebly-indirect block (which contains pointers to doubly-indirect blocks)."

So, there is a structure in ext2 that has 15 pointers, the first 12 are for direct blocks. Pointer number 13 points to an indirect block, number 14 to a doubly-indirect block and number 15 to a trebly-indirect block.

Directories

Each directory is a list of directory entries. Each directory entry associates one file name with one inode number, and consists of the inode number, the length of the file name, and the actual text of the file name. To find a file, the directory is searched front-to-back for the associated filename. For reasonable directory sizes, this is fine. But for huge large directories this is inefficient, and ext3 offers a second way of storing directories that is more efficient than just a list of filenames.

The root directory is always stored in inode number two, so that the file system code can find it at mount time. Subdirectories are implemented by storing the name of the subdirectory in the name field, and the inode number of the subdirectory in the inode field. Hard links are implemented by storing the same inode number with more than one file name. Accessing the file by either name results in the same inode number, and therefore the same data.

The special directories "." and ".." are implemented by storing the names "." and ".." in the directory, and the inode number of the current and parent directories in the inode field. The only special treatment these two entries receive is that they are automatically created when any new directory is made, and they cannot be deleted.

Allocating Data

When a new file or directory is created, the EXT2 file system must decide where to store the data. If the disk is mostly empty, then data can be stored almost anywhere. However, performance is maximized if the data is clustered with other related data to minimize seek times.

The EXT2 file system attempts to allocate each new directory in the group containing its parent directory, on the theory that accesses to parent and children directories are likely to be closely related. The EXT2 file system also attempts to place files in the same group as their directory entries, because directory accesses often lead to file accesses. However, if the group is full, then the new file or new directory is placed in some other non-full group.

The data blocks needed to store directories and files can found by looking in the data allocation bitmap. Any needed space in the inode table can be found by looking in the inode allocation bitmap.

File system limits

The reason for some limits of the ext2-file system are the file format of the data and the operating system's kernel. Mostly these factors will be determined once when the file system is built. They depend on the block size and the ratio of the number of blocks and inodes. In Linux the block size is limited by the architecture page size.

There are also some userspace programs that can't handle files larger than 2 GB.

The maximum file size is limited to min( ((b/4)³+(b/4)²+b/4+12)*b, 2³²*b ) due to the i_block (an array of EXT2_N_BLOCKS) and i_blocks( 32-bits integer value ) representing the amount of b-bytes "blocks" in the file.

The limit of sublevel-directories is 31998 due to the link count limit. Directory indexing is not available in ext2, so there are performance issues for directories with a large number of files (10,000+). The theoretical limit on the number of files in a directory is 1.3 × 10²⁰, although this is not relevant for practical situations.

Note: In Linux kernel 2.4 and earlier block devices were limited to 2 TB, limiting the maximum size of a partition regardless of block size.

Compression extension

e2compr is a modification to the ext2 file system driver in the Linux kernel to support online compression and decompression of files on file system level without any support by user applications.

e2compr is a small patch against the ext2 file system that allows on-the-fly compression and decompression. It compresses only regular files; the administrative data (superblock, inodes, directory files etc.) are not compressed (mainly for safety reasons). Access to compressed blocks is provided for read and write operations. The compression algorithm and cluster size is specified on a per-file basis. Directories can also be marked for compression, in which case every newly created file in the directory will be automatically compressed with the same cluster size and the same algorithm that was specified for the directory.

e2compr is not a new file system. It is only a patch to the ext2 file system made to support the EXT2_COMPR_FL flag. It does not require you to make a new partition, and will continue to read or write existing ext2 file systems. One can consider it as simply a way for the read and write routines to access files that could have been created by a simple utility similar to gzip or compress. Compressed and uncompressed files coexist nicely on ext2 partitions.

The latest e2compr-branch is available for current releases of 2.6 and 2.4 Linux kernels, but development is stalled. There are also older branches for older 2.0 and 2.2 kernels, which are more stable.

See also

• e2fsprogs
• ext : ancestor of ext2
• ext3 : extended version of ext2
• ext4 : fourth extended file system
• StegFS : a steganographic file system based on ext2
• cloop
• SquashFS
• List of file systems
• Comparison of file systems
• Filesystem in Userspace (FUSE)

References

Notes

Further reading

• John Newbigin. John's spec of the second extended filesystem. http://uranus.chrysocome.net/explore2fs/es2fs.htm. 
• Dave Poirier (2009). The Second Extended File System: Internal Layout. http://www.nongnu.org/ext2-doc/index.html. 
• Theodore Ts'o and Stephen Tweedie (June, 2002). "Planned Extensions to the Linux Ext2/Ext3 Filesystem". USENIX 2002 Annual Technical Conference. http://e2fsprogs.sourceforge.net/extensions-ext23/. 
• Larry Ayers (June 1997). "E2compr: Transparent File Compression for Linux". Linux Gazette (18). http://linuxgazette.net/issue18/e2compr.html. 
• Charles Cong and Jeremy H. Brown (1997-12-26) (PDF). A System for Transparent File Compression With Caching Under Linux. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.51.4780&rep=rep1&type=pdf. 
• Charles Cong and Jeremy H. Brown (1997-10-22) (PDF). A Survey of Modern File Compression Techniques. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.50.9847&rep=rep1&type=pdf. 

External links

• ext2fs user-space tools
• Read only software for ext2, ext3 on Windows
• Ext2read A windows application to read/copy ext2/ext3/ext4 files with extent and LVM2 support.
• A Free ext2 driver for Windows NT4.0/2000/XP/2003/Vista/2008 with full access to Linux Ext2 volumes (read access and write access). (Read-only access for ext3, too - see the FAQ)(Freeware)
• Ext2Fsd GPL ext2/ext3 file system driver for Windows 2000/XP/2003/VISTA/2008 (opensource, supports read & write, works with FreeOTFE)

exFAT (확장 파일 할당 테이블, Extended File Allocation Table, 줄여서 FAT64)는 특허 출원 중인^[1]사유 파일 시스템으로, 마이크로소프트사가 윈도 CE 6.0 장치와 데스크톱 운영 체제 윈도 비스타 서비스 팩 1^[2] 및 윈도 7, 그리고 자사의 서버 운영 체제인 윈도 서버 2008에 도입하기 위해 만든 것이다.^[3].

exFAT는 NTFS 파일 시스템이 자료 구조 오버헤드 등의 문제로 적절치 못할 경우, 또는 이전 버전인 FAT 파일 시스템의 파일 크기/디렉터리 제약이 문제가 되는 경우에 사용될 수 있다.

윈도 XP와 윈도 서버 2003 (둘다 x86, x64) 사용자들은 마이크로소프트사로부터 업데이트를 내려받아 설치하면 exFAT 지원을 사용할 수 있다.^[4] exFAT 파일 읽기를 지원하는 실험적인 오픈 소스 리눅스 커널 모듈은 현재 개발 중이다 ^[5]. 마이크로소프트 exFAT 드라이버로부터 라이선스 받아 전달된 클로즈드 소스의 읽기/쓰기 리눅스 드라이버는 Tuxera를 통해 구매하여 사용할 수 있다^[6]

이점

이전 파일 할당 테이블 (FAT) 파일 시스템 버전과 견주어 나아진 점은 다음과 같다:

• 대용량으로 크기를 넓힐 수 있음: 이론 상 최대 64 ZiB, 권장 최대 512 TiB 지원 - 이는 기존 FAT32 파티션의 2 TiB의 제한에서 상승한 것임. 다만 윈도 XP에 내장된 포맷 유틸리티는 새로운 FAT32 파티션을 32 GiB로까지 제한한다.^[4]
• 2⁹ (512)와 2¹² (4,096) 바이트의 섹터 크기
• 최대 32 MiB의 클러스터 ^[4]
• 파일 한 개 당 최대 64 ZiB (512 TiB 권장 최대) 지원 - 이는 FAT32에서 4 GiB에서 상승한 것임.^[4]
• 자유 공간 비트맵의 도입으로 자유 공간 할당 및 삭제 성능 개선
• 한 디렉터리에 최대 2,796,202개의 파일을 담을 수 있음^[1] - 이는 기존의 65,536개에서 상승한 것임.
• 접근 제어 목록 지원 (윈도 비스타 SP1에서는 아직 지원 안 함)^[7]
• TFAT 지원 - 트랜잭션 파일 시스템 표준 (WinCE 활성 기능은 선택 사항)
• OEM 정의 가능 변수 예비로 특정 드라이브 특성을 위한 파일 시스템의 사용자 지정 가능
• UTC 시간표 지원 (비스타 SP2부터 지원)^[8]
• 시간표 정밀도 10 ms (기존의 FAT 버전의 2 초보다 좋지만 NTFS의 100 ns보다는 나쁨)^[1]

단점

이전 FAT 버전과 견주어 나빠진 점은 다음과 같다:

• 윈도 XP, 윈도 서버 2003 사용자들은 exFAT 지원을 위하여 서비스 팩 2 이상 또는 별도의 업데이트를 설치하여야 함
• 윈도 비스타 사용자들은 exFAT 지원을 위하여 서비스 팩 1 이상을 설치하여야 함
• exFAT를 사용하여 포맷한 장치는 윈도 XP 이전의 버전, 도스, OS/2에 읽히지 않음
• exFAT를 사용하는 장치는 윈도 비스타의 레디부스트 기능을 사용할 수 없음 (윈도 7은 exFAT로 포맷한 드라이브에 대한 레디부스트 기능을 지원하며 기존 FAT32의 4GB 크기 제한이 없어짐으로써 더 넓은 레디부스트 캐시를 사용할 수 있음)^[9]
• 마이크로소프트사는 exFAT 파일 규격을 공개하지 않고 있고 exFAT 기능을 만들어 배포하려면 마이크로소프트로부터의 라이선스가 필요하다^[10]
• 현재 PC 환경 밖에서는 제한되거나 지원되지 않고 있음 — 텔레비전 및 A/V 수신기와 같은 대부분의 전자 기기는 이전의 FAT 버전만 다룰 수 있음 (이는 새로운 exFAT를 요구하는 SDXC 카드 및 메모리 스틱 XC와 함께 쓸 경우 달라질 수 있음)

라이선스

회사들은 exFAT를 사진기, 캠코더, 디지털 사진틀 등의 특정 그룹의 전자 기기에 통합할 수 있다. 다만 휴대 전화, 개인용 컴퓨터, 네트워크는 다른 가격 모델을 가진다. ^[11]

주석

Digital forensics (sometimes Digital forensic science) is a branch of forensic science encompassing the recovery and investigation of material found in digital devices, often in relation to computer crime.^[1][2] The term digital forensics was originally used as a synonym for computer forensics but has expanded to cover all devices capable of storing digital data and is now used to describe the entire field.^[1] The discipline evolved in a haphazard manner during the 1990s and it was not until the early 2000s that national policies were created.

Investigations can fall into four categories. The most common category is forensic analysis, where evidence is recovered to support or oppose a hypothesis before a criminal court, this is closely related to intelligence gathering, where material is intended to identify other suspects/crimes. eDiscovery is a form of discovery related to civil litigation and intrusion investigation is a specialist investigation into the nature and extent of an unauthorized network intrusion. The technical side of investigations is divided into several sub-branches; computer forensics, network forensics, database forensics and mobile device forensics. Any number of the fields may be utilised in an investigation.

As well as identifying direct evidence of a crime, digital forensics can be used to attribute evidence to specific suspects, confirm alibis or statements, determine intent, identify sources (for example, in copyright cases) or authenticate documents.^[3] Investigations are much broader in scope than other areas of forensic analysis (where the usual aim is to provide answers to a series of simpler questions) often involving complex time-lines or hypothesis.^[4]

The digital forensic process encompasses the seizure, forensic imaging (acquisition) and analysis of digital media. Finally producing a report of the digital evidence for the courts or an employer. Computer devices tend to store large amounts of information in cache/log files and deleted space and forensic examiners can recover this data as part of the analysis process.

History

Aerial photo of FLETC, where US digital forensics standards were developed in the 80s and 90s

Prior to the 1980s crimes involving computers were dealt with using existing laws. The first computer crimes were recognized in the 1978 Florida Computer Crimes Act, which included legislation against the unauthorized modification or deletion of data on a computer system.^[5][6] Over the next few years the range of computer crimes being committed increased, and laws were passed to deal with issues of copyright, privacy/harassment (e.g., cyber bullying, cyber stalking, and online predators) and child pornography.^[7][8] It was not until the 1980s that federal laws began to incorporate computer offences. Canada was the first country to pass legislation in 1983.^[6] This was followed by the US Federal Computer Fraud and Abuse Act in 1986, Australian amendments to their crimes acts in 1989 and the British Computer Abuse Act in 1990.^[6][8]

1980s-1990s: Growth of the field

The growth in computer crime during the 1980s and 1990s caused law enforcement agencies to begin establishing specialized groups, usually at the national level, to handle the technical aspects of investigations. For example, in 1984 the FBI launched a Computer Analysis and Response Team and the following year a computer crime department was set up within the British Metropolitan Police fraud squad. Besides being law enforcement professionals many of the early members of these groups were also computer hobbyists and became responsible for the fields initial research and direction.^[9][10]

One of the first practical (or at least publicised) examples of digital forensics was Cliff Stoll's pursuit of hacker Markus Hess in 1986. Stoll, whose investigation made use of both computer and network forensic techniques, was not a specialised examiner.^[11] Many of the earliest forensic examinations followed the same profile.^[12]

Throughout the 1990s there was high demand for the these new, and basic, investigative resources. The strain on central units lead to the creation of regional, and even local, level groups to help handle the load. For example, the British National Hi-Tech Crime Unit was set up in 2001 to provide a national infrastructure for computer crime; with personnel located both centrally in London and with the various regional police forces (the unit was folded into the Serious Organised Crime Agency (SOCA) in 2006).^[10]

During this period the science of digital forensics grew out of ad-hoc tools and techniques developed by these hobbyist practitioners. This is in contrast to other forensics disciplines which developed from work by the scientific community.^[1][13] It was not until 1992 that the term "computer forensics" was used in academic literature (although prior to this it had been in informal use); a paper by Collier and Spaul attempted to justify this new discipline to the forensic science world.^[14][15] This swift development resulted in a lack of standardization and training. In his 1995 book, "High-Technology Crime: Investigating Cases Involving Computers", K Rosenblatt wrote:^[6]

Seizing, preserving, and analyzing evidence stored on a computer is the greatest forensic challenge facing law enforcement in the 1990s. Although most forensic tests, such as fingerprinting and DNA testing, are performed by specially trained experts the task of collecting and analyzing computer evidence is often assigned to patrol officers and detectives^[16]

2000s: Developing standards

Since 2000, in response to the need for standardization, various bodies and agencies have published guidelines for digital forensics. The Scientific Working Group on Digital Evidence (SWGDE) produced a 2002 paper, "Best practices for Computer Forensics", this was followed, in 2005, by the publication of an ISO standard (ISO 17025, General requirements for the competence of testing and calibration laboratories).^[17][6][18] A European lead international treaty, the Convention on Cybercrime, came into force in 2004 with the aim of reconciling national computer crime laws, investigative techniques and international co-operation. The treaty has been signed by 43 nations (including the US, Canada, Japan, South Africa, UK and other European nations) and ratified by 16.

The issue of training also received attention. Commercial companies (often forensic software developers) began to offer certification programs and digital forensic analysis was included as a topic at the UK specialist investigator training facility, Centrex.^[6][10]

Since the late 1990s mobile devices have become more widely available, advancing beyond simple communication devices, and have been found to be be rich forms of information, even for crime not traditionally associated with digital forensics.^[19] Despite this, digital analysis of phones has lagged behind traditional computer media, largely due to problems over the proprietary nature of devices.^[20]

Focus has also shifted onto internet crime, particularly the risk of cyber warfare and cyberterrorism. A February 2010 report by the U.S. Joint Forces Command concluded:

Through cyberspace, enemies will target industry, academia, government, as well as the military in the air, land, maritime, and space domains. In much the same way that airpower transformed the battlefield of World War II, cyberspace has fractured the physical barriers that shield a nation from attacks on its commerce and communication.^[21]

The field of digital forensics still faces outstanding issues. A 2009 paper, "Digital Forensic Research: The Good, the Bad and the Unaddressed", by Peterson and Shenoi identified a bias towards Windows operating systems in digital forensics research.^[22] In 2010 Simson Garfinkel identified issues facing digital investigations in the future; including the increasing size of digital media, the wide availability of encryption to consumers, a growing variety of operating systems and file formats, an increasing number of individuals owning multiple devices and legal limitations on investigators. The paper also identified continued training issues, as well as the prohibitively high cost of entering the field.^[11]

Investigative tools

During the 1980s very few specialised digital forensic tools existed and investigators often performed live analysis on media, examining computers from within the operating system using existing sysadmin tools to extract evidence. This risked modifying data on the disk (inadvertently or otherwise) leading to claims of evidence tampering. In the early 1990s a number of tools were created to allow investigations to take place without the risk of altering data.

The need for such software was first recognised in 1989 at the Federal Law Enforcement Training Center, resulting in the creation of IMDUMP (by Michael White) and in 1990, SafeBack (developed by Sydex). Similar pieces of software were produced in other countries; DIBS (a hardware and software solution) was released commercially in the UK in 1991 and Rob McKemmish released Fixed Disk Image free to Australian law enforcement.^[9] These tools allowed examiners to create an exact copy of a piece of digital media to work on; leaving the original disk intact for verification. By the end of the 90s, as demand for digital evidence grew more advanced, commercial tools (EnCase, FTK, etc.) were developed that allowed analysts to examine copies of media without using any live forensics.^[6]

More recently the same progression of tool development has occurred for mobile devices; initially investigators accessed data directly on the device, these were soon replaced with specialist tools (such as XRY or Radio Tactics Aceso).^[6]

Forensic Process

A portable Tableau write-blocker attached to a Hard Drive

A digital forensic investigation commonly consists of 3 stages; acquisition or imaging of exhibits, analysis and reporting.^[6][23] Acquisition involves creating an exact sector level duplicate (or "forensic duplicate") of the media, often using a write blocking device to prevent modification of the original. Both acquired image and original media are hashed (using SHA-1 or MD5) and the values compared to verify the copy is accurate.^[24]

During the analysis phase an investigators recover evidence material using a number of different methodologies and tools. In 2002 and article in the International Journal of Digital Evidence referred to this step as "an in-depth systematic search of evidence related to the suspected crime".^[1] In 2006, forensics researcher Brian Carrie described an "intuitive procedure" in which obvious evidence is first identified and then "exhaustive searches are conducted to start filling in the holes."^[4]

The actual process of analysis can vary between investigations, but common methodologies include conducting keyword searches across the digital media (within files as well as unallocated and slack space), recovering deleted files and extraction of registry information (for example to list user accounts, or attached USB devices).

Once evidence is recovered it is analysed to reconstruct events or actions and to reach conclusions, work that can often be performed by less specialist staff.^[1] When an investigation is complete the data is presented, usually in the form of a written report, in lay persons terms.^[1]

Forms and uses

An example of an image's Exif metadata that might be used to prove its origin

Investigations can take one of four forms. Firstly as a forensic examination, traditionally associated with criminal law, where evidence is collected to support or oppose a hypothesis. Like other areas of forensics this is often part of a wider investigation spanning a number of disciplines. A related form is "intelligence gathering", functionally identical to a forensic examination the digital evidence is intended to be used as intelligence to (for example) locate, identify or halt other crimes. As a result intelligence gathering is sometimes held to a less strict forensic standard.

In civil litigation or corporate matters the process is referred to as electronic discovery (or eDiscovery). The forensic procedure is similar to that used in criminal investigations but with different legal requirements and limitations. Finally, intrusion investigation is a specialist examination into the nature and extent of an unauthorized network intrusion. Intrusion analysis is usually performed as a damage limitation exercise after an attack, both to establish the extent of any intrusion and to try and identify the attacker.^[4][3] Such attacks were commonly conducted over phone lines during the 1980s, but in the modern era are usually propagated over the internet.^[25]

The main use of digital forensics is to recover objective evidence of a criminal activity (termed actus reus in legal parlance). However the diverse range of data held in digital devices can help with other areas of investigation.^[3]

Attribution

Meta data and other logs can be used to attribute actions to an individual. For example, personal documents on a computer drive might identify its owner.

Alibis and statements

Information provided by those involved can be cross checked with digital evidence. For example, during the investigation into the Soham murders, the offenders alibi was disproven when mobile phone records of the person he claimed to be with showed she was out of town at the time.

Intent

As well as finding objective evidence of a crime being committed, investigations can also be used to prove the intent (known by the legal term mens rea). For example, the Internet history of convicted killer Neil Entwistle included references to a site discussing How to kill people.

Evaluation of source

File artifacts and meta-data can be used to identify the origin of a particular piece of data; for example, older versions of Microsoft Word embedded a Global Unique Identifer into files which identified the computer it had been created on. Proving whether a file was produced on the digital device being examined or obtained from elsewhere (e.g., the Internet) can be very important.^[3]

Document authentication

Related to "Evaluation of Source", meta data associated with digital documents can be easily modified (for example, by changing the computer clock you can affect the creation date of a file). Document authentication relates to detecting and identifying falsification of such details.

Limitations

One major limitation to a forensic investigation is the use of encryption; this disrupts initial examination where pertinent evidence might be located using keywords. Laws to compel individuals to disclose encryption keys are still relatively new and controversial.^[11]

Legal considerations

The examination of digital media is covered by national and international legislation. For civil investigations, in particular, laws may restrict the abilities of analysts to undertake examinations. Restrictions against network monitoring, or reading of personal communications often exist.^[26] For criminal investigation, national laws restrict how much information can be seized.^[26] For example, in the United Kingdom seizure of evidence by law enforcement is governed by the PACE act.^[6] The "International Organization on Computer Evidence" (IOCE) is one agency that works to establish compatible international standards for the seizure of evidence.^[27]

In the UK the same laws covering computer crime can also affect forensic investigators. The 1990 computer misuse act legislates against unauthorised access to computer material; this is a particular concern for civil investigators who have more limitations than law enforcement.

An individuals right to privacy is one area of digital forensics which is still largely undecided by courts. The US Electronic Communications Privacy Act places limitations on the ability of law enforcement or civil investigators to intercept and access evidence. The act makes a distinction between stored communication (e.g. email archives) and transmitted communication (such as VOIP). The latter, being considered more of a privacy invasion, is harder to obtain a warrant for.^[6][16] The ECPA also affects the ability of companies to investigate the computers and communications of their employees, an aspect that is still under debate as to the extent to which a company can perform such monitoring.^[6]

In Europe Article 5 of the European Convention on Human Rights asserts similar privacy limitations to the ECPA. In addition it also limits the processing and sharing of personal data both within the EU and with external countries. In the UK the ability of law enforcement to conduct digital forensics investigations is legislated by the Regulation of Investigatory Powers Act.^[6]

Digital evidence

Digital evidence can come in a number of forms

Where it will be used in a court of law, digital evidence falls under the same legal guidelines as other forms of evidence, courts do not usually require more stringent guidelines.^[6][28] In the United States the Federal Rules of Evidence are used to evaluate the admissibility of digital evidence, the United Kingdom PACE and Civil Evidence acts have similar guidelines and many other countries have their own laws. US federal laws restrict seizures to items with only obvious evidential value. This is acknowledged as not always being possible to establish with digital media prior to an examination.^[26]

Laws dealing with digital evidence refer to two considerations; integrity and authenticity. Integrity is ensuring that the act of seizing and acquiring digital media does not modify the evidence (either the original or the copy). Authenticity refers to the ability to confirm the integrity of information; for example that the imaged media matches the original evidence.^[26] The ease with which digital media can be modified means that documenting the chain of custody from the crime scene, through analysis and, ultimately, to the court, (a form of audit trail) is important to establish the authenticity of evidence.^[6]

Attorneys have argued that because digital evidence can theoretically be altered it undermines the reliability of the evidence. US judges are beginning to reject this theory, in the case US v. Bonallo the court ruled that "the fact that it is possible to alter data contained in a computer is plainly insufficient to establish untrustworthiness".^[6][29] In the United Kingdom guidelines such as those issued by ACPO are followed to help document the authenticity and integrity of evidence.

Digital investigators, particularly in criminal investigations, have to ensure that conclusions are based upon factual evidence and their own expert knowledge.^[6] In the US, for example, Federal Rules of Evidence state that a qualified expert may testify “in the form of an opinion or otherwise” so long as:

(1) the testimony is based upon sufficient facts or data, (2) the testimony is the product of reliable principles and methods, and (3) the witness has applied the principles and methods reliably to the facts of the case.^[30]

Many of the sub-branches of digital forensics have their own specific guidelines for handling and investigating evidence. For example, mobile phones are often acquired inside a Faraday shield to stop radio traffic to the device. Or, in the UK forensic examination of computers in criminal matters is subject to ACPO guidelines.^[6]

Branches

Digital forensics includes several sub-branches relating to the investigation of various types of devices, media or artefacts.

Computer forensics

The goal of computer forensics is to explain the current state of a digital artifact; such as a computer system, storage medium or electronic document.^[31] The discipline usually covers computers, embedded systems (digital devices with rudimentary computing power and onboard memory) and static memory (such as USB pen drives).

Computer forensics can deal with a broad range of information; from logs (such as internet history) through to the actual files on the drive. In 2007 prosecutors used a spreadsheet recovered from the computer of Joseph E. Duncan III to show premeditation and secure the death penalty.^[3] Sharon Lopatka's killer was identified in 2006 after email messages from him detailing torture and death fantasies were found on her computer.^[6]

Mobile phones in a UK Evidence bag

Mobile device forensics

Mobile device forensics is a sub-branch of digital forensics relating to recovery of digital evidence or data from a mobile device. It differs from Computer forensics in that a mobile device will have an inbuilt communication system (e.g. GSM) and, usually, proprietary storage mechanisms. Investigations usually focus on simple data such as call data and communications (SMS/Email) rather than in-depth recovery of deleted data.^[6][32] SMS data from a mobile device investigation helped to exonerate Patrick Lumumba in the murder of Meredith Kercher.^[3]

Mobile devices are also useful for providing location information; either from inbuilt gps/location tracking or via cell site logs (which track the devices within their range). Such information was used to track down the kidnappers of Thomas Onofri in 2006.^[3]

Network forensics

Network forensics relates to the monitoring and analysis of computer network (both local network and WAN/internet) traffic for the purposes of information gathering, legal evidence or intrusion detection.^[33] Traffic is intercepted (usually at the packet level) and either stored for later analysis with specialist tools or filtered in real time for relevant information. Unlike other areas of digital forensics, network data is often volatile and seldom logged, making the discipline often reactionary.

In 2000, the FBI lured computer hackers Aleksey Ivanov and Gorshkov to the United States for a fake job interview. By monitoring network traffic from the pair's computers, the FBI identified passwords that let it collect evidence directly from Russia-based computers.^[6][34]

Database forensics

Database forensics is a branch of digital forensics relating to the forensic study of databases and their metadata.^[35] Investigations use database contents, log files and in-RAM data in order to build a time-line or recover relevant information.

See also

• Cyberspace
• Glossary of digital forensics terms

Related journals

• Journal of Digital Forensics, Security and Law
• International Journal of Digital Crime and Forensics
• Journal of Digital Investigation
• International Journal of Digital Evidence
• International Journal of Forensic Computer Science
• Journal of Digital Forensic Practice
• Small Scale Digital Device Forensic Journal

References

Further reading

Look up digital forensics in Wiktionary, the free dictionary.

Wikibooks has more on the topic of Digital forensics

• Carrier, Brian D. (February 2006). "Risks of live digital forensic analysis". Communications of the ACM 49 (2): 56–61. doi:10.1145/1113034.1113069. ISSN 0001-0782. http://portal.acm.org/citation.cfm?id=1113034.1113069. Retrieved 31 August 2010. 
• Kanellis, Panagiotis. Digital crime and forensic science in cyberspace. IGI Publishing. p. 357. ISBN 1591408733. http://books.google.co.uk/books?id=oK_oYhTPW2gC. 
• Jones, Andrew (2008). Building a Digital Forensic Laboratory. Butterworth-Heinemann. p. 312. ISBN 1856175103. http://books.google.co.uk/books?id=F5IU7XXKwCQC. 
• Marshell, Angus M. (2008). Digital forensics: digital evidence in criminal investigation. Wiley-Blackwell. p. 148. ISBN 0470517751. http://books.google.co.uk/books?id=MC0FPQAACAAJ. 
• "Timeline of computer forensics". http://www.pc-history.org/forensics.htm.