From Wikipedia, the free encyclopedia

Data remanence is the residual representation of data that has been in some way been nominally erased or removed. This residue may be due to data being left intact by a nominal delete operation, or through physical properties of the storage medium. Data remanence may make inadvertent disclosure of sensitive information possible, should the storage media be released into an uncontrolled environment (e.g., thrown in the trash, or given to a third-party).

Over time, various techniques have been developed to counter data remanence. Depending on the effectiveness and intent, they are often classified as either clearing or purging/sanitizing. Specific methods include overwriting, degaussing, encryption, and physical destruction.

Contents 1 Causes 2 Countermeasures 2.1 Clearing 2.2 Purging 3 Specific methods 3.1 Overwriting 3.1.1 Gutmann method 3.2 Degaussing 3.3 Encryption 3.4 Physical destruction 4 Complications 4.1 Inaccessible media areas 4.2 Advanced storage systems 4.3 Optical media 4.4 Data remanence in RAM 5 Standards 6 See also 6.1 Standards 6.2 Software 7 References

Many operating systems, file managers, and other software provide a facility where file are not immediately deleted when the user requests that action. Instead, the file is moved to a holding area, to allow the user to easily revert a mistake.

Even when an explicit deleted file retention facility is not provided, most computers do not actually remove the contents of a file when it is deleted. Instead, they simply remove the file's entry from the file system directory. The contents of the file -- the actual data -- remain on the storage medium. The data will remain there until the operating system reuses the space for new data. In some systems, enough filesystem metadata is also left behind to enable easy undeletion by commonly available utility software. Even when undelete is not possible, until the actual data is overwritten, it can be read by software that reads disk sectors directly. Computer forensics often employs such software.

Likewise, reformatting, repartitioning, reghosting or reimaging a system is not always guaranteed to write to every area of the disk, though all will cause the disk to appear empty to most software.

Finally, even when the storage medium is overwritten, physical properties of the medium may make it possible to recover the previous contents using laboratory techniques.

Clearing is the removal of sensitive data from storage devices in such a way that there is assurance, proportional to the sensitivity of the data, that the data may not be reconstructed using normal system functions. The data may still be recoverable, but not without unusual effort.

Clearing is typically considered an administrative protection against accidental disclosure within an organization. For example, before a floppy disk is re-used within an organization, its contents may cleared to prevent their accidental disclosure to the next user.

Purging or sanitizing is the removal of sensitive data from a system or storage device with the intent that the data can not be reconstructed by any known technique. Purging is generally done before releasing media outside of control, such as before discarding old media, or moving media to a computer with different security requirements.

A common method used to counter data remanence is to overwrite the storage medium with new data. This is often called a wiping or shredding a file or disk. Because such methods can often be implemented in software alone, and may be able to selectively target only part of a medium, it is a popular, low-cost option for some applications.

The simplest overwrite technique writes the same data everywhere -- often just a pattern of all zeros. At a minimum, this will prevent the data from being retrieved simply by reading from the medium again, and thus is often used for clearing.

To counter more advanced data recovery techniques, specific overwrite patterns are often prescribed. These may be generic patterns intended to eradicate any trace signatures. For example, writing repeated, alternating patterns of ones and zeros may be more effective than zeros alone. Patterns based on the existing data may also be used, such as the complement, or bitwise inverse or NOT, of the existing data. For example, if the existing data is 1101 0110, its complement would be 0010 1001. Combinations of patterns are frequently specified.

One challenge with an overwrite is that some areas of the disk may be inaccessible, due to media degradation or other errors. Software overwrite may also be problematic in high-security environments which require stronger controls on data commingling than can be provided by the software in use. The use of advanced storage technologies may also make file-based overwrite ineffective.

As of Nov 2007, overwriting is no longer a DoD-acceptable sanitization method for magnetic media. Only degaussing or physical destruction is acceptable.^[1]

Peter Gutmann investigated data recovery from nominally overwritten media in the mid-1990s. He suggested magnetic force microscopy may be able to recover such data, and developed specific patterns, for specific drive technologies, designed to counter such.^[2] These patterns have come to be known as the Gutmann method.

Daniel Feenberg, an economist at the private National Bureau of Economic Research, claims that the chances of overwritten data being recovered from a modern hard drive amount to "urban legend".^[3]

Degaussing is the removal or reduction of a magnetic field. Applied to magnetic media, degaussing may purge an entire media element quickly and effectively. A device, called a degausser, designed for the media being erased, is used.

Degaussing often renders hard disks inoperable, as it erases low-level formatting which is only done at the factory, during manufacture. Degaussed floppy disks can generally be reformatted and reused.

In high-security environments, one may be required to use degausser which has been approved for the task. For example, in US government and military jurisdictions, one may be required to use a deguasser from the NSA's "Evaluated Products List" ^[4].

Encrypting data before it is stored on the medium may mitigate concerns about data remanence. If the decryption key is strong and carefully controlled (i.e., not itself subject to data remanence), it may effectively make any data on the medium unrecoverable. Even if the key is stored on the medium, it may prove easier or quicker to overwrite just the key, vs the entire disk.

Encryption may be done on a file-by-file basis, or on the whole disk.

Physical destruction of the data storage medium is generally considered the most certain way to counter data remanence, although also at the highest cost. Not only is the process generally time-consuming and cumbersome, it obviously renders the media unusable. Further, with the high recording densities of modern media, even a small media fragment may contain large amounts of data.

Specific destruction techniques include:

• Physically breaking the media apart, by grinding, shredding, etc.
• Incinerating
• Phase transition (i.e., liquification or vaporization of a solid disk)
• Application of corrosive chemicals, such as acids, to recording surfaces
• For magnetic media, raising its temperature above the Curie point

Storage media may have areas which become inaccessible by normal means. For example, a magnetic disks may develop new "bad sectors" after data has been written, and tapes require inter-record gaps. Modern hard disks often feature automatic remapping of marginal sectors or tracks, which the OS may not even be aware of. Attempts to counter data remanence by overwriting may not be successful in such situations, as data remnants may persist in such nominally inaccessible areas.

Data storage systems with more sophisticated features may make overwrite ineffective, especially on a per-file basis.

Journaling file systems increase the integrity of data by recording write operations in multiple locations, and applying transaction-like semantics. On such systems, data remnants may exist in locations "outside" the nominal file storage location.

Some file systems implement copy-on-write or built-in revision control, with the intent that writing to a file never overwrites data in-place.

Technologies such as RAID and anti-fragmentation techniques may result in file data being written to multiple locations, either by design (for fault tolerance), or as data remnants.

Optical media are not magnetic and are not affected by degaussing. Write-once optical media (CD-R, DVD-R, etc.) also cannot be purged by overwrite. Read/write optical media, such as CD-RW and DVD-RW, may be receptive to overwriting.

Data remnance has been observed in static RAM, which is typically considered volatile (i.e., contents are erased with loss of electrical power). In the study, data retention was sometimes observed even at room temperature.^[5]

• US DoD DoD 5220.22-M: National Industrial Security Program Operating Manual (NISPOM)
  • Current editions no longer specify specific sanitization methods. Standards for sanitization are left up to the Cognizant Security Authority. ^[6]
  • Past editions (1995) explicitly gave sanitization methods.^[7]
  • The Defense Security Service provides a Clearing and Sanitization Matrix (C&SM) which does specify methods^[1].
  • As of the Nov 2007 edition of the DSS C&SM, overwriting is no longer acceptable for sanitization of magnetic media. Only degaussing (with an NSA approved degausser) or physical destruction is acceptable.
• NAVSO P5239-26 standards are more rigorous versions of the same procedure^[clarify].
• US NIST Special Publication 800-88: Guidelines for Media Sanitization^[8]
• RCMP G2-003: Hard Drive Secure Information Removal and Destruction Guidelines^[9]
  • Up to A/B/Confidential data: Triple overwrite using RCMP DSX software
  • C/Secret/Top Secret data: Physical destruction or degaussing

• File deletion
• Security
• Cryptography
• Encryption
• Physical information security
• Gutmann method
• Computer forensics
• Electronic waste
• Data recovery
• Forensic identification
• Paper shredder
• Plaintext (security discussion)
• Zeroisation
• Sanitization (classified information)

• NAVSO P5239-26
• AFSSI-5020
• AR380-19

• Darik's Boot and Nuke
• shred (part of the GNU Coreutils package)

There are dozens of other tools for various operating systems.

1. ^ ^{a b} DSS Clearing & Sanitization Matrix (PDF). DSS (2007-11-12). Retrieved on 2007-11-25. (89 KB)
2. ^ Peter Gutmann (July 1996). "Secure Deletion of Data from Magnetic and Solid-State Memory". Retrieved on 2007-12-10.
3. ^ Daniel Feenberg. "Can Intelligence Agencies Recover Overwritten Data?". Retrieved on 2007-12-10.
4. ^ Evaluated Products. NSA. Retrieved on 2007-12-10.
5. ^ Sergei Skorobogatov (June 2002). "Low temperature data remanence in static RAM". University of Cambridge, Computer Laboratory.
6. ^ Download NISPOM. DSS. Retrieved on 2007-11-25.
7. ^ Obsolete NISPOM (January 1995). Retrieved on 2007-12-07. Section 8-306, Clearing and Sanitization Matrix.
8. ^ Special Publication 800-88: Guidelines for Media Sanitization (PDF). NIST (September 2006). Retrieved on 2007-12-08. (542 KB)
9. ^ Hard Drive Secure Information Removal and Destruction Guidelines (PDF). Royal Canadian Mounted Police (October 2003).

• (Sep 1991) A Guide to Understanding Data Remanence in Automated Information Systems. National Computer Security Center. Retrieved on 2007-12-10.  (Rainbow Series "Forrest Green Book")