Data remanence
Data remanence is the residual representation of
Various techniques have been developed to counter data remanence. These techniques are classified as clearing, purging/sanitizing, or destruction. Specific methods include overwriting, degaussing, encryption, and media destruction.
Effective application of countermeasures can be complicated by several factors, including media that are inaccessible, media that cannot effectively be erased, advanced storage systems that maintain histories of data throughout the data's life cycle, and persistence of data in memory that is typically considered volatile.
Several standards exist for the secure removal of data and the elimination of data remanence.
Causes
Many
Even when an explicit deleted file retention facility is not provided or when the user does not use it, operating systems do not actually remove the contents of a file when it is deleted unless they are aware that explicit erasure commands are required, like on a
Likewise, reformatting, repartitioning, or reimaging a system is unlikely to write to every area of the disk, though all will cause the disk to appear empty or, in the case of reimaging, empty except for the files present in the image, to most software.
Finally, even when the storage media is overwritten, physical properties of the media may permit recovery of the previous contents. In most cases however, this recovery is not possible by just reading from the storage device in the usual way, but requires using laboratory techniques such as disassembling the device and directly accessing/reading from its components.
§ Complications below gives further explanations for causes of data remanence.
Countermeasures
There are three levels commonly recognized for eliminating remnant data:
Clearing
Clearing is the removal of sensitive data from storage devices in such a way that there is assurance that the data may not be reconstructed using normal system functions or software file/data recovery utilities. The data may still be recoverable, but not without special laboratory techniques.[1]
Clearing is typically an administrative protection against accidental disclosure within an organization. For example, before a
Purging
Purging or
Destruction
The storage media is made unusable for conventional equipment. Effectiveness of destroying the media varies by medium and method. Depending on recording density of the media, and/or the destruction technique, this may leave data recoverable by laboratory methods. Conversely, destruction using appropriate techniques is the most secure method of preventing retrieval.
Specific methods
Overwriting
A common method used to counter data remanence is to overwrite the storage media with new data. This is often called wiping or shredding a file or disk, by
The simplest overwrite technique writes the same data everywhere—often just a pattern of all zeros. At a minimum, this will prevent the data from being retrieved simply by reading from the media again using standard system functions.
In an attempt to counter more advanced data recovery techniques, specific overwrite patterns and multiple passes have often been prescribed. These may be generic patterns intended to eradicate any trace signatures; an example is the seven-pass pattern 0xF6, 0x00, 0xFF, <random byte>, 0x00, 0xFF, <random byte>, sometimes erroneously attributed to US standard
One challenge with overwriting is that some areas of the disk may be inaccessible, due to media degradation or other errors. Software overwrite may also be problematic in high-security environments, which require stronger controls on data commingling than can be provided by the software in use. The use of advanced storage technologies may also make file-based overwrite ineffective (see the related discussion below under ).
There are specialized machines and software that are capable of doing overwriting. The software can sometimes be a standalone operating system specifically designed for data destruction. There are also machines specifically designed to wipe hard drives to the department of defense specifications DOD 5220.22-M.[3]
Feasibility of recovering overwritten data
Daniel Feenberg, an economist at the private
As of November 2007, the United States Department of Defense considers overwriting acceptable for clearing magnetic media within the same security area/zone, but not as a sanitization method. Only degaussing or physical destruction is acceptable for the latter.[6]
On the other hand, according to the 2014
Degaussing
Degaussing often renders
In some high-security environments, one may be required to use a degausser that has been approved for the task. For example, in US government and military jurisdictions, one may be required to use a degausser from the NSA's "Evaluated Products List".[9]
Encryption
Encrypting data before it is stored on the media may mitigate concerns about data remanence. If the decryption key is strong and carefully controlled, it may effectively make any data on the media unrecoverable. Even if the key is stored on the media, it may prove easier or quicker to overwrite just the key, versus the entire disk. This process is called crypto-shredding.
Encryption may be done on a file-by-file basis, or on the whole disk. Cold boot attacks are one of the few possible methods for subverting a full-disk encryption method, as there is no possibility of storing the plain text key in an unencrypted section of the medium. See the section Complications: Data in RAM for further discussion.
Other
Media destruction
Thorough destruction of the underlying storage media is the most certain way to counter data remanence. However, the process is generally time-consuming, cumbersome, and may require extremely thorough methods, as even a small fragment of the media may contain large amounts of data.
Specific destruction techniques include:
- Physically breaking the media apart (e.g., by grinding or shredding)
- corrosivechemicals)
- Phase transition (e.g., liquefaction or vaporization of a solid disk)
- For magnetic media, raising its temperature above the Curie point
- For many electric/electronic volatile and non-volatile storage media, exposure to electromagnetic fields greatly exceeding safe operational specifications (e.g., high-voltage electric current or high-amplitude microwave or ionizing radiation)[citation needed]
Complications
Inaccessible media areas
Storage media may have areas which become inaccessible by normal means. For example,
Advanced storage systems
Data storage systems with more sophisticated features may make
Wear leveling can also defeat data erasure, by relocating blocks between the time when they are originally written and the time when they are overwritten. For this reason, some security protocols tailored to operating systems or other software featuring automatic wear leveling recommend conducting a free-space wipe of a given drive and then copying many small, easily identifiable "junk" files or files containing other nonsensitive data to fill as much of that drive as possible, leaving only the amount of free space necessary for satisfactory operation of system hardware and software. As storage and system demands grow, the "junk data" files can be deleted as necessary to free up space; even if the deletion of "junk data" files is not secure, their initial nonsensitivity reduces to near zero the consequences of recovery of data remanent from them.[citation needed]
Optical media
As
Data on solid-state drives
Research from the Center for Magnetic Recording and Research, University of California, San Diego has uncovered problems inherent in erasing data stored on solid-state drives (SSDs). Researchers discovered three problems with file storage on SSDs:[10]
First, built-in commands are effective, but manufacturers sometimes implement them incorrectly. Second, overwriting the entire visible address space of an SSD twice is usually, but not always, sufficient to sanitize the drive. Third, none of the existing hard drive-oriented techniques for individual file sanitization are effective on SSDs.[10]: 1
Solid-state drives, which are flash-based, differ from hard-disk drives in two ways: first, in the way data is stored; and second, in the way the algorithms are used to manage and access that data. These differences can be exploited to recover previously erased data. SSDs maintain a layer of indirection between the logical addresses used by computer systems to access data and the internal addresses that identify physical storage. This layer of indirection hides idiosyncratic media interfaces and enhances SSD performance, reliability, and lifespan (see wear leveling), but it can also produce copies of the data that are invisible to the user and that a sophisticated attacker could recover. For sanitizing entire disks, sanitize commands built into the SSD hardware have been found to be effective when implemented correctly, and software-only techniques for sanitizing entire disks have been found to work most, but not all, of the time.[10]: section 5 In testing, none of the software techniques were effective for sanitizing individual files. These included well-known algorithms such as the Gutmann method, US DoD 5220.22-M, RCMP TSSIT OPS-II, Schneier 7 Pass, and Secure Empty Trash on macOS (a feature included in versions OS X 10.3-10.9).[10]: section 5
The TRIM feature in many SSD devices, if properly implemented, will eventually erase data after it is deleted[11][citation needed], but the process can take some time, typically several minutes. Many older operating systems do not support this feature, and not all combinations of drives and operating systems work.[12]
Data in RAM
Data remanence has been observed in static random-access memory (SRAM), which is typically considered volatile (i.e., the contents degrade with loss of external power). In one study, data retention was observed even at room temperature.[13]
Data remanence has also been observed in
: 12Despite some memory degradation, authors of the above described study were able to take advantage of redundancy in the way keys are stored after they have been expanded for efficient use, such as in
Standards
- Australia
- Canada
- RCMP B2-002, IT Media Overwrite and Secure Erase Products, May 2009 [17]
- Communications Security Establishment Clearing and Declassifying Electronic Data Storage Devices, July 2006 [18]
- New Zealand
- GCSB NZISM 2016, New Zealand Information Security Manual v2.5, July 2016 [19]
- NZSIS PSM 2009, Protective Security Manual
- United Kingdom
- Asset Disposal and Information Security Alliance (ADISA), ADISA IT Asset Disposal Security Standard[20]
- United States
- NIST Special Publication 800-88, Guidelines for Media Sanitization, September 2006 [1]
- DoD 5220.22-M, National Industrial Security Program Operating Manual (NISPOM), February 2006 [21]
- Current editions no longer contain any references to specific sanitization methods. Standards for sanitization are left up to the Cognizant Security Authority.[21]
- Although the NISPOM text itself never described any specific methods for sanitization, past editions (1995 and 1997)[22] did contain explicit sanitization methods within the Defense Security Service (DSS) Clearing and Sanitization Matrix inserted after Section 8-306. The DSS still provides this matrix and it continues to specify methods.[6] As of the Nov 2007 edition of the matrix, overwriting is no longer acceptable for sanitization of magnetic media. Only degaussing (with an NSA approved degausser) or physical destruction is acceptable.
- Army AR380-19, Information Systems Security, February 1998 [23] replaced by AR 25-2 https://armypubs.army.mil/epubs/DR_pubs/DR_a/pdf/web/ARN17503_AR25_2_Admin_FINAL.pdf (Army Publishing Directorate, 2009)
- Air Force AFSSI 8580, Remanence Security, 17 November 2008[24]
- Navy NAVSO P5239-26, Remanence Security, September 1993 [25]
See also
- Computer forensics
- Cryptography
- Data erasure
- Data recovery
- Electronic waste
- Encryption
- File deletion
- Forensic identification
- Gutmann method
- Memory scrambling
- Palimpsest
- Paper shredder
- Physical information security
- Plaintext (security discussion)
- Remanence (magnetic retentivity)
- Sanitization (classified information)
- Secure USB drive
- Zeroisation
References
- ^ a b "Special Publication 800-88: Guidelines for Media Sanitization Rev. 1" (PDF). NIST. 6 September 2012. Retrieved 2014-06-23. (542 KB)
- )
- CiteSeerX 10.1.1.180.8813.
- ^ Peter Gutmann (July 1996). "Secure Deletion of Data from Magnetic and Solid-State Memory". Retrieved 2007-12-10.
{{cite journal}}
: Cite journal requires|journal=
(help) - ^ Daniel Feenberg. "Can Intelligence Agencies Recover Overwritten Data?". Retrieved 2007-12-10.
{{cite journal}}
: Cite journal requires|journal=
(help) - ^ DSS. 2007-06-28. Retrieved 2010-11-04.
- doi:10.6028/NIST.SP.800-88r1. Retrieved 2018-06-26.)
{{cite journal}}
: Cite journal requires|journal=
(help - ISBN 978-3-540-89861-0.
- ^ "Media Destruction Guidance". NSA. Archived from the original on 2012-09-28. Retrieved 2009-03-01.
- ^ a b c d Michael Wei; Laura M. Grupp; Frederick E. Spada; Steven Swanson (February 2011). "Reliably Erasing Data From Flash-Based Solid State Drives" (PDF).
{{cite journal}}
: Cite journal requires|journal=
(help) - ^ Homaidi, Omar Al (2009). "Data Remanence: Secure Deletion of Data in SSDs".
{{cite journal}}
: Cite journal requires|journal=
(help) - ^ "Digital Evidence Extraction Software for Computer Forensic Investigations". Forensic.belkasoft.com. October 2012. Retrieved 2014-04-01.
- )
- ^ a b c J. Alex Halderman; et al. (July 2008). "Lest We Remember: Cold Boot Attacks on Encryption Keys" (PDF).
{{cite journal}}
: Cite journal requires|journal=
(help) - ^ https://www.veracrypt.fr/en/Release%20Notes.html VeraCrypt release notes, version 1.24
- ^ "Australia Government Information Security Manual" (PDF). Australian Signals Directorate. 2014. Archived from the original (PDF) on 2014-03-27.
- ^ "IT Media Overwrite and Secure Erase Products" (PDF). Royal Canadian Mounted Police. May 2009. Archived from the original (PDF) on 2011-06-15.
- ^ "Clearing and Declassifying Electronic Data Storage Devices". Communications Security Establishment. July 2006. Archived from the original (PDF) on 2014-08-07. Retrieved 2016-10-09.
- ^ "New Zealand Information Security Manual v2.5" (PDF). Government Communications Security Bureau. July 2016.
- ^ "ADISA: ASSET DISPOSAL & INFORMATION SECURITY ALLIANCE". Archived from the original on 2010-11-01.
- ^ DSS. February 2006. Archived from the original(PDF) on 2011-05-24. Retrieved 2010-09-22.
- Defense Security Service(DSS) Clearing and Sanitization Matrix; includes Change 1, July 31, 1997.
- ^ "Information Systems Security" (PDF). February 1998.
- ^ AFI 33-106 Archived 2012-10-22 at the Wayback Machine
- ^ "Remanence Security Guidebook". September 1993.
- ^ "IEEE Standard for Sanitizing Storage".
- ^ "IEEE 2883 Standard On Data Sanitization Is A Path To Storage Reuse And Recycling as published on Forbes". Forbes.
- ^ "IEEE P2883™ Draft Standard for Sanitizing Storage on SNIA".
Further reading
- A Guide to Understanding Data Remanence in Automated Information Systems. National Computer Security Center. September 1991. Retrieved 2007-12-10. (Rainbow Series"Forrest Green Book")
- Tutorial on Disk Drive Data Sanitization Gordon Hughes, UCSD Center for Magnetic Recording Research, Tom Coughlin, Coughlin Associates