-
Notifications
You must be signed in to change notification settings - Fork 63
Sanitizing Storage Devices
Sanitizing, deleting, or erasing data is an important part of a storage device when you are done using it and ready to repurpose it or destroy or recycle it.
Over the years, advice on the best methods to remove all data from a device has changed, but often that information is still readily found searching the web. So what is the recommendation for today's storage devices?
To help with this, there are a couple of standards around sanitizing storage devices:
- NIST 800-88
- IEEE 2883r2022
The NIST 800-88 came around first and IEEE 2883 has continued that work to make sure all current storage device features and technology are handled.
Both of these standards focus on the most thorough way to remove all data from the device as well as describe different techniques to verify that data has been removed.
This page will review the various methods available to sanitize your storage device according to these standards and assist with clearing up old advice and misinformation as best it can.
One way to improve our understanding of sanitization is to use clear terminology describing what has been done to sanitize the data from it.
Below are terms to help best describe the method in which the device was sanitized of user data:
Clear - Logical techniques are applied to all addressable storage location, protecting against simple, non-invasive data recovery techniques.
Purge - Logical techniques that target user data, overprovisioning, unused space, and bad blocks rendering data recovery infeasible even with state-of-the-art laboratory techniques.
"This drive has been freshly formatted" is a statement seen on many web listings for used drives, but this is extremely vague on what the seller has done to the drive.
Have they done a full sanitization of all previous data?
Have they simply gone into the Windows disk manager and told it to create a new NTFS partition using the "quick format" option?
The term "format" is not nearly as useful as modern terms like "sanitize", "clear", or "purge" which are described above.
If you are seeing a listing for storage like this, ask a few questions to make sure you understand exactly what was done with previous user-data, and if you are sanitizing a drive yourself to resell, be descriptive about exactly what you have done!
Most old advice is not necessarily incorrect or even bad...just outdated with the advancements that have been made to modern storage technology.
Many of the old sanitization techniques focus on multiple overwrite passes with various patterns used on each pass over the drive. The reason for these techniques was that at the time these were developed HDDs had a small space between the tracks where the data would be read and written.
With the technology at that time, it was even possible to read data from these small gaps even after a single overwrite due to these spaces retaining some amount of a magnetic field from the original data written to the tracks nearby.
Modern HDDs do not have this problem as there is no longer any adjacent space between tracks, so there is nothing that can be recovered attempting to read between tracks. There are many people today who agree that a single overwrite pass is sufficient to remove all user data from the drive since these gaps no longer exist.
If you apply these old multiple-pass overwrite techniques on modern storage devices it still effectively removes the data from it, however it does so at the cost of your time. On today's HDDs it takes approximately 2 hours per Terabyte to erase, so on a 30TB drive, you will be waiting approximately 60 hours for one pass...some of these old techniques may require 10 passes to complete. That is a LONG time to wait for no additional security from removing old data.
There is also a lot of bad advice on the web or even something as a complete misunderstanding on what is necessary to really remove all user data from a device.
Delete the file-system
While this may seem like it removed your data, it most likely did not do that at all. Many filesystem tools will simply remove the information at the beginning of the drive, but they will not overwrite and of your files. The faster this completes, the more likely your data is still accessible by simple software tools you can find on the web free of charge.
Reformat the file-system
This is most likely the exact same situation as the previous piece of bad advice. While writing a new file system over the old one will appear to remove files, the data in those files will still be untouched on your drive. This is not a secure way to remove your data.
Smash it with a hammer
Physical destruction of a disk, without first overwriting it, is bad advice. While it is very unlikely that someone will be able to recover data from broken HDD platters, it is still possible. Always attempt a clear or purge before physical destruction!
Now that we have all the old and bad advice and some terminology to use, lets talk about and classify methods to sanitize your drive!
The best practice is to always use a method capable of meeting a purge in order to ensure all data is sanitized as securely as possible.
Be aware that businesses in some countries may have specific requirements for user-data erasure. For example, The EU's GDPR and Lot 9 set specific requirements for businesses to remove a user's data. This wiki will not be able to capture all of these requirements, but the methods described in here will help you figure out how to best meet these legal requirements.
| Sanitization Method | Drive Type Supported | Classification |
|---|---|---|
| Sanitize Crypto Erase | SAS, SATA, NVMe | Purge |
| Sanitize Block Erase | SAS, SATA, NVMe | Purge |
| Sanitize Overwrite | SAS, SATA, NVMe | Purge |
| ATA Enhanced Security Erase | SATA | Purge |
| Revert SP 1 | SAS, SATA, NVMe | Purge |
| Revert 1 | SAS, SATA, NVMe | Purge |
| ATA Normal Security Erase | SATA | Clear |
| NVM Format with Secure User Erase | NVMe | Clear |
| NVM Format with Secure Crypto Erase | NVMe | Clear 2 |
| Format Unit | SAS | Clear 3 |
| Format Unit with Security Initialize | SAS | Purge 4 |
| Write Same | SAS, SATA | Clear |
| Simple Overwrite | SAS, SATA, NVMe | Clear |
- Revert and RevertSP are only Purge on Opal, Opalite, Ruby, and Enterprise TCG devices. Pyrite devices do not perform a purge as the hardware is not capable of cryptographic erasure. Revert/RevertSP are recommended to be used on Pyrite as a step before performing another kind of erase to ensure any pyrite configuration settings have been removed first.
- NVM Format with secure erase set to cryptographic erase is not considered a
purgeunless the device vendor has provided supporting documentation otherwise. - Format Unit may not perform any data erasure unless an initialization pattern is provided in some cases. SSDs may perform a TRIM/Unmap instead of an overwrite which is not secure as the NAND blocks themselves may still contain user data. TRIM/Unmap/Deallocate operations are hints to a storage controller that the data in the specified logical blocks is no longer needed. The controller may or may not erase them leaving them able to be recovered in a data recovery lab.
- Format Unit with the security initialize bit set performs the same as a single-pass sanitize overwrite, however Seagate does not support this bit on any products. T10 standards recommend use of Sanitize instead of this method!
Each of the features listed above, other than simple overwrite, are controlled by a device's firmware and may be optional in their command standard. A vendor may support some of these methods, or they may support only the simple overwrite. In openSeaChest_Erase, the option --showEraseSupport will check what the device supports and provide a list of what methods are supported and whether they are capable of purge or clear.
With so many options one question that can come up is what method is the best?
There is no direct answer to this question as it may depend on other factors like whether the cryptographic erase is a trusted method or not (some people or organizations may not consider this good enough). Another factor is how long do you want to wait for the sanitization to complete? Cryptographic erase is instantaneous and overwrite can take hours or days to complete.
The best thing to do is first figure out what your device supports from the openSeaChest_Erase -d <handle> --showEraseSupport output.
If any of the supported methods are considered a purge as showing in that output or the table above, that is a preferred method above any clear. Some devices may not support purge and only a clear is available. This is still worth doing in any case as it removes easily accessible user data for you.
The output of the --showEraseSupport also attempts to organize the list from fastest method to slowest method. A time estimate is also provided, but be aware it is only an estimate and the real time may vary.
If cryptographic erase is supported on the device, but it is not trusted for one reason or another, another option you could do is first run the cryptographic erase, then follow it with another trusted method such as an overwrite. The benefit in this case is data is instantly changed by changing the cryptographic key and when an overwrite is started afterwards it will begin writing new data before the cryptographic key can be reversed (if it is even possible to reverse the cryptographic key).
For users with drives that require a write after cryptographic erase, write after block erase, or have a PI formatted drive the best option would be to first run the cryptographic or block erase for the quickest possible erase, then the following overwrite will make the drive completely ready for reading and writing again. While this overwrite is not required, some host operating systems and software expect to be able to read some portions of the disk without error to even write a partition table, so this overwrite will make it possible to bypass this issue.
Before you begin sanitizing your data, there are a few steps you should take in order to ensure as much data is removed as possible:
- Restore the Max LBA to original capacity
- Remove Security settings (Revert or RevertSP)
While restoring the Max LBA is not technically necessary for any purge level erase, it is still a best practice to ensure all data is sanitized and verify that all data is sanitized.
For a clear level erase, even one like ATA Security Erase, that is managed by the firmware of the device, you MUST restore the Max LBA before the erase. The standards only require ATA Security Erase (normal mode) to overwrite between LBA 0 and the last user-accessible LBA, which means if the drive has had a lower max LBA set, there may still be user data left on the device.
For SAS drives, this is easy as there is only one way to change the Max LBA: mode sense/mode select to modify the block descriptor.
For SATA it depends on what features the drive supports: HPA (Host Protected Area), DCO (Device Configuration Overlay), and/or AMAC (Accessible Max Address Configuration).
To keep things simple in openSeaChest_Erase, just use the --eraseRestoreMaxPrep option.
NOTE: You may need to reboot after running this option and run it again if DCO is active due to requirements from the ATA standards. In some cases, the system may block or freezelock these features. If this option is unable to restore the MaxLBA to the native maximum address, try using a USB adapter or move to another system.
For TCG enabled devices, performing a revert or revertSP may be necessary. If the TCG SSC is Opal v2, Ruby, Enterprise, or Opalite then this will also change the cryptographic key and perform a purge.
For Pyrite devices, revert and revertSP do not necessarily remove any user data...it might or it might not depending on the device's firmware and if it is new enough to specify what to do when these methods are sent to the device.
It is best to perform a revert or revertSP if at anytime this feature was activated, then follow it with one of the other sanitization methods in the table.
NOTE: openSeaChest tools do not currently support revert or revertSP, however these options are available in the closed-source SeaChest_Erase and SeaChest_Security tools. Seagate has it on the internal Jira tracker to make these part of openSeaChest, but a timeline is not currently available.