Alan, are you asking how to purge the MFT of non-used entries?
If you're looking to clear out the MFT you need to compact and compress it back to what it would be like when a new disk is formatted, but of course keeping the existing valid-in-use entries. The MFT only grows, NTFS/Windows never shrinks it on its own.
When you delete a file you mark a spot (in the mft) as being open) and when you create a new file, that spot (or others nearby) can be used to store the new location information. In the meantime, in that spot rests the previous filename of what you just deleted, and some of the location data!
Let's say you format a disk fresh. It now has only 10 MFT entries (an arbitrarily low number I made up). And as you add your music collection it goes to 5,000 entries. Then 23,456 entries with your photo collection. And up and up. If you decide to delete your music collection the count still remains at 23,456. Those 5,000 music spots would now get filled with CCleaner's zzz.z.z..zzz.zz files.
And since they were made and deleted, the zzz.z.z..zzz.zz entries are remant spots earmarked for re-use when you start creating files again. What used to be validfilename-1.mp3 validfilename-2.mp3 and so on and so forth are now zzz.z.z..zzz.zz. (after drive wipe).
What you need to do is remove any MFT entries that are un-used, by way of concatenating the branches of the MFT's B+ tree's structure and reconnecting the structure together over the open spots - those zzz.z.z..zzz.zz entries otherwise known as recently deleted entries. And as a result you have no filler space where stuff is hiding. And in the process you're shrinking the MFT down to size as a natural result.
The plain english analogy is compacting the dead air space out of something until only real substance remains. When CCleaner's drive wipe is used, it only fills the holes with unidentifiable grey "stuff". It does nothing to remove the holes and old content.
Hello Alan - I did a Google search using the phrase "purge entries for deleted files from MFT" and came up with three possibilities for you to consider.
Prevent recovery when you delete files. The file system records the names and attributes of files to a special area of your disk drive (so called 'directory entries' for FAT and MFT for NTFS). When a file is deleted, the corresponding directory entry is modified by the file system which makes it invisible to Windows and to you. However, most of the information still exists and the name and attributes can be restored using any recovery utility. BCWipe shreds directory entries and MFT so that the information can never be recovered.
Scroll down the page and find the Flash Demo for "NTFS Filename Purge", then watch the demo. It appears that this software can be used to filter then delete MFT entries based on user defined parameters, so caution is needed. As long as you get the filters right, everything is fine. Get it wrong and you wreck the MFT beyond repair. Your call if you want to consider using this one.
I'm not too sure about this one being able to do what you want. I downloaded it but haven't had the time to install it and try it out. I included it based on the developer's statement found in this link: http://eraser.heidi.ie/forum/viewtopic.php?f=2&t=8864 Look for Joel's reply:
"Eraser's erase features are non-destructive for data you want to keep. So although we do clean the MFT, we only erase entries which already are deleted. MFT erasure comes as part of doing an unused space erase. There's no option to turn it off or on at this point."
If I have read Alan's post correctly then he wants the live records in the MFT grouped consecutively, thus removing all the free records from the MFT. I don't think that any of the above software does this, or does any more than CC's Wipe MFT.
The records in the MFT are numbered in ascending sequence, and this number is used to access the records, so you can see the problem with compaction. I've always thought that the complexity of the MFT would prevent compaction, but Paragon apparently has some success with its File Manager (I think that's what it's called). Considering the links between MFT records, extension records, external index clusters, referbacks to folders, bitmaps etc I wouldn't touch any software that promised to compact the MFT with a barge pole. Not unless I had a pretty good explanation of how it works.
I will clone a well used partition onto a VHD and try Eraser on that, and post back.
There's Eraser Portable, that way if you don't like it there's no installation. Remember that Eraser isn't released by the original developer anymore and back then when he did manage it people really loved the tool, I personally won't use it anymore.
I do not care about how the MFT is grouped or modified.
Well, that's your prerogative. I think that if I were to download a free application written by somebody I've never heard of that made structural changes to the most critical metafile in an undocumented and proprietary file system, I would care. It's good that you are fond of backups.
Some of the mentioned software might serve your purpose, in as much that an MFT 'clean' would overwrite deleted records with zero length files (CC uses around 600-byte files). Then Recuva, with Show Zero Length Files unchecked, would not display these files. Doesn't compact the MFT though. I don't know which of the mentioned software does the zero byte overwrite.
FastCopy runs under Windows and is able to copy everything I want from one drive to another, but excludes the wretched $MFT and other $MetaData files,
and as each file is copied across,
the destination $MFT is updated by Windows with the relevant information.
It even correctly duplicates Reparse Points,
unlike Freeware Portable Teracopy v2.27 which mistakenly copies the contents of the Reparse Point destination.
Other benefits of FastCopy over Teracopy are that when the duplicated file is read back for hash check-sum verification against the source,
it actually reads back the file that was written to the destination and NOT the intermediate copy to RAM which Windows used as a write cache.
I am disappointed that this is not an in-situ solution that simply re-writes a small $MFT but requires the writing of all the contents of one partition to a new partition,
but thankful that I am not having to buy another 1000 GB HDD to temporarily hold this new partition.
All I am afflicted with is a 10 GB partition with thousands of real files and the ghostly remnants of a gazillion browser cache files which Firefox accumulated and deleted.
Hello Alan_B - I'm glad to see you found something that works for your situation. I tried out Eraser on my Windows 8.1 system by creating a small 20GB partition, loading 50 files onto it, doing a wipe, deleting all but one of the files, then doing a second wipe. Using Recuva, I could not find any of the files afterwards, so I can only assume it works.
Which leads me to these questions: How do you go about examining the MFT directly? Is there some software utility that allows you to do this? If there is, does it display the information using a table or graphic of some sort that's easy to understand? Or does it spit out a bunch of hexadecimal gobbledygook that's beyond my understanding? I'm just curious about this, that's all.
Good old Recuva will show the size and cluster allocations of all files, if you have Show undeleted files checked.
I use WinHex and the slightly more user friendly HxDen to poke around in the $MFT and other files. Don't forget little-endian numerical values. There is a utility from Sysinternals that produces a csv list of the MFT's contents, but I wouldn't say that it was very helpful.
If you read to the end of the Eraser forum link above, the more revealing quote from the developer is:
'It took me around 1 week to implement the directory entry cleaning for FAT since the specifications were rather readily available. In the same amount of time I did not even implement proper parsing of the NTFS MFT... which would give a hint on what scale NTFS is on.'
I think that puts it in context.
You can edit the MFT with gay abandon using a hex editor, but NFTS will back out your edits a few seconds later.