Jump to content

MFT and Zero length files


Willy2

Recommended Posts

I have two suggestions to improve CCleaner v2.29.

 

1. Add an option to search and wipe zero length files (when not locked by a program). Especially in the ""Temporary internet"" directories I always find a lot of zero length files.

 

 

2. After having read

 

http://docs.piriform.com/ccleaner/ccleaner...leaner-settings

 

it's still not quite clear how the option ""Wipe MFT Free Space"" in the Settings-menu really works. When this option is selected, does CCleaner always wipe the MFT entry of a wiped file independent of the option ""Wipe free space drives" ?

 

If so, then I think the text of this option and the (check-)box should be moved to the left and placed above the "Wipe free space drives"" option, in order to make it clear that this is a separate option which works independent of the ""Wipe free space drives"" option. This would eliminate any confusion. In the current situation it suggests it's subordinate to/it's a part of ""Wipe free space drives"".

 

Is there anyone out there with an answer ?

System setup: http://speccy.piriform.com/results/gcNzIPEjEb0B2khOOBVCHPc

 

A discussion always stimulates the braincells !!!

Link to comment
Share on other sites

  • Moderators

You can't physically overwrite zero length files. Perhaps you mean the filename info in the MFT?

 

As far as I can deduce, the wipe MFT option requires wipe free space to be enabled as well. My deductions are that without WFS being ticked there would be no indication of which drive the WMFT is to be run, and there's no separate option in Cleaner to initiate WMFT, as there is with WFS. But I could be wrong. I don't use either option, and those who do seem quite relectant to say what actually happens.

Link to comment
Share on other sites

Augeas,

 

Yes, your answer makes sense.

 

A directory file contains filenames (e.g. a file called "Hello"") and a pointer to the first entry in the MFT related to this file called "Hello"".

The entries in the MFT tell Windows where to find the all fragments of a file on a drive, where all the fragments are located.

 

I would like to see that zero length files entries can be deleted from the directory file and overwritten (provided one selects a ""Secure deletion"" option). Of course, when a file has a length of zero bytes then I would think there's no or only one entry in the MFT and then it certainly occupies no disk space.

System setup: http://speccy.piriform.com/results/gcNzIPEjEb0B2khOOBVCHPc

 

A discussion always stimulates the braincells !!!

Link to comment
Share on other sites

  • Moderators

When running CC with secure deletion enabled zero length files do have the entries in the MFT overwritten. WFS with the Wipe MFT option enabled should do the same thing: rather like a morning after pill following an evening of dissolute surfing.

Link to comment
Share on other sites

Augeas,

 

Are you familiar with the technical details of the NTFS ?

 

If both the directory file and the MFT contain a copy of the name of a particular file then CC certainly is forced to wipe/overwrite in both the file directory and the MFT the name of that particular file when a secure deletion option is enabled. Are you sure CC does precisely that ? And this is not limited to zero length files (as you suggest in your previous post) but it applies to every file ?

 

 

CC v2.29 creates an interesting situation. Because it looks like one can wipe the free/obsolete MFT entries only without wiping the free space on drives. It's less thorough but it would speed up the cleaning process significantly. Has this been done intentionally ?

When this is the case then it would be - IMO - comparatively easy to apply this option (""Wipe MFT Free Space"") for every file that's deleted with secure deletion enabled as well.

But this also creates an additional problem. One e.g. could choose both ""Normal Deletion"" and ""Wipe MFT Free Space"" and then this perhaps could lead to a breakdown of the file system.

 

I still would like to see a separate option to be able to search for and delete zero length files, with or without a secured deletion option enabled.

System setup: http://speccy.piriform.com/results/gcNzIPEjEb0B2khOOBVCHPc

 

A discussion always stimulates the braincells !!!

Link to comment
Share on other sites

I noticed the online CC manual has been rewritten. A new version appeared on sunday march 14, 2010. The new manual clearly suggests that cleaning the MFT is independent of the ""Wipe free space"" (WFS) option.

http://docs.piriform.com/ccleaner/ccleaner...leaner-settings

 

But then I think that the ""Wipe MFT free space"" (WMFS) option should be removed from the WFS entry and should become a separate entry in the ""Settings"" menu. This new entry should be placed above the WFS entry.

 

There's another thing that - IMO - should be adressed and is related to the things mentioned above. The user can select in the main screen an option called ""Wipe free space"". I think that if one hasn't selected this option then one simply shouldn't be allowed to access the options in the WFS entry in the ""Settings"" menu. This also would eliminate any remaining confusion concerning the WMFS option/entry.

System setup: http://speccy.piriform.com/results/gcNzIPEjEb0B2khOOBVCHPc

 

A discussion always stimulates the braincells !!!

Link to comment
Share on other sites

  • Moderators

Hi Willy,

 

I only know about the MFT from reading various technical articles on the net.

 

I'm not sure what you mean about the directory file. As far as I know directory entries are held in the MFT along with other file entries. And I'm still not sure what your concern is with zero length files. CC will treat them in the same way as other files: when secure delete is enabled the entry in the MFT is overwritten. All files, and directory names, are overwritten with secure delete.

 

However the Secure Delete and WFS options are separate: the secure delete overwrite methods apply to files only, not to WFS.

 

I don't think that the documentation specifically says that WFS and WMFT can be run separately, although it is a little ambiguous. All it says is that if you choose both WFS and WMFT, the WMFT will run first. If you could overwrite the MFT separately, why isn't WMFT a separate entry, as you say above? And if WMFT is separate, how do multi-partition/drive systems determine which MFT to wipe? Furthermore, how would you enable WMFT, as there isn't (as you also point out) an entry on the main page.

 

As it seems that you are keen to use WFS and WMFT why don't you run a WMFT on it's own, with nothing ticked in the WFS box and without ticking the Wipe Free Space box on the main page? I bet that nothing will happen.

Link to comment
Share on other sites

As it seems that you are keen to use WFS and WMFT why don't you run a WMFT on it's own, with nothing ticked in the WFS box and without ticking the Wipe Free Space box on the main page? I bet that nothing will happen.

I just checked CCleaner to see if, as suggested above, the wipe MFT can be run independantly of the wipe free space function and it can't - it only runs if one of the disks is ticked to wipe free space (and the wipe free space box is ticked under the cleaner section too obviously).

Link to comment
Share on other sites

Thanks for the replies and the suggestions.

 

I did what JDPower did and Augeas had suggested. In order to enable WMFS one certainly needs to select:

1. the WFS option in the main screen.

2. at least one drive from the WFS menu.

 

But then I think the folks from Piriform should modify or add one or more things to eliminate any remaining confusion concerning this topic. e.g.

1. add the text ""(this option is ignored when no drive is selected)"" to the WMFS option in the ""Settings"" screen, and/or

2. add the text (see above) to the webpage

http://docs.piriform.com/ccleaner/ccleaner...leaner-settings

3a. or make it impossible for the user to enable the WMFS option when no drive has been selected in the WFS menu and

3b. automatically disable the WMFS option when one deselects the last remaining selected drive in the WFS menu.

 

The issue with the zero length files is that:

1. a lot of these files hide in folders with attributes ""hidden"" and/or ""system"" set and CC ignores these folders/directories.

2. a lot of these files have (extremely) randomly generated filenames.

That's why it's (nearly) impossible to specify names and wild cards in the ""Include"" section.

 

I certainly hope the folks from Piriform read this discussion and will - at least - look into this matter in order to see if perhaps some changes/modifications in the program are necessary.

System setup: http://speccy.piriform.com/results/gcNzIPEjEb0B2khOOBVCHPc

 

A discussion always stimulates the braincells !!!

Link to comment
Share on other sites

When Augeas, JDPower and/or the Piriform folks read this post, then I think steam will come out of their ears ;-) but I'll post these thoughts anyway. After all, a discussion always stimulates the braincells. We'll see whether the proposed changes will be implemented or not in a next version of CC (v2.30 ???). The suggestoins below are MFT related.

 

1. Change the behaviour of the ""Cancel"" button during the Wipe Free Space (WFS) operation (Settings screen).

When one has selected ""Wipe MFT Free Space"" (WMFS) and two drives (e.g. C:, D:) then 4 separate operations will occur:

1. WMFS of drive C:

2. WMFS of drive D:

3. WFS of drive C:

4. WFS of drive D:

I think that clicking on the ""Cancel"" button should force CC to cancel the current operation and move on to the next operation (e.g from #2 to #3), provided there's a next one in a/this sequence. Currently CC v2.29 aborts the entire operation when the user clicks on ""Cancel"".

 

2. Isn't there a way of establishing whether an entry in the MFT has been wiped/cleaned (entirely) before ? Perhaps this could be determined by looking at the filename in that particular MFT entry (deleted/overwritten/zero bytes) ? If CC was able to establish this then CC could skip all previously wiped MFT entries and this would speed up the cleaning of the MFT significantly.

But then CC must also wipe/overwrite/clean the entire entry in the MFT as well when a ""secure deletion"" option has been selected.

 

3. Another way to speed up the cleaning of the MFT is cleaning of the filenames in every MFT entry only and not wipe an entire MFT entry.

System setup: http://speccy.piriform.com/results/gcNzIPEjEb0B2khOOBVCHPc

 

A discussion always stimulates the braincells !!!

Link to comment
Share on other sites

  • Moderators

Agggggghhhhhhhh!

 

1) Personally I don't think you'll get that. Cancel is cancel. You will have more chance asking for the WMFT to be a separate operation.

 

2) As far as I (and a few others) know the WMFT process is to create a number of small (600 byte?) files sufficient to use all the available deleted entries in the MFT, and then delete them. This doesn't allow for selective overwriting, as NTFS will decide which MFT entry gets used in new file creation.

 

3) You can't, for reasons in answer 2. Windows doesn't allow applications to poke around in the MFT, you have to ask it nicely.

Link to comment
Share on other sites

Augeas, prepare yourself for more steam coming out of your ears. ;)

 

 

I don't advocate circumventing the NTFS and poking directly in the MFT or onto one's harddisk. It's simply too risky. The entire NTFS could break down and take down Windows with it.

 

Did you run the Wipe MFT Free Space (WMFS) option yourself ? Because the behaviour of this option contradicts your opinion. CC knows how much entries the MFT contains. It looks like it wipes all MFT free entries including the related pointers in the MFT.

 

Selective overwriting of the MFT impossible ? I am not so sure.

1. There's a counter on the screen (in %) that clearly indicates that CC knows which entries of the MFT are to be skipped. It indicates that on a computerdisk of mine the first approx. 27% and the last approx. 30% of the MFT entries are occupied. (How odd !!).

2. Aren't there in Windows (a lot of) carefully predefined entry points that allow programmers to use a lot of the features of the Windows NTFS system. (like writing/reading/deleting files/directories/MFTs, extracting information). If so, then perhaps one is also able to access and extract (more) information concerning the MFT entries (occupied/wiped/free). The behaviour of CC - at least - seems to indicate just that.

 

There's a program called Clean Disk Security that also wipes free disk space and MFTs. This program does a similar job like CC. Although perhaps it does its job less thorough I prefer that program over CC's ""Wipe (MFT) free space"" options because it's significantly faster.

http://www.theabsolute.net/sware/clndisk.html

 

I hope, the folks from Piriform - at least - are willing to look into to suggestions made in this thread and incorporate any of the suggestions in a future version of CC (v2.30 ???).

System setup: http://speccy.piriform.com/results/gcNzIPEjEb0B2khOOBVCHPc

 

A discussion always stimulates the braincells !!!

Link to comment
Share on other sites

  • Moderators

No, I don't run either WFS or WMFT, and probably won't unless circumstances force me to, but I am interested in the technical side of it. On the other hand I don't run the country either, but I offer that nice Mr Brown plenty of advice. And threads like this give me an excuse to pontificate at length.

 

As we well know you can only write data to drives, not wipe or erase or delete or clean or any other word that makes you feel all warm and comfortable. So to wipe etc. you can only write, or overwrite. I've looked at a few disk wipers, and the general method seems to be to allocate files until the disk is full, forcing data into the MFT zone (in XP). Then zero length files are allocated until the MFT is full. Then the lot is deleted. There's no published information on the workings of Piriform products, so we have to assume that the WFS and WMFT processes in CC use a similar method to other commercial wipers.

 

The MFT can be wiped (now I'm using that silly word) on its own. You can even do it manually. and you do it in part every time you use your pc. All it needs is enough new files to be created to overwrite all the 'spare' entries, those from previously deleted files and directories. The number of spare entries is available from Windows defragger analysis, or Recuva, or other means. Possibly CC requests or calculates the number of free entries then runs an appropriate file create/delete process.

 

To run a selective MFT record overwrite is another matter. Perhaps internally to NTFS there is a Next Available Record function, so you could see which record is about to be reused. Fine, except that the record the user would like to be wiped is most likely not the next available. So how would an application position itself on the desired record, and how would it force NTFS to use it? I think that there are three reasons why selective MFT record wiping is at least not practicable. Recuva doesn't do it. Other disk wipers, more sophisticated than Recuva or CC, don't do it. And it just doesn't have any real point or purpose.

 

As all records in the MFT are a collection of what NTFS calls attributes, there would never be a way that any application could tell that an entry had been 'wiped'. Wiping is replacing one set of attributes with another - there's no such concept as a wiped attribute. Why should an application use complex code to check whether the attributes are shall we say 'safe' when it's easier, faster and less prone to error to just overwrite it, as it does with all the others? If Piriform even thinks about implementing selective MFT overwriting I'll eat my hat.

 

I wouldn't put too much faith in a progress bar indicating where the deleted records are held. With a constant file creation and deletion process in normal pc usage they are bound to be fairly widely distributed.

 

That's enough for today.

Link to comment
Share on other sites

Interesting story, but I am still not convinced for the full 100% that a direct access to or extracting information from the MFT is impossible. But I don't want to continue that discussion.

 

I have read the information from the website of Microsoft on the topic MFT. (GOOGLE the words ""support, Microsoft, NTFS, MFT""). Very interesting, but it doesn't give a clear answer to the question whether accessing the MFT directly is possible or impossible. I also believe Microsoft is deliberately hiding (parts of) the real truth. e.g. Microsoft states that the MFT can not shrink and will grow only in order to accommodate a growing number of files on a disk. But my experience (thanks to Piriform's Defraggler !!!) says that this is rubbish. The MFT can - IMO - both shrink and grow and that interferes with one or two features in Piriform's Defraggler.

 

 

I DO think the MFT can be wiped independently of the ""Wipe Free Space"" (WFS) feature.

 

Everytime CC scans a drive/drives for files to be wiped it reads a lot of and perhaps all directory files on one or more drives. And then, without a shadow of doubt, it comes across (a lot of) files that are simply marked ""deleted"" but have not been ""wiped"". When a user has enabled the ""Secure file deletion (slower)"" option or the WFS option then an additional option could order CC to ""wipe"" all these files, who were marked as ""deleted"", as well, on every drive CC has scanned. The advantage of this is that this thorough cleaning prevents the MFT from growing too much and occupying a lot of/too much diskspace.

 

In order to accomplish this particular task in both and/or either in the ""Secure Deletion"" and/or in the ""Wipe Free Space"" an additional option (called e.g. ""Wipe MFT"") could be introduced.

 

In the WFS entry, I think CC should give the user the opportunity to choose between a thorough cleaning of the MFT (like it has been programmed in v2.29) or a ""Quick and dirty"" option (as described above). By ""Quick and dirty"" I mean that only MFT entries of files that have been deleted with a ""normal deletion"" are ""wiped"".

 

I hope it's clear what I mean.

 

As mentioned before , I hope the folks at Piriform - at least - will look into this matter and make a decision whether to incorporate or not any suggestions made in this thread.

System setup: http://speccy.piriform.com/results/gcNzIPEjEb0B2khOOBVCHPc

 

A discussion always stimulates the braincells !!!

Link to comment
Share on other sites

And then, without a shadow of doubt, it comes across (a lot of) files that are simply marked ""deleted"" but have not been ""wiped"". When a user has enabled the ""Secure file deletion (slower)"" option or the WFS option then an additional option could order CC to ""wipe"" all these files, who were marked as ""deleted"", as well, on every drive CC has scanned.

 

I sense danger.

 

What you see as "deleted" is seen by the O.S. as unwanted and ready for re-use.

 

What happens to the MFT when the O.S. does decide to re-use the deleted file with a new file and a different name ?

Will the MFT "deleted" entry be erased, or will it still designate as deleted the sectors that are now re-used ?

 

What happens to the MFT when the disc is defragged ?

Does the choice of any defragger on the market result in a different answer ?

Will all affected MFT "deleted" entries be erased, or will they still designate as deleted the sectors that are now in use ?

 

Alan

Link to comment
Share on other sites

  • Moderators
Everytime CC scans a drive/drives for files to be wiped it reads a lot of and perhaps all directory files on one or more drives. And then, without a shadow of doubt, it comes across (a lot of) files that are simply marked ""deleted"" but have not been ""wiped"". When a user has enabled the ""Secure file deletion (slower)"" option or the WFS option then an additional option could order CC to ""wipe"" all these files, who were marked as ""deleted"", as well, on every drive CC has scanned. The advantage of this is that this thorough cleaning prevents the MFT from growing too much and occupying a lot of/too much diskspace.

I'm a little confused. Are we still talking about wiping the MFT records or 'files', i.e. the clusters associated with the MFT entry? I'll assume MFT and files. How would wiping either the MFT or files prevent the MFT from increasing in size? How would anything, apart from a disk reformat, prevent the MFT from growing?

 

As for the clusters that a deleted MFT record points to, I would guess that NTFS obtains a lock on the next available free record in the MFT, then uses whatever algorithm is necessary (depending on head position, file size etc) to find available space, writes the file, and updates the MFT. The cluster addresses in deleted MFT records are irrelevant, as shown when Recuva frequently finds overwritten clusters.

Link to comment
Share on other sites

Alan,

 

I think I need to elaborate a little bit. One can not, as Augeas has pointed out, directly access a MFT entry and wipe that entry (although I am still not convinced it's impossible). One has to create a (zero length) file and then delete it. In that way one can guarantee that information in a MFT entry is rendered worthless. And then the problem you're refering to doesn't occur. So, ""wipe a MFT entry"" actually consists of two separate actions and doesn't constitute ""wiping"" in the literal sense of the word.

 

 

There's - IMO (!!!)- a smart way of knowing how many MFT entries are to be ""wiped"" (notice the quotation marks !!!). Every valid entry (including those marked ""deleted"") in a directory file, contains a pointer, pointing to an entry in the MFT. Reading, sorting and processing these pointers could provide the number of MFT entries that are used and/or empty. Then this information could be used to perform an intelligent ""MFT wipe"" process by limiting the number of ""wipe MFT entry"" operations. This, most definitely, would speed up the ""wiping"" process. And this could be the basis for a ""Quick and dirty"" ""wiping"" of the MFT. Although I recognize that every once in a while one should ""wipe"" the entire MFT as well.

These pointers also could provide the basis for a program (like CC or Defraggler ???) to compact the MFT. i.e. move all the used MFT entries closer to the beginning of the MFT. And that would speed up a every future ""cleaning"" of the MFT as well.

 

But this depends on how much information of the NTFS Windows is willing and able to provide to a program (like e.g. CC). Perhaps Piriform already employs that information for CC and Defraggler ?

System setup: http://speccy.piriform.com/results/gcNzIPEjEb0B2khOOBVCHPc

 

A discussion always stimulates the braincells !!!

Link to comment
Share on other sites

  • 2 weeks later...

I noticed that in CC v2.30 there's an useful extra feature in CC. When CC cleans the MFT of a drive then CC now gives an estimate of how much time it will take to ""wipe"" that particular MFT. Although I have asked for a progress bar I think this addition is good enough. No need any more for a progress bar.

 

Thanks !!!

System setup: http://speccy.piriform.com/results/gcNzIPEjEb0B2khOOBVCHPc

 

A discussion always stimulates the braincells !!!

Link to comment
Share on other sites

2) As far as I (and a few others) know the WMFT process is to create a number of small (600 byte?) files sufficient to use all the available deleted entries in the MFT, and then delete them. This doesn't allow for selective overwriting, as NTFS will decide which MFT entry gets used in new file creation.

 

I'm no programmer, but if you used consistent filenames during wiping, and if you can poll deleted entries in the MFT (that is, pull information for deleted MFT entries) you could poll each entry in the MFT and ask it its filename. If the filename is a known "wiped" name or format, skip it. If not, wipe it.

 

Again I'm no programmer, and this all seems a bit too complex for very little gain. My guess is that overwriting the MFT entries is probably pretty darn fast already.

Link to comment
Share on other sites

  • 2 months later...

In CC v2.33 the user can select to wipe the MFT. But CC won't wipe the MFT when the option ""Wipe Free Space"" (WFS) hasn't been selected. When both options are selected then CC will wipe the MFT first and execute WFS afterwards. And both processes can take a (very) long time.

 

It may sound weird but the time it takes for these two processes can be reduced (significantly). Let a future version (v2.34 ?) of CC perform WFS first and then wipe the MFT afterwards.

System setup: http://speccy.piriform.com/results/gcNzIPEjEb0B2khOOBVCHPc

 

A discussion always stimulates the braincells !!!

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.