Wipe Free Space Running Slow

[SimonFastEddy]

Dear CCleaner,

I am experiencing a prolonged wipe free-space task with CCleaner and Windows 10 v1909.  This was initially reported as a bug here in 2013.

I run three Dell computers with the same programs and personal files.  The old Dell GX620 would not complete the wipe free-space task without reporting an insufficient resource.  This PC has 53805 folders, 313894 files and 195 MB MFT.  The task would fail after about 2 hours running, before 50% completed, although repeated attempts (about half a dozen) eventually brought success. 

The solution was to rebuild the software with a clean installation of Windows 10, loading back the programs and personal files and testing the free-space wipe at intervals.  The GX620 didn't report any issues and now runs the Wipe MFT free-space in 2 minutes; a full task completes in 2 hours 5 minutes.  In comparison the Dell 755 and Dell 760 running installations several years old are now returning wipe MFT times of 35 minutes and 60 minutes, respectively.

I am currently treating the installations as disposable with a rebuild required around 6 months or when the wipe task fails.  It looks like an O.S. issue as though a journal is filling and not clearing...

The GX620 is a test machine that has been stripped of software and rebuilt and it is a test-bed. No surprise it has issues.

Malware not detected.

Any thoughts?

SimonFastEddy

 

 

[nukecad]

Quote

Any thoughts?

I would question why you seem to be wiping free space on a regular basis?

It will make no difference whatsoever to the performance of your computer.
It is not going to 'speed up' your computer, or make any more space available on the drive.

So what does wiping free space do then?

When you delete a file in Windows rather than deleting the actual data Windows just marks the space that the file was occupying as 'free', so that another file can then be written to those blocks on your hard drive.
This means that the data from the deleted file is still there until it is overwritten by something new, and so it can sometimes be reconstructed by a recovery tool like Recuva.

What wiping free space does is to erase that old data from files that you have deleted in the past, so that it can't be recovered.
It does this by writing new files (usually just X's and zeros) into that free space and then deleting them all again - That can take a long time on large drives.

Wiping free space is typically something that you would do after deleting your files when selling, or otherwise getting rid of, a machine/drive.
You do it so that the new owner has no chance of recovering any of your old data.

Again- Wiping free space will make no difference whatsoever to the performance of your computer.

 

[SimonFastEddy]

Hello Nukecad,

Why do I seem to be wiping free-space on a regular basis?  It is because I can.

Technicians use technical books from secondhand bookshops: they lack curiosity.  I am curious enough to find out answers by experimentation.

The question can be framed as follows:

If wiping free space has no practical purpose, then why does Piriform include a facility to do just that? 

Why does Microsoft include a registry entry with a value that can be changed for NtfsMftZoneReservation ?

Try using:  fsutil behavior set mftzone 1

If you change the value from 1 to 2, 3, or 4.  What happens?  Have a look using UltraDefrag.

All attempts to adjust memory  resources (and there are a few options) gave no improvement in performance.

I am fortunate in having a computer that I can tear-down and rebuild.  The old Dell GX620 will outlive me - it is bomb proof.

Have any thoughts?

Your health!  Drink in moderation.

SimonFastEddy

 

 

 

 

 

 

 

 

 

[nukecad]

4 hours ago, SimonFastEddy said:

The question can be framed as follows:

If wiping free space has no practical purpose, then why does Piriform include a facility to do just that? 

Why does Microsoft include a registry entry with a value that can be changed for NtfsMftZoneReservation ?

Try using:  fsutil behavior set mftzone 1

If you change the value from 1 to 2, 3, or 4.  What happens?  Have a look using UltraDefrag.

All attempts to adjust memory  resources (and there are a few options) gave no improvement in performance.

As said above Wipe Free Space is included in CCleaner as an easy way that you can wipe it to prevent recovery of deleted files after you pass on your computer to a new user.
It is not intended to speed up your computer - wiping free space won't do that.
It's included twice, once as a tickbox in Custom Clean, and again on it's own in Tools as Drive Wiper.

Note that if you regularly use the tickbox in Custom Clean it will wipe the free space every time you run CCleaner.
Which means that you will have little chance (well none) of  recovering deleted files yourself if you ever have the need.

The Windows settings you refer to apply to the Master File Table (MFT).
MFT free space and disk/data free space are two different things.
MFT is where Windows stores the location(s) of files on the disk (which clusters they occupy), the file data itself is on the disk not in the MFT.
From the CCleaner documentation: https://www.ccleaner.com/docs/ccleaner/ccleaner-settings/changing-ccleaner-settings

Quote

 

To set CCleaner to clean MFT free space (NTFS drives only):

The Master File Table (MFT) contains an index of the files on an NTFS drive. As the computer overwrites and deletes files, the MFT can accumulate free space which may contain remnants of index pointers to deleted files. You can now set CCleaner to wipe the MFT free space, just as you have been able to set it to wipe free drive space.

 

As you have found wiping either of the free spaces is not going to alter the preformance of your computer, because that's not what it is meant to do.

[Andavari]

It's stressing the hard disk, and that's one reason alone to not use it all that often.

[nukecad]

That's another good point.

Accessing the disk for long periods at a time is going to shorten the life of it's bearings and other moving parts. Not normally.a problem but if you are wiping free space everytime you run CCleaner (daily?) then it will affect the life of the drive. Especially with todays terrabyte drives that often have more free space than used space.

The same consideration applies to defragging too often with large drives that have a lot of data on them.

[Augeas]

The Set MFTZone parameter, by the way, does not alter the MFT size but alters the MFT reserved Zone in whuch the MFT resides. The method of allocating the MFT Reserved Zone changed in Win Vista onwards so this parameter acts differently on later systems.

[SimonFastEddy]

Hello every body,

Opinions vary on the Internet as to the virtues of wiping the free space.  

My three machines are refurbished ex-leased, ex-commercial units with negligible intrinsic value.  I only wish they would wear out but they aren't anywhere near doing that yet. The Dell GX620 Pentium4 single core machine, was originally a Windows XP Professional installation.  Drives are still in good health!  Surfaces of the HDD still good. 

Given the age of my GX620 installation and the small MFT, it seems to me that they need servicing with a regular drive wipe.  Piriform offer two options one of which is a complete Erase which I am not considering. A recent clean installation of Windows 10 back onto the GX620 reduced a CCleaner 2 hour MFT wipe to 2 minutes, and that was after having to sequentially wipe some half-a-dozen times (almost a day servicing the volume) because the machine was complaining of insufficient resource to complete the task. I am postulating the shortage of resource was in the MFT or MFT Zone Reservation.  That saves a huge amount of drive wear, if that is your preference.  It indicates that servicing is overdue.

We haven't answered the original question that was posed in 2013.

Link: NTFS MFT Zone Reservation

http://systemmanager.ru/the_tweaking_experience.en/tweaks/file system tweaks/ntfs mft zone reservation.htm

Question:  Should a manager schedule a MFT free space wipe; a clean re-installation, or a resizing of a NTFS MFT Zone Reservation volume?  Would any of those solve the problem anyway?

A puzzle to finish:  if you had a perfectly balanced coin and you had already thrown 4 heads, would you declare a heads again or change your call?

Heads up... bottoms up... last orders.

SimonFastEddy

 

 

 

 

 

 

[nukecad]

18 minutes ago, SimonFastEddy said:

 Piriform offer two options one of which is a complete Erase which I am not considering.

Complete Erase is not selectable for the drive that has the OS on it, for obvious reasons.
Again It's meant for securely wiping secondary (external) drives before you give then to someone else.

18 minutes ago, SimonFastEddy said:

 Question:  Should a manager schedule a MFT free space wipe; a clean re-installation, or a resizing of a NTFS MFT Zone Reservation volume?  Would any of those solve the problem anyway?

No, and No.

You will always get differing opinions on the internet, whether those opinions are correct or not is another question.
They can't all be correct, so you have to decide which opinion you trust.
HInt, if Microsoft technical documents say one thing and Johnny's Blog says the opposite (and is trying to sell you an app which will do the opposite) then which is more likely to be correct?

[trium]

Quote

All attempts to adjust memory  resources (and there are a few options) gave no improvement in performance.

 

with this result ... why do you want to change something about it?

 

i have this also tryed in the past but no major improvements.

i see no practical advantage for increasing or decreasing the MFT allocation.

ad the end of the day i reinstalled my os and leave it with the standard settings for its mft. i personaly dont use wipe free space. i think thats wasted time for the hardware.

i think the time saved after tuning is out of proportion to the time spent on this topic, reading in and trying to use these things.

but ultimately it's your PC (Personal Computer) and you can do whatever you want with it...

[trium]

Quote

 

1 Limits MFT to 12.5% of the drive (default)

2 For Workstation - Limits MFT to 25% of the drive

3 For servers - Limits MFT to 37.5% of the drive

4 For High End Servers - Limits MFT to 45% of the drive

 

Values 1 or 2 are useful here, since the other two values are only required for a very large number of files.

 

Quote

Note This is a run-time parameter and does not affect the actual format of a volume. Rather, it affects the way NTFS allocates space on all volumes on a given system. Therefore, to be completely effective, the parameter must be in effect from the time that a volume is formatted and throughout the life of the volume. If the registry parameter is adjusted downward or removed, the MFT zone will be reduced accordingly, but this will not have any affect on MFT space already allocated and used.

 

Quote

Increasing the MFT Zone does not decrease in any way disk space that can be used by users for data files.

 

[trium]

https://support.microsoft.com/de-de/help/174619/how-ntfs-reserves-space-for-its-master-file-table-mft

 

Quote

How NTFS reserves space for its Master File Table (MFT)

---

Summary

---

The NTFS file system contains at its core, a file called the master file table (MFT). There is at least one entry in the MFT for every file on an NTFS volume, including the MFT itself.

Because utilities that defragment NTFS volumes cannot move MFT entries, and because excessive fragmentation of the MFT can impact performance, NTFS reserves space for the MFT in an effort to keep the MFT as contiguous as possible as it grows.

 

Change in Windows XP and in Windows Server 2003

In Windows XP and in Windows Server 2003, the defrag utility defrags the MFT.

A defrag operation on the MFT combines an MFT file into 1 and prevents it from being stored in multiple places that are not sequential on disk. In this class of operation, the MFT file is more sequential. However, it is exactly the size that the MFT file was before the defrag operation.

An MFT can be too big if a volume used to have lots of files that were deleted. The files that were deleted cause internal holes in the MFT. These holes are significant regions that are unused by files. It is impossible to reclaim this space. This is at least true on a live NTFS volume.

 

More Information

---

NTFS uses MFT entries to define the files to which they correspond. All information about a file, including its size, time and date stamps, permissions, and data content is either stored in MFT entries or in space external to the MFT but described by the MFT entries.

(Directory entries, external to the MFT, also contain some redundant information regarding files. But a full discussion of all the structures on NTFS is beyond the scope of this article.)

As files are added to an NTFS volume, more entries are added to the MFT and so the MFT increases in size. When files are deleted from an NTFS volume, their MFT entries are marked as free and may be reused, but the MFT does not shrink. Thus, space used by these entries is not reclaimed from the disk.

Because of the importance of the MFT to NTFS and the possible impact on performance if this file becomes highly fragmented, NTFS makes a special effort to keep this file contiguous. NTFS reserves 12.5 percent of the volume for exclusive use of the MFT until and unless the remainder of the volume is completely used up. Thus, space for files and directories is not allocated from this MFT zone until all other space is allocated first.

Note You can change the NtfsMFTZoneReservation registry key to increase the volume in Windows NT 4.0 Service Pack 4.

For more information about the MFT, please see the "Key elements in the disk defragmentation process" section of the following Microsoft Technet Web site:

http://technet.microsoft.com/en-us/library/bb742585.aspx

Depending on the average file size and other variables, either the reserved MFT zone or the unreserved space on the disk may be used up before the other as the disk fills to capacity.

Volumes with a small number of relatively large files exhaust the unreserved space first, while volumes with a large number of relatively small files exhaust the MFT zone space first. In either case, fragmentation of the MFT starts to take place when one region or the other becomes full. If the unreserved space becomes full, space for user files and directories starts to be allocated from the MFT zone competing with the MFT for allocation. If the MFT zone becomes full, space for new MFT entries is allocated from the remainder of the disk, again competing with other files.


A new registry parameter was introduced in Service Pack 4 for Windows NT 4.0 that can increase the percentage of a volume that NTFS reserves for its master file table. NtfsMftZoneReservation is a REG_DWORD value that can take on a value between 1 and 4, where 1 corresponds to the minimum MFT zone size and 4 corresponds to the maximum. If the parameter is not specified or an invalid value is supplied, NTFS uses a default value of 1 for this parameter. The exact ratios that correspond to each setting are undocumented because they are not standardized and may change in future releases. In order to know what setting is best for your environment, it may be necessary to experiment with different values.

To determine the current size of the MFT on a Windows NT-based computer, type the dir /a $mft command on an NTFS volume.

To determine the current size of the MFT on a Microsoft Windows 2000-based computer, use Disk Defragmenter to analyze the NTFS drive, and then click View Report This displays the drive statistics, including the current MFT size and number of fragments.

The Windows 2000 version of Disk Defragmenter displays "green" for what is called "system files" and on an NTFS formatted volume this is simply the combination of the MFT, pagefile.sys (if one exists on this volume) and what is called the "MFT Zone" or reserved space for "MFT Expansion". The defragmentation report only displays information about the pagefile and MFT; it does not mention the MFT Zone because it does not effect in any way disk utilization or capacity.

The MFT Zone is not subtracted from available (free) drive space used for user data files, it is only space that is used last. When the MFT needs to increase in size, for example, you created new files and directories, it is taken from the MFT Zone first, thus decreasing MFT fragmentation and optimizing MFT performance.

The default MFT Zone is calculated and reserved by Ntfs.sys when it mounts the volume, and is based on volume size. You can increase the MFT Zone by means of the registry entry documented below, but you cannot make the default MFT Zone smaller than what is calculated by Ntfs.sys. Increasing the MFT Zone does not decrease in any way disk space that can be used by users for data files.

Note The results returned by the dir command may not be current. The size reported by the dir command may reflect cached data that reflects the size of the MFT at the time the system was started following an orderly shutdown.

 

 

maybe microsoft should adapt this article to modern operating systems... :-)

[SimonFastEddy]

Hello trium,

Hello again nukecad,

11th February 2020

Thank you for being curious and questioning the central dogma - (SimonFastEddy)

I worked at a TV factory making Black & White televisions, during my college days when on student breaks in the 60s.  They had lots of televisions returning for repair because the On/Off switch on the front failed early.  It was discovered that the switch was designed to only be turned "On" once and then left.  It needed a new type of switch that could endure many On/Off cycles.

I am not an advocate of the philosophy that defragmenting/servicing a HDD will wear the drive out.  In my living room is an electric clock that has run continuously since the 1950s requiring just a bit of flex (original one had crumbling insulation) and two re-oils. It doesn't have bearings made on air-bearing lathes - all ex-WW2 - manual.  If the life of the electrolytic capacitors in the power supply is about the same as the On/Off switch and the mother-board, HDD or memory, then a user would be best advised to use all the facilities and discard the machine when it fails. So I say "service it" if you can.  

Trium, I like what you say: "maybe microsoft should adapt this article to modern operating systems... :-)".  I've read the same articles.  

PrivaZer improves things for a bit and ProcessLasso I am not sure about.  A Clean Install does improve responsiveness (usability, feel, with speed being difficult to assess) after which I can even push this old PC to do a virus scan during a defrag with a document open on the Desktop as well, which is asking a lot.   After time, when the PC becomes unresponsive, that is with just a document open, then it appears that the MFT also has issues requiring a lengthy free-space wipe (if that is what is required) but the O.S doesn't give clarity in it's messaging.  A Clean Install is about two days work re-loading programs and settings.  The typical user isn't us!  They will have a 1 TB HDD and don't even know how to do a restore point.  I looked at the Microsoft drive wiper "cipher". It started as an encryption tool.  It didn't stall like CCleaner; it took 7 hours to complete the allocated task but the MFT was still stubborn to complete with CCleaner after a pass with the tool.  All three PCs have the same characteristic: wipes MFT slowly to 50% completed than finishes very fast onto the main disc area the operation of which can be cancelled.

The problem (if it is one) is not a bug in CCleaner or a bug in Microsoft W10.  It is a design feature that needs developing resources.  In the 1960s it was called "Built-in Obsolescence". 

I wonder if the original author who posted this question in 2013 is still curious.

To the Moon, Mars, the stars and beyond...

SimonFastEddy

 

 

 

[Augeas]

Microsoft does update its documents. Despite my earlier post (does anyone actually read what I post?) Trium's extensive cut and paste concerns XP and Server 2003, and Eddy's link harks back to 2004. If you really want to know about the Set MftZone parameter of Fsutil in Windows 10 then look at  https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/fsutil-behavior 

 But all this is a distraction. The MFT Zone is a mythical, OK logical, area of disk space which is kept clear for MFT expansion. I can't see what it has to do with wipe free space speed. The MFT remains unchanged, the same bits in the same position, whatever the Zone parameter is.

An MFT wipe is a rather crude 'fill the empty records with files and then delete them' process. I would hope that CC would calculate the number of empty MFT records, allocate the same number of small files (I think it's abour 650 bytes) and then delete them. That file size is used as it does not require a cluster allocation, and it is large enough to ensure that the previous contents of the 1024 byte record are overwritten. The process is simple: CC allocates a number of small files: NTFS scans the MFT and reuses the free records. There is some directory maintenance - those new files have to go into a directory somewhere - and this could be complex if the number of free records is considerable. That number is likely to be in the tens of thousands. Does CC use one directory or multiple directories? I don't know. Furthermore directory names are held in ascending order, so if the new files are named in a way that is anything but an ascending sequence, for each file allocation all the directory entries will have to be moved down one place to fit the new filename in. An ascending file name will go on the end of the long name chain with minimum distruption. A descending file name will be a killer, with extensive updates through a chain of thousands of files.

Does this cause the slowing down of WFS? Quite possibly, I really don't know.

 

 

 

 

[nukecad]

9 hours ago, SimonFastEddy said:

I wonder if the original author who posted this question in 2013 is still curious

?? You are the OP here and you started this thread last Sunday.

Things that were posted in 2013 are now 7 years old, about older CC versions and older Windows OS's - they have limited relevance today.

I believe that we have established that this is not a bug, it's just the way Windows works, and possibly a missunderstanding of what Wipe Free Space is meant to accomplish.

As it isn't a bug then any further discussion may be better in the lounge:
https://community.ccleaner.com/forum/7-the-lounge/

Thanks for all contributions, but I think this thread has run it's course now and so I'll close it.