Defragging the MFT

Defraggler
1

I was recently most intrigued to see that Mark Russinovich

has updated his "contig.exe" command line defragging tool

to be able to defrag a number of metadata files, including

the MFT.

=> See here: http://technet.microsoft.com/en-us/sysinternals/default.aspx

Oddly enough, whilst this is mentioned [at the moment] on the

Sysinternals home page, it isn't on contig's own webpage.

Anyway, I downloaded it and tried it out, it did indeed defrag

the MFT of one of my drives. I must admit, I always thought that

this was infeasible [except possibly at boot time].

Anyway, I also recently tried out another well-known freeware

defragger on a drive and noticed that it fragmented the MFT

(as shown using Defraggler).

So, then I tried to fix that using contig, and it worked.

And then I refragmented it again using the other defragger,

and upon a whim tried Defraggler on that drive to see if it could

also defrag the MFT...and it did !

I've looked in the Defraggler history and can't find any reference

to it becoming capable of defragging the MFT, so I am somewhat surprised

at this ! Is this "for real", I wonder, or is Defraggler misreporting

the status of the MFT ?

Incidentally, I've tried Auslogics' DiskDefrag on the same drive when

the MFT is fragmented, and it's not managed to fix it.

So, over to the forum for comments....

- THM

------8<------------------- EDIT ------------------------->8----------

Well, I just googled about a bit and found this....

http://www.piriform.com/docs/defraggler/using-defraggler/understanding-the-drive-map

Which explains that Defraggler can indeed defrag the MFT.

Does anyone know when this capability was introduced ? :)

2

It depends. On what you mean by defrag, how many fragments the MFT has, and what op sys you're running for a start.

In XP the MFT lives in an MFT zone, and unless your disk becomes around 90% full then any expansion of the MFT would be contiguous, even though it may have more than one fragment (no semantics regarding the meaning of 'one fragment', please). An XP MFT will almost certainly have two fragments from the start, one being the (large) allocation of the MFT records and the other being the (small) allocation of the MFT bitmap. These allocations are most probably contiguous. If your disk has been under stress with lack of space then new files may force the MFT zone to be reduced, and any extensions of the MFT at this time might be outside the zone and not contiguous with the bulk of the MFT. In this case you would have to relieve the stress by bringing the space used down, defragging to clear the MFT zone, and then defrag the MFT.

In Vista onwards the MFT zone is allocated as needed in 200 mb chunks, sufficient for knocking on for 200,000 files to be recorded. When the first 200 mb is full then a new zone is allocated anywhere on the disk and the MFT, and life, continues. You can defrag these 200 mb chunks if you wish (i.e. make them contiguous), but M/S's opinion is that it isn't worth the bother.

I don't know when defragging the MFT was introduced, probably when M/S introduced the API to do it. It's mentioned in the XP file system info in M/S TechCenter.

An easy way to see how many fragments the MFT has and where they are is to run Recuva with Scan for non-deleted files checked, and then look at the Info panel for $MFT. You can always cancel after the first few seconds if you're impatient.

3

Thanks for the time to update info about Contig 1.6, I use it all the time to defrag folders before burning to discs. Although using it, it did nothing about my MFT fragments.

4

In Vista onwards the MFT zone is allocated as needed in 200 mb chunks, sufficient for knocking on for 200,000 files to be recorded. When the first 200 mb is full then a new zone is allocated anywhere on the disk and the MFT, and life, continues.

Nice theory, but on my original Vista 32 filesystem (which ran Vista since before SP1) is claimed to be 182,720 KB, and is in 1,673 fragments. It appears to be in (on average) 109 KB pieces, no where near 200 MB chunks which would indeed be very satisfactory, though there's only 4 regions containing it on the Drive Map and no obvious reason why it shouldn't be relatively contiguous. Yes, the Vista filesystem seems noticeably more sluggish for some things than the fresher Win 7 one, held on same disk.

Checking it via Highlighted Tab from Drive Map, and doing "Defrag Checked" does not successfully defrag the file though it does try it pops up "No files were defragmented". I have managed to defrag other internal NTFS files this way (think $Usn$Jnl).

Running an FS check did find some unallocated blocks marked as allocated in bitmap and fixed those but no change to the situation.

Have downloaded Contig 1.6 and will give it a try to see if it can improve that MFT file. Thanks to OP for posting about $MFT!

5

Hi Rob, I just faithfully rephrase what M/S says, as:

Windows Vista and Windows Server 2008 use a default size of 200MB for the initial MFT zone reservation. As the MFT outgrows the default zone due to more files being added to the volume, the MFT will create another 200MB zone to grow into. This change to fixed amount versus a percentage was done to deal with increasing size of volumes and create better efficiencies.

and...

For workstation or server profiles that have large MFT?s, it is possible that the MFT will be more ?fragmented? however these fragments will be 200MB in size a piece so the performance impact will be negligible.

Are you saying that your 1673 fragments are dotted all over the place or are they within a 200 mb zone, and if so are they contiguous? I don't know whether an MFT expansion within the zone counts as another fragment. I think that if the MFT acts like any other file then it would count as additional fragments. Maybe you've had a lot of expansion? With that many fragments you will have many MFT extension records and possibly a non-resident attribute cluster as well. I would think that performance is dire.

You could use Recuva (as above) to see the MFT cluster allocation.

6

Yes, I didn't doubt that the 200 MB is what's supposed to happen.

Contig defragged the $Mft moving it into middle of free space into 2 fragments, as shown by Analyse.

But guess what! Running Defraggler Full Defrag aferwards re-fragments it, putting it back into the 5 or so holes (in the generally compacted beginning of disk with just a few other files held in them) where it was before Contig defragmented it. This seems like a real bug, as it's deciding to re-locate a 182MB 2 fragment file, but apparently storing the relocated blocks are on a first fit basis where the fragmentation HAS to be greater than before.

So it's actually seems like Defraggler can indeed actually cause seriously increased fragmentation of the $Mft. I found a Defrag Freespace caused $Mft to be in 256 pieces afterwards partially moved back. Running the Defrag again, I got back to the very high defrag level. I can't exclude V:\$Mft either as the exclude dialog doesn't have a way to select, or let me type $Mft in it, that I can find.

The odd things is in most of the blocks according to Drive Map, $Mft was the only file in it and in another block there's about 20 files only, all non fragmented.

This is reproducible on my Vista file system at the moment, Defraggler 1.21 & 2.0.2 are affected. hopefully if the hole left is filled in by something else, $Mft which is in 2 fragments won't be moved back and split up.

I'm afraid I've never installed or used Recuva, so I'm going by what Drive Map tells me, rather than use that tool to look at the Cluster Allocation.

7

After having Win 7 defragger consolidate the file allocations, the holes have moved, doing a Defrag resulted in part of $Mft being put in first hole, then allocated interspersed with other files, as they were being compacted.

$Mft is now in 1692 fragments after Defraggler Defrag :)

So the good news is, this is repeatable for me, I presume it's a property of the $Mft causing this, as some report success defragmenting it withing Defraggler via checking it's box.

8

Thanks for the time to update info about Contig 1.6, I use it all the time to defrag folders before burning to discs. Although using it, it did nothing about my MFT fragments.

Ah, well it appears that you have to explicitly name the metafiles in order for

contig to do its work,

e.g. ....

contig $Mft

Wildcards just do not seem to work with metafiles [here].

If you just type "config" on its own it will give you a list of metafiles that

it can [supposedly] defrag, but I find that [with Vista] a number of them don't work.

I knocked up a trivial batch script to use contig to defrag metafiles on a nominated

drive [or drives], and have commented out those that appear not to work [access denied].

[Yes: I did run it in admin mode !]

- thm

Excerpt below....

==============================================================================================

@echo off

if "%1%" EQU "" goto Usage

set DO=contig.exe

REM note some metadata commented out, as these appear not to work !

:Loop

%DO% %1$Mft

:: %DO% %1$LogFile

%DO% %1$Volume

%DO% %1$AttrDef

:: %DO% %1$Bitmap

:: %DO% %1$Boot

:: %DO% %1$BadClus

%DO% %1$Secure

%DO% %1$UpCase

%DO% %1$Extend

:LoopTest

shift

if "%1" NEQ "" (goto Loop) else (goto TheEnd)

:Usage

echo This utility will attempt to defrag the metadata of the designated drives.

echo.

echo USAGE...

echo [Template] DefragMetaData drives...

echo [Example ] DefragMetaData C: D: E:

echo.

:TheEnd

exit/B

:: Note that this requires the Sysinternals "contig.exe" utility

:: to be on the path.

=== Edit ===

Apologies for the lack of formatting - the forum editor appears to have removed

white space...