slow Defragmenting and moving same files

[25G]

Nergal, using debug3. not sure, but after a quick try it seemed just debug didnt give enough details, but shld dblcheck that.

BUT!!, the log file quickly grows larger than a GIGAByte, making it almost impossible to open and view without special tricks.

Would be smart if defraggler wld, for example, open another log file after the analyse,etc basic stuff.

THis second run-time-log file cld then be closed when pausing giving the user the possibility to open, check and delete or copy that file to have defraggler start a new "fresh" log-file, keeping it small enough to handle without special tools.

 

Limiting size and then starting a new numbered file wld do the same.

Old trick is also to not print the same line thousand times, the log shld compare the new line to the old one and just count them, finally outputting just how many times that function happened.

 

Redirecting to a (live) console, DOS window wld also help

[Nergal]

don't run debug3

run debug and post entire log. Wait for developers to request debug 2 or 3. Don't try to interpret the log, that's not the point of it, you aren't a developer.

[25G]

debug-debug3/ cant see some of the problems in just debug

developer// I am a developer, failsafe embedded system

 

some of the problems

1. defraggler cant handle the $system files, tries to optimize them although it cant (maybe except first blocks, fixed)

2. additionally it cant handle "the last" and "the first" files around $MFT, USjrnl etc. Leaves one fragment in front and one after.

 

3. Defraggler has problems handling the "data runs", file partly in MFT file, partly as "normal oldtime clusters" Another major reason for "persistent red spots".

In the debug output these appear as two different files. Nothing wrong with that if they are handled correctly

example

[2013-09-03] [03:21:54.729] 01948 3 FileCollection::Dump#357 "\___arenprob\Amerikan frendi (12)-2013-09-01T22_45_00.flv", lcn: 263575104, vcn: 0, length: 1

[2013-09-03] [03:21:54.729] 01948 3 FileCollection::Dump#357 "\___arenprob\Amerikan frendi (12)-2013-09-01T22_45_00.flv", lcn: 263651892, vcn: 1, length: 35123

 

Lots of them, depending on how files were created, small first and then growing larger.

For those who are unfamiliar, this always difficult NTFS thingy

http://www.reddragon.../data_runs.html

 

4. the "display goes haywire" is probably only for the blue area and occures systematically when doing "move to end of disk". One cld describe as "all blue and white area is filled with repeating stripes of shades of blue" while the red areas, squares stay (mostly?) the same.

I first thought it was "a smart way to show possible places to move the file", a forgotten debug, developer feature?

 

5. WHy not use MFT BitMap to find last empty cluster, not ask windows, or whatever ridiculous slow, for every cluster starting from the end.

THat is one reason NTFS provides the (very small) BitMap system file.

 

6. Why doesnt "defrag freespace (allwng fragments)" directly start emptying, freeing the clusters at the very end (depending on defrag options), instead it starts "pushing holes" all thrgh the 4 billion clusters to sometimes get them to the end.

SOmething smart shld also be done for "free space defrag" when options say place large files at the end.

Ask the user, although defrag seem to already try to figure out borders when "small at start, large at end"

 

7. And finally that "days pushing holes through huge blue cluster areas defragged during latest run". SHld be possible to detect easily and, for example, instead of searching for one perfect match for the hole, try if two, three smaller files (fragmented or from end) can fill it.

 

8. Wld help if defraggler actually marked all locked system files, also other locked or "failed' move, defrag attempts.

Give the user a possibility to output to a text file, etc.

Wld help for almost all Defragler problems I have seen and debugged, sorted out.

For example, the "data run red spots" files are "fixed" if copied to another partition and back again.

 

Btw, my first "move red spot neighbors too" observation was partly the "data runs", partly "neighbors to locked and system files"

 

9. the "blue empty spots" miht have to do with problems handling "data runs", have not dblchecked it but seen many cases where a square is painted blue but still containes undefragged files, usually a "data run" prblem close by, or failed defrag attempt.

 

Btw, note that WIndows API although returning with "OK" hasnt always "done the job", just means "no serious error occured".

(Typical defragment problem with the APIs)

 

AND!! CLosing he debug file during PAUSE, giving user a chance to handle the file, wld be good for the developers too,

Just as a "dry run" mode, defragging without actually changing the hard disk. EXTREMELY USEFUL!!

(and tests problems when the actual result turns out different)

 

Lots of work for the developers

[25G]

Btw, the "benchmark files" is a good function, but I hope there will also be reporting on locked files, NTFS problems and the WORST, bad unreadable sectors. (not to be confused with clusters marked bad by windows, which is the often unnecessary result)

The sectors are additionally often still readable if one "dives directly at them", skipping high level WIndows functions with timeouts,bad nonsense or nonexisting error-handling,etc

 

WLd make defraggler into a more usable product than competitors, especially these days when large modern HDs tend to have sector-problms that still can be fixed (and replaced by spares by HD controller)

[25G]

Btw, most of the "Blue remaining lonely squares containing no files" are $ATTRIBUTES:LIST for files.

Defraggler really need to handle all the NTFS system and special files in a consistent way, show them as SYSTEM or LOCKED or something similar.

 

Whats worse, these "weird dots, red and blue" also tend to make defragmenting split an otherwise defragged file and then show it as a red square.

 

Even worse, when running (next run) into one of these cases defraggler usually goes into this "pushing small holes thrgh alrdy huge defragged space".

 

That is, if GUI of Defraggler wld be able to handle and present info on these cases correctly, the user wld not get totally frustrated.

[25G]

Defraggler need to decide on fragmented or not in cases like this

 

[2013-09-19] [15:47:34.218] 00a04 3 FileCollection::Dump#357 "\__CSPANARCH10\11_10\e111910_photo.flv.flv", lcn: 365201, vcn: 0, length: 1079

[2013-09-19] [15:47:34.218] 00a04 3 FileCollection::Dump#357 "\$Extend\$RmMetadata\$TxfLog", lcn: 366280, vcn: 0, length: 1

[2013-09-19] [15:47:34.218] 00a04 3 FileCollection::Dump#357 "\__CSPANARCH10\11_10\e111910_photo.flv.flv", lcn: 366281, vcn: 1079, length: 7191

 

That is, a file whci starts before a ONE CLUSTER system file and continues immediatly after.

 

This is maybe the most important of WEIRD STUFF with defraggler.

 

- cant move the one cluster NTFS SYSTEM file

- the file to be defragmented IS DEFRAGMENTED , taking into account that system fil

- defraggler GUI still PAINTS the square RED as fragmented.

- but does not show the reason, that locked NTFS system file "in the middle"

 

GET THE DEVELOPERS GOING!! shld be easy fix

 

Dont forget those blue squares which contain "no files".

Same problem, both defrag threads amd GUI have to at least agree and GUI+guys need to give user information on these "NTFS-WEIRDer things"

[25G]

Here is another example, in this case defraggler debuglist does not show the *secrete invisible one cluster NTFS system file" and shows another one of these "WEIRD RED ONES"

.

 

[2013-09-19] [15:47:34.261] 00a04 3 FileCollection::Dump#357 "\__CSPANARCH10\10_10\wj100810.flv.flv", lcn: 21998373, vcn: 0, length: 107216

[2013-09-19] [15:47:34.261] 00a04 3 FileCollection::Dump#357 "\__CSPANARCH10\10_10\wj100810.flv.flv", lcn: 22105590, vcn: 107216, length: 4789