thany

Or better yet, Defraggler should allow physically separate disks to be defragmented in parallel

We all know that Defraggler can also only defragment fragmented files. Doing only this step is actually very efficient. The actual placement of files that are not fragmented is much less a performance hit. Maybe because of this knowledge, Diskeeper performs a two-stage defrag when doing a regular manual defrag. The first stage is to defragment fragmented files, and place them whereever it's most convenient, i.e. a contiguous piece of free space closest to the largest fragment, or as close to the beginning of the volume, whichever turns out best. The placement of files in relation to each other does not matter at this stage. The second stage is to defragment "free space", i.e. shoving files together so that the beginning of a file always touches the end of another file. No space between files. This is done by taking files from as close to the end of a volume as possible, and placing them as close to the beginning of the volume as possible, of course matching file size against the amount of free space that needs fitting. A possible first-and-a-half stage could be placing files in the order dictated by layout.ini. This ensures the files are layed out in a way the OS thinks is most efficient. Only for XP/Vista/2003/2008. I know Diskeeper has been around for quite some years, and being commercial and all gives them the oppertunities to come up with very efficient algorithms, but all the better: let it be an example -- But right now, the two-stage can be simulated by defragging fragmented files manually and then doing a normal defrag. Very efficient, if the regular defrag could be tweaked a little to make it faster, because I've seen it juggling with relatively small files without getting much further.

I don't think that has to be a boot-time defrag. To my knowledge, it is possible to defragment folders (and, for example, place them at the beginning of a volume) online while Windows is running normally.

I don't exactly how defragging works in Windows, but from what I've read and tried myself, the physical configuration of a volume should not matter for defragmentation. Especially if your configuration uses hardware RAID (as opposed to firmware-RAID, or worse, software-RAID). Maybe the only thing that might trouble Defraggler (haven't tested it, so it's just a hunch) is "dynamic" disks (a Windows feature to allow software-RAID, and some other extra features). Because dynamic disks no longer have a traditional partition table and such. So I guess Defraggler doesn't care about the physical configuration. Having hardware-RAID (which I suspect is the case in a Proliant server) is no different from having regular SATA disks, non-RAID SCSI disks or even USB disks. The driver should handle all I/O transparently to any sort of application. Afaik, that is. -- About your Exchange database. That's an in-use file. I've seen Defraggler skipping files that are in-use, but not nearly all of them. I *think* Defraggler cannot defrag files that have an exclusive lock by another user. Exchange hopefully runs in the context of a different user than the one logged on, and I believe Exchange does put exclusive locks on its databases. So I think you need a commercial tool to do that job. Also, Exchange databases, and any other sort of database, can fragment from the inside as well. The big file on disk might be contiguous, but records inside the database may still be fragmented. This level of fragmentation can only be taken care of by special tools. Exchange has a tool for it (don't ask me) and some commercial defraggers also offer specialized plugins or addons to defragment the inside of such databases... So good luck with that

I just found out that the same issue is with sparse files. Less common, but from what I know, some BitTorrent clients (like uTorrent) create sparse files to save space without having to compress their files. Like compressed files, sparse files also take less space on disk than they appear to do from a file manager. The difference is the way sparse files are stored: a sparse file only takes disk space for the parts that are actually written. So in the case of a download application, if a 1GB file (created as a sparse file) is downloaded 50%, it takes about 500MB of actual space on disk. Luckily, a file cannot be both compressed and sparse (afaik) I have not yet verified if encrypted files also show weird behavior like compressed and sparse files, but I suspect they don't, since their size-on-disk is the same as if it were a nonencrypted file.

Confirmed. I have a VM that's running eMule inside of it and all it's temporary/incomplete downloads are compressed files. That way, it's easy to get files that are like 600MB in reported size, but closer to 100MB on disk. It's not a huge problem, just a bug that's hopefully easy to fix