Jump to content

Proper setting for 1.5Tb video server?


ldelie

Recommended Posts

Hi. I've been using Defragger for a while now, and love the ability to defrag individual files. Most tools are set up for a desktop computer, not a video server, and although they work, the parameters may be a bit less than optimum. I think thats the case here.

 

I was setting up a video server drive for a dedicated video server box. It runs a small Linux kernal off ROM, and mostly just plays MPEG or AVI (Divx/Xvid) files on demand. It doesn't do much else except copying files or acting like a really big USB drive. I am using a 1.5Tb Seagate drive with 32M cache (more than it needs cache wise), and set it up as a secondary drive on a desktop computer just to format the drive and copying files. I formatted using the Seagate utility (I think it's written by Acronnis) just fine, but used standard single drive parameters. I copied about 1 Tb of files from other drives attached to the desktop. Everything went very well as I copied from a single source at a time. I was over 800Gb with no fragmentation at all. Then I got lazy; I copied from two drives at the same time. Sure enough, at about 1Tb, I had some minor (5-10%) fragmentation. Since I was about to move the drive to the video server, I decided to defrag the drive now as I would not have SATA acess in the server so a defrage on a big drive like this would become hopeless. That was a mistake. The better move would have been to move any fragmented files off the drive again, then put them back one at a time. Defragger went about it's business just fine, but after about 3 hours we had a re-start of the desktop (not due to defragger) long before it was done. Oh-oh. After the re-start, the files all appeared intact, but Defragger now claimed the drive was 50% or more fragmented. Half red, with some blue scattered here and there. It also appeared to have a larger MFT (?!?) than before (but I have to admit, I hadn't paid much attention to the size of the MFT before) and I have no idea how that could happen. On a 1.5 Tb drive, even a 12% MFT is over 160Gb. On a video server you don't need more than probably 20Gb MFT under any conditions I can imagine; a 2 hour Xvid file is only about 1.5 Gb, and you can only play one at a time. I stated Defragger again, and after 3 days I am only 15% done (this is a dual-core machine running over 3GHz). Eventually, in about a week, I may finish. At this point, I'm thinking of using two 1 Tb external drives, writing all the files out to them, and reformatting. Any other ideas? If I re-format, should I try and make a smaller MFT? How would I set that up? Any ideas as to what is appropriate? This probably should just look like a really big USB drive.

 

I really did myself in this time!

 

Larry

Link to comment
Share on other sites

Update: It's a little worse than I thought. I tried to just move all the avi's off onto other large drives. A fraction of the files have CRC errors; they were corrupted. As far as I can tell, I can only move off non-corrupted files, and will have to re-generate all the files with corrupted CRC's. That involves a LOT of time. This will not be done in the near future. I really thought that couldn't happen....

 

Hi. I've been using Defragger for a while now, and love the ability to defrag individual files. Most tools are set up for a desktop computer, not a video server, and although they work, the parameters may be a bit less than optimum. I think thats the case here.

 

I was setting up a video server drive for a dedicated video server box. It runs a small Linux kernal off ROM, and mostly just plays MPEG or AVI (Divx/Xvid) files on demand. It doesn't do much else except copying files or acting like a really big USB drive. I am using a 1.5Tb Seagate drive with 32M cache (more than it needs cache wise), and set it up as a secondary drive on a desktop computer just to format the drive and copying files. I formatted using the Seagate utility (I think it's written by Acronnis) just fine, but used standard single drive parameters. I copied about 1 Tb of files from other drives attached to the desktop. Everything went very well as I copied from a single source at a time. I was over 800Gb with no fragmentation at all. Then I got lazy; I copied from two drives at the same time. Sure enough, at about 1Tb, I had some minor (5-10%) fragmentation. Since I was about to move the drive to the video server, I decided to defrag the drive now as I would not have SATA acess in the server so a defrage on a big drive like this would become hopeless. That was a mistake. The better move would have been to move any fragmented files off the drive again, then put them back one at a time. Defragger went about it's business just fine, but after about 3 hours we had a re-start of the desktop (not due to defragger) long before it was done. Oh-oh. After the re-start, the files all appeared intact, but Defragger now claimed the drive was 50% or more fragmented. Half red, with some blue scattered here and there. It also appeared to have a larger MFT (?!?) than before (but I have to admit, I hadn't paid much attention to the size of the MFT before) and I have no idea how that could happen. On a 1.5 Tb drive, even a 12% MFT is over 160Gb. On a video server you don't need more than probably 20Gb MFT under any conditions I can imagine; a 2 hour Xvid file is only about 1.5 Gb, and you can only play one at a time. I stated Defragger again, and after 3 days I am only 15% done (this is a dual-core machine running over 3GHz). Eventually, in about a week, I may finish. At this point, I'm thinking of using two 1 Tb external drives, writing all the files out to them, and reformatting. Any other ideas? If I re-format, should I try and make a smaller MFT? How would I set that up? Any ideas as to what is appropriate? This probably should just look like a really big USB drive.

 

I really did myself in this time!

 

Larry

Link to comment
Share on other sites

Update: It's a little worse than I thought. I tried to just move all the avi's off onto other large drives. A fraction of the files have CRC errors; they were corrupted. As far as I can tell, I can only move off non-corrupted files, and will have to re-generate all the files with corrupted CRC's. That involves a LOT of time. This will not be done in the near future. I really thought that couldn't happen....

 

"Linux" caught my eye.

Why on earth use a fragmentation-prone Microsoft file-system with Linux?

If you are building this disc in your PC with NTFS, do you know for sure that your Linux -ROM NAS unit will be able to use it?

If you are using FAT32, all videos will be truncated at 4GB, it will be extremely "fragile" and, worst of all, when you try to copy onto it a file bigger that 4GB, the copy will go on forever without telling you why!

XFS is the best filesystem for huge discs of huge files, ReiserFS the best for millions of tiny files. EXT3 is almost as good as either and much better than NTFS. With "Ext2IFS" you can use Ext2/3 partitions in Windows.

Link to comment
Share on other sites

as far as I know defraggler can only defragment ntfs / fat32, but not ext2/ext3.

btw i was just reading an article in a german magazine talking about recovery tools and they said recuva is the best

PC Specs: Microsoft Windows Vista Home Premium SP1, AMD Athlon 64 X2 6000+ (2x 3GHz), 1GB RAM, 500GB internal HDD & 500 GB external HDD (both kinda full)

Laptop Specs: eeePC 1000HE, Ubuntu 9.4 (OS), Intel Atom 1.66 GHz (2x 1.66GHz), 1GB RAM, 160GB Interal HDD

What I support: GNU, Free Software Foundation, Ubuntu (Linux-based free Operating system)

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.