Jump to content

Question about defrag


Nomen

Recommended Posts

I have a win-98 system with an 80 gb hard drive, partitioned as C and D (about 32 gb each). So there some unpartitioned space on this drive. Each volume has about 5 gb free space.

I connected the drive as a slave to a win-XP sp3 system, and told it to perform a drive-check on each of these volumes, correct any errors, and then performed a defrag on each volume. The D volume defragged without any files being fragmented. The C volume ended up having about a dozen files with fragments, some only a handful, others having a few hundred fragments. These are large files, from a few hundred mb to 1.2 gb in size.

I repeated the defrag on the C volume several times, but it didn't change anything (those files were still fragmented).

I then did a "cut and paste" of those files from the slave drive over to the "real" C drive (NTFS) and then defragged the C volume again, and it defragged completely (there were no fragmented files).

I then copied the problem files back to the C volume (back to the directories where they were cut from) and did a defrag "analyze" - and these files were showing up as being fragmented. (?!)

I would have thought that simply copying these files to an NTFS drive, and then back to the FAT32 drive would have removed any fragmentation issues in these files.

So I again performed a defrag on the C volume, and in the end only 4 files were showing as still being fragmented, so I'm going to repeat this cut-and-paste process on those 4 files and see if I can finally get those files to defragment.

But my primary question is why these files didn't defragment automatically by moving them to an NTFS file system, and then moving them back to a de-fragged FAT32 file system.

Link to comment
Share on other sites


But my primary question is why these files didn't defragment automatically by moving them to an NTFS file system, and then moving them back to a de-fragged FAT32 file system.

There is not an answer.

It greatly depends on a number of factors, including the specific way they were copied, the order on which this was done, the amount of free space on the volume and possibly some more.

BUT it is not clear (to me) which steps you took.

If you want to try again with moving back and forth the procedure is:

  1. move the files to another volume
  2. defrag the "original" volume (which has now more free space as you removed from it the files)
  3. move back the files from the "temporary volume"

If you did not perform step #2 above this is likely the reason, a file becoming fragmented is not a "characteristic of the file" but a "characteristic of the filesystem where it is copied" (I hope I made myself clear).

If the "original volume" has a fragemented file is normally because the continuity of the space is interrupted by other files, imagine you have something like:

FF1F111F11FF

wher F represents the fragmented file and 1 represents another allocated file (or a set of other allocated files).

When you move the File, it will look something *like*:

001011101100

where 0 means non-allocated space.

If you copy back the file now, it will fill the 0's and thus it will be fragmented "as before".

If instead you run defrag now, the situation will be *like*:

111111000000

so that when you copy back the file you will get:

111111FFFFFF

If you want a specific file (or a set of specific files) defragmented, you might want to use a file defragmenter, there are two commonly used (they work under XP not W9x):

Contig (Command line):

http://technet.microsoft.com/en-us/sysinternals/bb897428.aspx

Wincontig (GUI and command line):

http://wincontig.mdtzone.it/en/

jaclaz

Link to comment
Share on other sites

Well, after moving these files around, and de-fragging the destination drive lots of times in -between the moves, I am still stuck with some files that are fragmented. These are files that are anywhere from 400 mb to 1.7 gb in size. These are Netscape Navigator email and Outlook 2000 post-office (PST) files.

So to summarize:

- I slave a FAT32 drive to a system running XP-SP3, and use XP's native tools to check the FAT32 drive for errors, and perform a defrag on the 2 volumes on the FAT32 drive.

- After defrag, I have about a dozen large files that are still fragmented, even after multiple defrag runs.

- I move (cut-and-paste) these files to the XP's native hard drive (NTFS) and perform more defrag runs on the FAT32 volume. The FAT32 volume is now fully defragged (no fragged files) and according to the graphic indicator there are large areas of clear, open space on the volume.

- I copy the large files from the NTFS drive back to the "clean" FAT32 drive (back to the folders where they came from), and run defrag / analyze on the FAT32 drive - and it shows those large files TO STILL BE FRAGMENTED! I perform more defrag runs, but those files still remain fragmented.

I would have thought that copying files (even fragmented files) from a FAT32 volume to an NTFS volume would have resulted in a "clean" placement of the files on the NTFS file system (ie - the fragmentation would have disappeared) and so copying them back to their original source volume (FAT32) that had been "cleaned up" by a defrag run would have resulted in a "clean" placement of these files back to where they came from.

Of the original dozen files that did not defragment initially, I am down to about 6 that will not defrag after moving them between the NTFS and FAT32 volumes. The worst of them is about 1.7 gb in size and has a few hundred fragments. And one more thing - running defrag on the NTFS volume (it contains a complete copy of the two 32gb FAT32 volumes in addition to an extra copy of the problem files) resulted in a completely de-fragged drive. None of these files are listed as fragmented as they sit on the NTFS drive.

Edit:

One more thing. I re-named one of the fragmented files as it sits on the FAT32 drive and copied another instance of that file from the NTFS drive to the same directory where this file exists on the FAT32 volume. So the two files are sitting side-by-side in the same directory on the FAT32 volume (but have different names). Do you think that this second copy will also be fragmented like it's brother? Answer: YES. Even after defrag, the two files have the same number of fragments. How do you explain that? Does the "memory" of the fragmented structure of the file still remain after the file has been moved to an NTFS volume and back to a FAT32 volume?

I will next try the suggestion by jaclaz...

Edited by Nomen
Link to comment
Share on other sites

As said it may also depend on the exact way files are copied.

If a "multithreaded" copy is initiated by the "cut and paste" or "copy and paste" (for whatever reason - possibly because of the large file size) it would be normal to have a fragmented file as a result.

This would be "compatible" with the "same number of fragments" observed (a given file starts the same of number of copying threads and results in the same number of fragments :unsure: )

I would rather try a copy or xcopy from command line.

But still defragmenting the file with a file-oriented tool seems to me like the most straightforward solution.

As an alternative you can try using a third party program, Ultradefrag:

http://ultradefrag.sourceforge.net/en/index.html

which provides more "control" or "granularity" on the defragmentation process.

jaclaz

Link to comment
Share on other sites

Hmmm... minor (I hope) interjection.

Fragmentation vs Contiguous (along with directory entries and cluster flags) may be a "key" to understanding.

http://www.forensicswiki.org/wiki/FAT

I could (almost) guarantee that if you

1 - In XP, create a Folder on the NTFS

2 - XXCOPY the FAT32 files to the Folder

3 - Format the FAT32

4 - Copy IO.SYS, MSDOS.SYS, and COMMAND.COM from the Folder back to FAT32

5 - XXCOPY the Folder, without "Overwrite" option, back to FAT32

that it would be totally Defragged.

Refer to this (post by "steve" - same thing...)

http://www.tomshardware.com/forum/141199-32-contiguous-space-defrag

Copying back-and-forth-and-back-and-forth is what you're doing (see my first link).

As jaclaz said, it's the WAY you did it. Using (e.g.) GHOST to Backup then Restore does it (loosely described) based on recreating each FOLDER and then each File ONE TIME, thus no Fragmentation and completely Contiguous starting at the "front" of the Partition, and working "back".

Defrag (Wiki) backgound -

http://en.wikipedia.org/wiki/Disk_Defragmenter_%28Windows%29

All of MS' "defrag" fails in the above respect. I quit using it for that very reason. Jaclaz' suggestion (if you insist) is a pretty decent alternative.

Also see this (look for "SCANFRAG") -

http://www.mdgx.com/upd98me.php

The WinME version seems to do quite well in DOS-Only but the CONTIGuous problem will/may still exist.

[/interjection]

Link to comment
Share on other sites

Jaclaz' suggestion (if you insist) is a pretty decent alternative.

NO. :no:

Jaclaz's suggestion - INDEPENDENTLY from whether the OP will insist or not - is a pretty decent alternative. :yes: (personally I would call it "an excellent one" :whistle:)

jaclaz

Link to comment
Share on other sites

I could (almost) guarantee that if you

1 - In XP, create a Folder on the NTFS

2 - XXCOPY the FAT32 files to the Folder

3 - Format the FAT32

4 - Copy IO.SYS, MSDOS.SYS, and COMMAND.COM from the Folder back to FAT32

5 - XXCOPY the Folder, without "Overwrite" option, back to FAT32

that it would be totally Defragged.

For one thing, I'm not going to reformat the FAT32 drive as part of the maintenance I'm doing.

But here's something that I've not yet seen any explanation for. In the steps you give (above), if instead of steps 3 and 4, I substitute "perform defrag on FAT32 volume" - why wouldn't / doesn't that result in the problem files being contiguous (non fragmented)?

If the problem is that a copy operation under XP (in the GUI shell) is multi-threaded, and results in file fragmentation when the destination is a FAT32 volume, then isin't that a HUGE performance disadvantage for NT-based operating systems? Why would they perform file-copy operations in a multi-threaded way anyways? There's only ONE hard drive, and a SINGLE lane of access to it - so what sense does it make to make the file-copy process multi-threaded, especially if it results in a fragmented file?

Next thing I'm going to try is to use the command shell to copy these files (copy , not xcopy or xxcopy) and see if the files still end up being fragmented on the FAT32 volume.

Link to comment
Share on other sites

If the problem is that a copy operation under XP (in the GUI shell) is multi-threaded, and results in file fragmentation when the destination is a FAT32 volume, then isn't that a HUGE performance disadvantage for NT-based operating systems?

No, actually it is a not-so-slight performance bettering, see (examples):

http://stackoverflow.com/questions/1797113/why-does-multithreaded-file-transfer-improve-performance

http://www.codeproject.com/Articles/24984/MTCopy-A-Multi-threaded-Single-Multi-File-Copying

That the Explorer copy may "switch" to multithread is mentioned here:

http://www.codeproject.com/Articles/65158/The-Practical-Guide-to-Multithreading-Part-2

For instance, try opening Windows Explorer (explorer.exe) and copy some files. You'd see the 'Copying...' window. Look into Task Manager, explorer.exe has created two threads (or even more!) for handling this thing. One is for actually copying (worker-thread), and one is for the GUI. The GUI thread is the 'Copying...' window, which is independent of the Explorer's main window. That's the reason you can close the main window and keep the 'Copying' window open!

What I don' t know (besides the exact way you are copying back those files - i.e. selecting more than one or one by one) is "when" the multi-thread kicks in (at a given file size, when more than N files are selected for copy, etc.).

And still there is the possibility that the target FAT32 is not "consolidated" or not consolidated enough" by the action of MS defrag, as said I would also check the status of the target before copying back the files.

jaclaz

Link to comment
Share on other sites

So I can verify that even if I start with a defragged NTFS source drive (where I've placed the large, fragmented files) and a defragged FAT32 destination drive (that is showing no fragged files), when using the copy command from a command prompt to put a copy of the problem files on the FAT32 drive, the result is still that the files end up being fragmented.

So I did that for a 500 mb file and it had a few dozen fragments. I ran XP's defrag several times, and got it down to 22 fragments. I used the sys internals single-file defrag program, and it got the file down to 10 fragments.

I think the problem is that XP's defrag doesn't do a good job moving files around to create a single contiguous allocation for empty / unused clusters.

I obtained a program called jkdefrag and I'm letting it run over-night on the drive.

I've come across comments where people either know (or seem to think) that Win-98's native defrag did a better job on FAT32 drives vs XP's defrag.

If I don't get good results with jkdefrag then I'll try Norton's defrag from NSW 2002.

Edit: I also performed this command: Attrib d: -s -h -r

The point was to remove anything that might cause a file or a directory on the FAT32 volume to appear as "unmovable". But since there is no "unmovable" attribute that I'm aware of, I'm guessing that maybe the hidden or system attribute might function as an "unmovable" property.

Edited by Nomen
Link to comment
Share on other sites

The point was to remove anything that might cause a file or a directory on the FAT32 volume to appear as "unmovable". But since there is no "unmovable" attribute that I'm aware of, I'm guessing that maybe the hidden or system attribute might function as an "unmovable" property.

The "System" attribute makes a file unmoveable. This is from the days when some Programs implemented Copy Protection by recording the Cluster numbers of certain files so that copying would be detected by a change in the Cluster numbers used by these files. Also IO.SYS had to be locked down in early versions of DOS.

Link to comment
Share on other sites

So I did that for a 500 mb file and it had a few dozen fragments. I ran XP's defrag several times, and got it down to 22 fragments. I used the sys internals single-file defrag program, and it got the file down to 10 fragments.

When a file-oriented defragging utility fails to make a single contiguous file, besides attempting ot use the other one (Wincontig that also allows analyzing the disk) it is a sign that the filesystem "underneath" is still fragmented.

I obtained a program called jkdefrag and I'm letting it run over-night on the drive.

I've come across comments where people either know (or seem to think) that Win-98's native defrag did a better job on FAT32 drives vs XP's defrag.

If I don't get good results with jkdefrag then I'll try Norton's defrag from NSW 2002.

Why not the one that was specifically suggested to you? (ultradefrag, which has such things as the -r switch and "path only" defragging)

I love it :) when people asks for advice and then attempts other things instead :whistle: .

jaclaz

Link to comment
Share on other sites

Ummm... I wouldn't use NSW2002 Defrag if I were you (IMHO).

Also, guess you didn't read and understand this -

http://www.forensicswiki.org/wiki/FAT

Data Area -

The Boot Record, FATs, and Root Directory are collectively referred to as the System Area. The remaining space on the logical drive is called the Data Area, which is where files are actually stored. It should be noted that when a file is deleted by the operating system, the data stored in the Data Area remains intact until it is overwritten.

Remember "file-based"?

IOW, if the FILE already exists a COPY operation has no way of knowing it's the Exact Same File.

Your scenario -

1 - Back up to "elsewhere"

2 - Defrag Source and all OK

3 - Restore from "elsewhere"

4 - Source is now "fragged"

SO... in Step#2, the DIRECTORY entry gets a "brand new Starting Cluster" and all subsequent "clusters" will go "wherever", thus causing "refrag". I said so here

Copying back-and-forth-and-back-and-forth is what you're doing
Seriously, you're wasting a lot of time defeating your primary purpose of Defrag. Kind of like cutting your finger, putting a bandage on it, then cutting through the bandage and wonder "why am I still bleeding". Defrag with tool suggested and have done with it.

jaclaz/rloew - is that correct?

Link to comment
Share on other sites

Why not just use the Windows 98 defrag?

The XP defragmenter program doesn't defragment folders and skips some files for varying reasons. This is because it was designed for a true multi-tasking system that they presumed would usually run NTFS.

Win98 defrag also includes a feature that will profile applications and make their startup files sequential on the disk, potentially making startup times faster.

If you get an "out of memory" error on 98's defrag then you may need to drop in the defrag.exe and dskmaint.dll files from windows ME. But that usually only happens on much larger partitions.

Running the 9x Speed Disk from Norton Utilities 2001/2002 under 9x usually does a good job too. This also gives you the ability to wipe free space, sort filenames, and put certain files first or last.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...