Jump to content

Welcome to MSFN Forum
Register now to gain access to all of our features. Once registered and logged in, you will be able to create topics, post replies to existing threads, give reputation to your fellow members, get your own private messenger, post status updates, manage your profile and so much more. This message will be removed once you have signed in.
Login to Account Create an Account



Photo

Does Fat32 align its clusters?

- - - - -

  • Please log in to reply
15 replies to this topic

#1
pointertovoid

pointertovoid

    Advanced Member

  • Member
  • PipPipPip
  • 498 posts
  • Joined 16-January 09
Something of a perverse activity...

On a Raid-0 as well as on most SSD, aligning a volume improves speed, at least with Ntfs file system.

That is, most SSD have page size of 2n bytes, as do Raid-0 arrays.
If the volume begins (... for instance) at sector 63, then many files will have their first sector on one page or disk, and the next one on the other page or disk, needing to access both where one may suffice.
But if the volume begins at sector 64, or at 1MiB, or similar values, performance improves (... at least for some file and cluster sizes, at least in IOMeter and AttoDisk).

Though, I didn't see this improvement with Fat32 volumes.

So: could it be that Fat32 puts its cluster borders at a bizarre number of sectors from the beginning of the volume?
For instance because it puts the Fat first, whose size isn't a multiple of the cluster, and immediately thereafter it puts the clusters?

Thank you!


How to remove advertisement from MSFN

#2
dencorso

dencorso

    Iuvat plus qui nihil obstat

  • Supervisor
  • 5,842 posts
  • Joined 07-April 07
  • OS:98SE
  • Country: Country Flag

Donator

No. You're looking at it from the wrong side. FAT-32 is a filesystem, which must be created on a given medium, which is first partitioned. The partition containing FAT-32 can be aligned or not. It depends on which application creates it. And, BTW, FAT-32 is different from FAR-12/16 in that the root directory may be anywhere, and grow as needed.

#3
cluberti

cluberti

    Gustatus similis pullus

  • Supervisor
  • 11,021 posts
  • Joined 09-September 01
  • OS:Windows 8.1 x64
  • Country: Country Flag

Donator

No, FAT32 doesn't magically align, but the default cluster size for FAT32 volumes is 16K for any drive larger than 32GB, rather than NTFS's default of 4K for volumes up to 16TB in size. This could actually be why you don't see much difference, as you're already spanning multiple clusters in FAT32 with the default format options.
MCTS Windows Internals, MCITP Server 2008 EA, MCTS MDT/BDD, MCSE/MCSA Server 2003, Server 2012, Windows 8
--------------------
Please read the rules before posting!
Please consider donating to MSFN to keep it up and running!

#4
pointertovoid

pointertovoid

    Advanced Member

  • Member
  • PipPipPip
  • 498 posts
  • Joined 16-January 09
Thank you!
More precisely:

I aligned the volumes with various tools, like GPartEd and DiskPar. This improved speed when the volume was subsequently formatted in Ntfs, but not in Fat32.
I tried several stripe (in the case of Raid-0) and cluster sizes, both for Ntfs and Fat32 - not only the default sizes.

In Ntfs, the conditions over alignment, cluster size, read access size are consistent with the stripe size in Raid, and the line size in Compact Flash cards.
In Fat32, little happens.

Hence my interrogation, where Fat32 puts its cluster with respect to the beginning of the volume.
If the first cluster is 153th sector inb the volume, then aligning the volume doesn't help.
In Ntfs in contrast, is does make a difference, hence the first cluster seems to be at a distance from the beginning that is multiple of cluster size.

About directories: I believe all of them, including the root, consist of clusters, hence their size won't change the alignment within the volume.

Or?

#5
jaclaz

jaclaz

    The Finder

  • Developer
  • 15,501 posts
  • Joined 23-July 04
  • OS:none specified
  • Country: Country Flag

Hence my interrogation, where Fat32 puts its cluster with respect to the beginning of the volume.
If the first cluster is 153th sector inb the volume, then aligning the volume doesn't help.
In Ntfs in contrast, is does make a difference, hence the first cluster seems to be at a distance from the beginning that is multiple of cluster size.

About directories: I believe all of them, including the root, consist of clusters, hence their size won't change the alignment within the volume.

Or?


ONLY seemingly unrelated ;) SPRING EFFECT GRAPHICS!
http://reboot.pro/13466/

jaclaz

#6
pointertovoid

pointertovoid

    Advanced Member

  • Member
  • PipPipPip
  • 498 posts
  • Joined 16-January 09
Yes Jaclaz, this is one effect I suspect. And since storage media from the "Format C:" era had no binary page size as Compact Flash cards have now, designers of Fat32 had no physical reason to make the Fat size multiple of the cluster size. Worse, there is more information stored at the beginning of the Fat32 volume.

I don't even know how the part of an Ntfs volume located after the Mft behaves, as I measured performance on empty volumes, hence the bench file was created before the Mft...

Well, I want to align the clusters of only one single Compact Flash card with one single volume size, so I consider testing all possible positions of the volume until I see performance improve.

Here is what Atto sees with Ntfs, the all-important small writes improve a lot, and 8kiB clusters would improve a bit further:

Attached File  Atto815q01.png   12.01KB   16 downloads Attached File  Atto4kioNtfsQ01.png   12.41KB   12 downloads

#7
jaclaz

jaclaz

    The Finder

  • Developer
  • 15,501 posts
  • Joined 23-July 04
  • OS:none specified
  • Country: Country Flag
I think that in NTFS you don't have that problem at all, since *anyhting* is 4Kb in size (or multiples thereof).

You can go a different way. (for testing).
There a few "third party formatters" that do give you more "freedom".

We have:
http://www1.mager.org/mkdosfs/
http://www.ridgecrop...fat32format.htm
and, more recently:
http://reboot.pro/14354/

Particularly mkdosfs has a number of "tweakable" settings that could do in your case.

jaclaz

#8
pointertovoid

pointertovoid

    Advanced Member

  • Member
  • PipPipPip
  • 498 posts
  • Joined 16-January 09
Bingo!

I used Diskpar (but will try the suggested software) to let a Fat32 volume begin at various locations, and
- I got with Fat32 the big improvement observed with Ntfs
- It happened at a volume offset which is not a power-of-two
- This best volume offset depends on cluster size
- Compact Flash Card manufacturers don't use this better offset
- This effect explains easily the non-repeatable measurements by Atto

So the very likely explanation is indeed that Fat32 puts its clusters at an odd offset from the volume start, at a position that depends on the size of the Fat.

I make a few measurements more and I tell you.

#9
jaclaz

jaclaz

    The Finder

  • Developer
  • 15,501 posts
  • Joined 23-July 04
  • OS:none specified
  • Country: Country Flag
Very good. :)

I was trying to induce you to have start of the volume at a "non-queer" or "non-bizare" position and have the actual storage area aligned to cluster size (by making arbitrary FAT(s) sizes and/or hidden sectors), but of course the same result can be done as you did, by shifting the beginning of the volume until the beginning of the actual storage area (after a given size of FAT(s)+hidden sectors) gets aligned to cluster size.

Very nice find. :thumbup

jaclaz

Edited by jaclaz, 14 June 2011 - 01:54 PM.


#10
pointertovoid

pointertovoid

    Advanced Member

  • Member
  • PipPipPip
  • 498 posts
  • Joined 16-January 09
I confirm the whole story: clusters in a Fat32 volume can be aligned at will, it improves speed, and alignment is more difficult than with Ntfs because the Fat's bizarre size shifts the clusters.

My trials are finished, and:
- If the size of the Fat32 volume or clusters are changed, hence the Fat's size, a new best number of hidden sectors can be found.
- MkDosFs does it as well as Diskpar. I've had worries with Diskpar, but have used MkDosFs too little to tell. GPartEd is nice, uncheck the Align option.
- A rather slow SD card in a good reader, with a 2GB Fat16 volume, isn't very sensitive. But a P-Ata/66 CF card is!
- Bad alignment fully explains the odd and unrepeatable speed curves in Atto - with a Raid-0 as well, so it should be aligned.
- The X25-E is insensitive to alignment, due to its 10-way Raid. Other SSD must be sensitive.

Here's an example with a CF card, Transcend 300x 4GB (SLC, fixed, reliable buffers), connected to an ICH2, with W2k and Atto v2.02:
(click on the pictures to display in real size)

Attached File  Atto63sect.png   12.77KB   11 downloads Attached File  Atto88sect.png   12.88KB   12 downloads
The small writes, a big weakness of CF cards even if SLC, improve a lot. Very useful if you install an OS or normal data on the CF.
Even big writes improve markedly.

As bought, the SD card I tested began its Fat16 at sector 135... Sector 139 improved marginally, but other values degraded. A hint that SanDisk knows the story.
But the Transcend CF had its Fat32 at sector 63, and was as bad as on the left picture, so users should take action.

On this card, I got a small improvement with 8kiB and 32kiB cluster size over the 16kiB on the pictures here.
Write page size may exceed all these sizes, and the measurement keeps some random uncertainty due to the alignment of Atto's test file when clusters are smaller.
By the way, on IOMeter, the user should define the byte spacing of the queries to match the clusters he intends to use.

Proper alignment to 1, 2, 4, 8, 16, 32 sectors does matter, but little to 64 sectors and above with this CF, though clusters as big exist.
Either I reached the size of write pages, or latency gets less important for such sizes - this is how manufacturers choose page size anyway.

Should a user try all 32+ possible offsets to find the best one? Not necessarily.
Either you have some tool that tells the absolute LBA of a file - easy then. Suggestion welcome!
Or you look carefully at Atto's curve: if the writes show their first notch at 4kiB say, then you should shift the clusters by 2kiB (4 sectors if 512B), run Atto again and see where the next notch is.
And once you believe to have the optimum, you should try to shift it by all powers-of-two and see a notch at the corresponding write size.

Ntfs has big advantages, but should Fat32 be preferred for CF cards? Maybe...
- Clusters >4kiB let files align on 8kiB and 16kiB page boundaries, important. But W2k-Xp won't defrag Ntfs then.
- NtfsDisableLastAccessUpdate seems to mess with Avast here, though fewer small writes would be desirable.
- Write errors happen with most CF brands due to buffer weakness, very annoying on Fat32.
- You waste less time with Ntfs! Start the volume at sector 64 or at 1MiB, done.
- I ignore Linux's filesystems, sorry folks - but you're nice people, cheers.

Marc Schaefer, aka Pointertovoid

Edited by pointertovoid, 15 June 2011 - 06:47 PM.


#11
billtodd

billtodd

    Junior

  • Member
  • Pip
  • 53 posts
  • Joined 28-November 07
The data clusters of a FAT32 volume can be arbitrarily aligned on a disk using available (often free) tools. The explanation below describes using them on old-style MBR partition formats (where the first partition on the disk normally starts at sector offset 63 and logical cylinders are normally defined as comprising 255 heads covering tracks of 63 sectors apiece) for those who may want to multi-boot among both older and newer systems (the newer systems and third-party software can usually be tweaked to honor the old format, but older software often cannot be used without risk on disks using the newer format), but much of it is also applicable to the new-style (Vista/Win7) partition format.

One such tool is the free version of EASEUS Partition Master, which can tell you (in properties->FAT info) what the offset of the first data cluster is within a FAT32 partition that you create (the offset will vary based on the size of the partition and the size of its clusters, due to variations in the size of the FATs). Using that information, you can tweak the start-position of the partition (e.g., by first creating a dummy partition ending just before the desired start sector for a primary partition or 63 sectors before the desired start sector for a logical partition) such that the start position, plus the first data cluster offset, winds up on a suitable disk boundary. First, however, ensure that the logical disk geometry is seen as 255 heads with 63 sectors per track (if it isn't, it will usually become so - at least using Paragon Partition Manager - if all partitions on the disk are first deleted; the 255-head value comes from the BIOS int 13h 8-bit field and a Microsoft bug which had difficulty dealing with the full 256 heads): the total number of sectors in a logical cylinder needs to be an odd number if you're going to be able to adjust partition start locations to arbitrary points while still maintaining 'legal' (cylinder-aligned) partition locations for older software that may otherwise become destructively confused (the old IDE standard value of 16 heads with 63 sectors per track obviously doesn't meet this criterion, nor does the default 240-head geometry used on some old Thinkpads).

Alternatively, you can use another tool such as the free version of Paragon Partition Manager to adjust the 'number of sectors per boot' (in create partition or format partition 'more options' for a FAT32 partition) to a value which will move the start of the first data cluster to an offset which, when added to the partition start location, will produce the desired alignment. For some reason I had difficulty doing this on a Win2K system with PPM versions later than 8.5, even though they nominally offered that option, but presumably they work with XP and later systems.

NTFS and recent extx file systems seem to guarantee that the data clusters (if at least 4 KB in size) will be aligned at some multiple of 4 KB from the partition's start, so there's no need to play 'number of boot sector' games (which PPM doesn't support anyway for these file systems) with them if that alignment is tolerable. But the ability to place the partition start location at an appropriate alignment on the disk as described above can still be useful (both when dealing with RAID alignment and when dealing with 'advanced format' disks with 4 KB sectors when using XP and earlier systems). One reassuring thing about using this approach is that (with the possible exception of tweaking the 'number of boot sectors' in a FAT32 partition) the resulting disk layout is bog-standard, minimizing that possibility that any software (including random third-party software) will get confused by it.

Edited by billtodd, 16 June 2011 - 12:04 AM.


#12
jaclaz

jaclaz

    The Finder

  • Developer
  • 15,501 posts
  • Joined 23-July 04
  • OS:none specified
  • Country: Country Flag

Either you have some tool that tells the absolute LBA of a file - easy then. Suggestion welcome!


If this is what you need you can probably find it here:
http://www.partition...m/utilities.htm

You can use FINDPART FINDFAT, CHSDIR or CYLDIR utilities.

Something that may be of help is this batch:
http://www.msfn.org/...ey/page__st__85
(probably needing to be fixed/adapted/whatever, as it was left in a semi-abandoned state before testing/bugfixing could be carried)

jaclaz

#13
pointertovoid

pointertovoid

    Advanced Member

  • Member
  • PipPipPip
  • 498 posts
  • Joined 16-January 09
Many thanks for the suggestions of partition and formatting software!

Meanwhile I've made attempts with Windows on aligned volumes, and not everything is clear...

Both the W2k and Xp installers failed identically to reboot ("non-system disk") when the volume containing the boot sector (at least I suppose so) was created by a separate software.
That is:
- I could align the clusters with Diskpar or MkDosFs, got identical excellent performance
- Win would install and boot on an aligned volume containing the Win folder, the paging file, Arcldr, Boot.ini, Ntldr, Ntdetect and the like but NOT if this was the active volume.
- The Win installer could create an active volume beginning at 3MB and boot from it
- I've had the same worry on an ich10r and had solved it as well with a Win-made active volume, then third-party aligned volumes could launch W2k or Xp.

About performance, it makes little difference at boot...
- One volume in Ntfs beginning after 63 sectors, with 4kiB clusters. The other in Fat32, with its 16kiB clusters optimally aligned through reserved sectors
- CF is a Transcend 300x 4GB, SLC, fixed, Udma4. Chipset is i815ep with ich2, Cpu PIII Tualatin 1400MHz (@1400MHz for the trial), 512MB
- Win moderately MsConfig'd, no antivirus nor firewall, with Ati video driver adding 1s, measured between OS choice and startmenu responding
- XP starts in 14.2s on both volumes
- W2k starts in 14.7s on misaligned Ntfs and 13.9s on optimal Fat32...
- On an X25-E and E8600 (@3300MHz then), W2k starts in 15s as well, and Xp in 8s. This machine makes a difference only when the antivirus and firewall bloat the start.
- Once booted, alignment clearly improves file operations, and applications start somewhat faster.

Edited by pointertovoid, 24 June 2011 - 04:49 PM.


#14
GrofLuigi

GrofLuigi

    GroupPolicy Tattoo Artist

  • Member
  • PipPipPipPipPipPip
  • 1,361 posts
  • Joined 21-April 05
  • OS:none specified
  • Country: Country Flag
Now you've mentioned the word align too many times and it reminded me Microsoft has been aligning things way back... :whistle:

GL

#15
jaclaz

jaclaz

    The Finder

  • Developer
  • 15,501 posts
  • Joined 23-July 04
  • OS:none specified
  • Country: Country Flag

Now you've mentioned the word align too many times and it reminded me Microsoft has been aligning things way back... :whistle:

From the article :whistle: :

Alignment is not disk optimization
Don't confuse program file alignment with disk optimization. Both can improve performance, but they are not directly related.


:angel

;)

jaclaz

#16
pointertovoid

pointertovoid

    Advanced Member

  • Member
  • PipPipPip
  • 498 posts
  • Joined 16-January 09
This meant aligning the executable code with 4kiB pages so W98 could transfer them directly from HDD to L2.
It makes no difference to W95, and with W98 only at the older executable that were not compiled aligned, like Office97.
In these not quite common configuration, the executable alignment tool did improve application load time by 20%.




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users