Jump to content

Welcome to MSFN Forum
Register now to gain access to all of our features. Once registered and logged in, you will be able to create topics, post replies to existing threads, give reputation to your fellow members, get your own private messenger, post status updates, manage your profile and so much more. This message will be removed once you have signed in.
Login to Account Create an Account


Photo

FAT16 vs FAT32 vs NTFS speed on USB stick


  • Please log in to reply
31 replies to this topic

#1
ilko_t

ilko_t

    MSFN Addict

  • Super Moderator
  • 1,708 posts
  • OS:none specified
  • Country: Country Flag
I have done a few tests to measure copy speed during Text portion of setup. Here are the results:

NLited XP source, 5251 files occupying 399.7 MB. After the copy part on destination drive are found 2850 files, occupying 476MB.
Preparation made by WinSetupFromUSB 0.1.1, run on XP SP2 system, C2D 4300@2.9GHz.
USB stick- Lexar JD Lightening 1100 1GB, which is relatively fast:
Posted Image

In all cases stick was quick formatted using HP USB format tool 2.0.6.
To check if USB write protection via migrate.inf plays role I tested without it as well.
All tests performed intentionally on an old Celeron 1.7MHz/384MB RAM/VIA 266KM chipset, which supports USB 2.0 once the USB drivers are loaded.

FAT16:

Stick prepared for ~15 min. (899 secs.)
Text mode copy part with USB write-protection- 4:10
Text mode copy part without USB write-protection- 6:13

FAT32:

Stick prepared for ~21 min. (1253 secs.)
Text mode copy part with USB write-protection- 12:58
Text mode copy part without USB write-protection- 7:15

NTFS:
Stick prepared for ~4 min. (238 secs.)
Text mode copy part with USB write-protection- 1:42
Text mode copy part without USB write-protection- 1:45

It really puzzles me why FAT32 is so slow compared to FAT16 :blink:

If your system supports USB boot from NTFS formatted partitions AND you are not planning excessive usage of the USB stick in terms of write operations, where NTFS may wear it out (disputable with modern USB sticks), NTFS seems to be the way to go.

Edited by ilko_t, 24 October 2008 - 01:51 PM.

Install Windows from USB, boot Linux, multiboot and a lot more with WinSetupFromUSB



How to remove advertisement from MSFN

#2
jaclaz

jaclaz

    The Finder

  • Developer
  • 14,067 posts
  • OS:none specified
  • Country: Country Flag
If you have an occasion to do so :unsure:, I would like those tests repeated on a comparable machine running Windows 2K.

You may find some interesting differences. ;)

jaclaz

#3
ilko_t

ilko_t

    MSFN Addict

  • Super Moderator
  • 1,708 posts
  • OS:none specified
  • Country: Country Flag
Here there are- 2000 SP4, 5738 files / 611MB. After the copy part on destination drive are 2697 files, occupying 290MB.

FAT16:
Stick prepared for ~16 min. (987 secs.)
Text mode copy part without USB write-protection- 3:47

FAT32:
Stick prepared for ~21 min. (1282 secs.)
Text mode copy part without USB write-protection- 3:36 :blink:

NTFS:

Stick prepared for ~5 min. (297 secs.)
Text mode copy part without USB write-protection- 4:18 :wacko:

What the heck were MS screwing with those drivers?

Will test 2003 SP1 too, will it be the same as XP SP2...?

Install Windows from USB, boot Linux, multiboot and a lot more with WinSetupFromUSB


#4
FSK-JoKeR

FSK-JoKeR

    Newbie

  • Member
  • 16 posts
thanks for your short review... just jesterday i'm worried about the long setup time for my xp (because i formated the stick to fat32 because i want to test gparted (which won't work under ntfs)

now i new, that it's much better to use fat16.
Images in signatures shall NOT exceed a total of 300x100 and 80kb. Flash and animated signatures and avatars are not permitted. You can enter max 4 lines of text in signature or signature image.
http://www.msfn.org/...ead-t18408.html

#5
ilko_t

ilko_t

    MSFN Addict

  • Super Moderator
  • 1,708 posts
  • OS:none specified
  • Country: Country Flag
Here is 2003 SP2, after the copy part on destination drive are 3444 files, occupying 554MB.

FAT16:
Text mode copy part without USB write-protection- 8:04

FAT32:
Text mode copy part without USB write-protection- 9:18

NTFS:
Text mode copy part without USB write-protection- 2:46

FSK-JoKeR- as far as I remember the old GParted 0.3.4-8 does work from NTFS, but is a bit outdated. PartedMagic 3.0 also worked from NTFS.

Install Windows from USB, boot Linux, multiboot and a lot more with WinSetupFromUSB


#6
jaclaz

jaclaz

    The Finder

  • Developer
  • 14,067 posts
  • OS:none specified
  • Country: Country Flag
Since most USB sticks of nowadays common size come from manufacturer (NOT bootable) formatted as FAT32, you can now understand how I left a number of people bewildered by the speed of sticks on my good ol' Asus sub-notebook running 2K SP0. ;) (600 Mhz - 10 Gb HD - built in year 2000)

Typically I would go to a customer's office, make him copy the files I need (from a very fast - top end - desktop) on one of his sticks, (several tens of seconds), then in a flash read them on my small faithful thingy (also helped by the fact that reading is usually much faster than writing). B)

Just imagine the overall amount of time wasted by people waiting for their files to be copied to sticks thanks to the "stupid" upgrade to the drivers the good MS guys made. :whistle:

Unrelated, but maybe worth a read:
http://www.msfn.org/...-me-t99220.html


jaclaz

#7
mmg1818

mmg1818
  • Member
  • 1 posts

exFAT Versus FAT32 Versus NTFS


With Vista SP1 Microsoft has introduced a new file system. Extended File Allocation Table (exFAT) is the successor to the old FAT32 file system. What are the advanatages and disadvantages to this new file system? What are the differences between exFAT and FAT32? When is exFAT preferred over NTFS?

Microsoft has released the exFAT file system with Vista SP1. This file system that previously had been rumored to be released with the original Vista is finally available to the public on a wide scale. This article will explain what issues with FAT32 that exFAT fixes. Surprisingly to many people, exFAT even may be better than the much loved NTFS in some circumstances.

FAT32 is the file system with which most windows users are most familiar. Windows first supported FAT32 with Windows 95 OSR2 and has increased support for it through XP.

FAT32 has multiple issues that modern systems can experience: - By default windows systems can only format a drive up to 32 GB. Additional software works around this issue. When formatted at these bigger sizes, FAT32 becomes increasingly inefficient.

- The maximum file size on a FAT32 formatted drive is around 4 GB. With DVD and high resolution DVD formats now available, this limit is commonly noticed.

- Dealing with fragmentation and free disk space calculations can become painfully resource intensive in large FAT32 systems.

- A FAT32 directory can have 65,536 directory entries. Each file or subdirectory can take up multiple entries; therefore, FAT32 directories are limited with how many files it can hold.



exFAT was first released with CE 6.0 but will finally hit the mainstream with Vista SP1. exFAT has several advantages over FAT32: -File size limit is now 16 exabytes.

- Format size limits and files per directory limits are practically eliminated.

- Like HPFS, exFAT uses free space bitmaps to reduce fragmentation and free space allocation/detection issues.

- Like HTFS, permission systems should be able to be attached through an access control list (ACL). It is unclear if or when Vista will include this feature, however.

In the past most power-users of Microsoft systems have opted to format/convert to a NTFS file system instead.



Interestingly enough, exFAT is not used currently for formatting hard drives. It is being recommended in Flash memory storage and other external devices only. This is why it is currently not considered a huge competitor to NTFS on hard drives.

However, exFAT should be a true competitor to NTFS on systems with limited processing power and memory. NTFS on flash memory has been known to be inefficient for quite some time. exFAT's smaller footprint/overhead makes it ideal for this purpose. Of course, only if your definition of "ideal" allows software to be proprietary and not open source.

Vista will happily read FAT, exFAT, and NTFS from flash. ReadyBoost may not work with exFAT formatted flash drives, however.

In conclusion, basically, FAT is a simple system. This limits FAT system by losing efficiency at large sizes, but allows it to run with less resources. The complexity of NTFS increases features but requires more memory and processing power.



Extended file allocation table (exFAT) is a new file system that is better adapted to the growing needs of mobile personal storage. The exFAT file system not only handles large files, such as those used for media storage, it enables seamless interoperability between desktop PCs and devices such as portable media devices so that files can easily be copied between desktop and device. In addition, exFAT can be adopted with minimal effort. The exFAT file system encapsulates standard FAT and TFAT functionality.

The exFAT system offers the following advantages: Enables the file system to handle growing capacities in media, increasing capacity to 32 GB and larger. Handles more than 1000 files in a single directory. Speeds up storage allocation processes. Removes the previous file size limit of 4 GB. Supports interoperability with future desktop OSs. Provides an extensible format, including OEM-definable parameters to customize the file system for specific device characteristics.

In addition, you can choose to add support for TFAT to your exFAT implementation to ensure transaction-safe operations. As of Windows Embedded CE 6.0 and later, TFAT can only be supported in an exFAT environment.


copy paste by mmg1818



#8
ilko_t

ilko_t

    MSFN Addict

  • Super Moderator
  • 1,708 posts
  • OS:none specified
  • Country: Country Flag
mmg1818, this is good to know, but how it relates to the specific subject in the posts above yours?

Install Windows from USB, boot Linux, multiboot and a lot more with WinSetupFromUSB


#9
jaclaz

jaclaz

    The Finder

  • Developer
  • 14,067 posts
  • OS:none specified
  • Country: Country Flag
...and besides ilko_t's question....

copy paste by mmg1818


From where? :unsure:

jaclaz

#10
erpdude8

erpdude8

    MSFN Master

  • Member
  • PipPipPipPipPipPipPipPip
  • 2,139 posts

With Vista SP1 Microsoft has introduced a new file system. Extended File Allocation Table (exFAT) is the successor to the old FAT32 file system. What are the advanatages and disadvantages to this new file system? What are the differences between exFAT and FAT32? When is exFAT preferred over NTFS?

Microsoft has released the exFAT file system with Vista SP1. This file system that previously had been rumored to be released with the original Vista is finally available to the public on a wide scale. This article will explain what issues with FAT32 that exFAT fixes. Surprisingly to many people, exFAT even may be better than the much loved NTFS in some circumstances.

exFAT was first released with CE 6.0 but will finally hit the mainstream with Vista SP1. exFAT has several advantages over FAT32: -File size limit is now 16 exabytes.

...


There is a way to unofficially access exFAT partitions under Windows XP. Read here:
http://www.merawindo...ts/Default.aspx

#11
jaclaz

jaclaz

    The Finder

  • Developer
  • 14,067 posts
  • OS:none specified
  • Country: Country Flag

There is a way to unofficially access exFAT partitions under Windows XP. Read here:
http://www.merawindo...ts/Default.aspx


You mean an unofficial way to access..... :whistle:

:lol:


GOOD find! :)

Thanks fo sharing it.

jaclaz

#12
Snake7

Snake7

    Newbie

  • Member
  • 16 posts
  • OS:Windows 7 x64
  • Country: Country Flag
My Asus P5Q Pro refused to boot Vista setup from exFAT formatted stick. So it's not the way at the moment.

As to speed: all usb sticks copy big files fast, but small one very slow.
The good way i think is to reach the goal in another way:

1. Create iso image on usb-stick, copy it after boot from usb to hdd, mount to virtual cd-rom with letter Z: and run setup from it (of cource boot.ini shoul be edited to run virtual cd after reboot - this is '-')

2. Create archive on usb-stick, which will be copied on any partition and unpacked there, after runing setup there should be no problems.

So what do you think? My opinion that second way will be easier to get working and faster.

#13
jaclaz

jaclaz

    The Finder

  • Developer
  • 14,067 posts
  • OS:none specified
  • Country: Country Flag

The good way i think is to reach the goal in another way:

1. Create iso image on usb-stick, copy it after boot from usb to hdd, mount to virtual cd-rom with letter Z: and run setup from it (of cource boot.ini shoul be edited to run virtual cd after reboot - this is '-')

2. Create archive on usb-stick, which will be copied on any partition and unpacked there, after runing setup there should be no problems.

So what do you think? My opinion that second way will be easier to get working and faster.


It greatly depends on hardware.

#1 above is not possible AFAIK unless you use a RAMDISK setup (and have enough RAM)
#2 is perfectly possible using a variation of either:
http://www.msfn.org/...howtopic=121446
or the good ol' DOS way:
http://www.911cd.net...showtopic=16713

jaclaz

#14
BigDaddy

BigDaddy

    Senior Member

  • Member
  • PipPipPipPip
  • 532 posts
i got a 4GB usb drive.
Would it be faster to format it as FAT16 or not?
And if it is not possible, how would I use the FAT32 driver from Win2k which is much faster?

Edited by BigDaddy, 28 January 2009 - 06:50 AM.


#15
jaclaz

jaclaz

    The Finder

  • Developer
  • 14,067 posts
  • OS:none specified
  • Country: Country Flag

i got a 4GB usb drive.
Would it be faster to format it as FAT16 or not?

A FAT16 formatted 4GB filesystem will use "non-standard" 64Kbyte clusters, it may be speedier, but highly inefficient:
http://www.msfn.org/...howtopic=110779
and non compatible with "standard" DOS

And if it is not possible, how would I use the FAT32 driver from Win2k which is much faster?

GOOD question. :)

No pre-made answer, though. :(

Probably, but cannot say, disabling SFC/WFP and replacing the "new" driver with the "old" one may work, but not only cannot say, I don't even know in WHICH driver lies the difference, possible candidates :unsure::
  • fastfat.sys
  • disk.sys
  • usbstor.sys
  • usbhub.sys
  • usbehci.sys (usbohci.sys) (usbuhci.sys)
and it could also be a "combined effect"....


jaclaz

#16
BigDaddy

BigDaddy

    Senior Member

  • Member
  • PipPipPipPip
  • 532 posts
uf, a cluster size of 64kb that would really be a waste especially since I have lots of little files below the 64KB on the usb pen.
Have MS posted any info why FAT32 is slower on WinXP than Win2k?
That really seems counter-productive.

Edited by BigDaddy, 28 January 2009 - 08:21 AM.


#17
jaclaz

jaclaz

    The Finder

  • Developer
  • 14,067 posts
  • OS:none specified
  • Country: Country Flag

That really seems counter-productive.


Counter-productive to what? :unsure:

If the "base idea" from the good guys at MS is to NOT let people run NT based systems on USB flash devices, by:
  • making FAT32 slower
  • making "common" flash based devices unpartitionable (with the "Removable" vs. "Fixed" bit) in the controller
  • failing to supply a Filter Driver to workaround above
  • pushing NTFS as the preferred filesystem (and thus heightening the risk of premature wear)

it seems to me like they did a VERY good work.... :whistle:

jaclaz

#18
BigDaddy

BigDaddy

    Senior Member

  • Member
  • PipPipPipPip
  • 532 posts
I wonder how the vista's and Win7's driver performs with FAT32?

Edited by BigDaddy, 29 January 2009 - 01:54 AM.


#19
ilko_t

ilko_t

    MSFN Addict

  • Super Moderator
  • 1,708 posts
  • OS:none specified
  • Country: Country Flag
BTW you may try replacing the above mentioned driver(s) ONLY in the ~BT folder with those from 2000, keeping the originals in ~LS.
During Text mode the drivers in ~BT folder are loaded and used, but not copied to the target Windows being installed. The ones from ~LS folder are copied instead.
Will take some tests to determine the proper combination of drivers, if possible at all.

Install Windows from USB, boot Linux, multiboot and a lot more with WinSetupFromUSB


#20
jaclaz

jaclaz

    The Finder

  • Developer
  • 14,067 posts
  • OS:none specified
  • Country: Country Flag
Just for the record, MS released exFAT drivers for XP SP2 or SP3:
http://www.msfn.org/...o...128994&st=5

jaclaz

#21
cdob

cdob

    Friend of MSFN

  • Member
  • PipPipPipPipPip
  • 963 posts
Windows 2000 and XP use different cache behaviour.
I guess that's the main difference. I don't have a full explanation.

Uwe describes some XP USB behaviour.
http://www.uwe-siebe...usbstick_e.html

A average USB stick at default settings:
FAT, FAT32: no write cache
NTFS: with write cache

A USB stick is often marked removable.
As knwon a factory tool can flip this setting, e.g. Lexar BootIT.
Or a filter driver can flip this setting. Cfadisk.sys can be integrated at textmode.
http://www.msfn.org/...p...st&p=818429

Policy 'Optimize for quick removal' is stored in registry.
That's a hardware related setting:
[HKLM\SYSTEM\CurrentControlSet\Enum\USBSTOR\#specific HardwareID#\Device Parameters\Classpnp]
"UserRemovalPolicy"=dword:00000002
Setting can be integrated at migrate.inf or setupreg.hiv.

I wonder:
Does write cache enable a read cache too?

Given a USB stick and write cache active:
What's text mode copy time for FAT, FAT32 and NTFS?


Just for the record, MS released exFAT drivers for XP SP2 or SP3:

Well, how to build a bootable exFAT USB stick at XP?
Do you update mbrbatch.cmd and mkimg.cmd?

format /FS:exFAT creates 32kb cluster at a 2gb USB stick.
At first glance cache behaviour is not changed. Writing small files is slow, no write cache by default.

#22
jaclaz

jaclaz

    The Finder

  • Developer
  • 14,067 posts
  • OS:none specified
  • Country: Country Flag

Well, how to build a bootable exFAT USB stick at XP?
Do you update mbrbatch.cmd and mkimg.cmd?


I have no idea, don't even know if it will EVER be bootable.

I mean, exFAT support needs to be integrated in NTLDR/SETUPLDR.BIN, otherwise there is no way for intial stages of booting, maybe it is possible to "enhance" current "advanced" method "Fake Signature"/"XP Kansas City Shuffle" to support using a "normal filesystem" kicker image and switch during the shuffle the filesystem?

But anyway we'll need to wait for exFAT32 support in grub4dos....

About updating the batches, as soon as I find some time to do that, I hope I will be able to :), though if the above speculation is true there would be not much of a point, would it? :unsure:

jaclaz

#23
ilko_t

ilko_t

    MSFN Addict

  • Super Moderator
  • 1,708 posts
  • OS:none specified
  • Country: Country Flag

I wonder:
Does write cache enable a read cache too?

Maybe it's still all about the write cache? Setup attempts to delete the compressed files while expanding them to target directory. USB devices are write protected, but maybe those attempts for deletion are still speeded up by the enabled write-cache triggered by using NTFS:
http://www.uwe-siebe...usbstick_e.html

Obviously Windows XP doesn't enable a write cache for USB drives that appear as 'Removable'. The settings 'Optimize for quick removal' or 'Optimize for performance' doesn't seem to make any difference then, except that the latter enables the user to format 'Removable' USB drives with NTFS. But with NTFS Windows XP enables a write cache, writing small files becomes lightning fast.

Seems using NTFS triggers use of write cache even on removable drives.

Removing write-protection gives no speed difference in the tests above, write cache still matters.

Could this be the explanation?

A quick test using this approach (no deleted files during Text mode), may reveal if that's the case:
http://www.msfn.org/...howtopic=119742

I'll have no test rig around for some weeks, any taker?

Edited by ilko_t, 30 January 2009 - 02:16 PM.

Install Windows from USB, boot Linux, multiboot and a lot more with WinSetupFromUSB


#24
cdob

cdob

    Friend of MSFN

  • Member
  • PipPipPipPipPip
  • 963 posts

I mean, exFAT support needs to be integrated in NTLDR/SETUPLDR.BIN, otherwise there is no way for intial stages of booting

Yes, that's true.

maybe it is possible to "enhance" current "advanced" method "Fake Signature"/"XP Kansas City Shuffle" to support using a "normal filesystem" kicker image and switch during the shuffle the filesystem?

Yes, that's possible. I'm chainbooting exFAT at XP already. Did worked at first trial.
BTW: chainboot works at Windows 2000 and XP64 too, should work at all ntldr or setupldr.bin. Most likely at bootmgr too.

But anyway we'll need to wait for exFAT32 support in grub4dos....

Yes, current grub4dos at MBR: dosn't find grldr at exFAT.

@ilko_t
Yes, write cache maybe the main difference.

As for SetupSourceDevice harddisk:
I dislike this because BIOS may change harddisk number. That's not a universal solution.
Maybe use new grub4dos write: boot grub4dos, write txtsetup.sif, set SetupSourceDevice harddisk

#25
ilko_t

ilko_t

    MSFN Addict

  • Super Moderator
  • 1,708 posts
  • OS:none specified
  • Country: Country Flag

@ilko_t
Yes, write cache maybe the main difference.

As for SetupSourceDevice harddisk:
I dislike this because BIOS may change harddisk number. That's not a universal solution.
Maybe use new grub4dos write: boot grub4dos, write txtsetup.sif, set SetupSourceDevice harddisk

Yep, not universal, but should do the trick just to reveal if the write cache makes the difference, since there are no write attempts when using it.

As for the grub4dos write- I doubt, grub4dos uses BIOS disk numbering, boot disk is hd0. At partition screen the same disk is harddisk1, internal one is harddisk0, even though it's not the first in BIOS boot order.
The only semi-logical explanation I found was that disks numbering is in alphabetical order, as per driver name. I.e. disks on ATAPI.SYS get the first numbers, then on DISK.SYS, then SIL3112.SYS for example. Drive letter assignment still depends on BIOS numbering, fixed/removable is taken into account.
Compare:
http://www.msfn.org/...o...st&p=705697
http://www.msfn.org/...o...st&p=647581
Posted Image

USB disks are at DISK.SYS. I tried to rename it to _isk.sys and amend txtsetup.sif accordingly. This failed, disk.sys is hardcoded and used in other drivers, I did not go further.

Thanks for your input, as always you trigger a little brainstorm.

Install Windows from USB, boot Linux, multiboot and a lot more with WinSetupFromUSB





1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users



How to remove advertisement from MSFN