[Volatus]

Here's something you all might be interested in. It's something I've been refining ever since I put together my new Core 2 Duo system with a NCQ-supported SATA2 hard drive. How to make a fast computer faster?

Partition. But read on... as it doesn't involve installing things on D:\, E:\, and F:\...

See, when you make the computer run from one huge partition, over time (even immediately after installation), Windows decides to scatter stuff everywhere. Defragging won't help a thing because the individual files are all in inappropriate places - like one file from a program being placed (in its entirety) at the top of the drive and a support file, say from System32, at the bottom of the drive. It has to seek both places, often at the same time resulting in multiple read/write accesses for the same two files. Bad stuff.

My solution? Partition. Windows needs three major folders: C:\Windows, C:\Users (or the annoying "Documents and Settings" if you don't modify your folder names), and C:\Program Files. Each has a distinct function and a generally fixed size. Through the use of mounted volumes (a function of NTFS), you can accomplish something like this, which I'm currently using on a 2.0GHz P4 system with 512mb RAM...


As you can tell, each partition lies along an appropriate segment of the drive based on its transfer rate... higher transfer rate also means better seek time due to higher data density along the disc surface - less tracks to need to seek through (given that the drive uses CLV recording, which I'm assuming it does for the transfer rate to drop off at the end of the drive - the inner area of the discs)

(click for larger)

Each partition also has its own drive letter assigned, in reverse, from Z upwards. They don't need drive letters assigned, but it helps to allow you to see how much space is remaining in each partition at a glance.


Here's the partition layout in Partition Magic...


The result? It's nothing to laugh about. The Windows partition being located in the top part of the drive results in EXTREMELY good seek times, and the pagefile partition being located between the Windows and Program Files partitions gives an exceedingly good advantage in access times, being right between the two interactive areas. Loaded programs often do two things while loading - they read files from System32, and they write to the page file, and they read themselves. Having the page file between the two partitions makes it extremely snappy since it's "along the way".

Cold boot within, a rough guess, about 20 seconds, faster than my C2D system (that doesn't separate the Windows folder from programs, etc). It starts faster than anything I've ever seen and it no longer has the hard drive being a bottleneck.

I very recently rebuilt a 550MHz Pentium III system with the same layout, and it was the first system I did with the partition layout. It was the fastest P3 system I'd ever seen. It started up blazing fast and opened programs even faster. I can't really describe in (believable) words how fast this computer ran, but... believe me, it really blew my "client" away.

This system also helps ensure reliability, as the file system is no longer centralized either. If one partition fails (like the Windows partition, at the beginning of the drive), the others should still be readable to recover your data if some slip of the finger ends you up erasing the first few hundred megs of your drive. You can still recover it by using a partition recovery program.

The only potential downside is the partition's size limitation, but I haven't seen a Windows folder get over about 5 gigs (on a really bad computer in need of a cleaning). You'd need to adjust the sizes of the page file partition and program files partitions accordingly, but the Users (or Documents and Settings) folder should always be at the end, as it's only used to store static data like videos, music, documents, etc.

Doing this on a clean Windows installation should be done by first partitioning the drive (slaved, or using a boot CD or PartitionMagic boot disks), then installing Windows on the small partition. Then mount the drives in fake folders (I usually use the words "Program Fails" and "Usirs"). Shut down the computer, slave the drive (or boot from a PE boot CD like UBCD4Win), move the data from the "real" folder to the "fake" folder (which should retain its mount assignment in the host OS), delete the original Users and Program Files folders, and rename the mount points to their real names. Then reboot and enjoy!

You can even do this with an existing Windows installation. Just do it carefully, but do the same "fake folders" system. It may take careful planning if your drive is packed, or more than one take. The best way to do it though, is of course a clean install.

Any thoughts on this?

[GrofLuigi]

Volatus, on Apr 9 2008, 03:48 AM, said:

Any thoughts on this?

Very interesting... For those who can't afford/don't want multiple hdd's.

No, really, it's pretty good analysis, I just don't agree with one thing (in theory):

Volatus, on Apr 9 2008, 03:48 AM, said:

This system also helps ensure reliability, as the file system is no longer centralized either. If one partition fails (like the Windows partition, at the beginning of the drive), the others should still be readable to recover your data if some slip of the finger ends you up erasing the first few hundred megs of your drive. You can still recover it by using a partition recovery program.

That would need to be measured against the probability of windows "mount partition in a folder" feature failing. As I said, only theoretical - I haven't actually seen examples of that. Maybe it doesn't matter - if the hard drive fails, the installation is toast anyway.

Another suggestion: You should have mentioned how to do the mounting in case anyone wants to try. I know how, but not everyone does.

Nice job in any case!

GL

*edit: how to do the mounting = the actual command

This post has been edited by GrofLuigi: 08 April 2008 - 09:29 PM

[Volatus]

Ah, true. Well, Windows relies on some kind of weird GUID thing to mount volumes in folders (I believe it makes a junction folder pointing to the new volume), which, as anyone familiar with Windows Installer or InstallShield should know, is really unreliable. But as long as you don't go trashing your drive tables or whatever, it should be good to go

As for how to mount those drives?

First, create a new empty folder where you want the drive to go. Then go to Control Panel, Administrative Tools, Computer Management, then go to Disk Management (or just start, run, diskmgmt.msc, OK). Right click the partition you want to mount in a folder, and click "Change Drive Letter and Paths...". Click "Add...". In that window, pick your new folder and click OK on each of the dialogs. Now you have a mounted drive!


(edit: Also note the lack of a "(Page)" denotation on any of those partitions. Disable your page file if you've got enough memory, for the WIN.)

Mounted drives can be infinitely useful even outside of doing this trick. You can do what I've done and even mount separate drives within your main C:\ tree - for example, my Downloads drive takes the place of my old "Downloads" folder on my hard drive - so to avoid breaking links in uTorrent (my BitTorrent client of choice), I mounted my new drive as the "Downloads" folder and gave it a letter of Z:. It's an eSATA drive that acts like part of my main hard drive!

This post has been edited by Volatus: 08 April 2008 - 09:38 PM

[Volatus]

I'm now giving an older (1ghz Duron, 192mb RAM, XP Pro SP2 + nLite, about 4 month old install) computer the partitioning treatment. (edit: I first slaved the drive, of course... then...) I first moved all the data in the Users and Program Files folders off to another drive (to simplify the process). Then I used Paragon Total Defrag to shrink the MFT to a reasonable size, and defragment/compact the drive. Then I'm going to shrink the main partition to 3gb (it's a 20gb drive), and add partitions for swap, program files, and user data, as in the example above.

I'll update this post as I go.

edit: Okay, all defragged... and started repartitioning. Wow, Partition Magic blazed through the resizing, since it was so cleanly defragged. Now I'm just waiting on it to "update system information" over and over. *guh*
edit: Yay! Now with the chkdsk x: /l:2048 to set the log file size to the absolute minimum. Stupid log file wasting valuable space. Saves a maximum of 62mb.
edit: Alrighty. Now to compact and defrag the MFT on each drive, since they're always placed in the worst location - the center of each partition...
edit: That out of the way, now created the folders on the main drive, and mounted the drives in the right places (leaving swap to be assigned when I reboot the computer). Now, copying the files back to the drive. Needless to say it's going a lot faster.
edit: Already ran into my first quirk... TeraCopy (my file manipulation app of choice - check it out) was led to believe that there was only F:\'s free space available in F:\Users, ignoring that it was a mounted volume. Hm... I wonder if that may be a problem in the future. I was able to tell it to ignore the error though and try copying it anyway - which it's doing now quite nicely.
edit: Contrary to unpopular belief, you should, in fact, get yourself one of these. It's a self contained USB-IDE adapter and power brick. Infinitely useful.
edit...: Yay! Now to install a power supply in this thing (notably missing from the above image), and give it a try. Keep in mind... the last time I used this computer, it took a full FIVE MINUTES to finish starting up and loading the startup items (yahoo messenger and MySpaceIM. Hey, it wasn't a computer I used.)
edit: ****. I wasn't expecting that. Windows totally did not recognize the new path assignments I gave it while "slaved" to USB. So the "users" and "program files" folders were unusable folders and I'm surprised Windows even booted to a semiusable state. It gave me all sorts of nasty errors and hung up at the strangest things. I got to a command prompt via Ctrl+Shift+Del (task manager) and used Sysinternals' Junction to change the "program files" and "users" mount points to junctions that pointed to the drive letters. Then I logged out and logged back in, and found that, to Disk Management's point of view, not even drive C: had a drive letter. Wow. That's messed up. Now I'm rebooting it to see if it can pull itself together, where I'll create the fake folders I originally mentioned, re-slave it, rename them, and try this again.
edit: Wow, that fubbed up. Bad. It worked, but I was tied to nonexistent drive letters! Both Disk Management _and_ diskpart agreed on one thing: that none of my partitions (including drive C:) had drive letters. I could reassign any of them to any other letter, even while Windows was running off drive C. I could change my C letter to H: on the fly. And it would still deny that a letter was assigned to it. That is the STRANGEST problem I'd ever experienced. And in one FAIL swoop of experimentation, I typed "clear" into diskpart, got a flash of BLUE, and the drive would no longer boot. Wow, thanks, MS, for the confirmation that I was going to brick the system. Grr... okay, so back to the drawing board.

Word of caution: don't do what I did.
That's me: f*cking up, and learning to fix, Windows problems, since 1996!

final edit: Wow. I was able to rebuild the MBR that diskpart erased without warning, and got it to boot again after a fixboot and fixmbr in recovery console, but Disk Management still REFUSES TO SHOW ME ANY DRIVE LETTERS! They're ASSIGNED drive letters, but Disk Management (and hence Diskpart) tell me there are none! So frustrating! So the lesson of today is: DON'T DO THIS WITHOUT FIRST CREATING THE MOUNT POINTS ON THE BOOTED COMPUTER!

This post has been edited by Volatus: 09 April 2008 - 03:10 AM

[frodo]

I thought id give this a try, despite thinking it would make no difference as i already partition my drive similarly and install windows, apps, pagefile and docs to different partitions.

There is no performance gain if you already partition your drive sensibly like a sane person and move your pagefile, apps and docs to offworld partitions.

The benefit though, and the only reason im sticking to it, is the mount points Far easier to mount a drive for apps into c:\program files\ than trying to re-path all your apps to any drive other than c:\program files\ !

Im glad that the post exists, it might get people into using sane data separation strategies!

This post has been edited by frodo: 14 April 2008 - 06:16 PM

[Volatus]

The problem with moving things to different partitions (drive letters) is that letters are inflexible - if you end up shuffling around drive letters (maybe by accident or by slaving a drive), you're screwed - all the files linking to the old letter don't get that drive any more. The advantage of mount points is that you don't need to tweak a single thing within Windows. Additionally, ill-coded programs that hard code a "Program Files" destination, for example, won't have trouble. You also don't need to manually tweak the installation path of each installed program, nor do you need to keep an additional "C:\Program Files" folder when your typical-usage path is really "D:\" or the like.

I also made a little better progress on another computer I was doing a separation on. Since you always need to install Windows first, you can always do this to a running (live) system with no damage. The method I chronicled earlier was just a bad idea - creating the partitions, mount points, and moving the data, all in one go while "slaved". If you make the partitions and mount points outside of the booted system, it won't recognize it when it goes to boot up again. You've got to make the partitions and mount points (folders) first - then slave (or boot-CD) it and move the data and rename the folders. Best way to do it.

And of course, glad I could help

edit: By the way, moving your page file to another partition will typically decrease your performance, unless it's between two partitions of frequently accessed data! For example, if you think that just putting it on its own partition will help, and hence just resize your main partition down by a few gigs and put a partition at the end of the drive, you'll end up with a sloppy, slow as hell system that's a headache to use (thinking "oh, your advice blows, n00b!"). So... well... I dare say that none of this is what a "sane" person would do - either that or most people are insane (more likely). Hopefully this will show people what is sane though

This post has been edited by Volatus: 14 April 2008 - 10:09 PM

[gendouhydeist]

this really interesting, but could this apply FAT32 filesystem?? do you think that NTFS faster than FAT32.. to tell you the truth i don't like NTFS cause it create SYSTEM VOLUME INFORMATION and i also think that this decrease read/write speed operation cause it's backup files, and i only use NTFS for greater 4GB file

[Volatus]

Jesus God no, don't use FAT32! FAT32 is so old and outdated, it's a slap in the face of logic to say "i don't like NTFS because of the system volume information folder".

That folder is annoying (with its default security options - just erase them and enable inherited permissions on it), but it only holds less than 1mb of filesystem data. Your FAT tables are larger than the NTFS MFT and bloated log file combined. NTFS has the added benefit of being able to store very small files in the file table itself, as opposed to needing to read the file table ("where is it?") then go fetch the data ("what is it?") like in FAT32. NTFS, when properly optimized, is orders of magnitude better (and not to mention more reliable) than FAT32.

The only things NTFS does wrong is the ESN journal (created if you install Live Messenger), which can be deleted and disabled, and the obscenely large "log file" that serves absolutely no purpose (64mb by default). You can reduce the log file size to the minimum (2mb) by doing "chkdsk X: /l:2048" and rebooting (where X is your drive letter). Most of the problems with NTFS is in Windows' default implementation of it - much like most things in Windows.

So ditch that old FAT32 piece of junk and upgrade yourself to NTFS. Properly optimized, it'll be a _lot_ quicker than FAT. Not to mention more reliable, and give you ever so slightly more disk space. NTFS also has filesystem-level compression as well, which, as opposed to DriveSpace-like crap, is actually reliable.

Oh... and maybe you were referring to the fact that System Restore stores its data in the System Volume Information folder. That's a reason I hate System Restore. Just disable it. That's part of Windows, not NTFS. You'll see that on your FAT drive as well.

[GrofLuigi]

Actually, Volatus (I don't want to quote your entire message of FAT vs NTFS because it's long), I'm starting to question all that information lately. I have read a long time ago about all this and agree with most of your points (or facts about NTFS vs FAT, but are they really facts?), but recently I am wondering if FAT could be used on a small (boot) partition efficiently, to bypass the NTFS security permissions overhead (no mater what anyone says, there's got to be overhead - on any computer, if you perform additional tasks you must use CPU time). Of course, this is of one doesn't care about security permissions - on a nonnetworked or specialized PC for example. Maybe this scenario even fits your method described in this thread?

One thing is for sure, I would never label NTFS as faster than FAT. More secure, reliable, efficient... probably, but faster? I myself always use it (if not only to avoid the chkdsk at startup).

Tip (from my long-time tinkering with filesystems): I've found that increasing the MFT zone too much (with the well known registry entry NtfsMftZoneReservation) actually hampers performance when there are many small files (all of them go into MFT and it grows and it fragments and then: 1. It takes eternity to defragment; 2. even defragmented, it is still too large/slow - I guess it acts as a partition of its own). This is only my observation (no scientific methods applied).

About the NTFS logfile: I don't think it's ordinary log file, I THINK it's used for file operations not yet committed to the table and I am afraid if reduced too much data loss would occur (in a case of power loss/really bad crash during intensive file operations). I don't know the actual parameters needed for it (like how much space one operation takes approx. = how much is really needed), but in the past I have reduced it (albeit not to the minimum) and noticed no difference whatsoever. I might be completely wrong about this tho.

*Edit: Link1 , Link2 , Link3 .

GL

This post has been edited by GrofLuigi: 17 April 2008 - 10:11 AM

[Volatus]

Well, I may opt to give that a try. Right now I'm still fighting with that failed conversion I outlined in painful detail above.

Problem is, I cannot for the life of me get that computer's Disk Management to assign a drive letter. I even backed up the whole drive, erased it, then repartitioned it (with one single 4gb partition to save time), then restored the Windows system onto that partition. It still boots as drive C: and is visible in My Computer, but Disk Management claims there is no drive letter attached to it.

If I create a new partition in the empty space, it will claim the letter it chose (E, for example) is already in use and assigns no letter. If I pick a high letter like U or V, it flashes that letter on the partition for a split second before it disappears again. I've tried deleting the drive, deleting all volumes, deleting any references to drives or volumes in the registry, deleting everything I possibly can, and it still has that same problem.

What I'm going to try now is creating a microscopic nLite installation, barely enough to install and boot. Then I'll export the registry to a ".reg" file, pull the ol' switcharoo on the partitions like I did on this computer, boot it again (band-aid if needed), and export it again. Then I'll compare the two reg files and see what changed.

That, I'm all but certain, will reveal the problem and allow me to fix it on this trouble computer (and I'll, of course, post the results). Man, I can't believe Windows can be such a pain, but browsing through the registry tonight kinda clued me in as to why it is such a pain... =(

edit: Yeah, I can't reproduce it. I just did exactly the same thing to that test installation (on the same base system as well) on a separate hard drive, and it failed just as catastrophically - locking up during the first startup with the "can't find profile, using default...", etc, error messages, had to Ctrl+Alt+Del x2 the login screen, but I was able to - just as before, "junction" the users/programs folders to their drive letters. Then I went to diskmgmt.msc (which hung for a while), and it had all 4 (primary!) partitions assigned letters. I don't get it. Maybe it's a drive thing and it needs to be erased. Sigh...

This post has been edited by Volatus: 27 April 2008 - 02:48 AM

[GrofLuigi]

Have you tried tinkering with this: http://support.microsoft.com/kb/223188 ?

HKEY_LOCAL_MACHINE\SYSTEM\MountedDevices

Warning: you could lose all your mount points!

GL

[Volatus]

Yeps. I don't understand why that didn't fix it. I deleted everything in there, and when I rebooted, it rebuilt itself but I still couldn't change anything

Strangest problem ever. I wonder if there's a Windows system file that got corrupted somehow. =\

edit:
I've given up... I have finally exhausted the last possible option. I grabbed a drive that the computer had never seen before, erased it, gave it a single active NTFS partition, then copied all the files from the "problem" drive (with no admitted drive letter) to the new drive using two IDE<->USB adapters. Then I booted it. It still assigned no letter to the C: drive.

And I do mean the C: drive. In My Computer, it will actually take any letter I assign it - that is, even while running, I can switch the C: drive to be X: and all shortcuts will break, running programs will crash, etc... but Disk Management still says there's no letter assigned to it. It flashes the new drive letter for a split second over the partition label, and that's it, it goes back to unassigned. The strange thing is, everything still works fine. The system runs perfectly fine. Not a single glitch or problem other than the drive letter in Disk Management.

Strange thing is, all other drives can be assigned letters. I plugged in a flash drive and it gave it a letter. I plugged in a second hard drive to the controller and it got a letter too. Everything but partitions on the boot drive can get a letter.

I have _no idea_ what is going on here, and I doubt anyone else does either. I guess I just stumbled on one of those genuine Windows flukes. I absolutely hate giving up on solving a problem, but I guess I have no choice.

This post has been edited by Volatus: 27 April 2008 - 06:39 PM

[TranceEnergy]

I used to to this procedure several years ago, actually 8years, on windows 2000/xp.
However, if you have a farely new computer there is a much faster way to increase overrall system/io performance.
Use a good partition manager to set cluster size to 64KB, disable pagefile first tho and hibernation if u have it,enable large memory pages on 64bit 2gbram+, increase critical workerthreads to 16(both). Then defragement harddrive. C:\ should be somewhere between 40GB to 60GB, suitable to keep many to insane many programs. Games should be kept at a completely different harddrive, because then games dont have to read/write to same place as windows/os has to.
Also prefetch should be disabled as it slows down system boot, unless you are going to manually optimize the layout.ini, just forget prefetch.

Also good practise is to use portable software, making the y:\program files here mentioned in #1 bloat, as you will NOT need it, since all programs should preferrably be portable and on different harddrive for optimized performance.

For even further i/o speed consider ramdisk software by superspeed software, or getting some expensive SSD sata drivers, which you place directly on sata-port on mainboard, making it a truly fast 0.1ms system access speed.

If that aint enough yet, start looking at memory latency timings.

Your ddr2 800mhz cl4.4.4.12t2 average good memory might do good in 3dmark benchmarks and the like, but ddr2 400 mhz at 3.3.3.9.t1 will make it look retarded when it comes to f.example boot times.

If your system when optimized as you like it cant boot under 10 seconds, it aint optimized in my book.

It's always a matter of how far you are going to go, but i think the whole repartitiong scheme is really outdated, unless your computer hardware is even MORE outdated, the performance gains from it are truly neglible, at BEST.

Please note, changing cluster size on c:\ in Windows Vista with latest acronis tools doesnt actually work, it will make system boot totally crashnburn.

[JuMz]

Now all you need is to publish a guide on how you accomplished what you did in your first post

[quasar_9000]

Volatus...... would you mind posting a pdf document, outlining your partioning steps, so that I (and others) can better understand the theory of how it works? One for a "clean install", and one for an "existing install".

Thanks a bunch.

[r0sWell]

very interesting idea :-)

I'm just wondering if it's possible to "set" these custom partitions/mounts in an installation of a fresh unattended WinXP ?

(and if it is, how we do that, in a small guide for example, i'd love to test this in a VM at 1st :-) )

[quasar_9000]

Hey all........ after much testing in VMWare, I have posted a "pdf" file that will guide everyone on how to properly map the partitions to the appropriate folder. Feel free to give me some feedback. Enjoy!!

 NTFS_Mounting.pdf (85.88K)
Number of downloads: 107

This post has been edited by quasar_9000: 08 June 2008 - 04:58 PM

[subassy]

Hey I'm a little new hear, this is a great board by the way.

Firs thing I thought I'd mention is that the attached PDF above downloads as ~44KB and when I try it to open it in FoxIt PDF reader I am told it is not a PDF or that the PDF is corrupt.
edit: Ok maybe that only happens if you're not logged in. I just tried downloading again (I'm logged in now) and it worked. Never mind then.

One thing I'll point out about C: drives, make sure they are big enough to store your page file in case something happens. For instance recently I ghosted my whole C:/S: partitions (the one HDD) so I could go to a LAN party and be only a little paranoid. When I restored this image my S: drive had reverted to D: for some reason. And since the system couldn't find S: it defaulted it to C:. I have two gigs of memory which works out to... (1.5 x 2048 = 3072) a 3,072 megabyte page file which would hardly fit on a more ~4 gig C: drive. Of course if you have 256MBs of memory you don't really have to worry about it.

In conclusion I recommend making your C: partition a little larger than four gigs or taking other precautions.

As far as installing applications to a drive other than C: I did find the extra utility SETX that will permanently change the value of the %ProgramFiles% environment variable to another value. You can get here, there's probably another link some where: MS's download page)

As for my own set up I am think of implementing something like setting up my first hard drive as C: and Page partitions and then mounting a 400 gig drive to C:\Program Files so that I don't have to worry about resetting the path of everything I install (which will help with silent installs). I'll have to SUBST to create some drives or something. That would take some getting used to.

This post has been edited by subassy: 25 June 2008 - 05:28 PM

[BRegi]

Experts pl forgive me if this sounds quite absurd but i would like to clear. Will it be possible mount the Programs in to a different directory by booting the system into safe mode and moving the folders into the desired drive and giving a mount diskmgmt of C:\Program Files to the desired partition. Kindly help.

[deda]

OK, I agree that this kind of research must be done, for future purposes and better answers for the future, but in the other hand, I don't understand this obssession on to be faster. Overclocking, partitons, etc. Almost all the available software still running on the same way of, at least, one decade ago. Maybe "the face and the way you move the mouse changed" but...
Most of all gains, you can't feel really. Compare a sheet or a graphic table with senses of a human body don't make sense. Put 2 machines side by side, same hardware and different configuration, and ask someboy what is the faster?
You'll be very surprised. Even identically machines have different behavior. There's a lot of things without answers, about circuits, electricity, etc.

Of course, English isn't my native idiom.