Jump to content

Moving the beginning of a Partition


Recommended Posts

Hi,

Under Win7, Disk Management (DM) allows you to shrink and extend basic partitions (primary, extended and logical partitions). Usually shrinking and extending happens on the right side of a partition in DM, that is, towards the innermost tracks of the drive.

I assume that the left side of a partition in DM represents the beginning of the partition and is located at the outermost (fast) tracks of the drive, and the right side of the partition represents the end of the partition and is located at the innermost (slow) tracks of the drive---correct me if I am wrong. I also assume that the data and filesystem bookkeeping information (Master File Table---MTF) is always at the left side, so the beginning of the partition in the case of Win7 (NTFS).

As far as I understood DM can move the end of a partition in the shrinking and extending process, but moving the beginning of a partition does not seem to work in Win7 DM. Maybe Diskpart could do the job, but I'm not sure.

However, there are third party tools out there that can do the job and move the beginning of a basic partition. I have a question though: if, in the case of extending the partition, the beginning of the partition is moved towards the outside of the disk (meaning adding more outermost tracks to the partition), then I assume the filesystem bookkeeping data has to be moved also towards the outside along with the beginning of the partition. But what about the many GBytes of user data in the partition?? Can that stay, which would imply that the relative position of the files would have to be recalculated in the MTF, or will that huge block of GBytes of user data also be moved towards the outside of the disk along with the beginning of the partition and the filesystem bookkeeping data?

Ofcourse, best would be that the huge block of user data would simply stay and that the relative file position of all files is recalculated, but I am not sure if that is the case. I am really looking for a conclusive answer on the question because that wil have an impact on the partition layout of my 2TB drive.

johan

Link to comment
Share on other sites


Hi,

Under Win7, Disk Management (DM) allows you to shrink and extend basic partitions (primary, extended and logical partitions). Usually shrinking and extending happens on the right side of a partition in DM, that is, towards the innermost tracks of the drive.

I assume that the left side of a partition in DM represents the beginning of the partition and is located at the outermost (fast) tracks of the drive, and the right side of the partition represents the end of the partition and is located at the innermost (slow) tracks of the drive---correct me if I am wrong. I also assume that the data and filesystem bookkeeping information (Master File Table---MTF) is always at the left side, so the beginning of the partition in the case of Win7 (NTFS).

Remember, you asked for it (you are WRONG! :w00t:)

Mainly you are assuming :ph34r: that the visual representation of Disk Management (and it's left or right) has actually any direct connection with inner or outer tracks of the disk drive and that *somehow* inner tracks are slower and and outer tracks are faster.

On modern hard disks this may (or may not be) the case.

Where is "sector 0" (i.e. first sector of a disk)?

Please choose one:

  1. innermost track
  2. outermost track

Any you choose might be either right or wrong.

Are you really sure that the hard disk you are actually using is organized like here?

http://www.pcguide.com/ref/hdd/geom/tracks-c.html

(and following)

So, assumed that the sector 0 is on the outermost track and that each track is one after the other the disk management representation is nothing but that of a spiral UNreeled into a line.

Whether the left means "outermost" and the right "innermost" or that "outermost" means "faster" and "innermost" means "slower" are just "opinions" or, better "guesses".

The $MFT is a LOGICAL index/structure (peculiar to NTFS ONLY) part of the filesystem and it's position can vary (and for the record it is NEVER at the beginning of a volume).

Everything inside a filesystem or volume is relative to the begginning (and end) of the filesystem or volume.

When you move a filesystem/partition/volume "to the right" (or "to the left") the indexing structure contents are left as they are and the whole content is shifted.

Example:

You have a disk with three (primary, to keep this simple) small partitions.

1st partitions starts at CHS 0/1/1 LBA 63 and ends at CHS 0/254/63 LBA 16064

2nd partitions starts at CHS 1/0/1 LBA 16065 and ends at CHS 1/254/63 LBA 32129

3rd partition starts at CHS 2/0/1 LBA 32130 and ends at CHS 2/254/63 LBA 48194

2nd and 3rd partition are identical in size, 8,225,280 bytes.

If they are formatted with NTFS they will get (under XP) the $MFT at (relative) offset Dec 5355 (cluster), i.e. since the cluster size will be 1 (512 bytes) sector the $MFT beginning will be 5355 sectors from the beginning of the volume.

So, if you access the PHYSICALDRIVE (i.e. the disk), you will find the beginning of the $MFT of second partiion at sector 16065+5355=21420 and the $MFT of third partition at sector 32130+5355=37485

In this particular case, since 2nd and third partitions are identical, the $MFT offset is the same.

So, if you delete second partition and "move to the left" the third partition, it's filesystem structures will occupy the same absolute position of the previous ones.

If you delete both partitions, and recreate a single "second" one that starts at CHS 1/0/1 LBA 16065 and ends at CHS 2/254/63 LBA 48194 and format it, you will see how the $MFT offset will be 10710, i.e. the $MFT will begin at sector 16065+10710=26775

And no, when you move/shift partitions, the WHOLE partition (and thus also data in them) is moved, WHILST when you enlarge partitions, normally data remains where it is and simply UNallocated sectors are added to the filesystem, the example above with the very small image was given to teach you how to never assume anything, when partitions grow bigger the $MFT offset tend to get to cluster #786432 (with 8 sectors per cluster) and remain there for a very wide range of sizes.

But filesystem structures have nothing to do with actual disk/partition management.

jaclaz

Edited by jaclaz
Link to comment
Share on other sites

A bit like adding a floor to a building.

It would be very different to to add a floor on top than digging the ground under the building watch the whole building shifts down and end up creating the new floor, ...still on top, for a same final height. :D

Link to comment
Share on other sites

Hi,

Well, the quessing work of mine was not just really wild guessing, there was some reasoning behind it. In Disk Management (DM) Disk 0 (boot disk) has its C partition on the left side in DM. I assume (I know, I know "assume" again) the folks from ASUS are smart enough to put the OS (Win7) on the outermost tracks, which are supposed to be the high data rate tracks (in the assumption the boot drive is recording in zones: http://www.pcguide.com/ref/hdd/geom/tracks_ZBR.htm ). That should results in the fastest booting possible with that particular drive.

So, implicitly and tacitly I assumed (sorry again) that the left side in DM represents the outermost (high data rate) tracks for ALL drives and parititions visible in DM--- I might be wrong.

But, you were very right by raising your point. The notions outermost, innermost, fast, slow, beginning, end, ...they are ambiguous and susceptible to interpretation.

So, I've examined the 2TB drive of mine in the following way. I created 3 primary partitions (cluster size = 64 KB) on the 2TB drive with the following asymmetric partition layout:

| (F) 10 GB || (H) 10 GB || UNallocated 1833 GB || (I) 10 GB |

Volume F is the leftmost volume in DM, Volume I is the rightmost Volume in DM. I then copied a test folder of 4 large video files (total 9.68 GB) onto Volume F, and that took 109 seconds. From that I calculated a write speed of 88.9 MB/sec. I repeated the same procedure for volume I.

Summarizing the Write speeds:

Volume F: 88.9 MB/sec

Volume I: 43.1 MB/sec

The same procedure was then repeated (again) with 3 video files in the test folter (7.26 GB), and that resulted in Write speeds:

Volume F: 90.8 MB/sec

Volume I: 45.4 MB/sec

I also did the test with Atto Disk Benchmark, that gave me the following results:

Volume F: 145 MB/sec (Write speed) ; 128 MB/sec (Read speed)

Volume I: 67 MB/sec (Write speed) ; 67 MB/sec (Read speed)---the same speed as the write speed, is that strange?---

Temporary conclusion:

In my particular case the left side in DM represents the high data rate (outermost) tracks, and the right side in DM represents the low data rate (innermost) tracks.

There is still the position of the MBR te be determined. Is the MBR located at the side of the outermost tracks or at the side of the innermost tracks?

To find that out, I used PartitionInfo 8 on Win7 (but in Compatible Mode WinXP-SP3). As the disk is asymmetrically partitioned, one can clearly see the one and the other end of the disk in the sector numbering in PartitionInfo 8. Volume F started at sector 2048, and Volume I started at sector 123886544896. As the MBR is usually located on sector 1, or LBA 0, the MBR is neighboring Volume F. Hence the MBR must be on the very outermost track, meaning the very left side in DM.

Still a question: is if fair to call the outermost track in the partition (which presumably is also the highest data rate track in the partition, and also the nearest to the MBR) then the "beginning" of the partition??

----------------

Concerning the Master File Table (MFT) issue, did some reading on:

http://en.wikipedia.org/wiki/Master_File_Table#Internals

http://www.pcguide.com/ref/hdd/file/ntfs/archMFT-c.html

In the example in your post (jaclaz), the $MFT sits at a (relative) offset of Dec 5355 (clusters) from the "beginning" of the partition. Do I understand it correctly that this is 30 to 35% into the partition size? Or, do I interpret this wrong somehow? In your example that would be an initial "MFT zone" of 30-35% ?! In pcguide.com they talk about a MFT zone of only 12%.

I have a question about the MFT. Just to be sure, in what direction does the MFT structure initially grow in case the partition is nearly empty (right after its creation)? I am not sure but I would "assume" (yes, yes, sorry) that the MFT initially grows towards the beginning of the partition inwhich the MFT sits. It looks like the MFT has "loosely" reserved some kind of smaller partition within its partition.

Next is really relevant for my problem.

Did I understand it correctly?? If an NTFS partition gets expanded by extending the "end" of the partition with UNallocated clusters, then nothing is being copied into the new part. If the old partition part was totally filled up, further filling up of the newly added part to the partion causes the "tail" of the MFT to jump at some time into the new part where it just keeps on growing there alongside the newly added (very) large files (which are nothing but external attributes of the MFT).

Do I understand this one correctly??; and this is maybe the most important part of all for me!! If the partition gets expanded with UNallocated clusters at the side which represent the "beginning" of the partition (by adding more outermost tracks to the partition), then neither the user data (with large files), nor any MFT info from the old part will be moved during the expansion process of the partition. Increasing the partition at its beginning is basically enlarging the MFT zone (which can be used, as far as I understood, for exactly the same purpose as the zone on the "other side" of the MFT in the partition). Do I interpret all this correctly?

(I seem to make a distiction between user data and the MFT, but I am getting more and more conscious that the (very large) user data files are a fully integral part of a dynamically growing and expanding structure called the MFT.)

johan

Edited by DiracDeBroglie
Link to comment
Share on other sites

Yes, the outermost track are normally the left side, but you cannot swear by it in every case, that was the point.

A modern hard disk is a "box" which you CANNOT look inside, and that you can inspect only by trusting what the "box" says about itself.

The MBR is FIRST absolute sector of a disk, CHS 0/0/1 LBA 0.

So it is (theoretically) the first sector of the outermost track (but again for all what you - or we - know it could well reside on sector - say 17489 - and being remapped by the integrated device controller to sector 0).

So yes, normally left=beginning=faster but never assume that this info is "universal" for all devices.

The $MFT is "tricky business".

It's start location is connected to size of the volume, but, as said, it is not a linear function of it, once you reach something like 8 Gb (if I recall correctly) it places itself at cluster #786432 and remains there up to at least 500 or 750 Gb.

It's size is an alltogether different thing, it is normally expressed in percentage of the volume size and there are switcheds in the Registry to set this percentage to either:

  1. 12.5%
  2. 25.0%
  3. 37.5%
  4. 50.0%

http://support.microsoft.com/kb/174619/en-us

http://www.ntfs.com/ntfs_optimization.htm

Please be aware that info about NTFS (as well as other filesystems) that you can find, including (actually EXPECIALLY) the ones on MS sites is often incorrect, partial, misleading or plainly wrong, you have to be VERY careful with whatever you find and always verify what is written.

As an example according to MS the example I posted above would not be true as per here:

http://support.microsoft.com/kb/140365/en-us

the default NTFS cluster size on a volume in the range 7 MB–512 MB is given as 4 KB, whilst in real word it is 512 byte.

Correct info is given here:

http://www.ntfs.com/ntfs_optimization.htm

Please keep in mind that what the filesystem (and the mentioned Registry settings) do is NOT (as it may seem semantically) to "reserve" that size to the $MFT, it actually sets a given amount of space to be "preferred".

To clear the concept, if you start filling a volume, at a given point the "reserved" $MFT area will be used to store files allright ;).

The $MFT is nothing but a "special" file, organized as a database index, so when you actually simply format a volume it will be very small and will grow in time.

Newish NT systems (since 2k, if I recall correctly) defrag operation will also defrag the $MFT.

The $MFT will also CONTAIN directly any file up to (around) 900 bytes in size!

The given "standard" address of cluster #786432 represents the BEGINNING of the $MFT, it will always expand "on the right" when it grows.

Do I understand this one correctly??; and this is maybe the most important part of all for me!! If the partition gets expanded with UNallocated clusters at the side which represent the "beginning" of the partition (by adding more outermost tracks to the partition), then neither the user data (with large files), nor any MFT info from the old part will be moved during the expansion process of the partition. Increasing the partition at its beginning is basically enlarging the MFT zone (which can be used, as far as I understood, for exactly the same purpose as the zone on the "other side" of the MFT in the partition). Do I interpret all this correctly?

I am not sure to understand this one correctly, it seems to me you are getting it exactly the other way round.

If you expand a partition "on the right side" (given that the $MFT is already in it's standard position) it WILL NOT be moved, as well as any data in the volume.

The only practical difference will be that the bootsector mirror will be moved to last sector of the partition slot (or first sector outside the filesystem, at the end or "on the right").

If you expand the partition "on the left" what you are actually doing is normally "moving" or "shiftng" the whole partition (and the $MFT and all data in the volume) "to the left" AND expand it "on the right". Cannot really say if some software will instead leave the $MFT in it's place and update it's offset in the bootsector(s) changing it from the the "standard", but I doubt it. :unsure:

I guess some experiments are needed.

Just for the record, if you think how "queer" is number #786432, try writing it in Hex: 0xC0000 ;).

jaclaz

Link to comment
Share on other sites

Some definitions:

-Disk Management (DM):

-Beginning of a volume = left(side) in DM = outermost track = highest data rate;

-End of a volume = right(side) in DM = innermost track = lowest data rate.

I read the content of all the links you posted, and also some content that was provided in secondary links. Thanks for the links.

It's start location is connected to size of the volume, but, as said, it is not a linear function of it, once you reach something like 8 Gb (if I recall correctly) it places itself at cluster #786432 and remains there up to at least 500 or 750 Gb.

Clusters?? So, if cluster size is 512 bytes then the 8 GB volume counts 15625000 clusters (=sectors in this case). If the beginning of the MFT is located at cluster #786432, the MFT zone occupies something like 5% of the volume space (in bytes)---I assume (yes "assume" again, sorry) that the 5% is between the beginning of the volume and the beginning of the MFT; so the MFT zone is on the outermost tracks of the volume. If, on the other hand, the cluster size on the 8 GB volume would be 8 KiB, then the MFT zone would occupy 80% to 90% of the volume space asuming the MFT still starts at cluster #786432? For the case were the cluster size would be 64 KiB on the same 8 GB volume, the MFT zone would hypothetically occupy 640% of the volume space, if the MFT still were to start at cluster #786432. Of course, this cannot be the case. There must be a limiting/correcting factor somewhere (I think?).

According to

http://www.pcguide.com/ref/hdd/file/ntfs/archMFT-c.html

the maximum cluster size of the MFT itself would be 4 KiB, but the pcguide.com article doesn't mention anything about the cluster size of the MFT zone. I hope and assume the cluster size of the MFT zone and the non-reserved area on the "other side" of the MFT always have the same cluster size (correct me if am wrong). Theoretically, with a cluster size of 8 (or higher) KiB during formatting of the entire 8 GB volume, one could still have a situation where the MFT zone would start at the end of the drive, despite the MFT itself having a cluster size of only 4 KiB.

Furthermore, with a maximum cluster size of 4 KiB for the MFT, it looks like one could have two (2) different cluster-size systems on the volume; e.g. 4 KiB for the MFT, and 64 KiB for the user data files!?

Do I really interpret all this correctly? I must've overlooked something. What did I miss?

---------------

The given "standard" address of cluster #786432 represents the BEGINNING of the $MFT, it will always expand "on the right" when it grows.

Are you sure that in a newly created volume (with an almost empty MFT zone and empty non-reserved area on the "other side") the MFT has the preference to grow towards the right, meaning towards the innermost tracks, away from the MFT zone??

From the links you gave me, I inferred that the MFT has the "tendence or preference" to grow from the beginning of the MFT zone (which lays (in disk-space terms) 12.5%, 25.0%, 37.5% or 50.0% from the beginning of the volume) towards the beginning of the volume, so towards the outermost track in the volume, meaning to the LEFTward direction (so not to the right). Maybe I interpreted all this not quite right, please correct me if I am wrong.

The given "standard" address of cluster #786432 represents the BEGINNING of the $MFT, it will always expand "on the right" when it grows.

Just realized something. With [expand "on the right" when it grows], maybe you meant "on the right SIDE" of the MFT, so growing in leftward direction?? Did you? If yes, then ignore previous paragraph of mine.

If you expand a partition "on the right side" (given that the $MFT is already in it's standard position) it WILL NOT be moved, as well as any data in the volume.

Ok good, expanding the partition towards the right side in DM (towards innermost (slower) tracks) does not entail copying of the MFT nor copying of any userdate. That is what I wanted to hear.

The only practical difference will be that the bootsector mirror will be moved to last sector of the partition slot (or first sector outside the filesystem, at the end or "on the right").

"bootsector mirror"? Did some Googling but couldn't really find that much info about it. Something with NTFSdos errors came up. I recently learned GPT has, apart from its primary version on the first 34 sectors, also a secundary GPT, which is a copy of the primary, and starts at the very last sector going towards lower LBA numbering, so 34 sectors towards the beginning of the volume, contrary to the primary GPT that goes upwards toward higher LBA numbering. So one could consider the secundary GPT as a mirror of the primary one. Is a bootsector mirror something similar then? Some extra safety in case the first bootsector gets damaged? I thought that, for recovery purposes, there was already a copy of the bootsector in the second sector (sector 2, = LBA 1).

If you expand the partition "on the left" what you are actually doing is normally "moving" or "shiftng" the whole partition (and the $MFT and all data in the volume) "to the left" AND expand it "on the right". Cannot really say if some software will instead leave the $MFT in it's place and update it's offset in the bootsector(s) changing it from the the "standard", but I doubt it.

If I interpret this correctly, expanding a volume (partition) by moving the beginning towards the left and leaving the end where it was, will likely entail that the MFT and all userdata will be copied towards the left! (Damm!) That is not really what I wanted to hear, but that's how it is now. I believe it is very important to know this kind of information when one needs to take decisions about partition layout prior to formatting a drive.

About the MFT zone, if the non-reserved area is filled up with userdata, then next files are dropped in the MFT zone. Question is, how is the MFT zone being filled up then? Does the dropping of userdata starts somewhere at the very beginning of the volume, so at the end of the MFT zone, where the userdata then grows towards the MFT? Or, does NTFS starts writing the userdata at the beginning of the MFT zone, right behind the MFT itself, where the userdata then grows towards the beginning of the volume? The latter scenario would consequently lead to fragmentation (and performance degradation) of the MFT itself.

I guess some experiments are needed.

I couldn't agree more. I have to do some experiments on my drive. Question is, do you (or anybody else reading this post) know any good utilities or tools (according to your personal experience), which could return information (or could give me at least some idea) about the beginning and ending of the MFT, the MFT zone, the non-reserved area, the location of areas with high concentration of occupied clusters, etc. A tool like that might have a graphical 1-dimensional (as in WinXP defragmentor) or 2-dimensional (Old Macintosh software) representations of the location of the MFT, system and userdata files, or could also return a numerical representation (a bit in the style of PTedit32 and PartitionInfo 8 for instance) of the location of the MFT, system and userdata files.

As for graphical tools, on pages

http://technet.microsoft.com/en-us/library/cc767961.aspx

http://technet.microsoft.com/en-us/library/Cc767961.opt05_big(en-us,TechNet.10).gif

I've found a tool called "diskeeper IRISCRIMSON". That tool clearly shows the MFT. On the webpage of diskeeper,

http://www.diskeeper.com/business/diskeeper/professional/features/default.aspx ,

I noticed there is no mentioning anymore of any MFT representation.

I remember that many years ago (When hard drives where still small) I had some utility on my Macinstosh, which represented the (or groups of) clusters in a 2-D matrix on the screen. If I recall correctly, some group of clusters was represented by a little square in a particular color, depending if the clusers belonged to userdata or system data. Lower left corner of the 2-D matrix represented LBA 0 and the upper right corner represended the last LBA. It was great stuff those days, but I can't remember that I have ever seen anything similar for the Windows environment.

Just for the record, if you think how "queer" is number #786432, try writing it in Hex: 0xC0000

HEX C0000 is lovely number (I think). I don't know why they picked exactly *that* number, though. Conversion both ways between the numbers is straight forward.

Johan

Edited by DiracDeBroglie
Link to comment
Share on other sites

Do I really interpret all this correctly? I must've overlooked something. What did I miss?

No :ph34r: . Yes :yes: . The basic notion that the $MFT is actually a file.

It begins at it's begin ;) and ends some bytes "later" or "on the right".

A cluster size is "defined" as part of the filesystem "earlier" than the $MFT, or, if you prefer, you know how much a foot is, but you first need to establish a length (say 1 yard equals 3 feet) and only after you can say that something is 10 yards from you.

In the bootsector you first write how big in sectors a cluster is, and only later you can use that unit of measure to actually find the address where to start writing the $MFT.

The equation is:

y=ax

where:

  • $MFT Start Address (in sectors)=y
  • Cluster size (in sectors)=a (Constant)
  • $MFT Start Address (in clusters)=x

you cannot find y if you don't know (besides x) the value of the multiplier a (and you can only count sectors until you devine cluster size)

Maybe I interpreted all this not quite right, please correct me if I am wrong.

You are wrong :ph34r:, see above.

"bootsector mirror"? Did some Googling but couldn't really find that much info about it. Something with NTFSdos errors came up. I recently learned GPT has, apart from its primary version on the first 34 sectors, also a secundary GPT, which is a copy of the primary, and starts at the very last sector going towards lower LBA numbering, so 34 sectors towards the beginning of the volume, contrary to the primary GPT that goes upwards toward higher LBA numbering. So one could consider the secundary GPT as a mirror of the primary one. Is a bootsector mirror something similar then? Some extra safety in case the first bootsector gets damaged? I thought that, for recovery purposes, there was already a copy of the bootsector in the second sector (sector 2, = LBA 1).

You still have some confusion between MBR (which is ALWAYS on LBA sector 0) and bootsector (or PBR or VBR) whiich is first sector of the volume (see below for definitions).

On a disk there is only one MBR and as many bootsectors as there are volumes, WHICH of the n bootsectors would you think should be written to sector LBA 1? :unsure:

grub4dos does normally (and IF installed to the MBR+hidden sectors AND IF properly installed) store a copy of the MBR on LBA 1, though.

I know of no other software that does the same, let alone MS OS's.

NTFS has this feature, when you format a partition (or volume) with NTFS the volume size will be 1 sector less than the actual space in the MBR (or EPBR) partition table, this extra sector, inside the partition or volume but outside of the actual filesystem, holds a copy of the first sector of the NTFS formatted partition/volume, see:

http://thestarman.pcministry.com/asm/mbr/NTFSBR.htm

http://thestarman.pcministry.com/asm/mbr/NTFSBR.htm#BSback

(and you also have another example on how confusing the terms partition, volume and filesystem can be, I use them in a slightly different way from there)

To me:

  • filesystem=volume for FAT1x/32/64 (but we can say that filesystem is INSIDE the volume, , actually being EXACTLY the volume, or if you prefer the filesystem and volume start and end at the SAME address)
  • filesystem is INSIDE volume for NTFS, as filesystem and volume start at the SAME address BUT filesystem ends 1 sector before volume
  • if the partition is primary, partition=volume (but we can say that volume is INSIDE partition, actually being EXACTLY the partition, or if you prefer partition and volume start and end at the SAME address)
  • if the partition volume is a logical volume inside extended partition, volume=volume and it is of course inside the (extended) partition, but it starts at a given address that is NEVER the same address as the (extended) partition and ends at an address that may (or may not) be the same as the (extended) partition

If I interpret this correctly, expanding a volume (partition) by moving the beginning towards the left and leaving the end where it was, will likely entail that the MFT and all userdata will be copied towards the left! (Damm!) That is not really what I wanted to hear, but that's how it is now. I believe it is very important to know this kind of information when one needs to take decisions about partition layout prior to formatting a drive.

Yep. :)

About the MFT zone, if the non-reserved area is filled up with userdata, then next files are dropped in the MFT zone. Question is, how is the MFT zone being filled up then? Does the dropping of userdata starts somewhere at the very beginning of the volume, so at the end of the MFT zone, where the userdata then grows towards the MFT? Or, does NTFS starts writing the userdata at the beginning of the MFT zone, right behind the MFT itself, where the userdata then grows towards the beginning of the volume? The latter scenario would consequently lead to fragmentation (and performance degradation) of the MFT itself.

Never happened to actually fill a NTFS volume "up to the brim" (or at least I don't remember having studied the "filling area progression").

Logically the MFT area should be eroded "from the right", as to leave as much "reserved" area as possible at each file written, most probably this is done in "chunks", but really cannot say.

That's why I said:

I guess some experiments are needed.

I couldn't agree more. I have to do some experiments on my drive. Question is, do you (or anybody else reading this post) know any good utilities or tools (according to your personal experience), which could return information (or could give me at least some idea) about the beginning and ending of the MFT, the MFT zone, the non-reserved area, the location of areas with high concentration of occupied clusters, etc. A tool like that might have a graphical 1-dimensional (as in WinXP defragmentor) or 2-dimensional (Old Macintosh software) representations of the location of the MFT, system and userdata files, or could also return a numerical representation (a bit in the style of PTedit32 and PartitionInfo 8 for instance) of the location of the MFT, system and userdata files.

I'm more like a hex/command line only kind of guy, but you could have a look at ultradefrag:

http://ultradefrag.sourceforge.net/

jaclaz

Link to comment
Share on other sites

grub4dos does normally (and IF installed to the MBR+hidden sectors AND IF properly installed) store a copy of the MBR on LBA 1, though.

I know of no other software that does the same, let alone MS OS's.

Ranish Partition Manager lets you have a backup (and more) at a place of your choice. Drawback is that it's user interface is not intuitive at all, not to say user unfriendly. And as we saw, support for bigger disks is dodgy.

Link to comment
Share on other sites

grub4dos does normally (and IF installed to the MBR+hidden sectors AND IF properly installed) store a copy of the MBR on LBA 1, though.

I know of no other software that does the same, let alone MS OS's.

Ranish Partition Manager lets you have a backup (and more) at a place of your choice. Drawback is that it's user interface is not intuitive at all, not to say user unfriendly. And as we saw, support for bigger disks is dodgy.

Sure, and you can also use dd or any other direct disk access software, to make a copy of a sector and store it on another sector, including DEBUG on DOS/Win9x to make a copy of it.

I'll rephrase ;):

The particular code grub4dos grldr.mbr uses is made of 18 sectors:

  1. first one is the grldr MBR code
  2. second one is space reserved for a copy of the MBR as it was before the grldr.mbr install
  3. remaining sectors are the rest of grldr.mbr code
  4. grub4dos has a provision to load the copy of the MBR stored on second sector

I know of no other software that does the same, BTW grub4dos can make a copy of any sector and store it on any other sectr and also chainload any sector, but this is completely OT.

My understanding is that actuator type drives (non stepper) land on the inner part of the drive.

Drives that I have pulled apart have had their heads landed on the inner part of the platter.

And this article indicates the same:-

http://www.datarecoverysoftware1.com/data-recovery-glossary/landing-zone.html

Landing or parking zone is - as clearly stated on the given link - a "no-data" zone, it has nothing to do with where data is stored.

Additionally a number of disks have a "head parking zone" OUTSIDE the disk platter on a ramp, so called UNLOADING:

http://en.wikipedia.org/wiki/Hard_disk_drive#Landing_zones

jaclaz

Link to comment
Share on other sites

Hi Jaclaz,

The equation is:

y=ax

where:

•$MFT Start Address (in sectors)=y

•Cluster size (in sectors)=a (Constant)

•$MFT Start Address (in clusters)=x

Normally (as far as I understood), a cluster comprises a number of sectors: a= 1, 2, 4, ... 128 (=equivalent with 64 KiB cluster); here a is always >1.

Maybe we could try the following exercise. Take the 8GB volume from your previous post, it contains 15625000 sectors (512 byte sectors). Now I would like to know at what SECTOR the MFT starts on the volume for several cases.

Suppose now we format the volume with clusters of just one (1) sector (0.5 KiB); the clusters contain each 512 bytes, and the MFT would then start at SECTOR #786432. Q:Is that correct?

Take now clusters of 8 KiB on the 8GB volume. According to y=ax, a=16 and the start SECTOR of the MFT is now #12582912. Q:Is that correct?

Assume the cluster size is 64 KiB on the 8GB volume. This would result in a=128 and the MFT start SECTOR becomes #100663296. Q:Is that correct? Note: the latter would entail that the MFT zone would occupy 6.44 times the 8GB volume capacity.

Previous exercise is maybe not *that* important for solving my particular problem but it can, however, give me more insight in the internals of a filesystem and its bookkeeping, insight that could be of use to me at a later stage.

--------------------------------

Yes, I got it confused. MBR and VBR (first bootsector in primary partition, or in EBR chain) are not to be confused. Thanks for the links btw. Also good that you pointed out the subtle difference between filesystem and volume (I didn't know). However, in

http://thestarman.pcministry.com/asm/mbr/NTFSBR.htm#BSback

they seem to make a distiction, in particular, between volume and partition.

Btw, if I understand correctly, each primary partition has its bootsector mirrored at the end of the partition outside the volume or filesystem. As far as I understood, this is also the case with logical partitions (encapsulated in an extended partition) where each logical partition has its EBR mirrored in the back of its logical partition just outside its logical volume (or filesystem)---please correct me if I am wrong.

I haven't found any confirmation on the net, but I assume there is nothing about any (boot)sector being mirrored to the back of the Extended encapsulating partition or volume, which would sit one level higher above the logical partitions (volumes), is there??

Still have to examine the tool(s) in

http://ultradefrag.sourceforge.net/

Johan

Link to comment
Share on other sites

Maybe we could try the following exercise. Take the 8GB volume from your previous post, it contains 15625000 sectors (512 byte sectors). Now I would like to know at what SECTOR the MFT starts on the volume for several cases.

Suppose now we format the volume with clusters of just one (1) sector (0.5 KiB); the clusters contain each 512 bytes, and the MFT would then start at SECTOR #786432. Q:Is that correct?

Take now clusters of 8 KiB on the 8GB volume. According to y=ax, a=16 and the start SECTOR of the MFT is now #12582912. Q:Is that correct?

Maybe, or maybe NOT. :sneaky:

Just try doing some experiments, there is no way you can learn more than from direct experience.

You can use IMDISK:

http://www.ltr-data.se/opencode.html/

and since the experiments are about volumes and not "full disks" a straight sparse file generator, like mksparse:

http://reboot.pro/3191/page__st__42

via Wayback Machine:

http://wayback.archive.org/web/*/http://www.acc.umu.se/~bosse/mksparse.zip

will be more than enough:

mksparse 8gbvolume.ima 8gb

Mount it in IMDISK.

Format it with "default allocation size" as NTFS.

You will have 8Kb 4 Kb clusters and $MFT @cluster #786432

Re-format with 512 byte cluster and you will get $MFT @ cluster #6291456

Re-format with 1024 byte cluster and you will get $MFT @ cluster #3145728

Re-format with 2048 byte cluster and you will get $MFT @ cluster #1572864

Re-format with 4096 byte cluster and you will get $MFT @ cluster ...... <find it yourself, then try to make some sense of the results....

(you can use a hex editor with a bootsector viewer to make sure)

Tiny Hexer is recommended:

http://reboot.pro/8734/page__st__17

Assume the cluster size is 64 KiB on the 8GB volume. This would result in a=128 and the MFT start SECTOR becomes #100663296. Q:Is that correct? Note: the latter would entail that the MFT zone would occupy 6.44 times the 8GB volume capacity.

Previous exercise is maybe not *that* important for solving my particular problem but it can, however, give me more insight in the internals of a filesystem and its bookkeeping, insight that could be of use to me at a later stage.

Exactly, that's why it is important that you carry them experiments.

--------------------------------

Yes, I got it confused. MBR and VBR (first bootsector in primary partition, or in EBR chain) are not to be confused. Thanks for the links btw. Also good that you pointed out the subtle difference between filesystem and volume (I didn't know). However, in

http://thestarman.pcministry.com/asm/mbr/NTFSBR.htm#BSback

they seem to make a distiction, in particular, between volume and partition.

Yes, there is a lot of confusion, and you still have missed the point I was trying to make.

Btw, if I understand correctly, each primary partition has its bootsector mirrored at the end of the partition outside the volume or filesystem. As far as I understood, this is also the case with logical partitions (encapsulated in an extended partition) where each logical partition has its EBR mirrored in the back of its logical partition just outside its logical volume (or filesystem)---please correct me if I am wrong.

I haven't found any confirmation on the net, but I assume there is nothing about any (boot)sector being mirrored to the back of the Extended encapsulating partition or volume, which would sit one level higher above the logical partitions (volumes), is there??

The issue is the following, you have now a correct "mental map" where an Extended Partition is a container for (one or more) volume(s).

Let's take the simplest possible Extended Partition, one that has inside a single big volume, and let's us assume that it is formatted with a FAT filesystem.

You will have this container with:

  1. beginning of Extended Partition
  2. the EPBR and a few hidden sectors (the EPBR contains the start address of the volume and the end of Extended Partition)
  3. beginning of volume (first sector of the volume is the bootsector or VBR, it contains the start address of the volume and the end of the volume)
  4. end of volume
  5. end of Extended Partition

(end of volume is the SAME as end of Extended partition)

Now, compare with a whole disk (again the simplest possible setup, one single primary partition occupying all the space), and consider it as a container also:

  1. the MBR and a few hidden sectors (the MBR contains the start and end addresses of the Primary partition)
  2. beginning of Primary Partition (first sector of the Partition is the bootsector or VBR or PBR, it contains the start address of the volume and the end of the volume)
  3. beginning of volume (first sector of the volume is the bootsector or VBR or PBR, it contains the start address of the volume and the end of the volume)
  4. end of volume
  5. end of Extended Partition

(BOTH beginning and end of volume are the SAME as beginning and end of Extended partition)

And in both cases above the volume contains a filesystem (and the filesystem beginning and end are the SAME as those of the "container" volume).

We can say that in this case the container (partition) is exactly the same of the contents (volume) and that the container (volume) is exactly the same of the contents (filesystem).

Now, when you format with NTFS you are in this situation (the way the Starman :thumbup represents it):

Extended:

  1. beginning of Extended Partition
  2. the EPBR and a few hidden sectors (the EPBR contains the start address of the volume and the end of Extended Partition)
  3. beginning of volume (first sector of the volume is the bootsector or VBR, it contains the start address of the volume and the end of the volume)
  4. end of volume
  5. one sector holding a copy of the VBR
  6. end of Extended Partition

Disk/Primary:

  1. the MBR and a few hidden sectors (the MBR contains the start and end addresses of the Primary partition)
  2. beginning of Primary Partition (first sector of the Partition is the bootsector or VBR or PBR, it contains the start address of the volume and the end of the volume)
  3. beginning of volume (first sector of the volume is the bootsector or VBR or PBR, it contains the start address of the volume and the end of the volume)
  4. end of volume
  5. one sector holding a copy of the VBR
  6. end of Extended Partition

And the way I like to represent it:

Extended:

  1. beginning of Extended Partition
  2. the EPBR and a few hidden sectors (the EPBR contains the start address of the volume and the end of Extended Partition)
  3. beginning of volume (first sector of the volume is the bootsector or VBR, it contains the start address of the volume and the end of the volume, but, IF the filesystem is NTFS the value is end of volume minus one sector)
  4. one sector holding a copy of the VBR
  5. end of volume
  6. end of Extended Partition

Disk/Primary

  1. the MBR and a few hidden sectors (the MBR contains the start and end addresses of the Primary partition)
  2. beginning of Primary Partition (first sector of the Partition is the bootsector or VBR or PBR, it contains the start address of the volume and the end of the volume)
  3. beginning of volume (first sector of the volume is the bootsector or VBR or PBR, it contains the start address of the volume and the end of the volume, but, IF the filesystem is NTFS the value is end of volume minus one sector)
  4. one sector holding a copy of the VBR
  5. end of volume
  6. end of Extended Partition

As you can see both representation may be valid, the issue is only when you go and backup such a thing, in my view, when I say that I backup a volume I actually backup from the beginning VBR to the End of the allocated space for the volume, including, in the case of NTFS the "extra sector".

If you prefer, in my mental map the volume extends over the addresses stored in it's VBR, but in the case of a NTFS formatted filesystem, I know that the end address must be increased by one, or that the addresses in the VBR represent the filesystem and not the volume. And that while with FAT filesystem and volume begin and end at the same address, with NTFS the filesystem is one sector less than the volume.

So i say that the volume contains the filesystem and in the case of NTFS it contains the filesystem + a backup bootsector.

Just for the record, early versions of NTFS had the backup bootsector in the MIDDLE of the volume :w00t:

http://support.microsoft.com/kb/153973/en-us

which, while making slightly more difficult to find the backup bootsector and making it prone to be deleted by some "deviating" software, did not create the above confusion between the addresses stored in the VBR and actual extents on disk.

jaclaz

Edited by jaclaz
Link to comment
Share on other sites

  • 1 month later...

Hi,

Did some tests (among so many other tests) under Windows 7 on several volumes sizes (250MB to 100GB) with cluster sizes going from 4KiB to 64KiB. Whatever the cluster size, the $MFT offset varies around 30% of the volume size with a maximum of 3 to 3.5GB. So a 1GB volume has an MFT offset of approximately 300MB; a 10GB has an MFT offset of 3GB or so; a 100GB has an offset of also 3 to 3.5GB. Those offsets did not change whatever the cluster value.

As for the MFT zone reservation, I performed tests on the same volumes with the registry key NtfsMftZoneReservation set to 4. For the volume of 250MB the MFT zone was 30 to 40MB; for the 5GB volume the MFT zone was 4.6% of the volume size, or 220MB; for the 10GB volume it was 2.5% or 250MB; for the 100GB volume the MFT zone was 0.22% of the volume size, or 220MB. The MFT zone reservation was insensitive to cluster size. Did the test with 4KiB and 64KiB cluster size and there was practically no difference. I've also done tests with NtfsMftZoneReservation set to 1, but the MFT zone was not that much smaller than in the case NtfsMftZoneReservation was set to 4. Had to reboot the computer each time I changed NtfsMftZoneReservation. The figures for MFT offset and MFT zone are an estimate from what I saw on the screen using UltraDefrag 5.0.2.

So, what I've found under Win7 is not compliant with http://support.microsoft.com/kb/174619 which mentions that for NtfsMftZoneReservation=1 12.5 percent of the volume should be reserved. I did the same tests on my 2TB drive connected to my WinXP laptop, and there the MFT zone reservation (1->4) worked as predicted in http://support.microsoft.com/kb/174619 ; for NtfsMftZoneReservation=4 halve of my volume was MFT zone. It is a bit strange that exactly the same 2TB drive with exactly the same NtfsMftZoneReservation values, results in such a huge difference in MFT zone size between WinXP and Win7.

(see http://sourceforge.net/projects/ultradefrag/files/stable-release/5.0.2/

and

http://ultradefrag.sourceforge.net/en/index.html )

There was also a peculiar thing, though. I filled up the 10GB volume (cluster size = 4KiB). As the volume got filled up, the MFT zone started to shrink, still leaving the MFT itself and the remaining part of the MFT zone contiguous; that was good. However, that was not the case for cluster size 64KiB. There the *beginnig* of the MFT zone started to fill up going towards the end, thereby chopping the MFT from its MFT zone; that resulted in a strongly fragmened MFT. After the volume was completely filled up, there were MFT droplets scattered all over the initial MFT zone. That was under Win7, I didn't do the same test under WinXP.

Noticed some very strange behavior when switching NtfsMftZoneReservation back and forth between 1 and 4, but I will open a new thread for that.

Did some tests with paritition software in an attempt to extend a volume at its beginning (in stead of extending its end), and that indeed moved (copied) the entire content of the volume at the very (newly created) beginning of the volume. I tested the free edition MiniTool Partition Wizard Home Edition 7.1 (PWWin7), and MiniTool Partition Wizard Bootable CD 7.1 (PWBootCD). The results under Win7 were very catastrophic: see

http://www.sevenforums.com/software/212288-partition-wizard-win7-non-compliant-advance-format-drive-mis-aligned.html

Johan

Edited by DiracDeBroglie
Link to comment
Share on other sites

Interesting report! :thumbup:

JFYI ;):

http://support.microsoft.com/?id=961095

Windows Vista and Windows Server 2008 use a default size of 200MB for the initial MFT zone reservation. As the MFT outgrows the default zone due to more files being added to the volume, the MFT will create another 200MB zone to grow into. This change to fixed amount versus a percentage was done to deal with increasing size of volumes and create better efficiencies.

For example, in previous versions of Windows a 500GB NTFS volume would have a ~62GB MFT Zone Reservation (500MB x 12.5%). So if the MFT only grew to the size of 100MB, the first 62GB of the volume would be “reserved” and data would not start until the end of the 62GB mark.

Starting with Windows Vista and Windows Server 2008, the MFT uses a fixed 200MB zone reservation so as not to reserve space that will never actually be used (note that the zone reservation can be used by data once the remainder of the volume is filled up - this is the same as in previous versions of Windows).

It is possible to adjust the size the MFT zone reservation uses. If not changed, the default is set to 1. The valid range is 1-4 where this is a multiplier of 200MB. For example, if the data is set to 4, any newly create volumes initial MFT Zone Reservation will be 800MB and if it grows beyond this will grow in 800MB segments. If this setting is created after a volume is created, the initial MFT zone reservation will be 200MB but future MFT zone reservations will be based on the multiplier.

Translated from MSish :w00t: to layman's terms:

12.5%, 25%, etc. make NO sense whatsoever as they are a-dimensional and the more the hard disk drive sizes will grow up in size the more this will have less sense, we were scolded by this combination of inevitable growth of disks several times in the past (CHS, Int13, etc.) and apparently we have now (at the nth practical demonstration of the foolishness of assuming that this won't happen) learned our lesson. :unsure:

Paradoxically hard disk grows because we put every day more bloat in our OS and apps and hard disk manufacturers are forced to invent new ways to fit more data on a disk. :whistle:

:lol:

Seriously, now, my experience with the location of $MFT is different, when a given size is reached the offset becomes "fixed" to cluster #786432 or 0xC0000.

jaclaz

Edited by jaclaz
Link to comment
Share on other sites

I've just done the test again with a 100GB and a 1.7TB volume, with two different cluster sizes: 4KiB and 64KiB. The offset of the MFT and the size of the MFT zone are insensitive to cluster size; in UltraDefrag the MFT remains at a fixed position.

If the start location would be fixed at cluster #786432, then with increasing clustersize, the start location would shift closer to the "end" of the volume, but that is clearly not so in my case. I did the test in Win7.

Do you have the time and infrastructure to try to reproduce my findings, jaclaz?

johan

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...