[joakim]

I've been annoyed by the +3Mb size of the supplied boot.sdi that is shipped with nt6.x. It is really a size overkill! My tweaked boot.sdi is now at 960 kb, or 983 040 bytes to be exact. The boot.sdi is basically an sdi with just a tiny ntfs partition image injected and a marker for the "mysterious" wim blob at the very end. Download at; http://www.mediafire...lb/boot_sdi.zip

If anyone can reduce the size further, I would be happy to inspect the file.

Why on earth did MS make it so big? Only answer can be that they are lazy and don't care about a few MB of wasted space

Joakim

Updated version 300 kB; http://mft2csv.googl...oot_sdi_300.zip

Details of sectors inside the latest partition image;

$Boot 0 
$MFT 1-65 
Root directory 66-73 
$LogFile 74-586 
$MFTMirr 136-143 
$UpCase 136 
$Secure:$SDS 137-392 
$Bitmap 138 
$AttrDef 139-143 

The rest of the systemfiles are fully contained within $MFT

The reorganization of the ntfs metafiles was done to make it easier for those that want to decode and understand it.

This post has been edited by joakim: 27 August 2012 - 03:05 PM

[jaclaz]

NTFS is a rather "bulky" filesystem for very little sized volumes.

Strip from the file first 0x2000 or 8192 bytes and save as bottsdi.raw.
Open this file with 7-zip and you will see the NTFS filesystem structures that occupy a lot of space.

The problem was talked about many years ago when Mark Russinovich made possible to have NTFS formatted floppies:
http://freewareapp.c...fsflp_download/
http://web.archive.o...com/ntfsflp.htm
http://web.archive.o...com/ntfsflp.zip

It seems like the Log file can be reduced from the current 524,288 bytes to 262,144, that at the time was found to be the bare minimum.

Compare the NTFSIMG in the download with the bootsdi.raw file in 7-zip .

jaclaz

[joakim]

Cool, I did not know. Will investigate it soon and see what we get.

[joakim]

Here is a new one at 696 kB; http://www.mediafire...oot_sdi_696.zip

It is a modified version of Mark's ntfs partition. The partition image is only 708 608 bytes.

I wonder why Microsoft made their boot.sdi almost 4,5 times the size of what is necessary? It is really just a waste of space..

Joakim

This post has been edited by joakim: 18 July 2010 - 12:20 PM

[joakim]

Last shot at 692 kB; http://www.mediafire...oot_sdi_692.zip

By shrinking the metafile $Boot from 8 192 bytes to 512 bytes, the total partition size is now 684 kB or 700 928 bytes. The partition has no free space and only contains ntfs systemfiles. It will only work for booting nt6.x based winpe (does not write anything to the partition, just mounting the wim). I doubt it is possible to reduce the size of it any further. Partition image is also included in download.

Joakim

[joakim]

Size of boot.sdi is now 319 488 bytes: http://www.mediafire...oot_sdi_312.zip

The interesting stuff is that some metafiles only need to have some of the first bytes present. And some of them are not read at all. But since their reference can't be removed from $MFT, their occupied sectors can be filled with 0's. Namely $MFTMirr and $Bitmap are not read at all. $MFT and $AttrDef are not modded at all. For $UpCase only the first sector needs to be present. As mentioned before, the $Boot can be reduced to 1 sector (IBL not needed). The $LogFile only requires the first 8 sectors to be present (rest can be 0's). However the size of the LogFile can't be reduced from its current size of 262 656 bytes. Now the most interesting bit is that some metafiles can be cross referenced, meaning they can occupy the same sectors. This will be true when certain sectors are not read, or their content are the same (like with the 0's). In this boot.sdi, as much as 3 metafiles are located on some common sectors. The $Secure metafile effectively only takes 1 sector (rest can be 0's). It is entirely located inside $LogFile, and because $LogFile can't be shrinked, I did not bother much about its size.

If the $LogFile can be reduced in size, it is likely that the total partition image size can be reduced to the crazy size of 73 216 bytes!!

This must only be considered as research and a PoC. It works, but has not been extensively tested.

Joakim

[jaclaz]

VERY GOOD WORK!

About this:

Quote

I wonder why Microsoft made their boot.sdi almost 4,5 times the size of what is necessary?

There is an easy answer:

Quote

This is by design.

http://www.boot-land...?showtopic=3541


A seemingly OT (but not much) idea:
Is there any real *need* for the filesystem inside the boot.sdi to be NTFS?
Or FAT 12 would do?


jaclaz

This post has been edited by jaclaz: 27 July 2010 - 02:46 AM

[joakim]

jaclaz, on 27 July 2010 - 02:40 AM, said:

Is there any real *need* for the filesystem inside the boot.sdi to be NTFS?
Or FAT 12 would do?

According to their own doc, it must be ntfs for rw's; http://technet.micro.../cc722145(WS.10).aspx

Quote

You can mount a .wim file with read/write permissions only on an NTFS file system.

Joakim

[jaclaz]

Yep, I completely missed that it was used as a mounting point.

But in the original use the .wim is not Written, is it?
Still I don't think you can mount anyhting on FAT at all.

I guess that this is - as often happens - a bit misleading:
http://technet.micro...145(WS.10).aspx

Quote

You can mount a .wim file with read/write permissions only on an NTFS file system. This avoids the 2 gigabyte (GB) barrier that is imposed by FAT file systems and prevents data loss that is possible with FAT or other non-NTFS file systems.

jaclaz

This post has been edited by Tripredacus: 18 January 2011 - 09:08 AM
Reason for edit: fixed link

[joakim]

jaclaz, on 27 July 2010 - 04:36 AM, said:

But in the original use the .wim is not Written, is it?
Still I don't think you can mount anyhting on FAT at all.

That's right. It will bsod 0x..ED with a fat partition.

Joakim

[joakim]

Size now at 307 200 bytes; http://www.mediafire...oot_sdi_300.zip

The partition image is now basically constructed like this;

Sector 0:         $Boot
Sector 1 - 64:    $MFT
Sector 65:        $MFT:$Bitmap
Sector 66 - 73:   Root directory
Sector 74 - 586:  $LogFile

The rest of the systemfiles are found "inside" the $LogFile starting at sector 136.

It is now much easier to read it too.

Also tried putting the $LogFile in sector 1 and placing all other systemfiles inside it, but that did not work.

Btw, it is a really good way to learn about ntfs when working with such a small partition image.

Joakim

This post has been edited by joakim: 28 July 2010 - 01:53 AM

[jaclaz]

Good.

Now it comes to mind a question:
if a single guy in a few days of his spare time can take a "random" item and reduce it's size tenfold what could do the full-time MS guys if they wanted to?



As a dinosaur, in my simplicity I continue thinking that smaller things are faster, no matter how faster is your hardware, managing less bytes it will make it faster!

jaclaz

This post has been edited by jaclaz: 28 July 2010 - 01:19 AM

[Kelsenellenelvian]

Quote

As a dinosaur, in my simplicity I continue thinking that smaller things are faster, no matter how faster is your hardware, managing less bytes it will make it faster!

AMEN!

[wimb]

jaclaz, on 28 July 2010 - 12:57 AM, said:

Now it comes to mind a question:
if a single guy in a few days of his spare time can take a "random" item and reduce it's size tenfold what could do the full-time MS guys if they wanted to?

Reduction of Windows 7 size by factor 40 would be possible and interesting .....
Functional Size can be around 200 MB .....

[Tripredacus]

Kelsenellenelvian, on 28 July 2010 - 01:00 AM, said:

Quote

As a dinosaur, in my simplicity I continue thinking that smaller things are faster, no matter how faster is your hardware, managing less bytes it will make it faster!

AMEN!

This exact idea is also present in the web design community. Us "old schoolers" were taught that you needed to make your code as small as possible so that the page would load fast because most people had dial-up. Now that most people have broadband, no one cares that their code is all bloated because no one is going to notice.

[joakim]

Microsoft still uses the same file they created for Longhorn on 15.02.2005, over 5 years ago (assuming their timestamps are more trustworthy than mine).

Three severely retarded things are worth noting about the original boot.sdi;

1. The $LogFile is 2 Mb
2. Free space is set to 0,6 Mb
3. Page alignment is set to 8192 bytes (instead of 4096) in the sdi header

Strange that they still keep this one.

Joakim

[jaclaz]

joakim, on 28 July 2010 - 12:38 PM, said:

Strange that they still keep this one.

Well, if you compare with the Windows ME floppy disk "embedded" in XP, you wiil see how they used a floppy they had around, deleted a few files (most of them recoverable/undeletable) and simply "dded" them inside an .exe.

So, they are not new to this kind of behaviour.

JFYI:
http://www.911cd.net...showtopic=16745

jaclaz

[Bilou_Gateux]

joakim, on 27 July 2010 - 02:41 PM, said:

Size now at 307 200 bytes; http://www.mediafire...oot_sdi_300.zip

The partition image is now basically constructed like this;

Sector 0:         $Boot
Sector 1 - 64:    $MFT
Sector 65:        $MFT:$Bitmap
Sector 66 - 73:   Root directory
Sector 74 - 586:  $LogFile

The rest of the systemfiles are found "inside" the $LogFile starting at sector 136.

It is now much easier to read it too.

Also tried putting the $LogFile in sector 1 and placing all other systemfiles inside it, but that did not work.

Btw, it is a really good way to learn about ntfs when working with such a small partition image.

Joakim

How do i mount this file as virtual disk?

With the initial release, i can mount boot.sdi with imdisk:

imdisk -a -t vm -f boot.sdi -b 8192 -m C:\Dev\boot

Now your archive contains both 300.part file and boot.sdi
I tried mounting boot.sdi but can't find NTFS bootrecord which starts with EB5290

This post has been edited by Bilou_Gateux: 23 August 2010 - 07:24 AM

[joakim]

That is expected behaviour since the ntfs filesystem is invalidated. No existing tool will be able to mount that partition image normally. You must use a hex editor and/or a similar low level reader (like ntfs diskexplorer). Btw, the starting offset is 4096 and not 8192 (determined by the page size of the sdi). The bootsector is completely removed.

The first version that you mention is mountable, has a good and valid filesystem, and therefore is possible to mount using "normal" tools.

Joakim

This post has been edited by joakim: 23 August 2010 - 09:50 AM

[Bilou_Gateux]

Thanks for the clarification.

Is there any practical use for smallest release?