Jump to content

Welcome to MSFN Forum
Register now to gain access to all of our features. Once registered and logged in, you will be able to create topics, post replies to existing threads, give reputation to your fellow members, get your own private messenger, post status updates, manage your profile and so much more. This message will be removed once you have signed in.
Login to Account Create an Account



Photo

The smallest possible size of boot.sdi

- - - - -

  • Please log in to reply
48 replies to this topic

#1
joakim

joakim

    Member

  • Member
  • PipPip
  • 154 posts
  • Joined 18-November 09
  • OS:none specified
  • Country: Country Flag
I've been annoyed by the +3Mb size of the supplied boot.sdi that is shipped with nt6.x. It is really a size overkill! My tweaked boot.sdi is now at 960 kb, or 983 040 bytes to be exact. The boot.sdi is basically an sdi with just a tiny ntfs partition image injected and a marker for the "mysterious" wim blob at the very end. Download at; http://www.mediafire...lb/boot_sdi.zip

If anyone can reduce the size further, I would be happy to inspect the file.

Why on earth did MS make it so big? Only answer can be that they are lazy and don't care about a few MB of wasted space

Joakim

Updated version 300 kB; http://mft2csv.googl...oot_sdi_300.zip

Details of sectors inside the latest partition image;
$Boot 0 
$MFT 1-65 
Root directory 66-73 
$LogFile 74-586 
$MFTMirr 136-143 
$UpCase 136 
$Secure:$SDS 137-392 
$Bitmap 138 
$AttrDef 139-143 

The rest of the systemfiles are fully contained within $MFT

The reorganization of the ntfs metafiles was done to make it easier for those that want to decode and understand it.

Edited by joakim, 27 August 2012 - 03:05 PM.



How to remove advertisement from MSFN

#2
jaclaz

jaclaz

    The Finder

  • Developer
  • 14,567 posts
  • Joined 23-July 04
  • OS:none specified
  • Country: Country Flag
NTFS is a rather "bulky" filesystem for very little sized volumes.

Strip from the file first 0x2000 or 8192 bytes and save as bottsdi.raw.
Open this file with 7-zip and you will see the NTFS filesystem structures that occupy a lot of space.

The problem was talked about many years ago when Mark Russinovich made possible to have NTFS formatted floppies:
http://freewareapp.c...fsflp_download/
http://web.archive.o...com/ntfsflp.htm
http://web.archive.o...com/ntfsflp.zip

It seems like the Log file can be reduced from the current 524,288 bytes to 262,144, that at the time was found to be the bare minimum.

Compare the NTFSIMG in the download with the bootsdi.raw file in 7-zip ;).

jaclaz

#3
joakim

joakim

    Member

  • Member
  • PipPip
  • 154 posts
  • Joined 18-November 09
  • OS:none specified
  • Country: Country Flag
Cool, I did not know. Will investigate it soon and see what we get.

#4
joakim

joakim

    Member

  • Member
  • PipPip
  • 154 posts
  • Joined 18-November 09
  • OS:none specified
  • Country: Country Flag
Here is a new one at 696 kB; http://www.mediafire...oot_sdi_696.zip

It is a modified version of Mark's ntfs partition. The partition image is only 708 608 bytes.

I wonder why Microsoft made their boot.sdi almost 4,5 times the size of what is necessary? :ph34r: It is really just a waste of space..

Joakim

Edited by joakim, 18 July 2010 - 12:20 PM.


#5
joakim

joakim

    Member

  • Member
  • PipPip
  • 154 posts
  • Joined 18-November 09
  • OS:none specified
  • Country: Country Flag
Last shot at 692 kB; http://www.mediafire...oot_sdi_692.zip

By shrinking the metafile $Boot from 8 192 bytes to 512 bytes, the total partition size is now 684 kB or 700 928 bytes. The partition has no free space and only contains ntfs systemfiles. It will only work for booting nt6.x based winpe (does not write anything to the partition, just mounting the wim). I doubt it is possible to reduce the size of it any further. Partition image is also included in download.

Joakim

#6
joakim

joakim

    Member

  • Member
  • PipPip
  • 154 posts
  • Joined 18-November 09
  • OS:none specified
  • Country: Country Flag
Size of boot.sdi is now 319 488 bytes: http://www.mediafire...oot_sdi_312.zip

The interesting stuff is that some metafiles only need to have some of the first bytes present. And some of them are not read at all. But since their reference can't be removed from $MFT, their occupied sectors can be filled with 0's. Namely $MFTMirr and $Bitmap are not read at all. $MFT and $AttrDef are not modded at all. For $UpCase only the first sector needs to be present. As mentioned before, the $Boot can be reduced to 1 sector (IBL not needed). The $LogFile only requires the first 8 sectors to be present (rest can be 0's). However the size of the LogFile can't be reduced from its current size of 262 656 bytes. Now the most interesting bit is that some metafiles can be cross referenced, meaning they can occupy the same sectors. This will be true when certain sectors are not read, or their content are the same (like with the 0's). In this boot.sdi, as much as 3 metafiles are located on some common sectors. The $Secure metafile effectively only takes 1 sector (rest can be 0's). It is entirely located inside $LogFile, and because $LogFile can't be shrinked, I did not bother much about its size.

If the $LogFile can be reduced in size, it is likely that the total partition image size can be reduced to the crazy size of 73 216 bytes!!

This must only be considered as research and a PoC. It works, but has not been extensively tested.

Joakim

#7
jaclaz

jaclaz

    The Finder

  • Developer
  • 14,567 posts
  • Joined 23-July 04
  • OS:none specified
  • Country: Country Flag
VERY GOOD WORK! :thumbup

About this:

I wonder why Microsoft made their boot.sdi almost 4,5 times the size of what is necessary?

There is an easy answer:

This is by design.

;)

http://www.boot-land...?showtopic=3541
:lol:

A seemingly OT (but not much) idea:
Is there any real *need* for the filesystem inside the boot.sdi to be NTFS?
Or FAT 12 would do? :unsure:


jaclaz

Edited by jaclaz, 27 July 2010 - 02:46 AM.


#8
joakim

joakim

    Member

  • Member
  • PipPip
  • 154 posts
  • Joined 18-November 09
  • OS:none specified
  • Country: Country Flag

Is there any real *need* for the filesystem inside the boot.sdi to be NTFS?
Or FAT 12 would do? :unsure:


According to their own doc, it must be ntfs for rw's; http://technet.micro.../cc722145(WS.10).aspx

You can mount a .wim file with read/write permissions only on an NTFS file system.


Joakim

#9
jaclaz

jaclaz

    The Finder

  • Developer
  • 14,567 posts
  • Joined 23-July 04
  • OS:none specified
  • Country: Country Flag
Yep, I completely missed that it was used as a mounting point. :blushing:

But in the original use the .wim is not Written, is it?
Still I don't think you can mount anyhting on FAT at all. :unsure:

I guess that this is - as often happens - a bit misleading:
http://technet.micro...145(WS.10).aspx

You can mount a .wim file with read/write permissions only on an NTFS file system. This avoids the 2 gigabyte (GB) barrier that is imposed by FAT file systems and prevents data loss that is possible with FAT or other non-NTFS file systems.



jaclaz

Edited by Tripredacus, 18 January 2011 - 09:08 AM.
fixed link


#10
joakim

joakim

    Member

  • Member
  • PipPip
  • 154 posts
  • Joined 18-November 09
  • OS:none specified
  • Country: Country Flag

But in the original use the .wim is not Written, is it?
Still I don't think you can mount anyhting on FAT at all. :unsure:

That's right. It will bsod 0x..ED with a fat partition.

Joakim

#11
joakim

joakim

    Member

  • Member
  • PipPip
  • 154 posts
  • Joined 18-November 09
  • OS:none specified
  • Country: Country Flag
Size now at 307 200 bytes; http://www.mediafire...oot_sdi_300.zip

The partition image is now basically constructed like this;
Sector 0:         $Boot
Sector 1 - 64:    $MFT
Sector 65:        $MFT:$Bitmap
Sector 66 - 73:   Root directory
Sector 74 - 586:  $LogFile

The rest of the systemfiles are found "inside" the $LogFile starting at sector 136.

It is now much easier to read it too.

Also tried putting the $LogFile in sector 1 and placing all other systemfiles inside it, but that did not work.

Btw, it is a really good way to learn about ntfs when working with such a small partition image.

Joakim

Edited by joakim, 28 July 2010 - 01:53 AM.


#12
jaclaz

jaclaz

    The Finder

  • Developer
  • 14,567 posts
  • Joined 23-July 04
  • OS:none specified
  • Country: Country Flag
Good. :)

Now it comes to mind a question:
if a single guy in a few days of his spare time can take a "random" item and reduce it's size tenfold what could do the full-time MS guys if they wanted to? :unsure:

;)

As a dinosaur, in my simplicity I continue thinking that smaller things are faster, no matter how faster is your hardware, managing less bytes it will make it faster! :thumbup

jaclaz

Edited by jaclaz, 28 July 2010 - 01:19 AM.


#13
Kelsenellenelvian

Kelsenellenelvian

    WPI Guru

  • Developer
  • 8,822 posts
  • Joined 18-September 03
  • OS:Windows 7 x64
  • Country: Country Flag

As a dinosaur, in my simplicity I continue thinking that smaller things are faster, no matter how faster is your hardware, managing less bytes it will make it faster!


AMEN!

#14
wimb

wimb

    Senior Member

  • Developer
  • 679 posts
  • Joined 21-March 07

Now it comes to mind a question:
if a single guy in a few days of his spare time can take a "random" item and reduce it's size tenfold what could do the full-time MS guys if they wanted to? :unsure:

Reduction of Windows 7 size by factor 40 would be possible and interesting .....
Functional Size can be around 200 MB .....

#15
Tripredacus

Tripredacus

    K-Mart-ian Legend

  • Super Moderator
  • 9,902 posts
  • Joined 28-April 06
  • OS:Server 2012
  • Country: Country Flag

Donator

As a dinosaur, in my simplicity I continue thinking that smaller things are faster, no matter how faster is your hardware, managing less bytes it will make it faster!


AMEN!


This exact idea is also present in the web design community. Us "old schoolers" were taught that you needed to make your code as small as possible so that the page would load fast because most people had dial-up. Now that most people have broadband, no one cares that their code is all bloated because no one is going to notice. :realmad: :angel
MSFN RULES | GimageX HTA for PE 3-5 | lol probloms
msfn2_zpsc37c7153.jpg

#16
joakim

joakim

    Member

  • Member
  • PipPip
  • 154 posts
  • Joined 18-November 09
  • OS:none specified
  • Country: Country Flag
Microsoft still uses the same file they created for Longhorn on 15.02.2005, over 5 years ago (assuming their timestamps are more trustworthy than mine).

Three severely retarded things are worth noting about the original boot.sdi;

1. The $LogFile is 2 Mb
2. Free space is set to 0,6 Mb
3. Page alignment is set to 8192 bytes (instead of 4096) in the sdi header

Strange that they still keep this one.

Joakim

#17
jaclaz

jaclaz

    The Finder

  • Developer
  • 14,567 posts
  • Joined 23-July 04
  • OS:none specified
  • Country: Country Flag

Strange that they still keep this one.

Well, if you compare with the Windows ME floppy disk "embedded" in XP, you wiil see how they used a floppy they had around, deleted a few files (most of them recoverable/undeletable) and simply "dded" them inside an .exe.

So, they are not new to this kind of behaviour.

JFYI:
http://www.911cd.net...showtopic=16745

jaclaz

#18
Bilou_Gateux

Bilou_Gateux

    Powered by Windows Embedded

  • Member
  • PipPipPipPipPip
  • 768 posts
  • Joined 03-January 04

Size now at 307 200 bytes; http://www.mediafire...oot_sdi_300.zip

The partition image is now basically constructed like this;

Sector 0:         $Boot
Sector 1 - 64:    $MFT
Sector 65:        $MFT:$Bitmap
Sector 66 - 73:   Root directory
Sector 74 - 586:  $LogFile

The rest of the systemfiles are found "inside" the $LogFile starting at sector 136.

It is now much easier to read it too.

Also tried putting the $LogFile in sector 1 and placing all other systemfiles inside it, but that did not work.

Btw, it is a really good way to learn about ntfs when working with such a small partition image.

Joakim


How do i mount this file as virtual disk?

With the initial release, i can mount boot.sdi with imdisk:
imdisk -a -t vm -f boot.sdi -b 8192 -m C:\Dev\boot

Now your archive contains both 300.part file and boot.sdi
I tried mounting boot.sdi but can't find NTFS bootrecord which starts with EB5290

Edited by Bilou_Gateux, 23 August 2010 - 07:24 AM.

OS Version = 5.1.2600 Service Pack 3
Platform ID = 2 (NT)
Service Pack = 3.0
Suite = 0x0140
Product Type = 1
Architecture = x86

#19
joakim

joakim

    Member

  • Member
  • PipPip
  • 154 posts
  • Joined 18-November 09
  • OS:none specified
  • Country: Country Flag
That is expected behaviour since the ntfs filesystem is invalidated. No existing tool will be able to mount that partition image normally. You must use a hex editor and/or a similar low level reader (like ntfs diskexplorer). Btw, the starting offset is 4096 and not 8192 (determined by the page size of the sdi). The bootsector is completely removed.

The first version that you mention is mountable, has a good and valid filesystem, and therefore is possible to mount using "normal" tools.

Joakim

Edited by joakim, 23 August 2010 - 09:50 AM.


#20
Bilou_Gateux

Bilou_Gateux

    Powered by Windows Embedded

  • Member
  • PipPipPipPipPip
  • 768 posts
  • Joined 03-January 04
Thanks for the clarification.

Is there any practical use for smallest release?
OS Version = 5.1.2600 Service Pack 3
Platform ID = 2 (NT)
Service Pack = 3.0
Suite = 0x0140
Product Type = 1
Architecture = x86

#21
joakim

joakim

    Member

  • Member
  • PipPip
  • 154 posts
  • Joined 18-November 09
  • OS:none specified
  • Country: Country Flag
No, not except preventing an insignificant waste of resources. That's why I said only consider it as a PoC.

Joakim

#22
Metzen

Metzen

    Member

  • Member
  • PipPip
  • 135 posts
  • Joined 19-March 04

No, not except preventing an insignificant waste of resources. That's why I said only consider it as a PoC.

Joakim


Not true.

A smaller file size can help speed up PXE booting over a slow link. Sometimes I have to work with a link that takes ~30mins to boot WinPE over PXE. Any reduction is a huge boon!

In addition, a smaller boot.sdi will kick off downloading the WIM faster as well.

#23
joakim

joakim

    Member

  • Member
  • PipPip
  • 154 posts
  • Joined 18-November 09
  • OS:none specified
  • Country: Country Flag
Yes I am aware of the pxe fact, but such slow links are getting rather unusual nowadays I think.

If this boot.sdi will help then very good to hear. But keep in mind it has not been extensively tested. I cannot guarantee that it works in any environment.

If you run into problems with it, let me know.

Joakim

#24
joakim

joakim

    Member

  • Member
  • PipPip
  • 154 posts
  • Joined 18-November 09
  • OS:none specified
  • Country: Country Flag
If someone with very good knowledge about the $LogFile is aroung here, then please join this thread. If we could decode the structure of that metafile (or just the header of it), then I am quite sure boot.sdi could be reduced close to 60 Kb.

Joakim

#25
rpaz

rpaz
  • Member
  • 3 posts
  • Joined 05-July 10
  • OS:none specified
  • Country: Country Flag
Hi Joakim,

Thanks for sharing this small boot.sdi file it works fine booting PE2 and PE3 images from a PXE server.

Every size reduction is good to make PXE boot faster. :thumbup




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users