[rloew]

The 24 Partitions don't have to be on one drive either.
My current prototype DDO is limited to 512TiB, so I would need six drives with four Partitions each to support 3PiB.

[Mijzelf]

jaclaz, on 30 April 2010 - 02:02 AM, said:

Mijzelf, on 30 April 2010 - 01:37 AM, said:

rloew said:

With 24 Partitions

How is the partition table organized? The original IBM partitiontable is not usable, is it? Is there another way to define a partition on an internal (boot) harddisk?

Why not?

I'm under the impression that the default partition table is depends on a 512 byte sectorsize.

Mijzelf, on 11 April 2007 - 04:40 AM, said:

rloew, on Jan 8 2007, 01:37 AM, said:

I have already done the reverse (split one large hard drive into smaller drives) in the experimental versions of my High Capacity Disk Patch for Hard Drives larger than 2200GB. A matching DOS DDO, similar to my BOOTMAN packages, will be required for DOS support and to pass the protected mode validation precedure in the IOS.VXD and ESDI_506.PDR code.

Wouldn't is be more elegant to increase the sectorsize, instead of splitting the drive? AFAIK the sectorsize doesn't need to be 512 bytes, and since FAT32 creates clusters of at least 4kB, sectors could get this size without any penalty.

rloew, on 11 April 2007 - 01:10 PM, said:

More elegant maybe, but it would require a major redesign. A new partition table format would be required making it incompatable with all existing operationg systems.

Mijzelf, on 10 January 2010 - 12:06 PM, said:

I own a 1024 byte sectorsize usb-stick (actually an el-cheapo mp3 player), which is formatted FAT16. W98 has no problems accessing it.

But I couldn't create a partitiontable on it, using linux, while other usb sticks are no problem.

[rloew]

Mijzelf, on 01 May 2010 - 02:02 AM, said:

jaclaz, on 30 April 2010 - 02:02 AM, said:

Mijzelf, on 30 April 2010 - 01:37 AM, said:

rloew said:

With 24 Partitions

How is the partition table organized? The original IBM partitiontable is not usable, is it? Is there another way to define a partition on an internal (boot) harddisk?

Why not?

I'm under the impression that the default partition table is depends on a 512 byte sectorsize.

A standard Partition Table and Boot Sector Data are 512 Bytes, but the size of the Sector is not important. The rest will just be ignored.

Mijzelf, on 11 April 2007 - 04:40 AM, said:

rloew, on Jan 8 2007, 01:37 AM, said:

I have already done the reverse (split one large hard drive into smaller drives) in the experimental versions of my High Capacity Disk Patch for Hard Drives larger than 2200GB. A matching DOS DDO, similar to my BOOTMAN packages, will be required for DOS support and to pass the protected mode validation precedure in the IOS.VXD and ESDI_506.PDR code.

Wouldn't is be more elegant to increase the sectorsize, instead of splitting the drive? AFAIK the sectorsize doesn't need to be 512 bytes, and since FAT32 creates clusters of at least 4kB, sectors could get this size without any penalty.

Three years ago I was not as familiar with the internals of IO.SYS and VFAT.VXD. I was not aware that their code could be adapted without a major rewrite.
As it is, several modifications were required to both of them and IOS.VXD as well. This still tops out at 16TiB of course.

rloew, on 11 April 2007 - 01:10 PM, said:

More elegant maybe, but it would require a major redesign. A new partition table format would be required making it incompatable with all existing operationg systems.

Subsequently I developed a modified Partition table Format to get past 2TiB, but the caveat above still applies.

Mijzelf, on 10 January 2010 - 12:06 PM, said:

I own a 1024 byte sectorsize usb-stick (actually an el-cheapo mp3 player), which is formatted FAT16. W98 has no problems accessing it.

But I couldn't create a partitiontable on it, using linux, while other usb sticks are no problem.

Unmodified VFAT.VXD and the 32-Bit portion of IOS.VXD can handle up to 2KB Sectors, so it is possible that an 1KB Sector USB Device would work.
I would be interested in more details about that MP3 Player. My latest RFDISK Program should be able to Partition it.

[dencorso]

RLoew, since you've studied IO.SYS possibly more deeply than anyone else here, could you perhaps write a standalone decompressor for Win ME's IO.SYS, please? I was on the verge of doing it myself several times, but never found continuous spare time enough to actually do it. It would be of great help to develop and/or port patches to Win ME's IO.SYS. And would also help settling (in case anything can settle it) an open discussion elsewhere, as well!

[rloew]

dencorso, on 01 May 2010 - 02:34 PM, said:

RLoew, since you've studied IO.SYS possibly more deeply than anyone else here, could you perhaps write a standalone decompressor for Win ME's IO.SYS, please? I was on the verge of doing it myself several times, but never found continuous spare time enough to actually do it. It would be of great help to develop and/or port patches to Win ME's IO.SYS. And would also help settling (in case anything can settle it) an open discussion elsewhere, as well!

I have been concentrating on the Windows 98 IO.SYS, mainly the Boot sequence and the resident code.
I took a quick look at the ME IO.SYS. It appears to only load the initial 2K block uncompressed. The decompressor should be inside.
IO.SYS loads itself in multiple stages so analysis is difficult. I am still looking at the loading sequence for the Windows 98 Version to see if it can be made bootable with larger sectors and/or 64KB or larger Clusters.

[Mijzelf]

rloew, on 01 May 2010 - 02:00 PM, said:

I would be interested in more details about that MP3 Player.

What do you want to know? It's this device. When you think it can help you to Support the Community, pm me your address, and I will send it to you. (I just connected it, and saw I was mistaken. It has a 2k sectorsize)

[rloew]

Mijzelf, on 02 May 2010 - 07:11 AM, said:

rloew, on 01 May 2010 - 02:00 PM, said:

I would be interested in more details about that MP3 Player.

What do you want to know? It's this device. When you think it can help you to Support the Community, pm me your address, and I will send it to you. (I just connected it, and saw I was mistaken. It has a 2k sectorsize)

You may be looking at the USB CDROM component.
Many MP3 Players mount as two devices. A Mass Storage Device where your files go and a simulated CDROM with a Setup Program on it.

[Mijzelf]

I don't think so. This is the output of dmesg after inserting:

Quote

usb 2-1: new full speed USB device using uhci_hcd and address 2
usb 2-1: New USB device found, idVendor=0639, idProduct=6639
usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
usb 2-1: Product: MP3 Player      
usb 2-1: Manufacturer: KONIG  
usb 2-1: SerialNumber: F03A2468774C1194
usb 2-1: configuration #1 chosen from 1 choice
Initializing USB Mass Storage driver...
scsi2 : SCSI emulation for USB Mass Storage devices
usbcore: registered new interface driver usb-storage
USB Mass Storage support registered.
usb-storage: device found at 2
usb-storage: waiting for device to settle before scanning
scsi 2:0:0:0: Direct-Access     KONIG    MP3 Player       0100 PQ: 0 ANSI: 4
sd 2:0:0:0: Attached scsi generic sg1 type 0
usb-storage: device scan complete
sd 2:0:0:0: [sdb] 496000 2048-byte logical blocks: (1.01 GB/968 MiB)
sd 2:0:0:0: [sdb] Write Protect is off
sd 2:0:0:0: [sdb] Mode Sense: 3e 00 00 00
sd 2:0:0:0: [sdb] Assuming drive cache: write through
sd 2:0:0:0: [sdb] 496000 2048-byte logical blocks: (1.01 GB/968 MiB)
sd 2:0:0:0: [sdb] Assuming drive cache: write through
sdb:
sd 2:0:0:0: [sdb] 496000 2048-byte logical blocks: (1.01 GB/968 MiB)
sd 2:0:0:0: [sdb] Assuming drive cache: write through
sd 2:0:0:0: [sdb] Attached SCSI removable disk

and this the output of fdisk -l:

Quote

Sectorsize is 2048 (not 512)

Disk /dev/sdb: 1015 MB, 1015808000 bytes
32 heads, 61 sectors/track, 254 cilinders
Unit = cilinders of 1952 * 2048 = 3997696 bytes

mount:

Quote

/dev/sdb on /media/KONIG type vfat (rw,nosuid,nodev,noatime,uhelper=hal,shortname=lower,uid=10001,flush,utf8)

[rloew]

dencorso, on 01 May 2010 - 02:34 PM, said:

RLoew, since you've studied IO.SYS possibly more deeply than anyone else here, could you perhaps write a standalone decompressor for Win ME's IO.SYS, please? I was on the verge of doing it myself several times, but never found continuous spare time enough to actually do it. It would be of great help to develop and/or port patches to Win ME's IO.SYS. And would also help settling (in case anything can settle it) an open discussion elsewhere, as well!

I was able to decompress the Windows ME IO.SYS file and combine the data to produce an uncompressed IO.SYS file. It turned out that the entire file is decompressed in one operation. This made it a lot simpler.


@Mijzelf: I don't have a Linux system to test my MP3 Player on. Windows ME USB Drivers mount mine as two separate Devices.

I was able to mount a FAT Filesystem on a CD-ROM so I already have a way of setting up a 2KB Logical Sector Device.

[dencorso]

rloew, on 03 May 2010 - 12:10 AM, said:

I was able to decompress the Windows ME IO.SYS file and combine the data to produce an uncompressed IO.SYS file. It turned out that the entire file is decompressed in one operation. This made it a lot simpler.

Wow! That was fast! Way to go, RLoew! You do rock!
I presume you did it by hand... Please, do consider creating a decompressor tool to automate the process. Besides the default IO.SYS for hard disks and the one that comes in the EBD floppy, three other flavours of the Win ME IO.SYS are known: the one in the bootable floppy image contained in Win ME's CD, and those embedded in discopy.dll, retrievable with DiskExtract, thanks to Nuno Brito (there are two flavors of it, one in the discopy.dll from Win XP and another in the same dll from Vista or 2k3), not to mention those in KB311561... so that an automated solution would be a wonderful tool to have handy.

BTW, if I'm not mistaken, the compressed part starts with the identifier MSCM (which is reminescent of MSCF, the .cab file identifier), right?

[rloew]

dencorso, on 03 May 2010 - 01:23 AM, said:

rloew, on 03 May 2010 - 12:10 AM, said:

I was able to decompress the Windows ME IO.SYS file and combine the data to produce an uncompressed IO.SYS file. It turned out that the entire file is decompressed in one operation. This made it a lot simpler.

Wow! That was fast! Way to go, RLoew! You do rock!
I presume you did it by hand... Please, do consider creating a decompressor tool to automate the process. Besides the default IO.SYS for hard disks and the one that comes in the EBD floppy, three other flavours of the Win ME IO.SYS are known: the one in the bootable floppy image contained in Win ME's CD, and those embedded in discopy.dll, retrievable with DiskExtract, thanks to Nuno Brito (there are two flavors of it, one in the discopy.dll from Win XP and another in the same dll from Vista or 2k3), not to mention those in KB311561... so that an automated solution would be a wonderful tool to have handy.

Done.

Quote

BTW, if I'm not mistaken, the compressed part starts with the identifier MSCM (which is reminescent of MSCF, the .cab file identifier), right?

Close, but no cigar. Even though next to each oter, the 'MS' and 'CM' as tested separately for different purposes. The 'MS' appears to be an integrity check. The 'CM' indicates the Compressed Data Header. It also appears at the start of the Decompression Code Header.

[dencorso]

Awesome!

[rloew]

dencorso, on 03 May 2010 - 09:27 PM, said:

Awesome!

I have published Version 1.0 of my Decompressor on my Website. It is in the Software Catalog under the Prerelease and Beta Section.

[Marius '95]

rloew, on 14 January 2010 - 02:22 AM, said:

I have done some experimenting using a DDO that simulates a 4KB Logical Sector Hard Drive.

Sorry for the delayed response.

If you need to test different sector sizes for real (not using a DDO) I recommend using a SCSI HDD. SCSI HDDs support low level formatting with different sector sizes. You just need an old HDD (most 9 GB HDDs support this feature), an old SCSI adapter and Bart's SCSI Tool.

Run Bart's SCSI Tool and set a different sector size in drive geometry page, then low-level-format the drive. It takes 30min - 1h depending on drive speed and size. There is no progress indicator.

[rloew]

Marius, on 19 May 2010 - 04:46 AM, said:

rloew, on 14 January 2010 - 02:22 AM, said:

I have done some experimenting using a DDO that simulates a 4KB Logical Sector Hard Drive.

Sorry for the delayed response.

If you need to test different sector sizes for real (not using a DDO) I recommend using a SCSI HDD. SCSI HDDs support low level formatting with different sector sizes. You just need an old HDD (most 9 GB HDDs support this feature), an old SCSI adapter and Bart's SCSI Tool.

Run Bart's SCSI Tool and set a different sector size in drive geometry page, then low-level-format the drive. It takes 30min - 1h depending on drive speed and size. There is no progress indicator.

I am aware that SCSI Drives can be set for different Sector Sizes.
The DDO is more than adequate for testing DOS.
I had to add extra Patches to ESDI_506.PDR to simulate in 32-Bit mode but SCSI would not help since I could not use ESDI_506.PDR with it.
I was able to use a real 2KB Sector FileSystem by putting a FAT FileSystem on a CD.

[Ninho]

Mijzelf, on 10 January 2010 - 12:06 PM, said:

 There are no compatible biosses, so you won't be able to boot from a 4k sector disk.

This is plainly false. A standard AT BIOS will read one (1) sector from disk to a fixed place in memory, then jump to execute code from the start of that buffer provided the required signature, hex AA 55 is present at offset 510 (1fe hex). It doesn't matter if the sector length isn't 512, provided it is /larger/ (if it were smaller than 512, /then/ you'd have a problem because the boot sector signature wouldn't be found after reading the sector and so, BIOS would think there was a read error and the boot process wouldn't go on.

[rloew]

Ninho, on 20 May 2010 - 03:39 PM, said:

Mijzelf, on 10 January 2010 - 12:06 PM, said:

 There are no compatible biosses, so you won't be able to boot from a 4k sector disk.

This is plainly false. A standard AT BIOS will read one (1) sector from disk to a fixed place in memory, then jump to execute code from the start of that buffer provided the required signature, hex AA 55 is present at offset 510 (1fe hex). It doesn't matter if the sector length isn't 512, provided it is /larger/ (if it were smaller than 512, /then/ you'd have a problem because the boot sector signature wouldn't be found after reading the sector and so, BIOS would think there was a read error and the boot process wouldn't go on.

Reading the MBR would not be the issue. Unless the BIOS recognizes larger sector Hard Drives, it would only read 256 Words (512 Bytes) from the sector. This would leave the request incomplete with the drive waiting to return the rest of the sector. The drive would not be ready for the next request. Even if the BIOS managed to continue to boot by repeatedly resetting the drive between reads, it would fail as soon as the loader expected a larger sector.

[Ninho]

rloew, on 20 May 2010 - 06:24 PM, said:

Reading the MBR would not be the issue. Unless the BIOS recognizes larger sector Hard Drives, it would only read 256 Words (512 Bytes) from the sector. This would leave the request incomplete with the drive waiting to return the rest of the sector.

Hmm, I suppose the low level BIOS routines would be accessing the HD controller in the old AT-compatible way, i.e. request a number of /sectors/ not /bytes/. But I haven't got 1024 byte per sector disks yet, so I could've held my mouth shut :~{ --- May I assume your reply was backed on actual trials you made ?

W/ best regards,

--
Ninho

This post has been edited by Ninho: 23 May 2010 - 12:19 PM

[rloew]

Ninho, on 23 May 2010 - 12:18 PM, said:

rloew, on 20 May 2010 - 06:24 PM, said:

Reading the MBR would not be the issue. Unless the BIOS recognizes larger sector Hard Drives, it would only read 256 Words (512 Bytes) from the sector. This would leave the request incomplete with the drive waiting to return the rest of the sector.

Hmm, I suppose the low level BIOS routines would be accessing the HD controller in the old AT-compatible way, i.e. request a number of /sectors/ not /bytes/. But I haven't got 1024 byte per sector disks yet, so I could've held my mouth shut :~{ --- May I assume your reply was backed on actual trials you made ?

W/ best regards,

--
Ninho

Requests to Hard Drives are always based in Sectors. The Data transfer is in Bytes, Words or DWords. The BIOS would have to know the Sector Length to compute the number of transfer requests to perform or to tell the DMA controller to perform. The earliest versions of the protocol allowed specifying a Sector size, but I don't think it was ever used. The LBA protocol completely breaks this protocol. Sector size information was not added to the standard until very recently so only the newest BIOSes could possibly use it. Only by reading a Sector and monitoring the Data Ready Flag could an older BIOS determine the size of a Sector.

Since there are no ATA Hard Drives, that I know about, to test with, I could not run trials. As even the latest BIOSes I have seen, do not support the full 48-Bit LBA, but only 32 Bits, I doubt they would have Large Sector support.

This post has been edited by rloew: 23 May 2010 - 01:34 PM

[Ninho]

You're probably right - I was thinking of (very) old AT disks, chipsets and BIOSes - that did not use DMA for access to HD.