Petr

137GB limit - ESDI_506.PDR and other limits

173 posts in this topic

Since I have invented how to break 34GB and 64GB barrier in Award BIOS several years ago (details here) I tried to summarize the knowledge about 137GB barrier and suggest some solutions.

At first, it has to be expressed that 32/34 GB limit (old Award BIOS, sysdm.cpl in W98/SE/ME) and 64 GB limit (old Award BIOS) are caused by overflow condition in the software (BIOS or CPL) and may be considered as bugs - the programer just used 16-bit integer value for disk capacity in megabytes => 32GB limit for signed int (sysdm.cpl), 64GB limit for unsigned int (Award BIOS), or divide overflow in translated geometry calculation (Award BIOS).

137GB limit has different nature. It is caused by different logic of the hardware interface of the hard disk drive. Now I will forget all older HDDs that do not support LBA and I will describe the difference. Classic HDDs use 28-bit addressing of the sector, each sector has 512 bytes, and therefore the maximum size is 0xfffffffh * 512 = 137,438,952,960 bytes. New 48-bit LBA addressing has theoretical maximum size 0xffffffffffff * 512 = 144,115,188,075,855,360 bytes. Detail description of the difference and how it works is in ATA/ATAPI-7 specification, especially chapter 4.14 on page 38/58. Another source of information is http://www.48bitlba.com/ .

I will describe it in short.

How ithe IDE (ATA) interface works? It is controlled by 8+1 registers addressed as ports (page 71 of ATA/ATAPI-7 spec), given example is for the most standard base address of the primary IDE controller 0x1f0 to 0x1f7 and 0x3f6.

When read:

0x1f0: Data

0x1f1: Error

0x1f2: Sector Count

0x1f3: LBA low

0x1f4: LBA mid

0x1f5: LBA high

0x1f6: Device

0x1f7: Status

0x3f6: Alternate Status

When written:

0x1f0: Data

0x1f1: Features

0x1f2: Sector Count

0x1f3: LBA low

0x1f4: LBA mid

0x1f5: LBA high

0x1f6: Device

0x1f7: Command

0x3f6: Device Control

For 28-bit LBA addressing, the highest 4 bits are part of the Device register.

For example, read of a sector works so that Sector Count, LBA Low, Mid, High, Device register are filled with the right data and then the Command is issued. Please also see table 6, p. 39/59.

48-bit LBA uses the same interface, but Sector Count, LBA Low, Mid, High are 2-bit FIFO registers, it means that the first write to these register contain upper 24 bits and the second write lower 24 bits. The 4 bits of Device register used by 28-bit LBA for are not used by 48-bit LBA. Please see table 5, p. 38/58.

This description clearly shows that there has to be no special hardware controller to support 48-bit LBA HDD, only the software has to be modified to address the sectors properly. To boot from such disk, either the BIOS has to have 48-LBA support, or some boot manager has to be used. Also if you want to use DOS based tools that use BIOS.

And how it is with Windows 98?

The maximum FAT32 partition size is 2 TB, it is (2^32-1) * 512 = 2,199,023,255,552 bytes. Windows seems to use 32-bit addressing of sectors. If the upper 4 bits are zero, 28-bit LBA will be used and no problem arise. But for size above 137GB, 48-bit LBA addressing has to be used and it means the above described two writes to each address register, in this case it means to write the upper 8 bits to the LBA Low register and zeroes to LBA Mid and LBA High in first step, and in the second step to write remianing 24 bits to LBA Low-Mid-High. Also Sector Count register has to be written in 2-byte sequence if programmed.

And this is the modification that has to be done with ESDI_506.PDR driver. There is a commercial patch made by Rudolph R. Loew but if we are looking for something to add to the SE SP, it has to be free. I'd strongly recommend that if anybody will want to write the patch for ESDI_506.PDR, he should never look at Mr. Loew patch to avoid possible license problems.

So what has to be done. ESDI_506.PDR is linear executable (LE) file, there are symbols available and also debug version + symbols is available in Windows 98 DDK.

All routines that has to be modified are in the first code segment, it is located at address 0x00800 in the ESDI_506.PDR file and this segment has size 0x2000 = 8192 bytes. In version 4.10.2225, 0x1dec = 7660 bytes are used, it means there are 532 bytes free for the patch. The code that needs to be modified seems to be in the following 4 procedures:

ESDI_Start_Hardware

ESDI_Func_Read

ESDI_Func_Write

ESDI_Func_Verify

So if the sector address is lower than 0xfffff00, old routine will be used, if it is higher or equal, new routine with LBA-48 addressing has to be used. In fact, this new routine can to be simplified to writing 8 highest bits to LBA Low and zeroes to LBA Mid and High and then the old procedure can be called - but with changed command number. It looks like the following commands are used, the first command number is for 28-bit LBA, the second for 48-bit LBA:

0xc8 -> 0x25 − READ DMA EXT

0xc4 -> 0x29 − READ MULTIPLE EXT

0x20 -> 0x24 − READ SECTOR(S) EXT

0x40 -> 0x42 − READ VERIFY SECTOR(S) EXT

0xca -> 0x35 − WRITE DMA EXT

0xc5 -> 0x39 − WRITE MULTIPLE EXT

0x30 -> 0x34 − WRITE SECTOR(S) EXT

There may be other tests needed, but this seems to be essential.

Regarding utilities, some of them use BIOS, some of them use Windows, and they has to be tested for compatibility with 137GB+ drives. I see no technical reason for 137GB limit here because this limit is caused by 28-bit hardware addressing and none of these utilities access hardware directly, always via BIOS or Windows calls, and there seems to be no 28-bit limit, the sector number is 32-bit. I suppose that if there is really 137GB limit, it could be easily removed, it is probably artificial only.

Short Q&A:

Q: Why the highest sector number for 28-bit LBA is 0xffffffe and not 0xfffffff?

A: Because the maximum number of sectors is 0xfffffff and they are numbered from 0, not from 1.

Q: Why the 28-bit LBA maximum sector number in ESDI_506.PDR proposed modification is 0xffffe0f and not 0xffffffe?

A: Because the sector count many be up to 255, and even the last sector shoud fit into 28-bit addressing.

Q: Why new HDD controller is often recommended as a solution? You wrote that no hardware changes are necessary.

A: This is not related to hardware. The only reason is that new controller comes with its own new BIOS and Windows driver that supports 48-bit LBA, this eliminates the need to have the 48-bit compatible BIOS or boot manager, and the need of patched ESDI_506.PDR.

Q: When the ESDI_506.PDR driver is used?

A: It is for standard controllers, like Intel PIIX, most newer controllers uses its own driver, like Intel Application Accelerator, these new drivers usually support 48-bit LBA.

Q: How to test whether the 48-bit LBA wirks as expected?

A: One possibility is just to fill the disk beyond 137GB and see whether the disk content got corrupted. Programmers could try to write small utility that would read and write data beyond the 137GB limit using registry access, using BIOS calls and using Windows calls and compare if the same sector is read and written.

I wrote this to enable somebody to write the necessary patches - I personally have no 137GB+ disk, not I'm skilled coder, nor I have time enough.

But this could help to make some overview what the 137GB limit is and what is not, and what has to be done to fully suport it in Windows 98 SE.

Petr

0

Share this post


Link to post
Share on other sites
And this is the modification that has to be done with ESDI_506.PDR driver. There is a commercial patch made by Rudolph R. Loew but if we are looking for something to add to the SE SP, it has to be free. I'd strongly recommend that if anybody will want to write the patch for ESDI_506.PDR, he should never look at Mr. Loew patch to avoid possible license problems.

Ah, Petr, in your wisdom you point out a rare occasion when re-inventing the wheel is a good idea! It would be nice to add that to the list of fixes for good ol' 9x.

Uh.... the wheel is open source, isn't it?

0

Share this post


Link to post
Share on other sites

What's the 137 GB limit anyway?

harddrive.png

My harddrive is properly detected :)

0

Share this post


Link to post
Share on other sites

I to have a harddrive bigger than 137GB (mines FAT32 format though), but I dont think the actual drive size is the problem, rather the amount of data thats on it. What happens when the drive fills beyond 137GB?.

0

Share this post


Link to post
Share on other sites
What's the 137 GB limit anyway?

And what is not clear to you?

Petr

0

Share this post


Link to post
Share on other sites

Seems only problems trying to write to harddisks larger than 137 GB then?

0

Share this post


Link to post
Share on other sites

Seems only problems trying to write to harddisks larger than 137 GB then?

The problem is in the file ESDI_560.PDR. The code it uses to access the hard disk does not support 48-bit LBA. As soon as you try to write beyond the 137 GB point of your hard disk, it will wrap around and access the area at 0 GB, most likely completely trashing your data. Reading beyond 137 GB will most likely cause crashes. Partitions don't cure this.

You will be using this file and it will appear under Hard Disk Controllers unless 1. you booted to DOS 2. you are in safe mode or 3. you installed special drivers for your motherboard or HD controller (like me; I use VIADSK.MPD).

0

Share this post


Link to post
Share on other sites
I to have a harddrive bigger than 137GB (mines FAT32 format though), but I dont think the actual drive size is the problem, rather the amount of data thats on it. What happens when the drive fills beyond 137GB?.
Indeed I had successfully formatted a 200GB FAT32 drive and the problem arises when you break through the 128/137 GB barrier. I have tested it. It terminates abruptly your windows session and prevent you to reaccess windows by corrupting data.
As soon as you try to write beyond the 137 GB point of your hard disk, it will wrap around and access the area at 0 GB, most likely completely trashing your data. Reading beyond 137 GB will most likely cause crashes.

Not exactly it seems. In fact the data that has been destroyed on my disk has been the one that had been written the last to the disk. For example, the filling I copied to reach the barrier got integrally trashed. And I suspect the session did terminate only because the registry got thus destroyed as it is almost always the last thing written to the disk. Barely any of the old data I had got damaged. Additionally I did run several scandisk sessions from DOS on that drive and corrected several millions wrong entries (not all of them because I gave up). I did then delete all the data that was intact and I am now using the trashed drive as Disk D and I have on it my desktop, swap file and temporary folders, all perfectly working apparently beyond the 137 GB barrier. See screenshots :

1) Filling the Drive

2) All the latest filling is trashed

3) A recent folder that had originally only a few small files in it. Notice the reported size of it.

4) The current state of the drive

5) Partition Manager view. Note that the reported size is beyond the 128 GB limit and seems consistent with the 2.2 GB of workable data I have currently on it.

6) The workable data

Hi Petr, If you want some more details such as inspecting the FAT, just tell me what I must do to copy it to a file. I can do other crash tests if you want and forward you the precise results you might need so that it'll help you figure out even better what goes on and make it easier for you to write that patch that I am willing to help testing.

PS : I am currently on Windows ME but can install 98 for that purpose.

Regards.

0

Share this post


Link to post
Share on other sites

hmmm... got a similar problem some years ago on the 64gb barrier...but after patching the problem was solved...

totay i use an 3/4 filled up 200gb fat32 partitioned drive with an external pci-idecontroller successful and without corruption...

problem circumvented with the external controller?

0

Share this post


Link to post
Share on other sites
totay i use an 3/4 filled up 200gb fat32 partitioned drive with an external pci-idecontroller successful and without corruption...

problem circumvented with the external controller?

Yes and no. The PCI IDE controller hardware itself is not necessary, but the BIOS on this board and Windows drivers for this controller support 137GB+ disks.

Petr

0

Share this post


Link to post
Share on other sites
Partitions don't cure this.

I can't speak for 98SE but with Windows ME and the native Esdi_506.pdr 4.90.3000, there is no problem for handling more than 128/137 GB of data on a single disk provided the partitions are smaller than 128/137 GB. I just tested it. See screenshots :

1) Filling the drive with data

2) Result

0

Share this post


Link to post
Share on other sites

Are USB "type" harddrives affected by this limit?. :unsure:

0

Share this post


Link to post
Share on other sites
And this is the modification that has to be done with ESDI_506.PDR driver. There is a commercial patch made by Rudolph R. Loew but if we are looking for something to add to the SE SP, it has to be free. I'd strongly recommend that if anybody will want to write the patch for ESDI_506.PDR, he should never look at Mr. Loew patch to avoid possible license problems.

Dear Petr,

The License agreement on all versions of my "High Capacity Disk Patch" prohibit the disassembly or reverse engineering of my software or the Patches it installs.

If any of the information you posted was obtained by examining my code, or a patched ESDI_506.PDR file, it could be considered an illegal disclosure of trade secrets. In addition, anyone who uses any such trade secrets to write Software could be found to be in violation of my Copyright even if they personally have not examined my Software.

Rudoph R. Loew

http://rloew1.no-ip.com

Edited by rloew
0

Share this post


Link to post
Share on other sites
If any of the information you posted was obtained by examining my code, or a patched ESDI_506.PDR file, it could be considered an illegal disclosure of trade secrets. In addition, anyone who uses any such trade secrets to write Software could be found to be in violation of my Copyright even if they personally have not examined my Software.

Dear Rudolph,

I have posted publicly available information only, no information posted was obtained by examining of your code. I just summarized the information I already had about 48-bit LBA because many people asked me for BIOS patch to break the 137GB barrier. Plus added short inormation that I obtained when I was preparing localized language versions of VXD files. That's all. Do you see any of your trade secrets in my description?

BTW, do you know how the first Phoenix BIOS were made not to violate the IBM license?

Petr

0

Share this post


Link to post
Share on other sites

petr:

BTW, do you know how the first Phoenix BIOS were made not to violate the IBM license?

Way to go, Petr! If Mr. Loew had read this thread, he would know that we are looking for free solutions for improving Win 9x. So he is either:

A: uninformed

B: bragging

C: protecting (what he thinks is) his turf.

All of us are here to help or not here at all.

I'm not much of a flamer, but this one got me. petr, I've followed your posts. You're more then smart enough to figure out your own solutions and have been a great help to this board. Let us all follow your example!

nitebat

P.S. There's more then one way to skin a rat...

0

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.