[Petr]

Since I have invented how to break 34GB and 64GB barrier in Award BIOS several years ago (details here) I tried to summarize the knowledge about 137GB barrier and suggest some solutions.

At first, it has to be expressed that 32/34 GB limit (old Award BIOS, sysdm.cpl in W98/SE/ME) and 64 GB limit (old Award BIOS) are caused by overflow condition in the software (BIOS or CPL) and may be considered as bugs - the programer just used 16-bit integer value for disk capacity in megabytes => 32GB limit for signed int (sysdm.cpl), 64GB limit for unsigned int (Award BIOS), or divide overflow in translated geometry calculation (Award BIOS).

137GB limit has different nature. It is caused by different logic of the hardware interface of the hard disk drive. Now I will forget all older HDDs that do not support LBA and I will describe the difference. Classic HDDs use 28-bit addressing of the sector, each sector has 512 bytes, and therefore the maximum size is 0xfffffffh * 512 = 137,438,952,960 bytes. New 48-bit LBA addressing has theoretical maximum size 0xffffffffffff * 512 = 144,115,188,075,855,360 bytes. Detail description of the difference and how it works is in ATA/ATAPI-7 specification, especially chapter 4.14 on page 38/58. Another source of information is http://www.48bitlba.com/ .
I will describe it in short.
How ithe IDE (ATA) interface works? It is controlled by 8+1 registers addressed as ports (page 71 of ATA/ATAPI-7 spec), given example is for the most standard base address of the primary IDE controller 0x1f0 to 0x1f7 and 0x3f6.
When read:
0x1f0: Data
0x1f1: Error
0x1f2: Sector Count
0x1f3: LBA low
0x1f4: LBA mid
0x1f5: LBA high
0x1f6: Device
0x1f7: Status
0x3f6: Alternate Status
When written:
0x1f0: Data
0x1f1: Features
0x1f2: Sector Count
0x1f3: LBA low
0x1f4: LBA mid
0x1f5: LBA high
0x1f6: Device
0x1f7: Command
0x3f6: Device Control
For 28-bit LBA addressing, the highest 4 bits are part of the Device register.
For example, read of a sector works so that Sector Count, LBA Low, Mid, High, Device register are filled with the right data and then the Command is issued. Please also see table 6, p. 39/59.

48-bit LBA uses the same interface, but Sector Count, LBA Low, Mid, High are 2-bit FIFO registers, it means that the first write to these register contain upper 24 bits and the second write lower 24 bits. The 4 bits of Device register used by 28-bit LBA for are not used by 48-bit LBA. Please see table 5, p. 38/58.

This description clearly shows that there has to be no special hardware controller to support 48-bit LBA HDD, only the software has to be modified to address the sectors properly. To boot from such disk, either the BIOS has to have 48-LBA support, or some boot manager has to be used. Also if you want to use DOS based tools that use BIOS.

And how it is with Windows 98?

The maximum FAT32 partition size is 2 TB, it is (2^32-1) * 512 = 2,199,023,255,552 bytes. Windows seems to use 32-bit addressing of sectors. If the upper 4 bits are zero, 28-bit LBA will be used and no problem arise. But for size above 137GB, 48-bit LBA addressing has to be used and it means the above described two writes to each address register, in this case it means to write the upper 8 bits to the LBA Low register and zeroes to LBA Mid and LBA High in first step, and in the second step to write remianing 24 bits to LBA Low-Mid-High. Also Sector Count register has to be written in 2-byte sequence if programmed.

And this is the modification that has to be done with ESDI_506.PDR driver. There is a commercial patch made by Rudolph R. Loew but if we are looking for something to add to the SE SP, it has to be free. I'd strongly recommend that if anybody will want to write the patch for ESDI_506.PDR, he should never look at Mr. Loew patch to avoid possible license problems.

So what has to be done. ESDI_506.PDR is linear executable (LE) file, there are symbols available and also debug version + symbols is available in Windows 98 DDK.
All routines that has to be modified are in the first code segment, it is located at address 0x00800 in the ESDI_506.PDR file and this segment has size 0x2000 = 8192 bytes. In version 4.10.2225, 0x1dec = 7660 bytes are used, it means there are 532 bytes free for the patch. The code that needs to be modified seems to be in the following 4 procedures:
ESDI_Start_Hardware
ESDI_Func_Read
ESDI_Func_Write
ESDI_Func_Verify
So if the sector address is lower than 0xfffff00, old routine will be used, if it is higher or equal, new routine with LBA-48 addressing has to be used. In fact, this new routine can to be simplified to writing 8 highest bits to LBA Low and zeroes to LBA Mid and High and then the old procedure can be called - but with changed command number. It looks like the following commands are used, the first command number is for 28-bit LBA, the second for 48-bit LBA:
0xc8 -> 0x25 − READ DMA EXT
0xc4 -> 0x29 − READ MULTIPLE EXT
0x20 -> 0x24 − READ SECTOR(S) EXT
0x40 -> 0x42 − READ VERIFY SECTOR(S) EXT
0xca -> 0x35 − WRITE DMA EXT
0xc5 -> 0x39 − WRITE MULTIPLE EXT
0x30 -> 0x34 − WRITE SECTOR(S) EXT
There may be other tests needed, but this seems to be essential.

Regarding utilities, some of them use BIOS, some of them use Windows, and they has to be tested for compatibility with 137GB+ drives. I see no technical reason for 137GB limit here because this limit is caused by 28-bit hardware addressing and none of these utilities access hardware directly, always via BIOS or Windows calls, and there seems to be no 28-bit limit, the sector number is 32-bit. I suppose that if there is really 137GB limit, it could be easily removed, it is probably artificial only.

Short Q&A:
Q: Why the highest sector number for 28-bit LBA is 0xffffffe and not 0xfffffff?
A: Because the maximum number of sectors is 0xfffffff and they are numbered from 0, not from 1.

Q: Why the 28-bit LBA maximum sector number in ESDI_506.PDR proposed modification is 0xffffe0f and not 0xffffffe?
A: Because the sector count many be up to 255, and even the last sector shoud fit into 28-bit addressing.

Q: Why new HDD controller is often recommended as a solution? You wrote that no hardware changes are necessary.
A: This is not related to hardware. The only reason is that new controller comes with its own new BIOS and Windows driver that supports 48-bit LBA, this eliminates the need to have the 48-bit compatible BIOS or boot manager, and the need of patched ESDI_506.PDR.

Q: When the ESDI_506.PDR driver is used?
A: It is for standard controllers, like Intel PIIX, most newer controllers uses its own driver, like Intel Application Accelerator, these new drivers usually support 48-bit LBA.

Q: How to test whether the 48-bit LBA wirks as expected?
A: One possibility is just to fill the disk beyond 137GB and see whether the disk content got corrupted. Programmers could try to write small utility that would read and write data beyond the 137GB limit using registry access, using BIOS calls and using Windows calls and compare if the same sector is read and written.

I wrote this to enable somebody to write the necessary patches - I personally have no 137GB+ disk, not I'm skilled coder, nor I have time enough.
But this could help to make some overview what the 137GB limit is and what is not, and what has to be done to fully suport it in Windows 98 SE.

Petr

[nitebat]

Quote

And this is the modification that has to be done with ESDI_506.PDR driver. There is a commercial patch made by Rudolph R. Loew but if we are looking for something to add to the SE SP, it has to be free. I'd strongly recommend that if anybody will want to write the patch for ESDI_506.PDR, he should never look at Mr. Loew patch to avoid possible license problems.

Ah, Petr, in your wisdom you point out a rare occasion when re-inventing the wheel is a good idea! It would be nice to add that to the list of fixes for good ol' 9x.

Uh.... the wheel is open source, isn't it?

[Acheron]

What's the 137 GB limit anyway?



My harddrive is properly detected

[Vectis]

I to have a harddrive bigger than 137GB (mines FAT32 format though), but I dont think the actual drive size is the problem, rather the amount of data thats on it. What happens when the drive fills beyond 137GB?.

[Petr]

hp38guser, on May 24 2005, 11:50 PM, said:

What's the 137 GB limit anyway?

And what is not clear to you?

Petr

[Acheron]

Seems only problems trying to write to harddisks larger than 137 GB then?

[azagahl]

Seems only problems trying to write to harddisks larger than 137 GB then?

The problem is in the file ESDI_560.PDR. The code it uses to access the hard disk does not support 48-bit LBA. As soon as you try to write beyond the 137 GB point of your hard disk, it will wrap around and access the area at 0 GB, most likely completely trashing your data. Reading beyond 137 GB will most likely cause crashes. Partitions don't cure this.

You will be using this file and it will appear under Hard Disk Controllers unless 1. you booted to DOS 2. you are in safe mode or 3. you installed special drivers for your motherboard or HD controller (like me; I use VIADSK.MPD).

[eidenk]

Quote

I to have a harddrive bigger than 137GB (mines FAT32 format though), but I dont think the actual drive size is the problem, rather the amount of data thats on it. What happens when the drive fills beyond 137GB?.

Indeed I had successfully formatted a 200GB FAT32 drive and the problem arises when you break through the 128/137 GB barrier. I have tested it. It terminates abruptly your windows session and prevent you to reaccess windows by corrupting data.

Quote

As soon as you try to write beyond the 137 GB point of your hard disk, it will wrap around and access the area at 0 GB, most likely completely trashing your data. Reading beyond 137 GB will most likely cause crashes.

Not exactly it seems. In fact the data that has been destroyed on my disk has been the one that had been written the last to the disk. For example, the filling I copied to reach the barrier got integrally trashed. And I suspect the session did terminate only because the registry got thus destroyed as it is almost always the last thing written to the disk. Barely any of the old data I had got damaged. Additionally I did run several scandisk sessions from DOS on that drive and corrected several millions wrong entries (not all of them because I gave up). I did then delete all the data that was intact and I am now using the trashed drive as Disk D and I have on it my desktop, swap file and temporary folders, all perfectly working apparently beyond the 137 GB barrier. See screenshots :

1) Filling the Drive
2) All the latest filling is trashed
3) A recent folder that had originally only a few small files in it. Notice the reported size of it.
4) The current state of the drive
5) Partition Manager view. Note that the reported size is beyond the 128 GB limit and seems consistent with the 2.2 GB of workable data I have currently on it.
6) The workable data

Hi Petr, If you want some more details such as inspecting the FAT, just tell me what I must do to copy it to a file. I can do other crash tests if you want and forward you the precise results you might need so that it'll help you figure out even better what goes on and make it easier for you to write that patch that I am willing to help testing.

PS : I am currently on Windows ME but can install 98 for that purpose.

Regards.

[shaddam]

hmmm... got a similar problem some years ago on the 64gb barrier...but after patching the problem was solved...

totay i use an 3/4 filled up 200gb fat32 partitioned drive with an external pci-idecontroller successful and without corruption...
problem circumvented with the external controller?

[Petr]

shaddam, on May 25 2005, 10:09 AM, said:

totay i use an 3/4 filled up 200gb fat32 partitioned drive with an external pci-idecontroller successful and without corruption...
problem circumvented with the external controller?
<{POST_SNAPBACK}>

Yes and no. The PCI IDE controller hardware itself is not necessary, but the BIOS on this board and Windows drivers for this controller support 137GB+ disks.

Petr

[eidenk]

Quote

Partitions don't cure this.

I can't speak for 98SE but with Windows ME and the native Esdi_506.pdr 4.90.3000, there is no problem for handling more than 128/137 GB of data on a single disk provided the partitions are smaller than 128/137 GB. I just tested it. See screenshots :

1) Filling the drive with data
2) Result

[Vectis]

Are USB "type" harddrives affected by this limit?.

[rloew]

Quote

And this is the modification that has to be done with ESDI_506.PDR driver. There is a commercial patch made by Rudolph R. Loew but if we are looking for something to add to the SE SP, it has to be free. I'd strongly recommend that if anybody will want to write the patch for ESDI_506.PDR, he should never look at Mr. Loew patch to avoid possible license problems.

Dear Petr,

The License agreement on all versions of my "High Capacity Disk Patch" prohibit the disassembly or reverse engineering of my software or the Patches it installs.

If any of the information you posted was obtained by examining my code, or a patched ESDI_506.PDR file, it could be considered an illegal disclosure of trade secrets. In addition, anyone who uses any such trade secrets to write Software could be found to be in violation of my Copyright even if they personally have not examined my Software.

Rudoph R. Loew

http://rloew1.no-ip.com

This post has been edited by rloew: 30 August 2009 - 12:37 AM

[Petr]

rloew, on May 31 2005, 03:40 AM, said:

If any of the information you posted was obtained by examining my code, or a patched ESDI_506.PDR file, it could be considered an illegal disclosure of trade secrets. In addition, anyone who uses any such trade secrets to write Software could be found to be in violation of my Copyright even if they personally have not examined my Software.

Dear Rudolph,

I have posted publicly available information only, no information posted was obtained by examining of your code. I just summarized the information I already had about 48-bit LBA because many people asked me for BIOS patch to break the 137GB barrier. Plus added short inormation that I obtained when I was preparing localized language versions of VXD files. That's all. Do you see any of your trade secrets in my description?

BTW, do you know how the first Phoenix BIOS were made not to violate the IBM license?

Petr

[nitebat]

petr:

Quote

BTW, do you know how the first Phoenix BIOS were made not to violate the IBM license?

Way to go, Petr! If Mr. Loew had read this thread, he would know that we are looking for free solutions for improving Win 9x. So he is either:

A: uninformed
B: bragging
C: protecting (what he thinks is) his turf.

All of us are here to help or not here at all.

I'm not much of a flamer, but this one got me. petr, I've followed your posts. You're more then smart enough to figure out your own solutions and have been a great help to this board. Let us all follow your example!

nitebat

P.S. There's more then one way to skin a rat...

[MDGx]

Not to mention Mr. Loew is in direct violation of MS copyrights + trademark agreements, because if Mr. Loew would have read the EULA/License, the mandatory document to be read upon installation of every MS OS and software package, he would have noticed that:

Quote

Limitations on Reverse Engineering, Decompilation and Disassembly.
You may not reverse engineer, decompile, or disassemble the SOFTWARE PRODUCT, except and only to the extent that such activity is expressly permitted by applicable law notwithstanding this limitation.

which means that anybody who wants to modify any MS file in any way must contact them first to obtain their written permission, and which afterwards must be posted/bundled by the respective person(s)/business(es) together with the modified file(s)/software so everybody can read it and understand that MS formally approved of this modified software/file(s)/etc.

This fragment is quoted directly from LICENSE.TXT, the text file installed by MS Windows 98 SE setup into %windir% [usually C:\WINDOWS].
Note that all Windows OS releases install this file or similar.

[StateS]

Is this a multichoice question cause I got the answers, I choose A: uninformed abd B: Bragging, it appears that Mr. Loew has only posted two posts, the first one was on this thread: http://www.msfn.org/...showtopic=30809 and the second one was here in this thread, and guess what the posts were about...

Mr. Loew, no offense but this thread was made so that people who enjoy windows 98se can have a chance to let it keep on living, to make it better and more stable... in short this thread and forum for that matter is made for people who want to either contribute to make windows better and more custimizable, or to help others with problems they might be having with their windows, not to brag or to flame...

This forum is made for people who want to help others and for those who need help, not for people who want to help themselves...

P.S.
Sorry if I sound a bit rude or conceited people... (I only have two posts myself)

[MDGx]

StateS, on May 31 2005, 08:07 AM, said:

(I only have two posts myself)

Yeah, you should be banned from the internet for having "just" 2 posts. How dare you?

[rloew]

Dear Petr,

Some of the information you posted is not public knowledge and would require substantial reverse engineering of the ESDI_506.PDR file to identify. You said you are not a serious coder, which is why I questioned your sources. Neither the major hard drive manufacturers nor the driver writers at VIA seem aware of some of your info so testing actual hard drives is unlikely to produce the results you posted.

Anyone who is able to solve the 137GB limit on their own is free to do so. But as you yourself stated, they should not look at my code. This includes information posted by anyone who has. I have already found postings elsewhere by people attempting to reverse engineer my demo copy to distribute warez copies.

The Phoenix BIOS, as well as my Patch, come under the interoperability exemption to the D.M.C.A. but reverse engineering my code would not since it would add no new interoperability.

Rudolph R. Loew

http://rloew1.no-ip.com

P.S.

As for the A, B and C comments by other posters. The answer is C.
I know the intent of this forum, so A is not true. Otherwise I woud have done B.

The other posting was in response to a list of existing approaches that only covered a limited number of chipsets. I offered the only solution that covered all of them.

If anybody objects to "Commercial" code, they can write their own, independently of course. My Patch is not suited for any of the Open Source Business models.

This post has been edited by rloew: 30 August 2009 - 12:41 AM

[nitebat]

Quote

rloew
P.S.

As for the A, B and C comments by other posters. The answer is C.
I know the intent of this forum, so A is not true. Otherwise I woud have done B.

Logged & noted, with respect, rloew.

Part of keeping Win 9x alive is compatability with new hardware & software. This forum, it seems, will leave no stone unturned in that regard.

There are many ways to build a house;

the 'other' poster