IPB

Google Frontpage Forums Unattended CD/DVD Guide
6 Pages V   1 2 3 > »   
Reply to this topicStart new topic
> Copy2Gb, Attempt at a fix to copy files > 2Gb with Explorer
LLXX
post Aug 21 2006, 08:15 PM
Post #1


MSFN Junkie
*********

Group: Banned
Posts: 3482
Joined: 4-December 05
Member No.: 81511
OS: none
Country Flag


Version 1.1 available for download

As mentioned in http://support.microsoft.com/?id=318293 Windows 98SE shell fails at trying to copy files over 2Gb. According to M$...
QUOTE
This problem can occur because of a problem in the versions of the Shell32.dll file that are included with Windows 98 and Windows 98 Second Edition. The root cause is a generic file operation which is used by the shell in Windows 95/98 and Microsoft Windows NT 4.0 which interprets values that are greater than 2 GB as negative numbers.
...the problem is with shell32.dll.

Unfortunately, M$ is only partially right. I've inspected and traced through the copying code in shell32.dll, and the actual problem seems to lie within the kernel itself, at the _llseek API.
CODE
7FCEB349     mov     [ebp+nNumberOfBytesToWrite], eax
7FCEB34C     mov     eax, [edi+20h]
7FCEB34F     cmp     eax, 30000h; over 192Kb in size?
7FCEB354     jbe     short 7FCEB386; no pre-enlargement if not
7FCEB356     push    ebx; iOrigin
7FCEB357     mov     ebx, ds:_llseek
7FCEB35D     push    eax; lOffset
7FCEB35E     push    [ebp+hDestFile]; hFile
7FCEB361     call    ebx; _llseek apparently has problems seeking over 2Gb
7FCEB363     cmp     eax, 0FFFFFFFFh
7FCEB366     jz      loc_0_7FCEB55D; fails here
7FCEB36C     push    [ebp+hDestFile]; hFile
7FCEB36F     call    ds:SetEndOfFile
7FCEB375     test    eax, eax
7FCEB377     jz      loc_0_7FCEB55D
7FCEB37D     push    0; iOrigin
7FCEB37F     push    0; lOffset
7FCEB381     push    [ebp+hDestFile]; hFile
7FCEB384     call    ebx; _llseek
For those that don't understand the above code, what it does is attempt to enlarge the destination file to the correct size by seeking to the source filesize and then setting the file's end there. Unfortunately, if the file is >2Gb, _llseek fails with "invalid parameter" error. Also note that unless the file is over 192Kb in size, no attempt to pre-enlarge is made - the file just enlarges automatically as data is written to it.

Either the bug is with the _llseek function in the kernel and the code in shell32 is fine, or _llseek was designed to do that (i.e. interprets >2Gb as moving the file pointer backwards, but the pointer is already at the start of the file so it complains) and shell32 is flawed. I can see several possible fixes:

1. Patch shell32.dll to use newer SetFilePointer function which uses 64-bit signed integers - difficult, but possibly the most "correct" solution.
2. Patch kernel32.dll _llseek to interpret seeking from the beginning with a negative offset as a positive offset - moderate, might break some other apps, but what program would want to move the file pointer past the start of the file?
3. Change conditional jump at 7FCEB354 to a permanent jump - easiest, but no more pre-enlargement - is pre-enlargement really needed?

-------------------------------------------------------------------------------------------------
Update:

Fixed kernels:
4.10.1998
4.10.2001
4.10.2222
4.10.2225
4.90.3000

(kernels are too big to attach)

Verification Tool:
Make2Gb 1.0

This post has been edited by LLXX: Aug 25 2006, 12:03 AM
Go to the top of the page
 
+Quote Post
wizardofwindows
post Aug 21 2006, 08:38 PM
Post #2


Wizard of Windows
****

Group: Banned
Posts: 532
Joined: 17-June 05
From: Ontario Canada
Member No.: 60904
Country Flag


thumbup.gif conquering the 2gb barrier would be a plus with dvd use today.noting many new games etc. are well over 2gig.u da man.
Go to the top of the page
 
+Quote Post
Petr
post Aug 22 2006, 12:11 AM
Post #3


MSFN Expert
******

Group: Members
Posts: 1000
Joined: 15-April 05
Member No.: 52191
Country Flag


In Windows 2000 (beta), it seems to be this part of the source code:
CODE
// initialzie the file to the full size
    // this takes 3 dos calls, so only do it if the file is big
    if (pfd->nFileSizeLow > (COPYMAXBUFFERSIZE * 3))
    {
        // if there's a problem, bail
        if ((_llseek(hDest, pfd->nFileSizeLow, 0L) == HFILE_ERROR) ||
            (!SetEndOfFile((HANDLE)hDest)))
        {
            iLastError = GetLastError();
            goto ErrorOnWrite;
        }
        else
        {
            _llseek(hDest, 0, 0L);
        }
    }


Petr
Go to the top of the page
 
+Quote Post
LLXX
post Aug 22 2006, 12:37 AM
Post #4


MSFN Junkie
*********

Group: Banned
Posts: 3482
Joined: 4-December 05
Member No.: 81511
OS: none
Country Flag


QUOTE
This problem was corrected in Windows Millennium Edition, Windows 2000, and Windows XP.
If Windows 2000 uses_llseek and is able to copy files > 2Gb correctly, then the problem must reside in _llseek function in kernel32.dll.

Also, would increasing the buffer size beyond the default 64k make for faster copying?
Go to the top of the page
 
+Quote Post
LLXX
post Aug 22 2006, 03:12 AM
Post #5


MSFN Junkie
*********

Group: Banned
Posts: 3482
Joined: 4-December 05
Member No.: 81511
OS: none
Country Flag


I've fixed _llseek, and it seems to work fine biggrin.gif



Maybe I'll experiment with different buffer sizes next...

As usual, if you want a newer version or already modified kernel to be patched, just post.

Once Kernel Update Project stabilises I'll probably fix the modified kernel from it.
Go to the top of the page
 
+Quote Post
the_guy
post Aug 22 2006, 04:20 AM
Post #6


Creator of the Windows ME Service Pack
*****

Group: Members
Posts: 944
Joined: 15-July 05
Member No.: 64572
Country Flag


Hey LLXX,

If you get it working from the 313829 version, can you try patching the NT4 version from MS04-037 (with Active Desktop) as well?

the_guy
Go to the top of the page
 
+Quote Post
Tihiy
post Aug 22 2006, 06:26 AM
Post #7


Revolutions Pack Creator
******

Group: Members
Posts: 1488
Joined: 19-November 04
From: NeverSleep
Member No.: 36858
OS: none
Country Flag


I haven't checked code, but project seems excellent.
It is not needed to patch Kernel Update because it is patch itself and does not provide kernel32.dll

But patch 4.10.2225!
Go to the top of the page
 
+Quote Post
wizardofwindows
post Aug 22 2006, 10:12 AM
Post #8


Wizard of Windows
****

Group: Banned
Posts: 532
Joined: 17-June 05
From: Ontario Canada
Member No.: 60904
Country Flag


thx u this is great u rock!!

This post has been edited by wizardofwindows: Aug 22 2006, 10:14 AM
Go to the top of the page
 
+Quote Post
erpdude8
post Aug 22 2006, 10:21 AM
Post #9


MSFN Addict
*******

Group: Members
Posts: 1983
Joined: 24-November 04
Member No.: 37246
Country Flag


QUOTE (LLXX @ Aug 21 2006, 09:15 PM) *
As mentioned in http://support.microsoft.com/?id=318293 Windows 98SE shell fails at trying to copy files over 2Gb. According to M$...
QUOTE
This problem can occur because of a problem in the versions of the Shell32.dll file that are included with Windows 98 and Windows 98 Second Edition. The root cause is a generic file operation which is used by the shell in Windows 95/98 and Microsoft Windows NT 4.0 which interprets values that are greater than 2 GB as negative numbers.
...the problem is with shell32.dll.



You also need to patch kernel32.dll version 4.10.1998 (and version 4.10.2001 from Q320798) for Win98 FE, LLXX. Let's not forget, the 2GB+ copy problem also occurs under Win98 FE as noted in MS article 318293.

patching kernel32.dll files from Win95? I believe Win95 also has the 2GB+ copy problem but the Win95 kernel32.dll files are missing a bunch of functions that were included in the Win98 FE/SE versions of kernel32.dll files.

This post has been edited by erpdude8: Aug 22 2006, 10:25 AM
Go to the top of the page
 
+Quote Post
MDGx
post Aug 22 2006, 10:39 AM
Post #10


creator of 98SE2ME
Group Icon

Group: Moderator
Posts: 2403
Joined: 22-November 04
Member No.: 37121
Country Flag


Great work.
Keep it up. newwink.gif
Added link here [scroll under "Windows 98/98 SE/ME Updates + Patches"]:
http://www.mdgx.com/

Does it make sense to patch WinME kernel32.dll ?

And I think it's a good idea to patch kernel32.dll 4.10.2225 .

IMHO:
When patching a system file that has already been patched previously, it's best to patch the newest build, which contains all previous patches.
Patching the oldest file only takes care of the newest issue, but not any previous ones.

Thanks for your time.
Go to the top of the page
 
+Quote Post
eidenk
post Aug 22 2006, 01:30 PM
Post #11


MSFN Expert
******

Group: Members
Posts: 1351
Joined: 28-March 05
Member No.: 49647
Country Flag


QUOTE (MDGx @ Aug 22 2006, 10:39 AM) *
Does it make sense to patch WinME kernel32.dll ?


I don't think so as it does not appear there is a problem with WinME. I have just created a 3.8GB archive and copied it over without problems.

I have been using Windows ME since a few years in the erroneous belief that it wouldn't handle files over 2GB.
Go to the top of the page
 
+Quote Post
winxpi
post Aug 22 2006, 03:57 PM
Post #12


Member
**

Group: Members
Posts: 167
Joined: 22-September 05
Member No.: 74059
Country Flag


QUOTE (LLXX @ Aug 21 2006, 08:15 PM) *
[1. Patch shell32.dll to use newer SetFilePointer function which uses 64-bit signed integers - difficult, but possibly the most "correct" solution.
2. Patch kernel32.dll _llseek to interpret seeking from the beginning with a negative offset as a positive offset - moderate, might break some other apps, but what program would want to move the file pointer past the start of the file?
3. Change conditional jump at 7FCEB354 to a permanent jump - easiest, but no more pre-enlargement - is pre-enlargement really needed?

Super another barrier fell.
I have some questions :
Not so important but did you try the most "correct" solution modifing shell32.dll ?

A better question :
How can I "Change conditional jump at 7FCEB354 to a permanent jump" and witch file would I have to change to do this?

thx again
Go to the top of the page
 
+Quote Post
LLXX
post Aug 22 2006, 05:50 PM
Post #13


MSFN Junkie
*********

Group: Banned
Posts: 3482
Joined: 4-December 05
Member No.: 81511
OS: none
Country Flag


The following patch requests have been listed:

- NT4 kernel - I don't have NT4, if you have the file PM me and deposit it somewhere.
- 4.10.2225 - Will do, http://www.mdgx.com/files/Q320798.EXE
- 4.10.1998 - I don't have this file, nor 98FE.
- 4.10.2001 - Will do, http://www.mdgx.com/files/Q320798.EXE
- Win95 kernels - probably pointless, as 95's explorer.exe doesn't work correctly with large files anyway.
- WinME - SHELL32.DLL probably changed to use SetFilePointer instead of flawed _llseek, may patch.

I have created a simple test to see if _llseek API in your kernel is flawed:

http://z11.zupload.com/file.php?filepath=40201

All it does is attempt to create a file and then enlarge it to 2147483648 bytes via _llseek. It will report success or error depending on behavior of _llseek. (Delete the file after testing, it contains no useful data. Also ensure there is more than 2Gb of free space).

QUOTE
A better question :
How can I "Change conditional jump at 7FCEB354 to a permanent jump" and witch file would I have to change to do this?
Open shell32.dll in a hex editor and go to .7FCEB354 (or 3B354, depending on the editor). The following bytes should be there:
CODE
76 30 53 8b 1d 44 16 cb 7f
Change it to
CODE
eb 30 53 8b 1d 44 16 cb 7f
(This is version 4.72.3612.1700, I don't know the exact location in other versions).

Edit: fixed link

This post has been edited by LLXX: Aug 23 2006, 03:47 PM
Go to the top of the page
 
+Quote Post
Philco
post Aug 23 2006, 12:34 AM
Post #14


Member
**

Group: Members
Posts: 162
Joined: 24-June 06
From: czech republic
Member No.: 99711
OS: 98SE
Country Flag


QUOTE (LLXX @ Aug 22 2006, 05:50 PM) *
....

I have created a simple test to see if _llseek API in your kernel is flawed:

http://www.upload2.net/page/download/VYkEc...AKE2GB.ZIP.html

...


Please, file corrupted, hosting other server?
Go to the top of the page
 
+Quote Post
LLXX
post Aug 23 2006, 03:40 AM
Post #15


MSFN Junkie
*********

Group: Banned
Posts: 3482
Joined: 4-December 05
Member No.: 81511
OS: none
Country Flag


It seems so. I found another filehost... seems to be working

http://z11.zupload.com/file.php?filepath=40201
Go to the top of the page
 
+Quote Post
eidenk
post Aug 23 2006, 03:51 AM
Post #16


MSFN Expert
******

Group: Members
Posts: 1351
Joined: 28-March 05
Member No.: 49647
Country Flag


I have tested your tool in WinME and I get an error dialog with the following message : Issuing_llseek call.

As you've read above, I am able to create and copy files bigger than 2GB on that platform.

So what is the meaning of failing this test ?
Go to the top of the page