Jump to content

Welcome to MSFN Forum
Register now to gain access to all of our features. Once registered and logged in, you will be able to create topics, post replies to existing threads, give reputation to your fellow members, get your own private messenger, post status updates, manage your profile and so much more. This message will be removed once you have signed in.
Login to Account Create an Account



Photo

ImportPatcher - Find and fix dependency problems

- - - - - IP.38_(3/29/2013) IP.39_(7/06/2013)

  • Please log in to reply
128 replies to this topic

#101
jumper

jumper

    2014 All-American Masters HJ'er

  • Member
  • PipPipPip
  • 498 posts
  • Joined 21-January 11
  • OS:98SE
  • Country: Country Flag
In W2K and later, many NTDLL.DLL apis are made available in Kernel32.dll via export forwarders.

Three types of dependencies are still not supported: import forwarders, export forwarders, and dynamic delay-loads. I've been learning a lot about export forwarders lately and if import forwarders work the same way, both of these should be relatively simple to implement.

I know exactly how export forwards are implemented, but it's not a small change for Import Patcher to support them. IP is architectured to patch Imports, not Exports. "It's impossible. But doable."

KernelEx 4.52 runs as an MPR service, thus it loads after MPR. I assume this is why you're trying to patch MPR instead of using Kexstubs.

NTDLL.DLL interfaces directly to drivers and the OS, so it really is an impossible substitution. Kernel32.dll is also extremely OS-centric and off-limits for substitution. Also Kernel32.dll is non-relocatable, so two versions can't be loaded at the same time.

The best solution I see is to rebuild the Wine or ReactOS sources for MPR.dll and static link the necessary functions from the other librarys.
Design feedback requested:
IHAtool - IpHlpApi tester; call various functions and report results
--status-> framework is solid; 22 api's fully supported; preview release coming soon
ComDlg32 wrapper - ComDlgEx meets IpHlpApi wrapper
--status-> PrintDlgExW working in latest SumatraPDF 8^)
Future projects: ImportPatcher40 - dialog interface; Kexter - IP40+Ktree+Kexstubs


How to remove advertisement from MSFN

#102
jds

jds

    -DOS+

  • Member
  • PipPipPipPip
  • 603 posts
  • Joined 03-June 08
  • OS:98SE
  • Country: Country Flag

In W2K and later, many NTDLL.DLL apis are made available in Kernel32.dll via export forwarders.


Three types of dependencies are still not supported: import forwarders, export forwarders, and dynamic delay-loads. I've been learning a lot about export forwarders lately and if import forwarders work the same way, both of these should be relatively simple to implement.

I know exactly how export forwards are implemented, but it's not a small change for Import Patcher to support them. IP is architectured to patch Imports, not Exports. "It's impossible. But doable."

Hi jumper.

Ah, I see. And if I'm interpreting this correctly, the "ntdll.RtlDeleteCriticalSection" and similar functions seen in 'k2rnel32.dll' (W2K 'kernel32.dll') are export forwarder thingies. So what we actually need (ignoring the other issues here) is an Export Patcher tool?

KernelEx 4.52 runs as an MPR service, thus it loads after MPR. I assume this is why you're trying to patch MPR instead of using Kexstubs.

NTDLL.DLL interfaces directly to drivers and the OS, so it really is an impossible substitution. Kernel32.dll is also extremely OS-centric and off-limits for substitution. Also Kernel32.dll is non-relocatable, so two versions can't be loaded at the same time.

The best solution I see is to rebuild the Wine or ReactOS sources for MPR.dll and static link the necessary functions from the other librarys.

I didn't try the KexStubs path because there were so many missing dependencies involved and because there was no chance that they could all simply be stubs.

As regards the 'ntdll.dll' and 'kernel32.dll' OS compatibility, alas, I do believe you're right. I guess that's why the whole thing came crashing down in the end. As regards not being able to have two versions of 'kernel32.dll' loaded at the same time, does not renaming the W2K version (theoretically) make this possible?

Joe.

Edited by jds, 15 February 2013 - 01:22 AM.


#103
jumper

jumper

    2014 All-American Masters HJ'er

  • Member
  • PipPipPip
  • 498 posts
  • Joined 21-January 11
  • OS:98SE
  • Country: Country Flag
> Ah, I see. And if I'm interpreting this correctly, the "ntdll.RtlDeleteCriticalSection" and similar functions seen in 'k2rnel32.dll' (W2K 'kernel32.dll') are export forwarder thingies.

I think so.

> So what we actually need (ignoring the other issues here) is an Export Patcher tool?

Today (Feb 19) is the one-year anniversary a short-lived Export Patcher project according to my folder creation date. I dropped it in favor of fwd: DLL forwarder which came out about a week later.

> As regards not being able to have two versions of 'kernel32.dll' loaded at the same time, does not renaming the W2K version (theoretically) make this possible?

The two versions both want to be loaded in overlapping address spaces, so one of them must be relocated. The 9x version is not relocatable, I don't think the NT version is either (look for a .reloc section with Quick View). The ReactOS version is, however (and also doesn't overlap)! :w00t:
Design feedback requested:
IHAtool - IpHlpApi tester; call various functions and report results
--status-> framework is solid; 22 api's fully supported; preview release coming soon
ComDlg32 wrapper - ComDlgEx meets IpHlpApi wrapper
--status-> PrintDlgExW working in latest SumatraPDF 8^)
Future projects: ImportPatcher40 - dialog interface; Kexter - IP40+Ktree+Kexstubs

#104
jds

jds

    -DOS+

  • Member
  • PipPipPipPip
  • 603 posts
  • Joined 03-June 08
  • OS:98SE
  • Country: Country Flag

> As regards not being able to have two versions of 'kernel32.dll' loaded at the same time, does not renaming the W2K version (theoretically) make this possible?

The two versions both want to be loaded in overlapping address spaces, so one of them must be relocated. The 9x version is not relocatable, I don't think the NT version is either (look for a .reloc section with Quick View). The ReactOS version is, however (and also doesn't overlap)! :w00t:

Hi jumper,

Wow! I didn't realize 'kernel32.dll' has to load at a specific address. That's a fly in the ointment! Hmmm ... back to ReactOS ... I had tried a similar experiment with those DLLs in the past with the same apparent result. But I only half know what I'm doing, so another attempt may be worthwhile ...

Joe.

#105
jumper

jumper

    2014 All-American Masters HJ'er

  • Member
  • PipPipPip
  • 498 posts
  • Joined 21-January 11
  • OS:98SE
  • Country: Country Flag
Searching the 279 files in my C:\WINDOWS\KernelEx folder (and its 5 subfolders!), I found 3 versions of Kernel32.dll that contained the text ".reloc"

+ bwc13i .... 699KB ... 5.00.2195.7152
+ bwc20a .... 703KB ... 5.00.2195.7173
+ ReactOS .. 1428KB ... 42.3.14
I suggest trying one of the blackwingcat versions.

Edited by jumper, 22 February 2013 - 02:28 AM.

Design feedback requested:
IHAtool - IpHlpApi tester; call various functions and report results
--status-> framework is solid; 22 api's fully supported; preview release coming soon
ComDlg32 wrapper - ComDlgEx meets IpHlpApi wrapper
--status-> PrintDlgExW working in latest SumatraPDF 8^)
Future projects: ImportPatcher40 - dialog interface; Kexter - IP40+Ktree+Kexstubs

#106
dencorso

dencorso

    Iuvat plus qui nihil obstat

  • Supervisor
  • 6,013 posts
  • Joined 07-April 07
  • OS:98SE
  • Country: Country Flag

Donator

Win XP SP3 kernel32.dll v. 5.1.2600.6293 (xpsp_sp3_qfe.121001-1624), which is the latest one available, does have .reloc information, and I presume previous versions might do, too, at least in the QFE branch...

#107
jumper

jumper

    2014 All-American Masters HJ'er

  • Member
  • PipPipPip
  • 498 posts
  • Joined 21-January 11
  • OS:98SE
  • Country: Country Flag
> Win XP SP3 kernel32.dll v. 5.1.2600.6293

Link is to a patcher. No Kernel32.dll file. :(
Design feedback requested:
IHAtool - IpHlpApi tester; call various functions and report results
--status-> framework is solid; 22 api's fully supported; preview release coming soon
ComDlg32 wrapper - ComDlgEx meets IpHlpApi wrapper
--status-> PrintDlgExW working in latest SumatraPDF 8^)
Future projects: ImportPatcher40 - dialog interface; Kexter - IP40+Ktree+Kexstubs

#108
dencorso

dencorso

    Iuvat plus qui nihil obstat

  • Supervisor
  • 6,013 posts
  • Joined 07-April 07
  • OS:98SE
  • Country: Country Flag

Donator

> Win XP SP3 kernel32.dll v. 5.1.2600.6293
Link is to a patcher. No Kernel32.dll file. :(


Of course the kernel32.dll (both the gdr and the qfe versions) are there! With all due respect, haven't you ever heard about Intra-Package Deltas-Aware Packages? :blink:

Do the following: under win 2k or higher, download file, put it in an arbitrary empty directory, then open a DOS-box there and run:

WindowsXP-KB2758857-x86-ENU /x:..\thefiles <Enter>

It'll fully unpack to the (newly created) thefiles subdirectory of your arbitrary empty directory.

#109
bphlpt

bphlpt

    MSFN Addict

  • Member
  • PipPipPipPipPipPipPip
  • 1,799 posts
  • Joined 12-May 07
  • OS:none specified
  • Country: Country Flag
The free app Universal Extractor can also easily open IPD's - Intra-Package Deltas-Aware Packages.

Home - http://www.legroom.n...ware/uniextract
Gora (part of their design/test team) modded discussion - http://www.ryanvm.ne...opic.php?t=9771
Repacked installer of Gora's mod (by Ricktendo) - http://www.wincert.n...6166d-gora-mod/

Cheers and Regards

Posted Image


#110
dencorso

dencorso

    Iuvat plus qui nihil obstat

  • Supervisor
  • 6,013 posts
  • Joined 07-April 07
  • OS:98SE
  • Country: Country Flag

Donator

But, for me at least, it's far from clear whether it's possible to correctly unpack Intra-Package Deltas-Aware Packages from 9x/ME. And from the NT-family, it's one of those rare cases in which the native functionality is quite enough, so I really see no advantage in using the Universal Extractor for this purpose. I do find the Universal Extractor awesome and wonderful, and do use it for lots of things, constantly, but not for this.

#111
jumper

jumper

    2014 All-American Masters HJ'er

  • Member
  • PipPipPip
  • 498 posts
  • Joined 21-January 11
  • OS:98SE
  • Country: Country Flag
> Of course the kernel32.dll (both the gdr and the qfe versions) are there!
gdr? qfe? What are those? :unsure:

> With all due respect, haven't you ever heard about Intra-Package Deltas-Aware Packages? :blink:
Sorry, but no. I'm a Win9x-only guy! :yes:

> Do the following: under win 2k or higher....
:puke: Surely you jest! :lol: Will it run on SE with KernelEx?

Using 7-zip, I can extract what appears to be a series of patch files from the package. Last year I studied one of these install packages and concluded that it must be run on a system that already contained the original file. I don't give this type of package a second look anymore.

If the Kernel32.dll file isn't going to be available to ordinary 9x users, using it as part of a general solution isn't really an option. Perhaps jds can try it in his current endeavor.
Design feedback requested:
IHAtool - IpHlpApi tester; call various functions and report results
--status-> framework is solid; 22 api's fully supported; preview release coming soon
ComDlg32 wrapper - ComDlgEx meets IpHlpApi wrapper
--status-> PrintDlgExW working in latest SumatraPDF 8^)
Future projects: ImportPatcher40 - dialog interface; Kexter - IP40+Ktree+Kexstubs

#112
dencorso

dencorso

    Iuvat plus qui nihil obstat

  • Supervisor
  • 6,013 posts
  • Joined 07-April 07
  • OS:98SE
  • Country: Country Flag

Donator

In a nutshell, gdr versions incorporate cumulatively all previous security patches including the latest mentioned in the respective KB document. Now, qfe versions include all that plus all the previous hotfixes (functionality patches, not security patches). Hardcore users prefer the qfe branch updates, but MS recommends the better tested gdr for general use (and that's what windows update site intalls by default, in principle, unless it already finds qfe files present). To make things still more complicated, of late MS decided to call qfe versions ldr, so that now there are three possible labels around, for the same version numbers, although files are issued either as qfe or ldr but never as both (which, I insist, at the end of the day, mean the same). :yes:

#113
loblo

loblo

    Oldbie

  • Member
  • PipPipPipPipPip
  • 763 posts
  • Joined 12-January 10
  • OS:ME
  • Country: Country Flag

In a nutshell, gdr versions incorporate cumulatively all previous security patches including the latest mentioned in the respective KB document. Now, qfe versions include all that plus all the previous hotfixes (functionality patches, not security patches). Hardcore users prefer the qfe branch updates, but MS recommends the better tested gdr for general use (and that's what windows update site intalls by default, in principle, unless it already finds qfe files present). To make things still more complicated, of late MS decided to call qfe versions ldr, so that now there are three possible labels around, for the same version numbers, although files are issued either as qfe or ldr but never as both (which, I insist, at the end of the day, mean the same). :yes:

Oh my! I'm glad I am a Win9x only user too. :lol:

#114
Drugwash

Drugwash

    MSFN Expert

  • Member
  • PipPipPipPipPipPip
  • 1,259 posts
  • Joined 21-June 06
  • OS:98SE
  • Country: Country Flag
I've been playing with Delta packages back in 2011, it's not that big of a deal once you get the idea. And actually, the idea is very clever: take a certain file, patch it and out goes another file that may have no relation whatsoever with the former.
The required API can be found in PatchApi.h, part of Platform SDK. I had built an AHK wrapper for the API but it was never completed, however it did work at the time. I'll be reviewing the code and maybe I can come up with a small toy to automatically unpack and apply the patch under Win9x.
BTW, the package can easily be unpacked with 7-zip, which is what I'm using in the script.

#115
bphlpt

bphlpt

    MSFN Addict

  • Member
  • PipPipPipPipPipPipPip
  • 1,799 posts
  • Joined 12-May 07
  • OS:none specified
  • Country: Country Flag
@Drugwash

On my Win7 x64 Ultimate system, I tried:

7-Zip (v9.20)
WinRAR (v4.20 64-bit)
FreeArc (v0.666)
WinZip Pro (v16.0 b9715 64-bit)
HaoZip (v2.6 b8450 x64)
PeaZip (v4.6)
PowerArchiver Pro 2011 (v12.12.02)

to extract the IPD that dencorso referenced:

Win XP SP3 kernel32.dll v. 5.1.2600.6293 (xpsp_sp3_qfe.121001-1624)


and, unless you know an extraction trick that I missed, none of them would extract it correctly. However, Universal Extractor (v1.6.1.62) and the Windows native method that decorso described both worked.

You should get the following file structure:

SP3GDR
...kernel32.dll
SP3QQFE
...kernel32.dll
update
...branches.inf
...eula.txt
...KB2758857.CAT
...spcustom.dll
...update.exe
...update.ver
...update_SP3GDR.inf
...update_SP3QFE.inf
...updatebr.inf
...updspapi.dll
spmsg.dll
spuninst.exe

Did I miss something?

Cheers and Regards

Posted Image


#116
PROBLEMCHYLD

PROBLEMCHYLD

    The Resurrector for old Windows OS

  • Member
  • PipPipPipPipPipPipPipPip
  • 2,531 posts
  • Joined 07-October 05
  • OS:98SE
  • Country: Country Flag

@Drugwash

On my Win7 x64 Ultimate system, I tried:

7-Zip (v9.20)
WinRAR (v4.20 64-bit)
FreeArc (v0.666)
WinZip Pro (v16.0 b9715 64-bit)
HaoZip (v2.6 b8450 x64)
PeaZip (v4.6)
PowerArchiver Pro 2011 (v12.12.02)

to extract the IPD that dencorso referenced:

Win XP SP3 kernel32.dll v. 5.1.2600.6293 (xpsp_sp3_qfe.121001-1624)


and, unless you know an extraction trick that I missed, none of them would extract it correctly. However, Universal Extractor (v1.6.1.62) and the Windows native method that decorso described both worked.

You should get the following file structure:

SP3GDR
...kernel32.dll
SP3QQFE
...kernel32.dll
update
...branches.inf
...eula.txt
...KB2758857.CAT
...spcustom.dll
...update.exe
...update.ver
...update_SP3GDR.inf
...update_SP3QFE.inf
...updatebr.inf
...updspapi.dll
spmsg.dll
spuninst.exe

Did I miss something?

Cheers and Regards

Me too :thumbup
WindowsXP-KB2758857-x86-ENU\SPMSG.DLL
WindowsXP-KB2758857-x86-ENU\SPUNINST.EXE
WindowsXP-KB2758857-x86-ENU\SP3GDR\KERNEL32.DLL
WindowsXP-KB2758857-x86-ENU\SP3QFE\KERNEL32.DLL
WindowsXP-KB2758857-x86-ENU\update\BRANCHES.INF
WindowsXP-KB2758857-x86-ENU\update\EULA.TXT
WindowsXP-KB2758857-x86-ENU\update\KB2758857.CAT
WindowsXP-KB2758857-x86-ENU\update\SPCUSTOM.DLL
WindowsXP-KB2758857-x86-ENU\update\UPDATE.EXE
WindowsXP-KB2758857-x86-ENU\update\UPDATE.VER
WindowsXP-KB2758857-x86-ENU\update\UPDATE_SP3GDR.INF
WindowsXP-KB2758857-x86-ENU\update\UPDATE_SP3QFE.INF
WindowsXP-KB2758857-x86-ENU\update\UPDATEBR.INF
WindowsXP-KB2758857-x86-ENU\update\UPDSPAPI.DLL

Believe God is the Alpha and Omega.
Believe Jesus Christ died for our sins.
Repent for your sins now or there will be
BLOOD

The Path to God


U98SESP3 03-11-2013


#117
bphlpt

bphlpt

    MSFN Addict

  • Member
  • PipPipPipPipPipPipPip
  • 1,799 posts
  • Joined 12-May 07
  • OS:none specified
  • Country: Country Flag
What method did you use to extract PROBLEMCHYLD?

Cheers and Regards

Edited by bphlpt, 23 February 2013 - 02:45 PM.

Posted Image


#118
Drugwash

Drugwash

    MSFN Expert

  • Member
  • PipPipPipPipPipPip
  • 1,259 posts
  • Joined 21-June 06
  • OS:98SE
  • Country: Country Flag
Any archiver will only unpack the Delta files. You get a bunch of _sfx_xxxx._p files (where xxxx can be 0000 to 9999), plus _sfx_manifest_._p and _sfx_.dll. Those are the source files that will further be processed to create the final files.
Win9x doesn't know how to natively apply the patches, therefore an external tool is needed, which is what I've been working on. Using it, one will end up with the correct folder structure, plus the original patch files.
There's a little more work to do before I can release the basic version. Stay tuned!

#119
dencorso

dencorso

    Iuvat plus qui nihil obstat

  • Supervisor
  • 6,013 posts
  • Joined 07-April 07
  • OS:98SE
  • Country: Country Flag

Donator

Way to go, Drugwash! I'd love to be able to do it on 9x/ME! :thumbup

@all: most of the time, the goodies are in the SP3QQFE subdirectory, and the rest can be ignored. Sometimes, however, there are .INFs worth looking at elsewhere in the final directory structure.

#120
PROBLEMCHYLD

PROBLEMCHYLD

    The Resurrector for old Windows OS

  • Member
  • PipPipPipPipPipPipPipPip
  • 2,531 posts
  • Joined 07-October 05
  • OS:98SE
  • Country: Country Flag

Way to go, Drugwash! I'd love to be able to do it on 9x/ME! :thumbup

@all: most of the time, the goodies are in the SP3QQFE subdirectory, and the rest can be ignored. Sometimes, however, there are .INFs worth looking at elsewhere in the final directory structure.

The skills that I have learned over the years is to use GDR versions. This is what Microsoft and MDGx uses, depending on the situation. This is what Unofficial Windows 98 Second Edition Service Pack 3.x consist of :w00t: This is why I trust Microsoft and MDGx most of the time, not cause they are always right (MOST OF THE TIME), but because they are(ACCURATE) :yes: :w00t: :D :)

Edited by PROBLEMCHYLD, 23 February 2013 - 05:05 PM.

Believe God is the Alpha and Omega.
Believe Jesus Christ died for our sins.
Repent for your sins now or there will be
BLOOD

The Path to God


U98SESP3 03-11-2013


#121
bphlpt

bphlpt

    MSFN Addict

  • Member
  • PipPipPipPipPipPipPip
  • 1,799 posts
  • Joined 12-May 07
  • OS:none specified
  • Country: Country Flag
Here's a quick summary of differences between GDR and QFE as I understand it:

GDR means General Distribution Release. They are released to the public on Windows Update and other places.

LDR/QFE updates ( Limited Distribution Release and Quick Fix Engineering ) are on another "branch" of development, and sometimes have to be requested. They usually are created to fix specific problems.
Think of it as "testing": you have the latest fixes, but also potentially new bugs or regressions.

When fixes of LDR/QFE are considered stable, they are merged in GDR branch. (the "stable" version )
The KB posted on Windows Update contains the two branches, so once you have installed LDR/QFE hotfixes, you're staying on this branch. (until the next service pack or if you uninstall any updates on LDR/QFE branch).

As far as which branch to use, it's really a matter of personal preference. Some prefer the GDR branch as there are fewer updates to integrate and they are slightly more stable, though they are not immune to issues. Others figure that all of the LDR/QFE updates will eventually be rolled into the GDR branch anyway, or the next SP if there ever is one, so you might as well be prepared. While there is a very small risk that new bugs are introduced, the cases where that has occurred are rare and they are usually corrected quickly with yet another update. LOL I personally have not run into a problem using the LDR/QFE branch.

Cheers and Regards

Edited by bphlpt, 24 February 2013 - 04:16 AM.

Posted Image


#122
Drugwash

Drugwash

    MSFN Expert

  • Member
  • PipPipPipPipPipPip
  • 1,259 posts
  • Joined 21-June 06
  • OS:98SE
  • Country: Country Flag
Guys, let me know if it works correctly or there's anything to change/add. Please note it only uses the ANSI versions of the API and some of the functions are not complete, so there's no testing available yet.

Here you are: MSPatchGUI 1.1.0.0

#123
dencorso

dencorso

    Iuvat plus qui nihil obstat

  • Supervisor
  • 6,013 posts
  • Joined 07-April 07
  • OS:98SE
  • Country: Country Flag

Donator

This is what Microsoft and MDGx uses, depending on the situation.

MS, that's for sure. MDGx , I don't think so... see this page. However it's true that the GDR branch is the safer one to use (and bphlpt's post two posts above this one explains in more detail what's involved).
@Drugwash: that's great news! I'll be testing it as soon as I manage to reboot into 98SE. Thanks a lot! :thumbup

#124
PROBLEMCHYLD

PROBLEMCHYLD

    The Resurrector for old Windows OS

  • Member
  • PipPipPipPipPipPipPipPip
  • 2,531 posts
  • Joined 07-October 05
  • OS:98SE
  • Country: Country Flag

This is what Microsoft and MDGx uses, depending on the situation.

MS, that's for sure. MDGx , I don't think so... see this page. However it's true that the GDR branch is the safer one to use.

If you notice I said depending on the situation. MDGx had some Unofficial hotfixes that was GDR. Not (ALL) but I know Microsoft hotfixes have both on most occasions. I guess its a toss up if you are specifically referring to Windows update or hotfixes. Windows update don't offer both versions but most hotfixes do.

Edited by PROBLEMCHYLD, 23 February 2013 - 11:20 PM.

Believe God is the Alpha and Omega.
Believe Jesus Christ died for our sins.
Repent for your sins now or there will be
BLOOD

The Path to God


U98SESP3 03-11-2013


#125
dencorso

dencorso

    Iuvat plus qui nihil obstat

  • Supervisor
  • 6,013 posts
  • Joined 07-April 07
  • OS:98SE
  • Country: Country Flag

Donator

True enough. Myself, I use preferentially QFE. I've had to fall back to GDR in one case already, but it really is a very rare happening.




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users