[RetroOS]

RetroOS, on Jan 24 2010, 09:26 AM, said:

...
Bugs:

SSL (https) pages in Internet Explorer 6 SP1 do not work. They start loading and then hang, or don't load at all.
The same pages in Firefox 3 work with no problem.

Also, in Internet Explorer 6 SP1, after browsing for only a short time, going to a new web page does not work.
The IE globe spins and nothing happens. At all... A restart of Windows is needed to get it working again.

The Save As... dialog box is having problems.
In Internet Explorer 6 SP1, it will hang for about a minute before showing the folder list.
In Office Word XP (2002), it hangs indefinitely.
...

The Save As.../Save dialog box problem seems to be gone now.
I'm not sure why, but I have never seen it before I updated to KernelEx 4.5 Beta 1.
It may be an intermittant KernelEx 4.5 Beta 1 problem.

The other IE problems are still outstanding.

[Tihiy]

IE bugs and other reported crashes were all confirmed and being fixed, don't worry.

[SlugFiller]

Woo! Inkscape finally works! Pango-Cairo finally works!

The bad news are, I don't need it anymore. I've found a much better SVG library for Java, and for my browser I've switched to K-Meleon, the Win32 port of the Gecko engine, which works just fine on 9x (actually, I should say "it works faster than any browser I've used to date", but, you know...).

Well, apps have never been a good reason to do hard OS works. Someone, somewhere, has already made a more compatible and better featured version.

So I'll be testing out various games soon. Will report if I find anything of interest.

By the way, if I'm reading the source correctly, there is no more need for code generation to create a new API in v4. Rather, the APIs are compiled directly. Is that correct?
I may consider porting my filename extra compatibility filter to the new architecture. Actually, from the looks of it, it won't require too much porting.

[CharlesF]

Hi Tihiy and Xeno86,

nice job once again! KernelEx v 4.5 Beta 1 working nice here!

* No more crash with FSViewer

* Maybe a small glitch I didn't noticed before (so I'm not sure that it is coming from KernelEx, could be from RP9 or else): transparency of skins is broken in WMP9. Have a look at the screenshots.

* BTW, do we have to uninstall previous version of KernelEx first, or we can install the new one 'on the top'? (I didn't find an answer on this thread).

* @Chozo4: try this: in Compatibility tab of IEXPLORE.EXE, tick first (in Advanced options) 'Don't use these settings in child processes' before 'Disable KernelEx extensions'. I have no problem with Shockwave 10 and Flash 10 players, or to post here (but https pages still do not work).

* Is there a way to make last VLC 1.05 working?: the exe is running, but no more displaying the text of the menus.

Charles.

Attached File(s)

•  Wmp9_Splat.png (44.38K)
  Number of downloads: 5
•  Wmp9_Ducky.png (18.67K)
  Number of downloads: 4

[SlugFiller]

Hmm... I've been looking up information on VxDs recently, and found out something interesting. Apparently, Win9x's file system manager, ifsmgr.vxd, as part of Microsoft's effort to be backwards compatible with NT4, uses unicode strings to identify resources. The kernel-mode ZwCreateFile takes a unicode string, which may be generated from a wide-char string, and this apparently applies to 9x.

So, the theory is, if these calls are made directly from a KEx API, 9x can be made to support real unicode versions of CreateFile and similar functions. In other words, 9x can support filenames in any given locale on any given version. And this does not require any VxD patching or rewriting.

Now, obviously there are a few issues.
The first of all being, can Kernel32 truly call VxD or DDK functions directly?
Secondly, what sort of conversion might be required between a handle returned by the kernel-mode ZwCreateFile, and the handle used by user applications for calls such as FileReadA? Are they identical? Does Kernel32 keep its own objects and/or handles?

Short of decompiling Kernel32 (I've yet to find a half-descent PE-file disassembler), I guess the only way to test is with trial and error. Of course, kernel-level error can be very risky.

One thing I was wondering about is if the pre-application KEx configuration can be extended. If true unicode file access is a possibility, it would be nice to set the code page for ANSI file functions, such as CreateFileA, on a per-program basis. But there is a question of whether that configuration can be read from the overridden CreateFileA itself (or if that would create some sort of infinite loop).

P.S. Does anyone ever read my posts? I never seem to get a reply, and I can't help but wonder if they are even visible to other people.

[Tihiy]

Quote

The kernel-mode ZwCreateFile takes a unicode string, which may be generated from a wide-char string, and this apparently applies to 9x.

Yet to be tested. I'm afraid it's not that easy.

Quote

The first of all being, can Kernel32 truly call VxD or DDK functions directly?

Only those which "win32 services" are registered for. Not much for ntkern.

Quote

Secondly, what sort of conversion might be required between a handle returned by the kernel-mode ZwCreateFile, and the handle used by user applications for calls such as FileReadA? Are they identical? Does Kernel32 keep its own objects and/or handles?

9x already has hideous support for NT device namespace. For example, you can do CreateFileA \\?\C:\file.txt and you'll receive handle for file opened with NT minikernel. This handle is limited to only certain apis so reimplementing whole certain apis and kernel driver will be required.

[SlugFiller]

Tihiy, on Feb 4 2010, 10:17 AM, said:

Quote

The kernel-mode ZwCreateFile takes a unicode string, which may be generated from a wide-char string, and this apparently applies to 9x.

Yet to be tested. I'm afraid it's not that easy.

I'm basing this on the FileMon code. I've actually ran FileMon on my 9x, so I know it works.

If I knew how to load and call VxD functions directly from KernelEx, it wouldn't be too difficult for me to create my own VxD to do the Zw stuff, if necessary.

Hmm... In fact, maybe I can create a namespace which takes hexa-decimal strings, and converts them to unicode, then relays to the appropriate sub-driver. Then, the only thing I would have to do is add a bin2hex in CreateFileW, and pass the result to CreateFileA. The question is, would that method be enough to support all basic file operations?

[Tihiy]

Quote

Yet to be tested. I'm afraid it's not that easy.

I've tested and unfortunately it's a failure. Calling NtCreateFile with unicode characters just turns them into underscores (any non-english char).

[CharlesF]

taunus20m, on Feb 4 2010, 07:33 PM, said:

My Windows Media Player 9 crashes after few seconds I playa mp3-music.

Hi taunus20m,

I have tried with mine (WMP9 upgraded with MP9URP9X.EXE from erpdude8), and no problem here.
I have tried with two files: Mpeg1 layer3 and Mpeg2 layer3.

[SlugFiller]

What do you mean by "turns them"? Doesn't it call IFSMGR directly (Which, AFAIK, takes unicode device names)?
What did you test exactly?

Fat32 uses UTF16 long file names, so somewhere between CreateFileA and the IO subsystem, the ANSI filename is converted to UTF16 (using a distribution-dependent codepage). The only question is where. If I knew the answer to that, I could patch it area to use a more comprehensive codepage, such as UTF8. Then supporting arbitrarily localized filenames would be simple.

Since I've recently tested replacing my ifsmgr.vxd with localized versions (those were hard to track down), to no effect, I think it is safe to assume it is locale-independent.
This does leave the possibility that the conversion occurs -after- ifsmgr.vxd, inside vfat.vxd. I did try replacing it, to no effect, but I didn't choose highly varied locales there, so maybe it's just a coincidence. Still, it doesn't seem too likely, considering it gets its input as a unicode string already.
Unfortunately, I couldn't get a good description of the path taken between Kernel32 and ifsmgr. Like I've said, I've tried finding a good DLL-capable disassembler to get a better view of the CreateFileA implementation, with no success.

I did start poking around my copy of the DDK to try and see if I could write a file-system driver. I figured, I could, at the very least, write a driver that takes the filenames as hex-encoded or base64-encoded strings, and converts it to file name using some UTF convention. I could then use that to either call another driver (one which supports unicode filenames), or implement my own version of the FAT32 file-system.
Unfortunately, I could find any good examples of file system implementations using the DDK. I did find one which uses VtoolsD, so maybe that can help. It is also quite disturbing that not one of the DDK examples is written in pure C or C++, all use assembler code.

[Joseph_sw]

SlugFiller, on Feb 5 2010, 06:10 AM, said:

Fat32 uses UTF16 long file names, so somewhere between CreateFileA and the IO subsystem, the ANSI filename is converted to UTF16 (using a distribution-dependent codepage). The only question is where. If I knew the answer to that, I could patch it area to use a more comprehensive codepage, such as UTF8. Then supporting arbitrarily localized filenames would be simple.

Since I've recently tested replacing my ifsmgr.vxd with localized versions (those were hard to track down), to no effect, I think it is safe to assume it is locale-independent.

is theres also posibilty that conversion were done by kernel32.dll itself (somewhat semi hard-coded) ?

as in the kernel32.dll file also contain some data structure of those cp_####.nls files
the 8->16 CodePage to UTF16,
and 16->16 UTF16 to CodePage data structures (using to those three-layered lookup table)

thanks to your earlier post about the .nls files, i found that fact in kernel32.dll

[slhk]

Used 4.5 Beta 1 for a period of time. Here is my report

1. Previous working programs continue to work

2. Inkscape: It runs when compatability is set to 2000, but cannot open or save file (it hangs)

3. Foxit Reader: It runs but cannot print. It says "no printer is installed"

4. Arora: Cannot run. Page fault in QTWEBKIT4.DLL. Could someone tell me how to make it run?

Thank you

[Tihiy]

Quote

What do you mean by "turns them"? Doesn't it call IFSMGR directly (Which, AFAIK, takes unicode device names)? What did you test exactly?

I'm not sure how ntkern really works. But 98 kernel has (K32_)NtCreateFile which calls ntkern one and takes unicode strings. So no, your idea is not working. You have to call ifsmgr directly somehow.

[SlugFiller]

@Tihiy: I was actually wondering more about your experiment process. Did you try to create a unicode-named file, only to see it created with underscores? Did you verify the underscores were actually in the filename, and not just in display? (If it is just in display, the file would not load in any ANSI program)

@Joseph_sw: I've already replaced my Kernel32. I've also messed around with the NLS files. They effect the GUI behavior, but not the file system.
Having a mismatch between the GUI code page and the FS code page just causes files of either locale to be inaccessible.

Well, I've done some more research. Apparently, IFSMgr has an ANSI API which is used by Kernel32. (I think it also has a Unicode API, but it's hard to get concrete details on IFSMgr's API)
When it receives an ANSI filename, it makes a conversion using tables located in Unicode.bin (One file for all CPs, now there's a bottleneck). I've modified it to add a few extra code pages, and successfully gotten access to localized files. It uses the same registry key as the GUI code page, but is more limited since it uses a single data file (instead of the pluggable NLS files).
It does require restart every time I want to change code pages, and every time I only have access to some of my files. It only support encodings up to 2 bytes per character, so creating a UTF-8 code page is out of the question (unless I want to recode IFSMgr itself).

Still, there should be a way to access IFSMgr using Unicode calls, either directly from the kernel, or from a dedicated VxD. If I could find more information about its API, maybe I could get true Unicode file access to work.

One thing I did notice was that Explorer isn't exactly "well behaved" when it comes to double-byte filenames. It displays them okay in the list, but improperly when renaming. It also fails to start up localized files when clicked (DnD into an application works, though). I wonder if there's an XP Unicode version of Explorer I could use or something.

[Joseph_sw]

SlugFiller, on Feb 6 2010, 12:25 AM, said:

Having a mismatch between the GUI code page and the FS code page just causes files of either locale to be inaccessible.

Well, I've done some more research. Apparently, IFSMgr has an ANSI API which is used by Kernel32. (I think it also has a Unicode API, but it's hard to get concrete details on IFSMgr's API)

When it receives an ANSI filename, it makes a conversion using tables located in Unicode.bin (One file for all CPs, now there's a bottleneck). I've modified it to add a few extra code pages, and successfully gotten access to localized files. It uses the same registry key as the GUI code page, but is more limited since it uses a single data file (instead of the pluggable NLS files).
It does require restart every time I want to change code pages, and every time I only have access to some of my files. It only support encodings up to 2 bytes per character, so creating a UTF-8 code page is out of the question (unless I want to recode IFSMgr itself).

i'm definetely interested about modifiying the unicode.bin myself, as i still have grudge on CP-1252 about ansi/ascii character that can't be used as filenames.

now, where do i start? what i need to know about unicode.bin' data structures?

[SlugFiller]

I've made a small Java program for creating my unicode.bin. Attached below. It is roughly 400 lines of code, plus 60000 lines of CP data.
It would be easier than trying to make such a program yourself. The bin uses a tree structure to define lookup ranges, and my program automatically sorts it out as a binary tree.
It could be slightly improved by not creating a new range for a gap of less than 6 characters, but instead padding with underscores. This would create a slightly smaller bin.

 MakeUnicodeBin.rar (199.35K)
Number of downloads: 11

But, like I said, I'm looking for a more comprehensive solution. I am contemplating using the file system hook to convert base64 or hex filenames into unicode ones. To ensure compatibility, though, I want it to use a special device name which wouldn't collide with real devices. But I'm not sure whether the hook is even called for device names which don't exit.
The alternative is to use Ring0 FileIO. This is a bit more complex, since it requires quite some book-keeping, and consideration of edge-cases.
Unfortunately, developing VxDs is VERY complex. I still have a lot to learn before I can properly start testing.

If I am successful, though, patching KEx would be incredibly easy. The new architecture is even easier to extend and deploy than the previous one. Making file APIs which attempt to detect a hex-to-unicode helper device is actually quite simple.

[kocoman]

Does anyone experience graphics corruption with the 4.5 beta 1 and remote desktop connection for win98?

[SlugFiller]

I've finally managed to compile and run the VxD equivalent of "Hello world", coded in C++ using 98DDK and VS6's CL. This will give me the ability to experiment a bit, and see if I can get something akin to Unicode file access, one way or another.
One issue is that once the VxD is loaded, it cannot be unloaded or reloaded, even if the file is modified. The only solution is to restart the computer for every revision. That gets old really fast.

P.S. 8 downloads, 0 comments

[Joseph_sw]

SlugFiller, on Feb 7 2010, 10:35 AM, said:

P.S. 8 downloads, 0 comments

oh sorry, i have been busy trying to port that java into Delphi as that remaining programing language that were still installed on my computer, i lost my java se installer somehow.
i probably gonna need the unicode.bin result from that java program, just to compare wether my port attempt were done correctly or not.
currently i just finished porting the CodePage class, so i haven't tried how the new unicode.bin will works on the system.

question:
1. is there any limit in maximum size unicode.bin? i intend to add all codepage for every existing cp_###.nls in my computer.
original unicode.bin were only contain 30 codepages, you example was 9 codepages, in my system theres 36 cp_####.nls files.

2. in prgram given it seem you deliberately skip cp.addMapSingle() for cp 0x81, 0x8D, 0x8F, 0x90, 0x9D, for cp1252 why is that?
that were incidentaly chracters that can't be used as filenames in cp_1252.

the differences between cp_1252.nls and cp-1252 in original unicode.bin was 0x8E & 0x9E which also can't be used in filenames.


anyway that java code were very informative , thanks to that i now know about how unicode.bin should be read/inspected.
while unicode.bin contain 30 cps, i look in win98se .cab files, it only provided 20 cp_####.nls files, hmm...

This post has been edited by Joseph_sw: 07 February 2010 - 02:56 AM