• Announcements

    • xper

      MSFN Sponsorship and AdBlockers!   07/10/2016

      Dear members, MSFN is made available via subscriptions, donations and advertising revenue. The use of ad-blocking software hurts the site. Please disable ad-blocking software or set an exception for MSFN. Alternatively, become a site sponsor and ads will be disabled automatically and by subscribing you get other sponsor benefits.
Xeno86

KernelEx 4.5.2

1,561 posts in this topic

What do you mean by "turns them"? Doesn't it call IFSMGR directly (Which, AFAIK, takes unicode device names)?

What did you test exactly?

Fat32 uses UTF16 long file names, so somewhere between CreateFileA and the IO subsystem, the ANSI filename is converted to UTF16 (using a distribution-dependent codepage). The only question is where. If I knew the answer to that, I could patch it area to use a more comprehensive codepage, such as UTF8. Then supporting arbitrarily localized filenames would be simple.

Since I've recently tested replacing my ifsmgr.vxd with localized versions (those were hard to track down), to no effect, I think it is safe to assume it is locale-independent.

This does leave the possibility that the conversion occurs -after- ifsmgr.vxd, inside vfat.vxd. I did try replacing it, to no effect, but I didn't choose highly varied locales there, so maybe it's just a coincidence. Still, it doesn't seem too likely, considering it gets its input as a unicode string already.

Unfortunately, I couldn't get a good description of the path taken between Kernel32 and ifsmgr. Like I've said, I've tried finding a good DLL-capable disassembler to get a better view of the CreateFileA implementation, with no success.

I did start poking around my copy of the DDK to try and see if I could write a file-system driver. I figured, I could, at the very least, write a driver that takes the filenames as hex-encoded or base64-encoded strings, and converts it to file name using some UTF convention. I could then use that to either call another driver (one which supports unicode filenames), or implement my own version of the FAT32 file-system.

Unfortunately, I could find any good examples of file system implementations using the DDK. I did find one which uses VtoolsD, so maybe that can help. It is also quite disturbing that not one of the DDK examples is written in pure C or C++, all use assembler code.

0

Share this post


Link to post
Share on other sites
Fat32 uses UTF16 long file names, so somewhere between CreateFileA and the IO subsystem, the ANSI filename is converted to UTF16 (using a distribution-dependent codepage). The only question is where. If I knew the answer to that, I could patch it area to use a more comprehensive codepage, such as UTF8. Then supporting arbitrarily localized filenames would be simple.

Since I've recently tested replacing my ifsmgr.vxd with localized versions (those were hard to track down), to no effect, I think it is safe to assume it is locale-independent.

is theres also posibilty that conversion were done by kernel32.dll itself (somewhat semi hard-coded) ?

as in the kernel32.dll file also contain some data structure of those cp_####.nls files

the 8->16 CodePage to UTF16,

and 16->16 UTF16 to CodePage data structures (using to those three-layered lookup table)

thanks to your earlier post about the .nls files, i found that fact in kernel32.dll

0

Share this post


Link to post
Share on other sites

Used 4.5 Beta 1 for a period of time. Here is my report

1. Previous working programs continue to work

2. Inkscape: It runs when compatability is set to 2000, but cannot open or save file (it hangs)

3. Foxit Reader: It runs but cannot print. It says "no printer is installed"

4. Arora: Cannot run. Page fault in QTWEBKIT4.DLL. Could someone tell me how to make it run?

Thank you

0

Share this post


Link to post
Share on other sites
What do you mean by "turns them"? Doesn't it call IFSMGR directly (Which, AFAIK, takes unicode device names)? What did you test exactly?
I'm not sure how ntkern really works. But 98 kernel has (K32_)NtCreateFile which calls ntkern one and takes unicode strings. So no, your idea is not working. You have to call ifsmgr directly somehow.
0

Share this post


Link to post
Share on other sites

@Tihiy: I was actually wondering more about your experiment process. Did you try to create a unicode-named file, only to see it created with underscores? Did you verify the underscores were actually in the filename, and not just in display? (If it is just in display, the file would not load in any ANSI program)

@Joseph_sw: I've already replaced my Kernel32. I've also messed around with the NLS files. They effect the GUI behavior, but not the file system.

Having a mismatch between the GUI code page and the FS code page just causes files of either locale to be inaccessible.

Well, I've done some more research. Apparently, IFSMgr has an ANSI API which is used by Kernel32. (I think it also has a Unicode API, but it's hard to get concrete details on IFSMgr's API)

When it receives an ANSI filename, it makes a conversion using tables located in Unicode.bin (One file for all CPs, now there's a bottleneck). I've modified it to add a few extra code pages, and successfully gotten access to localized files. It uses the same registry key as the GUI code page, but is more limited since it uses a single data file (instead of the pluggable NLS files).

It does require restart every time I want to change code pages, and every time I only have access to some of my files. It only support encodings up to 2 bytes per character, so creating a UTF-8 code page is out of the question (unless I want to recode IFSMgr itself).

Still, there should be a way to access IFSMgr using Unicode calls, either directly from the kernel, or from a dedicated VxD. If I could find more information about its API, maybe I could get true Unicode file access to work.

One thing I did notice was that Explorer isn't exactly "well behaved" when it comes to double-byte filenames. It displays them okay in the list, but improperly when renaming. It also fails to start up localized files when clicked (DnD into an application works, though). I wonder if there's an XP Unicode version of Explorer I could use or something.

0

Share this post


Link to post
Share on other sites
Having a mismatch between the GUI code page and the FS code page just causes files of either locale to be inaccessible.

Well, I've done some more research. Apparently, IFSMgr has an ANSI API which is used by Kernel32. (I think it also has a Unicode API, but it's hard to get concrete details on IFSMgr's API)

When it receives an ANSI filename, it makes a conversion using tables located in Unicode.bin (One file for all CPs, now there's a bottleneck). I've modified it to add a few extra code pages, and successfully gotten access to localized files. It uses the same registry key as the GUI code page, but is more limited since it uses a single data file (instead of the pluggable NLS files).

It does require restart every time I want to change code pages, and every time I only have access to some of my files. It only support encodings up to 2 bytes per character, so creating a UTF-8 code page is out of the question (unless I want to recode IFSMgr itself).

i'm definetely interested about modifiying the unicode.bin myself, as i still have grudge on CP-1252 about ansi/ascii character that can't be used as filenames.

now, where do i start? what i need to know about unicode.bin' data structures?

0

Share this post


Link to post
Share on other sites

I've made a small Java program for creating my unicode.bin. Attached below. It is roughly 400 lines of code, plus 60000 lines of CP data.

It would be easier than trying to make such a program yourself. The bin uses a tree structure to define lookup ranges, and my program automatically sorts it out as a binary tree.

It could be slightly improved by not creating a new range for a gap of less than 6 characters, but instead padding with underscores. This would create a slightly smaller bin.

MakeUnicodeBin.rar

But, like I said, I'm looking for a more comprehensive solution. I am contemplating using the file system hook to convert base64 or hex filenames into unicode ones. To ensure compatibility, though, I want it to use a special device name which wouldn't collide with real devices. But I'm not sure whether the hook is even called for device names which don't exit.

The alternative is to use Ring0 FileIO. This is a bit more complex, since it requires quite some book-keeping, and consideration of edge-cases.

Unfortunately, developing VxDs is VERY complex. I still have a lot to learn before I can properly start testing.

If I am successful, though, patching KEx would be incredibly easy. The new architecture is even easier to extend and deploy than the previous one. Making file APIs which attempt to detect a hex-to-unicode helper device is actually quite simple.

0

Share this post


Link to post
Share on other sites

Does anyone experience graphics corruption with the 4.5 beta 1 and remote desktop connection for win98?

0

Share this post


Link to post
Share on other sites

I've finally managed to compile and run the VxD equivalent of "Hello world", coded in C++ using 98DDK and VS6's CL. This will give me the ability to experiment a bit, and see if I can get something akin to Unicode file access, one way or another.

One issue is that once the VxD is loaded, it cannot be unloaded or reloaded, even if the file is modified. The only solution is to restart the computer for every revision. That gets old really fast.

P.S. 8 downloads, 0 comments

0

Share this post


Link to post
Share on other sites
P.S. 8 downloads, 0 comments

oh sorry, i have been busy trying to port that java into Delphi as that remaining programing language that were still installed on my computer, i lost my java se installer somehow.

i probably gonna need the unicode.bin result from that java program, just to compare wether my port attempt were done correctly or not. :whistle:

currently i just finished porting the CodePage class, so i haven't tried how the new unicode.bin will works on the system.

question:

1. is there any limit in maximum size unicode.bin? i intend to add all codepage for every existing cp_###.nls in my computer.

original unicode.bin were only contain 30 codepages, you example was 9 codepages, in my system theres 36 cp_####.nls files.

2. in prgram given it seem you deliberately skip cp.addMapSingle() for cp 0x81, 0x8D, 0x8F, 0x90, 0x9D, for cp1252 why is that?

that were incidentaly chracters that can't be used as filenames in cp_1252.

the differences between cp_1252.nls and cp-1252 in original unicode.bin was 0x8E & 0x9E which also can't be used in filenames.

anyway that java code were very informative :thumbup , thanks to that i now know about how unicode.bin should be read/inspected.

while unicode.bin contain 30 cps, i look in win98se .cab files, it only provided 20 cp_####.nls files, hmm...

Edited by Joseph_sw
0

Share this post


Link to post
Share on other sites

Why are you using Delphi? JDK is available for free from the Sun Java website. And it's a much better language too, although that doesn't really say much. I mean Pascal+Visual+Proprietary-Any chance of cross-platform compatibility=Worthless language.

According to the documentation I got, the original 95 IFSMgr had a bug in that it cache the header of the unicode.bin, but its cache was only large enough to contain 18 language entries. The documentation was a bit unclear, but it might be able to work with more languages, provided the OEM and ANSI code pages are on the same 18-language block. Certain localized versions did not have the bug.

I'm not sure about the status of the 98 driver, although I imagine it probably fixed the bug already. The format itself should support up to a 4GB file.

I didn't feel like pushing it to the limit. My original unicode.bin only had 5 code pages, and I've added an extra 4 (all East Asians). Your 30-pages bin should probably go on mgdx or something. Could you list the code pages you have there?

The code page data itself is from the Microsoft contribution to unicode.org. I made a small javascript page which converts the text files into the necessary Java commands. Obviously, I didn't sit down and manually input 60000 character codes.

So any "blanks" in the data are the official Microsoft stance on the code page. There are no coincidences there, if it matches what you already have.

0

Share this post


Link to post
Share on other sites

If you're programming something that is Windows-specific, cross-platform portability is useless. There are speed and resource use benefits to pretty much all "Pascal+Visual+Proprietary" languages over Java.

Queue

0

Share this post


Link to post
Share on other sites

Could someone make a step by step tutorial on how to install the Flash Player 10 plugin with Firefox 3.6? The KernelEx 4.5 Beta did install successfully but even with the compatibility, it just won't install. I tried installing it on Firefox from the Adobe site but all I got is the Adobe DLM extension (v1.6.2.60) and the getplus® packaged with it doesn't run on 98SE. I tried installing the Flash Player plugin beta but that doesn't work in IE6 OR Firefox 3.6 :angry:

Any help?

0

Share this post


Link to post
Share on other sites

I can't install all of the 4.x version KernelEx on Japanese edition of Windows ME.

I got the following error message:

exportfromX: pattern not found

The following pattern is in KERNEL32.DLL of Japanese WinME:

0F BF 40 10 8B 14 81 8B 44 24 14 3D 00 00 01 00 8B 4A 04 73 15 39 1D

The pattern of KernelEx source setup/setup.cpp line 369-370:

0x0F,0xBF,0x40,0x10,0x8B,0x14,0x81,0x8B,0x44,

0x24,0x14,0x3D,0x00,0x00,0x01,0x00,0x8B,0x4A,0x04,0x73,0x15,0x3B,0x1D,

Version 0.3.x installer works fine.

0

Share this post


Link to post
Share on other sites

@Queue: "Visual", by definition, denies any "speed and resource benefits". It is probably many times slower than JIT-compiled Java code, and while I can't testify as to the library size, the memory usage is undoubtedly much higher. It was one thing if this was C, then you'd have good performance, and it wouldn't be proprietary. It would also be easily portable, although not necessarily cross-platform (depends on whether the C runtime is used, or the Win32API).

Also, I hope I don't have to explain the issues with a language with a proprietary dynamically linked run-time (although I suppose Windows may also be counted as a "proprietary runtime"). Especially not on this forum, in this thread (*cough* VCRun2008 *cough*).

0

Share this post


Link to post
Share on other sites

If you like Pascal, I would recommend Free Pascal and Lazarus. You will have Pascal and Object Pascal, a Delphi-like GUI and a very wide portability between different platforms (Windows, Linux, Windows CE, BSD, MacOS X, AmigaOS... even Nintendo DS!) and processors (x86, AMD64 aka x86 64, ARM, PowerPC, 68k, SPARC...).

JDK? Seriously, Java is not a sane language for people that likes Pascal or C/C++ languages, despite being massively used by market(ing) reasons. The development methodology of Java is different and certain developers are not comfortable with it or feel a lot productive not using it.

I'm not against Java as a programming language, just I avoid apps running into bytecodes because generally being highly inefficient and prefer to use native code instead.

About support: What about Chromium? It's Open Source, so maybe it can be even patched for Win 9x support. Don't get me wrong, I'm not a developer and just suggesting it :)

0

Share this post


Link to post
Share on other sites

@Queue, SlugFiller and timofonic: This is the KernelEx thread, which is, by itself, already huge! Let's not get off-topic!

If you want it badly, let me know and I'll split your latest posts into a new thread (in this case do PM me about it). But I'd much rather avoid the potential development of yet another this-vs.-that flamefest, if you don't mind letting it go.

0

Share this post


Link to post
Share on other sites
I didn't feel like pushing it to the limit. My original unicode.bin only had 5 code pages, and I've added an extra 4 (all East Asians). Your 30-pages bin should probably go on mgdx or something. Could you list the code pages you have there?
well, the unicode.bin that i have was exactly came from the win98_22.cab file.

i also have my winME installer ready, unicode.bin from win_8.cab which also binary exact match with the win98se (win98_22.cab) counter part.

this the listing codepages found in that unicode.bin

 1 -  437 - 0034C 0022C 1 - 00578 0027D   12
 2 - 850 - 007F5 0022C 1 - 00A21 001D9 6
 3 - 1252 - 00BFA 0022C 1 - 00E26 001AA 10
 4 - 1250 - 00FD0 0022C 1 - 011FC 008C4 137
 5 - 1251 - 01AC0 0022C 1 - 01CEC 00643 88
 6 - 1253 - 0232F 0022C 1 - 0255B 006B5 99
 7 - 1254 - 02C10 0022C 1 - 02E3C 008A1 134
 8 - 1257 - 036DD 0022C 1 - 03909 00395 55
 9 - 737 - 03C9E 0022C 1 - 03ECA 003C1 56
10 - 852 - 0428B 0022C 1 - 044B7 00881 132
11 - 855 - 04D38 0022C 1 - 04F64 0061B 85
12 - 857 - 0557F 0022C 1 - 057AB 00890 133
13 - 866 - 0603B 0022C 1 - 06267 0061C 85
14 - 869 - 06883 0022C 1 - 06AAF 0064E 92
15 - 775 - 070FD 0022C 1 - 07329 00243 5
16 - 874 - 0756C 0022C 1 - 07798 00191 12
17 - 1258 - 07929 0022C 1 - 07B55 002E1 40
18 - 860 - 07E36 0022C 1 - 08062 0093C 144
19 - 861 - 0899E 0022C 1 - 08BCA 0093C 144
20 - 863 - 09506 0022C 1 - 09732 0093C 144
21 - 865 - 0A06E 0022C 1 - 0A29A 0093C 144
22 - 1255 - 0ABD6 0022C 1 - 0AE02 0023A 26
23 - 862 - 0B03C 0022C 1 - 0B268 009FA 160
24 - 1256 - 0BC62 0022C 1 - 0BE8E 0031F 44
25 - 708 - 0C1AD 0022C 1 - 0C3D9 001F0 9
26 - 710 - 0C5C9 0022C 1 - 0C7F5 00245 20
27 - 711 - 0CA3A 0022C 1 - 0CC66 001F9 14
28 - 720 - 0CE5F 0022C 1 - 0D08B 0020B 12
29 - 786 - 0D296 0022C 1 - 0D4C2 001F3 13
30 - 864 - 0D6B5 0022C 1 - 0D8E1 00218 14

listed in order of appearances in original unicode.bin that i have

Code page - Offset Length Ranges (cp to uni) - Offset Length Ranges (uni to cp)

i suspect that localized version of win9x might have different content of unicode.bin going on.

ack, its make me want to hunt for that files.

0

Share this post


Link to post
Share on other sites
...

* Is there a way to make last VLC 1.05 working?: the exe is running, but no more displaying the text of the menus.

Charles.

VLC 1.0.5 works for me.

However, I had the same problem you describe, back in 1.0.3.

It turned out to be the font I was using for Windows text.

After I changed it to another font, VLC displayed correctly.

Try changing to another desktop theme to test, or just change the fonts in display appearance settings.

I suspect that some 'custom' fonts may not support Unicode and therefore extended characters are blank/missing like in VLC.

I could be wrong... :whistle:

0

Share this post


Link to post
Share on other sites

Since it appears that Kernel EX was first designed to run Firefox 3 and above on W98SE machines why not address the biggest problem (the problem that forced me to go back to Firefox 2 with Kernel EX extensions disabled) which is that many of us are unable to Print from Firefox 3, and in my case the e-mail client (mailto) wouldn't work in Firefox 3 under any circumstances nor in Firefox 2 until I disabled Kernel EX when Firefox 2 loads?

Opera 10 worked fine.

John

Edited by craftech
0

Share this post


Link to post
Share on other sites
the e-mail client (mailto) wouldn't work in Firefox 3 under any circumstances nor in Firefox 2
I've researched this issue and - that may sound stupid - it's really mimicking XP behavior. SHParseDisplayName check they added in some security update actually may fail to process mailto: links. Nevertheless it's a serious issue and it will be fixed soon. And if you want printing, what you wanna print?
0

Share this post


Link to post
Share on other sites
After I changed it to another font, VLC displayed correctly.
Wonderful! I have only switched the title bar font from Tahoma to Verdana,

and VLC is now perfect. :thumbup

Thanks RetroOS.

I suspect that some 'custom' fonts may not support Unicode and therefore extended characters are blank/missing like in VLC.
I think Tahoma must support Unicode.

Oddly, I went to my Font folder and it is displaying two times the Tahoma font: same TAHOMA.TTF, same size and same time stamp.

It is the only font I see in double.

Do you know how to correct that?

0

Share this post


Link to post
Share on other sites
After I changed it to another font, VLC displayed correctly.
Wonderful! I have only switched the title bar font from Tahoma to Verdana,

and VLC is now perfect. :thumbup

Thanks RetroOS.

I suspect that some 'custom' fonts may not support Unicode and therefore extended characters are blank/missing like in VLC.
I think Tahoma must support Unicode.

Oddly, I went to my Font folder and it is displaying two times the Tahoma font: same TAHOMA.TTF, same size and same time stamp.

It is the only font I see in double.

Do you know how to correct that?

I noticed the two Tahoma fonts a while ago...

One is listed as TAHOMA.TTF and the other as tahoma.ttf. The same size, date, and time.

I also have some other duplicate fonts, some with identical font name and file name, and others with variations.

I changed all my fonts to Trebuchet MS and that worked.

0

Share this post


Link to post
Share on other sites
I suspect that some 'custom' fonts may not support Unicode and therefore extended characters are blank/missing like in VLC.
I changed all my fonts to Trebuchet MS and that worked.
Thanks RetroOS for your help!

The problem comes from the font 'MS Sans Serif' (SSERIFE.FON) which is not a TrueType but a Fixed font, and doesn't support Unicode.

This font is the default system font in native Windows 98!

I didn't found a workaround except to use another font (what you did) for each Item in Display Properties->Appearance.

If one want to keep the same font, he can switch to Microsoft Sans Serif (micross.ttf) from WinXP, which is the 'MS Sans Serif' in TrueType!

NB: I have tried to replace the default font in %windir%\WIN.INI:

[FontSubstitutes]

MS Shell Dlg=MS Sans Serif

MS Shell Dlg 2=MS Sans Serif

but it doesn't help.

Also, I didn't found out from how Revolutions Pack 9 is replacing that font with Tahoma (in Misc tab) for the whole system.

Regards.

Charles.

0

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.