Jump to content

Welcome to MSFN Forum
Register now to gain access to all of our features. Once registered and logged in, you will be able to create topics, post replies to existing threads, give reputation to your fellow members, get your own private messenger, post status updates, manage your profile and so much more. This message will be removed once you have signed in.
Login to Account Create an Account


Photo

KernelEx 4.5.2

- - - - -

  • Please log in to reply
1555 replies to this topic

#451
SlugFiller

SlugFiller

    Member

  • Member
  • PipPip
  • 166 posts
What do you mean by "turns them"? Doesn't it call IFSMGR directly (Which, AFAIK, takes unicode device names)?
What did you test exactly?

Fat32 uses UTF16 long file names, so somewhere between CreateFileA and the IO subsystem, the ANSI filename is converted to UTF16 (using a distribution-dependent codepage). The only question is where. If I knew the answer to that, I could patch it area to use a more comprehensive codepage, such as UTF8. Then supporting arbitrarily localized filenames would be simple.

Since I've recently tested replacing my ifsmgr.vxd with localized versions (those were hard to track down), to no effect, I think it is safe to assume it is locale-independent.
This does leave the possibility that the conversion occurs -after- ifsmgr.vxd, inside vfat.vxd. I did try replacing it, to no effect, but I didn't choose highly varied locales there, so maybe it's just a coincidence. Still, it doesn't seem too likely, considering it gets its input as a unicode string already.
Unfortunately, I couldn't get a good description of the path taken between Kernel32 and ifsmgr. Like I've said, I've tried finding a good DLL-capable disassembler to get a better view of the CreateFileA implementation, with no success.

I did start poking around my copy of the DDK to try and see if I could write a file-system driver. I figured, I could, at the very least, write a driver that takes the filenames as hex-encoded or base64-encoded strings, and converts it to file name using some UTF convention. I could then use that to either call another driver (one which supports unicode filenames), or implement my own version of the FAT32 file-system.
Unfortunately, I could find any good examples of file system implementations using the DDK. I did find one which uses VtoolsD, so maybe that can help. It is also quite disturbing that not one of the DDK examples is written in pure C or C++, all use assembler code.


How to remove advertisement from MSFN

#452
Joseph_sw

Joseph_sw

    Member

  • Member
  • PipPip
  • 217 posts
  • OS:98SE
  • Country: Country Flag

Fat32 uses UTF16 long file names, so somewhere between CreateFileA and the IO subsystem, the ANSI filename is converted to UTF16 (using a distribution-dependent codepage). The only question is where. If I knew the answer to that, I could patch it area to use a more comprehensive codepage, such as UTF8. Then supporting arbitrarily localized filenames would be simple.

Since I've recently tested replacing my ifsmgr.vxd with localized versions (those were hard to track down), to no effect, I think it is safe to assume it is locale-independent.

is theres also posibilty that conversion were done by kernel32.dll itself (somewhat semi hard-coded) ?

as in the kernel32.dll file also contain some data structure of those cp_####.nls files
the 8->16 CodePage to UTF16,
and 16->16 UTF16 to CodePage data structures (using to those three-layered lookup table)

thanks to your earlier post about the .nls files, i found that fact in kernel32.dll

#453
slhk

slhk

    Junior

  • Member
  • Pip
  • 68 posts
Used 4.5 Beta 1 for a period of time. Here is my report

1. Previous working programs continue to work

2. Inkscape: It runs when compatability is set to 2000, but cannot open or save file (it hangs)

3. Foxit Reader: It runs but cannot print. It says "no printer is installed"

4. Arora: Cannot run. Page fault in QTWEBKIT4.DLL. Could someone tell me how to make it run?

Thank you

#454
Tihiy

Tihiy

    the creator

  • Member
  • PipPipPipPipPipPipPip
  • 1,924 posts
  • OS:Windows 7 x64
  • Country: Country Flag

Donator

What do you mean by "turns them"? Doesn't it call IFSMGR directly (Which, AFAIK, takes unicode device names)? What did you test exactly?

I'm not sure how ntkern really works. But 98 kernel has (K32_)NtCreateFile which calls ntkern one and takes unicode strings. So no, your idea is not working. You have to call ifsmgr directly somehow.

#455
SlugFiller

SlugFiller

    Member

  • Member
  • PipPip
  • 166 posts
@Tihiy: I was actually wondering more about your experiment process. Did you try to create a unicode-named file, only to see it created with underscores? Did you verify the underscores were actually in the filename, and not just in display? (If it is just in display, the file would not load in any ANSI program)

@Joseph_sw: I've already replaced my Kernel32. I've also messed around with the NLS files. They effect the GUI behavior, but not the file system.
Having a mismatch between the GUI code page and the FS code page just causes files of either locale to be inaccessible.

Well, I've done some more research. Apparently, IFSMgr has an ANSI API which is used by Kernel32. (I think it also has a Unicode API, but it's hard to get concrete details on IFSMgr's API)
When it receives an ANSI filename, it makes a conversion using tables located in Unicode.bin (One file for all CPs, now there's a bottleneck). I've modified it to add a few extra code pages, and successfully gotten access to localized files. It uses the same registry key as the GUI code page, but is more limited since it uses a single data file (instead of the pluggable NLS files).
It does require restart every time I want to change code pages, and every time I only have access to some of my files. It only support encodings up to 2 bytes per character, so creating a UTF-8 code page is out of the question (unless I want to recode IFSMgr itself).

Still, there should be a way to access IFSMgr using Unicode calls, either directly from the kernel, or from a dedicated VxD. If I could find more information about its API, maybe I could get true Unicode file access to work.

One thing I did notice was that Explorer isn't exactly "well behaved" when it comes to double-byte filenames. It displays them okay in the list, but improperly when renaming. It also fails to start up localized files when clicked (DnD into an application works, though). I wonder if there's an XP Unicode version of Explorer I could use or something.

#456
Joseph_sw

Joseph_sw

    Member

  • Member
  • PipPip
  • 217 posts
  • OS:98SE
  • Country: Country Flag

Having a mismatch between the GUI code page and the FS code page just causes files of either locale to be inaccessible.

Well, I've done some more research. Apparently, IFSMgr has an ANSI API which is used by Kernel32. (I think it also has a Unicode API, but it's hard to get concrete details on IFSMgr's API)

When it receives an ANSI filename, it makes a conversion using tables located in Unicode.bin (One file for all CPs, now there's a bottleneck). I've modified it to add a few extra code pages, and successfully gotten access to localized files. It uses the same registry key as the GUI code page, but is more limited since it uses a single data file (instead of the pluggable NLS files).
It does require restart every time I want to change code pages, and every time I only have access to some of my files. It only support encodings up to 2 bytes per character, so creating a UTF-8 code page is out of the question (unless I want to recode IFSMgr itself).

i'm definetely interested about modifiying the unicode.bin myself, as i still have grudge on CP-1252 about ansi/ascii character that can't be used as filenames.

now, where do i start? what i need to know about unicode.bin' data structures?

#457
SlugFiller

SlugFiller

    Member

  • Member
  • PipPip
  • 166 posts
I've made a small Java program for creating my unicode.bin. Attached below. It is roughly 400 lines of code, plus 60000 lines of CP data.
It would be easier than trying to make such a program yourself. The bin uses a tree structure to define lookup ranges, and my program automatically sorts it out as a binary tree.
It could be slightly improved by not creating a new range for a gap of less than 6 characters, but instead padding with underscores. This would create a slightly smaller bin.

Attached File  MakeUnicodeBin.rar   199.35KB   12 downloads

But, like I said, I'm looking for a more comprehensive solution. I am contemplating using the file system hook to convert base64 or hex filenames into unicode ones. To ensure compatibility, though, I want it to use a special device name which wouldn't collide with real devices. But I'm not sure whether the hook is even called for device names which don't exit.
The alternative is to use Ring0 FileIO. This is a bit more complex, since it requires quite some book-keeping, and consideration of edge-cases.
Unfortunately, developing VxDs is VERY complex. I still have a lot to learn before I can properly start testing.

If I am successful, though, patching KEx would be incredibly easy. The new architecture is even easier to extend and deploy than the previous one. Making file APIs which attempt to detect a hex-to-unicode helper device is actually quite simple.

#458
kocoman

kocoman

    Newbie

  • Member
  • 40 posts
Does anyone experience graphics corruption with the 4.5 beta 1 and remote desktop connection for win98?

#459
SlugFiller

SlugFiller

    Member

  • Member
  • PipPip
  • 166 posts
I've finally managed to compile and run the VxD equivalent of "Hello world", coded in C++ using 98DDK and VS6's CL. This will give me the ability to experiment a bit, and see if I can get something akin to Unicode file access, one way or another.
One issue is that once the VxD is loaded, it cannot be unloaded or reloaded, even if the file is modified. The only solution is to restart the computer for every revision. That gets old really fast.

P.S. 8 downloads, 0 comments

#460
Joseph_sw

Joseph_sw

    Member

  • Member
  • PipPip
  • 217 posts
  • OS:98SE
  • Country: Country Flag

P.S. 8 downloads, 0 comments

oh sorry, i have been busy trying to port that java into Delphi as that remaining programing language that were still installed on my computer, i lost my java se installer somehow.
i probably gonna need the unicode.bin result from that java program, just to compare wether my port attempt were done correctly or not. :whistle:
currently i just finished porting the CodePage class, so i haven't tried how the new unicode.bin will works on the system.

question:
1. is there any limit in maximum size unicode.bin? i intend to add all codepage for every existing cp_###.nls in my computer.
original unicode.bin were only contain 30 codepages, you example was 9 codepages, in my system theres 36 cp_####.nls files.

2. in prgram given it seem you deliberately skip cp.addMapSingle() for cp 0x81, 0x8D, 0x8F, 0x90, 0x9D, for cp1252 why is that?
that were incidentaly chracters that can't be used as filenames in cp_1252.

the differences between cp_1252.nls and cp-1252 in original unicode.bin was 0x8E & 0x9E which also can't be used in filenames.


anyway that java code were very informative :thumbup , thanks to that i now know about how unicode.bin should be read/inspected.
while unicode.bin contain 30 cps, i look in win98se .cab files, it only provided 20 cp_####.nls files, hmm...

Edited by Joseph_sw, 07 February 2010 - 02:56 AM.


#461
SlugFiller

SlugFiller

    Member

  • Member
  • PipPip
  • 166 posts
Why are you using Delphi? JDK is available for free from the Sun Java website. And it's a much better language too, although that doesn't really say much. I mean Pascal+Visual+Proprietary-Any chance of cross-platform compatibility=Worthless language.

According to the documentation I got, the original 95 IFSMgr had a bug in that it cache the header of the unicode.bin, but its cache was only large enough to contain 18 language entries. The documentation was a bit unclear, but it might be able to work with more languages, provided the OEM and ANSI code pages are on the same 18-language block. Certain localized versions did not have the bug.
I'm not sure about the status of the 98 driver, although I imagine it probably fixed the bug already. The format itself should support up to a 4GB file.

I didn't feel like pushing it to the limit. My original unicode.bin only had 5 code pages, and I've added an extra 4 (all East Asians). Your 30-pages bin should probably go on mgdx or something. Could you list the code pages you have there?

The code page data itself is from the Microsoft contribution to unicode.org. I made a small javascript page which converts the text files into the necessary Java commands. Obviously, I didn't sit down and manually input 60000 character codes.
So any "blanks" in the data are the official Microsoft stance on the code page. There are no coincidences there, if it matches what you already have.

#462
Queue

Queue

    Member

  • Member
  • PipPip
  • 164 posts
If you're programming something that is Windows-specific, cross-platform portability is useless. There are speed and resource use benefits to pretty much all "Pascal+Visual+Proprietary" languages over Java.

Queue

#463
Siknik64

Siknik64
  • Member
  • 1 posts
Could someone make a step by step tutorial on how to install the Flash Player 10 plugin with Firefox 3.6? The KernelEx 4.5 Beta did install successfully but even with the compatibility, it just won't install. I tried installing it on Firefox from the Adobe site but all I got is the Adobe DLM extension (v1.6.2.60) and the getplus® packaged with it doesn't run on 98SE. I tried installing the Flash Player plugin beta but that doesn't work in IE6 OR Firefox 3.6 :angry:

Any help?

#464
yuuten

yuuten
  • Member
  • 1 posts
I can't install all of the 4.x version KernelEx on Japanese edition of Windows ME.
I got the following error message:

exportfromX: pattern not found


The following pattern is in KERNEL32.DLL of Japanese WinME:
0F BF 40 10 8B 14 81 8B 44 24 14 3D 00 00 01 00 8B 4A 04 73 15 39 1D

The pattern of KernelEx source setup/setup.cpp line 369-370:
0x0F,0xBF,0x40,0x10,0x8B,0x14,0x81,0x8B,0x44,
0x24,0x14,0x3D,0x00,0x00,0x01,0x00,0x8B,0x4A,0x04,0x73,0x15,0x3B,0x1D,


Version 0.3.x installer works fine.

#465
SlugFiller

SlugFiller

    Member

  • Member
  • PipPip
  • 166 posts
@Queue: "Visual", by definition, denies any "speed and resource benefits". It is probably many times slower than JIT-compiled Java code, and while I can't testify as to the library size, the memory usage is undoubtedly much higher. It was one thing if this was C, then you'd have good performance, and it wouldn't be proprietary. It would also be easily portable, although not necessarily cross-platform (depends on whether the C runtime is used, or the Win32API).
Also, I hope I don't have to explain the issues with a language with a proprietary dynamically linked run-time (although I suppose Windows may also be counted as a "proprietary runtime"). Especially not on this forum, in this thread (*cough* VCRun2008 *cough*).

#466
timofonic

timofonic
  • Member
  • 5 posts
If you like Pascal, I would recommend [url="http://<a%20href="http://www.freepascal.org"%20target="_blank"%20rel="nofollow">http://www.freepasca...org</a>"]Free Pascal[/url] and Lazarus. You will have Pascal and Object Pascal, a Delphi-like GUI and a very wide portability between different platforms (Windows, Linux, Windows CE, BSD, MacOS X, AmigaOS... even Nintendo DS!) and processors (x86, AMD64 aka x86 64, ARM, PowerPC, 68k, SPARC...).

JDK? Seriously, Java is not a sane language for people that likes Pascal or C/C++ languages, despite being massively used by market(ing) reasons. The development methodology of Java is different and certain developers are not comfortable with it or feel a lot productive not using it.

I'm not against Java as a programming language, just I avoid apps running into bytecodes because generally being highly inefficient and prefer to use native code instead.

About support: What about Chromium? It's Open Source, so maybe it can be even patched for Win 9x support. Don't get me wrong, I'm not a developer and just suggesting it :)

#467
dencorso

dencorso

    Adiuvat plus qui nihil obstat

  • Super Moderator
  • 5,793 posts
  • OS:98SE
  • Country: Country Flag

Donator

@Queue, SlugFiller and timofonic: This is the KernelEx thread, which is, by itself, already huge! Let's not get off-topic!
If you want it badly, let me know and I'll split your latest posts into a new thread (in this case do PM me about it). But I'd much rather avoid the potential development of yet another this-vs.-that flamefest, if you don't mind letting it go.

#468
Joseph_sw

Joseph_sw

    Member

  • Member
  • PipPip
  • 217 posts
  • OS:98SE
  • Country: Country Flag

I didn't feel like pushing it to the limit. My original unicode.bin only had 5 code pages, and I've added an extra 4 (all East Asians). Your 30-pages bin should probably go on mgdx or something. Could you list the code pages you have there?

well, the unicode.bin that i have was exactly came from the win98_22.cab file.

i also have my winME installer ready, unicode.bin from win_8.cab which also binary exact match with the win98se (win98_22.cab) counter part.

this the listing codepages found in that unicode.bin
 1 -  437 - 0034C 0022C 1 - 00578 0027D   12
 2 -  850 - 007F5 0022C 1 - 00A21 001D9	6
 3 - 1252 - 00BFA 0022C 1 - 00E26 001AA   10
 4 - 1250 - 00FD0 0022C 1 - 011FC 008C4  137
 5 - 1251 - 01AC0 0022C 1 - 01CEC 00643   88
 6 - 1253 - 0232F 0022C 1 - 0255B 006B5   99
 7 - 1254 - 02C10 0022C 1 - 02E3C 008A1  134
 8 - 1257 - 036DD 0022C 1 - 03909 00395   55
 9 -  737 - 03C9E 0022C 1 - 03ECA 003C1   56
10 -  852 - 0428B 0022C 1 - 044B7 00881  132
11 -  855 - 04D38 0022C 1 - 04F64 0061B   85
12 -  857 - 0557F 0022C 1 - 057AB 00890  133
13 -  866 - 0603B 0022C 1 - 06267 0061C   85
14 -  869 - 06883 0022C 1 - 06AAF 0064E   92
15 -  775 - 070FD 0022C 1 - 07329 00243	5
16 -  874 - 0756C 0022C 1 - 07798 00191   12
17 - 1258 - 07929 0022C 1 - 07B55 002E1   40
18 -  860 - 07E36 0022C 1 - 08062 0093C  144
19 -  861 - 0899E 0022C 1 - 08BCA 0093C  144
20 -  863 - 09506 0022C 1 - 09732 0093C  144
21 -  865 - 0A06E 0022C 1 - 0A29A 0093C  144
22 - 1255 - 0ABD6 0022C 1 - 0AE02 0023A   26
23 -  862 - 0B03C 0022C 1 - 0B268 009FA  160
24 - 1256 - 0BC62 0022C 1 - 0BE8E 0031F   44
25 -  708 - 0C1AD 0022C 1 - 0C3D9 001F0	9
26 -  710 - 0C5C9 0022C 1 - 0C7F5 00245   20
27 -  711 - 0CA3A 0022C 1 - 0CC66 001F9   14
28 -  720 - 0CE5F 0022C 1 - 0D08B 0020B   12
29 -  786 - 0D296 0022C 1 - 0D4C2 001F3   13
30 -  864 - 0D6B5 0022C 1 - 0D8E1 00218   14
listed in order of appearances in original unicode.bin that i have
Code page - Offset Length Ranges (cp to uni) - Offset Length Ranges (uni to cp)

i suspect that localized version of win9x might have different content of unicode.bin going on.
ack, its make me want to hunt for that files.

#469
RetroOS

RetroOS

    Senior Member

  • Member
  • PipPipPipPip
  • 531 posts
  • OS:98SE
  • Country: Country Flag

...
* Is there a way to make last VLC 1.05 working?: the exe is running, but no more displaying the text of the menus.

Charles.

VLC 1.0.5 works for me.
However, I had the same problem you describe, back in 1.0.3.
It turned out to be the font I was using for Windows text.
After I changed it to another font, VLC displayed correctly.

Try changing to another desktop theme to test, or just change the fonts in display appearance settings.

I suspect that some 'custom' fonts may not support Unicode and therefore extended characters are blank/missing like in VLC.
I could be wrong... :whistle:
retrofreak n a person who is very enthusiastic about something from the past.
retrology n the study of association with and revival from the past.
life n a series of near misses.

#470
craftech

craftech
  • Member
  • 7 posts
Since it appears that Kernel EX was first designed to run Firefox 3 and above on W98SE machines why not address the biggest problem (the problem that forced me to go back to Firefox 2 with Kernel EX extensions disabled) which is that many of us are unable to Print from Firefox 3, and in my case the e-mail client (mailto) wouldn't work in Firefox 3 under any circumstances nor in Firefox 2 until I disabled Kernel EX when Firefox 2 loads?

Opera 10 worked fine.

John

Edited by craftech, 08 February 2010 - 08:29 AM.


#471
Tihiy

Tihiy

    the creator

  • Member
  • PipPipPipPipPipPipPip
  • 1,924 posts
  • OS:Windows 7 x64
  • Country: Country Flag

Donator

the e-mail client (mailto) wouldn't work in Firefox 3 under any circumstances nor in Firefox 2

I've researched this issue and - that may sound stupid - it's really mimicking XP behavior. SHParseDisplayName check they added in some security update actually may fail to process mailto: links. Nevertheless it's a serious issue and it will be fixed soon. And if you want printing, what you wanna print?

#472
EvanD

EvanD

    KISS Advocate

  • Member
  • 32 posts
  • OS:98SE
  • Country: Country Flag

Many of us are unable to Print from Firefox 3. <snip>


See: http://www.msfn.org/...p...st&p=904976

It's not perfect but, at least for some of us, it always prints from Firefox 3.x.x.

Edited by EvanD, 09 February 2010 - 04:05 PM.


#473
CharlesF

CharlesF

    Advanced Member

  • Member
  • PipPipPip
  • 461 posts
  • OS:98SE
  • Country: Country Flag

After I changed it to another font, VLC displayed correctly.

Wonderful! I have only switched the title bar font from Tahoma to Verdana,
and VLC is now perfect. :thumbup
Thanks RetroOS.

I suspect that some 'custom' fonts may not support Unicode and therefore extended characters are blank/missing like in VLC.

I think Tahoma must support Unicode.
Oddly, I went to my Font folder and it is displaying two times the Tahoma font: same TAHOMA.TTF, same size and same time stamp.
It is the only font I see in double.
Do you know how to correct that?

#474
RetroOS

RetroOS

    Senior Member

  • Member
  • PipPipPipPip
  • 531 posts
  • OS:98SE
  • Country: Country Flag

After I changed it to another font, VLC displayed correctly.

Wonderful! I have only switched the title bar font from Tahoma to Verdana,
and VLC is now perfect. :thumbup
Thanks RetroOS.

I suspect that some 'custom' fonts may not support Unicode and therefore extended characters are blank/missing like in VLC.

I think Tahoma must support Unicode.
Oddly, I went to my Font folder and it is displaying two times the Tahoma font: same TAHOMA.TTF, same size and same time stamp.
It is the only font I see in double.
Do you know how to correct that?

I noticed the two Tahoma fonts a while ago...
One is listed as TAHOMA.TTF and the other as tahoma.ttf. The same size, date, and time.
I also have some other duplicate fonts, some with identical font name and file name, and others with variations.

I changed all my fonts to Trebuchet MS and that worked.
retrofreak n a person who is very enthusiastic about something from the past.
retrology n the study of association with and revival from the past.
life n a series of near misses.

#475
CharlesF

CharlesF

    Advanced Member

  • Member
  • PipPipPip
  • 461 posts
  • OS:98SE
  • Country: Country Flag

I suspect that some 'custom' fonts may not support Unicode and therefore extended characters are blank/missing like in VLC.

I changed all my fonts to Trebuchet MS and that worked.

Thanks RetroOS for your help!
The problem comes from the font 'MS Sans Serif' (SSERIFE.FON) which is not a TrueType but a Fixed font, and doesn't support Unicode.
This font is the default system font in native Windows 98!

I didn't found a workaround except to use another font (what you did) for each Item in Display Properties->Appearance.
If one want to keep the same font, he can switch to Microsoft Sans Serif (micross.ttf) from WinXP, which is the 'MS Sans Serif' in TrueType!

NB: I have tried to replace the default font in %windir%\WIN.INI:
[FontSubstitutes]
MS Shell Dlg=MS Sans Serif
MS Shell Dlg 2=MS Sans Serif
but it doesn't help.

Also, I didn't found out from how Revolutions Pack 9 is replacing that font with Tahoma (in Misc tab) for the whole system.

Regards.
Charles.




2 user(s) are reading this topic

0 members, 2 guests, 0 anonymous users



How to remove advertisement from MSFN