Day-to-day running Win 9x/ME with more than 1 GiB RAM

[rloew]

First off, I would like to excise the term "theoretical" from my terminology. Instead, I propose that we adopt a Boolean distinction along the lines of:

1) Non-Rloew Universe (NRU)

2) Rloew Universe (RU)

[edit by dencorso]

Not familiar with the terminology. Please clarify.

[bphlpt]

I think he meant:

1) system without rloew modified files

2) system with rloew modified files

So the rest of his post is pretty much asking for clarification of terms and measurements using a system that does not have rloew modified files in it, if I understood him correctly.

Cheers and Regards

[RetroWish]

Ok, Rloew

In the interest of *finally* doing away with a bunch of misconceptions and "urban legends", I will walk the proverbial extra mile here.

Non-Rloew Universe (NRU)

--------------------------------------

This is the state of affairs regarding Win98SE *without* a user doing anything like what you have done with your celebrated patch to address the RAM limitations under long-standing discussion. It certainly includes a technical understanding of why the limitations *do* exist in the NRU and, possibly, a conceptual as well as technical blueprint aiming at overcoming them.

Rloew Universe (RU)

------------------------------

The starting point is, of course, the NRU. The RU is characterized by the existence of individuals (like yourself ), who, armed with the requisite technical knowledge, proceeded to *do* something about the aforesaid NRU RAM limitations. It also includes all relevant experience relating to the *actually* overcome RAM limitations, whether as programmers/distributors of successful solutions (patches) as well as users of such tools (bug fixes).

I hope this sets the stage for an eagerly awaited discussion! :>)

[dencorso]

Observations

---------------------

System Properties reports 1,156 MB of physical RAM recognized by the OS. Adding the Onboard VGA Shared Memory Size of 64 MB yields 1,220 MB which precisely matches the theoretical maximum achievable without the "magic" of a certain rather well known patch. :>)

The VGA has nothing to do with with the reported size or the limit.

The Onboard VGA reserves its share of RAM for itself much before the time DOS boots, so by that time it aready is "not there". Nonetheless, part of it will be mapped within the System Arena and part at the A0000-BFFFF region of the Compatibility Arena of Virtual Machine #0 for Win 9x/ME.

Real Addresses are the dominion of the VMM (and of the hardware itself). All that is there, from the POV of the rest of the SO and all other applications are Virtual Addresses. The Swapfile is part of the implementation of Virtual Memory, but its size does not define the Virtual Address Map, which is predefined by the OS, taking into account the CPU's intrinsic limits to address space. The full Virtual Address Space used by Win 9x/ME is described in Q125691 - INFO: Overview of the Windows 95 Virtual Address Space Layout. Now, in a nutshell, it's set like this (and divided in four Arenas by MS):

You should, by all means, re-read Q125691, whence the figure above came, and also Igor Leyko's Article (in English, by GreyPhound) (link to the original Igor Leyko's Article in Russian). So, there are just so many adresses, and that's all. All things that go in the System Arena must fit within 1 GiB, else the OS must crash from lack of other options.

[rloew]

Going from Rloew to "Rolw" only made things unclear. It does not follow.

I suspected that you meant unpatched vs. patched but you could have been referring to something else.

To my statement that "System Properties reports 1,156 MB of physical RAM recognized by the OS" you partially responded:

[...with the reported size or the limit]

I assume, then, that System Properties is attempting to reflect *some* kind of NRU limit (not "theoretical", but a *practical* limit, nonetheless), right?

The System Properties shows the amount of Physical RAM Memory managed by Windows. There is no limit to what it can show.

The limit of interest is determined by an overflow in a table that is used to manage Physical Memory. There are other limits, but they are higher.

To my statement that "Adding the Onboard VGA Shared Memory Size of 64 MB yields 1,220 MB" you responded:

[The VGA has nothing to do with with the reported size or the limit]

If true, a lot of NRU technical information regarding this particular aspect that one comes across in these forums is obviously wrong!!

I don't recall seeing 1,220 MB being mentioned anywhere.

The VGA Shared Memory is taken from the Total Physical RAM, but is not related to the Maximum allowed RAM. So 1,220 MB has no meaning.

You also wrote:

[You set MaxPhysPage to only recognize 1,158 MB of RAM]

Well, yes. :>) This is *my* practical NRU limit that I zeroed in on by laborious trial and error. My situation is definitely not unique around here, I understand...

Lucky you. I only got 1,152 MB in my experiments.

Very importantly, you stated:

[MaxFileCache plus any XMS RAM Disk plus some Graphics Apertures determine the amount of System Arena space used for Memory Management. The Kernel and DOS Command Windows use more. The System Arena is limited to 1GB total of Virtual Memory.]

Thank you very much for explicitly specifying the important components applicable here. Clearly, the System Arena requires the "existence" of *some* Virtual Memory.

Here is where I need big-time enlightenment, I am afraid. To begin with, you stated:

[The System Arena is limited to 1GB total of Virtual Memory.]

Yes, under NRU conditions, right? What about *minimum* requirements, though? More to the point, you also wrote:

The System Arena is exactly 1GB of Virtual Memory, by definition.

This memory can be mapped to Physical Memory, Swapped to Disk, or unused.

[This limitation is why MaxFileCache needs to be specified when you exceed approx 768MB of RAM.]

MaxFileCache sets the amount of Virtual Memory reserved for the File Cache. The actual amount of Physical Memory used is dynamic, but the Virtual Memory usage never changes.

Ok, here is my *main* question. What is the relationship between physical RAM *installed* (not necessarily recognized by the OS as such) and Virtual Memory, always in an NRU sense? I am afraid that I cannot wade through all these "urban legends" all by myself. :>) To boot, aside from the Virtual Memory requirements that you have alluded to, do all (some of) these components utilize *physical* RAM as well, when active/invoked? If so, does the RAM come off the 1,158 MB that the OS recognizes?

Any active Code or Data must reside in Physical Memory. Windows allocates general RAM from the RAM it reserved at boot, 1,158 MB in your case.

Memory Mapped I/O, Adapter RAM (including VGA Apertures), and DOS based RAM Disks, are not allocated from this RAM.

But managing this Memory may tie up Virtual Memory in the System Arena to map to it.

More specifically, you wrote:

[Your Physical Memory allocation is as follows:

Windows: 1,158 MB

RAM Disk: 384 MB

Aperture: 64 MB

Unused: 441 MB

Reserved: 1 MB (May Vary depending upon motherboard)

Total: 2,048 MB ]

If "Unused" amounts to 441 MB, where does the MaxFileCache provision fit, if or when actually activated by the OS? Does it consume part of the 1,156 MB that the OS recognizes? (I have already asked this before in a more inclusive framework)

In your Computer, the File Cache premanently reserves 384MB of the System Arena in Virtual RAM. When in actual use, it will take up to 384MB of Physical RAM from the 1,156 MB recognized by Windows.

I am afraid that I am lost in a dual world where the same components appear to make separate (?) claims on installed/recognized physical RAM as opposed to the Virtual Memory which is the home of the System Arena. Any further enlightenment would greatly be appreciated.

With 4 different types of Virtual Arenas, 4 different usages of Physical RAM, and Memory Mapped I/O Space, it can get complicated.

Even the use of HIMEMX vs. HIMEM has an effect.

You should probably read up on Memory mapping and Virtual Memory.

Edited February 26, 2012 by rloew

[RetroWish]

Hello @rloew,

I am really indebted to you for your prompt and comprehensive reply to my posts. Now that you have pointed the way towards further specific technical enlightenment I will be hitting the books shortly. If I have further questions I know where to come for answers... I will try not to waste your time with information and conclusions that I can access/arrive at on my own, being scientifically meticulous etc.

Thanks Again

PS. Sorry about the unfortunate spelling mistake. Now I've got it. It is "RLOEW"! :>)

[dencorso]

Here are two good book recommendations:

Modern Operating Systems (by Andrew S. Tanenbaum) ISBN 978-0-13-600663-3 (not necessarily the 3rd ed., which is the current one)

or, if you're much more technically minded, the book that enabled/inspired Linus Torvalds to create LINUX:

Operating Systems: Design and Implementation, (by Andrew S. Tanenbaum and Albert Woodhull), ISBN 978-0-13-142938-3

[RetroWish]

Hello @Dencorso and @Rloew,

@Dencorso, thank you for taking the time to suggest those two books for further technical reading. They will have to wait until summer! :>) Also, thank you for entering my system into the ever expanding "hall of retro-fame"...

@Rloew, Igor Leyko's article was a conceptual game changer for me. A million thanks for pointing me in that direction! :>) If it is not too much to ask, could you comment in your usual precise language on the differences between HIMEM and HIMEMX, especially as they relate to VMM.VXD initialization? It appears that HIMEMX (which I *am* using) significantly expanded options, always in an NRU sense and in the context of *this* topic, right?

Cheers

[rloew]

Hello @Dencorso and @Rloew,

@Dencorso, thank you for taking the time to suggest those two books for further technical reading. They will have to wait until summer! :>) Also, thank you for entering my system into the ever expanding "hall of retro-fame"...

@Rloew, Igor Leyko's article was a conceptual game changer for me. A million thanks for pointing me in that direction! :>) If it is not too much to ask, could you comment in your usual precise language on the differences between HIMEM and HIMEMX, especially as they relate to VMM.VXD initialization? It appears that HIMEMX (which I *am* using) significantly expanded options, always in an NRU sense and in the context of *this* topic, right?

Cheers

I'm not sure why you are so obsessed with "NRU". The vast majority of the issues discussed are not affected by "RU" vs. "NRU".

Only the following are different:

With my Patch, Windows 9x is not limited to recognizing 1,158 MB of Physical RAM. It is able to use as much as 4GB without crashing.

There is also a DMA Memory Patch, which fixes a DMA Memory exhaustion issue that has not been discussed so far.

I also have a Non-XMS RAM Disk that is completely unmanaged by Windows, so no Virtual Memory in the System Arena is reserved at any time.

In addition I have a 64-Bit Version that can use 64-Bit Physical Memory, leaving all 32-Bit RAM available for Windows or other purposes.

All other matters, such as the System Arena issue are unaffected.

As to your current question.

The difference between HIMEM and HIMEMX, that I studied, is the way that Memory Allocated outside of Windows is handled. Since XMS RAM Disks are allocated from memory before Windows is loaded, they are affected by this choice.

HIMEM is understood by Windows, so Windows takes over the management of this Non-Windows Memory. To do so, Windows reserves an equal sized area of Virtual memory in the System Arena, to map the Physical RAM of the Non-Windows Memory into Virtual Memory, so accesses to this Memory can be handled within Windows.

HIMEMX is not understood by Windows, so Windows is not aware of the Allocated Non-Windows Memory, and does not reserve Virtual Memory in the System Arena. Calls to HIMEMX, to access this Memory, are passed to Interrupt 15H which is handled by Windows. Windows will then Allocate Virtual Memory from the System Arena to map the RAM being accessed.

In the case of a XMS RAM Disk, using HIMEM will end up Allocating System Arena Virtual Memory for the entire RAM Disk, immediately upon Startup. If the RAM Disk is too big, Windows will crash.

If HIMEMX is used, System Arena Virtual Memory is only Allocated when the RAM Disk is actually accessed. This Memory is never Released, so eventually the entire RAM Disk will be Allocated in System Arena Virtual Memory. With an oversized RAM Disk, Windows will crash as you fill it up.

[RetroWish]

Hello @Rloew,

As usual, your response to my latest post has been prompt and, most importantly, quite comprehensive. Thank you very much for staying with me so far! :>)

My references to "NRU"/"RU" do not aim at establishing some kind of objective dichotomy. Rather, they reflect my actual experiences and puzzlements with various aspects related to the topic under discussion. Please remember that, in your case, my subjective status may come across as decidedly "yesterday's news" since "you've been there, done that", so to speak. I am still struggling with basic moving parts that are super-foundational by the their very all-encompassing nature (e.g. XMS issues, 4th GB mapping issues etc.)

Here is an example. Leyko writes:

[What's needed done first is to forget, once and for all, EMM386 and other memory managers; more or less stable work, or even (system) booting can't be guaranteed with them]

What is one to make of this? Is he including such venerable "beasts" as HIMEM or HIMEMX ?!!? What aspects of EMM386 (QEMM etc) are potentially problematic here? Is it the Expanded Memory or, perhaps, the DEVICEHIGH/LOADHIGH invokations, or both? You see what I mean?

In any case, any and all assistance with such matters is greatly appreciated.

Cheers

[dencorso]

No. He meant EMM386, QEMM, CEMM, 386MAX and NETROOM... because they are simple VMMs which run DOS in V86 mode, and have to be suspended in order for Windows own full-fledged VMM to take over, while leaving lots of things for Win VMM to take care of that otherwise wouldn't exist.

[RetroWish]

Hello @Dencorso,

Thanks for the prompt response. That's what I thought too! :>) I speculate that, as experienced "old pros", @Rloew and you have seen your fair share of digital pathologies... To this effect, I would like to ask your opinion about a couple of things regarding HIMEMX and EMM386.

First off, when it comes to HIMEMX, is DOS=HIGH required, implied or, even understood? Also, how can one activate the DOS=UMB option? Does one have to use EMM386? Finally, I have read somewhere that there are some drawbacks to using HIMEMX in MS-DOS 7.1 real-mode. Is this true?

My second question has to do with the Win98se environment itself (not real-mode). Is there a drawback to utilizing the DOS=UMB option while making darn sure that EMM386 (or whatever works with HIMEMX) does not provide Expanded Memory services (i.e., NOEMS NOVCPI)?

Thanks for your patience.

[RetroWish]

Hello again,

I made some progress in dealing with my second question. I have experimented with EMM386 in the context of HIMEMX. My attempt at just allowing UMA/UMB utilization while blocking the provision of Expanded Memory services (NOEMS NOVCPI) led to quite a surprise. Namely, Win98SE just recognized 384 MB of physical memory!!

I wonder if HIMEMX is compatible with some other, hopefully unobtrusive, way to *just* allow UMA/UMB utilization *within* the Win98SE environment...

[dencorso]

First off, when it comes to HIMEMX, is DOS=HIGH required, implied or, even understood?

It's not required nor implied, but is understood and will work.

Also, how can one activate the DOS=UMB option? Does one have to use EMM386?

Either EMM386, or any of CEMM, 386MAX, QEMM or NETROOM. None are good ideas with Win 9x/ME so, if you cannot live without UMBs, and have an AMD based machine, the best solution (which is not so good) is to keep using EMM386. If you have an Intel based machine, UMBPCI is the way to go.

Finally, I have read somewhere that there are some drawbacks to using HIMEMX in MS-DOS 7.1 real-mode. Is this true?

HIMEMX has some bugs (at least four, I think, but I'm not finding my notes on it), but none are really serious. I don't think it'd cause any problems in Real Mode DOS. The problems it may cause in windows were already described by RLoew, some posts above.

I think this answers the questions your experimentation hasn't already answered. Now, enough of thread hijacking. If you want to discuss anything outside the scope of this thread's title, please do open another thread.

[loblo]

Since dencorso has updated the list with my new system without doing the usual thread bump ( ), please allow me guys to bump it myself:

This topic has been updated!

What's New?

on post #2:

loblo's 4GB system (the only known 4GB Windows Me system in the world ) has been added.

Edited April 23, 2012 by loblo