THE EVIL WIN9X REGISTRY NIGHTMARE
In a nutshell, this problem can be nothing short of Cruel and Unusual Punishment. So let's first remember how we got here. The problems can all be traced back to a single reckless decision made by someone on the Windows team during the Win95 Chicago
development. That decision was to encourage all software authors to store their settings in the System registry. Prior to this it was standard practice for software which required it, to maintain it's persistent settings in a private file. a .CFG or .INI or .SAV or .TXT or any other filetype or combination of files, binary or plaintext, whatever the author chose. Microsoft's rationalizations for the change included "There's all these INI files scattered all over the system!"
or "The registry is much faster!"
. Yeah, that's pretty much true but completely beside the point. The registry also contains 99% of the core system configuration, and that means no registry = no system. Inviting every single software application or utility to have read and write access to the SAME database that contains the system configuration, well, what could possibly go wrong? The answer of course would have been for Windows to maintain two registries, one for hardware and one for software ( other than the Windows OS ). The former would only be written to at SETUP and for patches, service packs, hotfixes, updates, hardware upgrades and device driver updates. Most importantly it should have been machine specific. The other registry should have been completely hardware independent. Not only would this have built a firewall between Windows and everyone else, it would have prevented that other disastrous problem from ever occurring - the necessity to reinstall every piece of software after reinstalling Windows since that operation creates a new registry, wiping out everything else.
The official sanctioned method for generating a new registry without the 'holes' is by using REGEDIT.EXE
with the /C
switch while in DOS ( FYI, REGEDIT is a dual-mode executable containing code for both 'real-mode 16-bit true DOS and protected-mode 32-bit Windows ). This official method was often found in many software packages that operated in 32-bit Windows but when you selected an option to compact the registry it would punch some commands into AUTOEXEC, reboot into DOSMODE and launch a batch file scripted to execute the REGEDIT /C
(yikes!). A failure here could be catastrophic. This is how things went until a handful of 32-bit programs like RegCompact made the logical step to creating the new registry while in Windows where a failure would be insignificant. Anyway. this is all that Microsoft gave us, and when it failed it was very painful as we'll see in a minute. F8 Command Line DOS
: that bare DOS environment is available via the F8
keypressing when the booting computer is just finishing the BIOS and about to start Windows, and is one of the favorite features of the Win9x environment ( unfortunately hidden
in WinME and completely lacking in the NT family ). It's availability is due to the bootstrap method of that era where some real-mode DOS files are first loaded prior to the switchover to protected mode Windows and at that specific point there is a small window of opportunity to intercept the sequence and tell the bootstrap to "stop here and let me into DOS please"
( hence the F8 keypressing to be sure to hit the right spot ). This naked DOS environment is especially useful to the expert user because they can replace any file existing on the system without interference from Windows file protection. So if you have a virus or are testing device drivers or patching low-level core files, or as in this case replacing the entire registry this can be easily done via traditional DOS commands after entering F8 Command Line DOS.
Anyway, it is what it is. We have a monolithic registry file ( from the user and from the software's point of view although it is multiple files on disk ) and it can be broken in several ways. Two manifestations of this breakage came up in another MSFN thread here
. What can happen in Win9x is that the registry grows in size and complexity and consequently it can both fail to be loaded
at bootstrap or it can fail to be created
when you use the proper REGEDIT /c
command from DOS to rebuild a new registry without 'holes'. My description and speculation of these two problems from that thread:
My understanding is that there are two major but separate registry problems in the Win9x family ...
... This occurs in DOS when the real-mode part of REGEDIT using /c
cannot create one of the DAT files It is trying to rebuild a new registry ( output binary ) from a previously successful export ( input ASCII ). I discovered this way back in Win95 before the OSR updates corrected it ( temporarily ), when certain software moved all of its settings from private files into the registry and doubled the size. It was cat and mouse for the next few years but eventually even for Win98se large exports couldn't be /c
created. Before WinME, the patching and backporting of a later REGEDIT version ( e.g., from 95osr to 95 ) allowed successful creation, but we ran out of luck when WinME changed REGEDIT ( it separated CLASSES out of the SYSTEM hive as a last ditch effort to make it manageable ). The reasons for this CREATION bug seem to be registry size, complexity, available RAM, ( and possibly CPU model, speed and cache ) so it is a chaotic and unpredictable problem, and pretty much unsolvable. Remember, this is the 16-bit universe and even before WinME Microsoft was already breaking things and leaving them broken.
Microsoft could have fixed this in the 32-bit portion instead, but I am guessing that they did not want to give REGEDIT an option to output binary DATs to an arbitrary selected folder which would be necessary since it cannot overwrite the current in-use registry files. A 32-bit registry tool that can do this ( output binary DATs to an arbitrary selected folder ) would be very desirable for Win9x, hence my previous comment.
... BSOD protection error preventing successful bootstrap because one of the DATs cannot be successfully read or loaded. I had long thought that this was for all the same reasons, registry size, complexity, available RAM, etc, but here on MSFN RLoew has also discovered a connection to networking in some way. Consequently, a surprised and unprepared Win9x user is screwed at this point. However a well-prepared user has saved many DAT backups and thankfully in Win9x it is simple to change registries in F8 DOS command line. It is very possible while in Windows to absolutely overload the registry with imports ( or just install every SDK ) and it will appear fine as it fits within the protected-mode available memory space, but then it BSOD's on reboot. With great perseverance a person could manage this situation with lots of REG files used when in Windows and have REG 'deleters' to remove them before restart, and/or have fallback DATs ready to be copied in F8 DOS.
These are two different problems entirely but as alluded to they likely have most of the same causes in common. Actual Example.
In my own case it was back on Win95 while working at a recording studio when I discovered this evil bug. What happened is that the software from an important hardware company ( ~cough~ Creative Labs ~cough~ SoundBlaster ) followed Microsoft's brilliant advice and moved all their EA presets ( environment presets ) into the system registry. This was many megabytes of strings of seldom used data that bloated the registry to twice the size. Most computers only had 8 or 16 MB total RAM at the time and when the registry was loaded at Windows start a variety of errors would show up. This is the LOADING
bug in action. Yes, I thankfully had many backups of the DATs. The workaround was to delete out all the presets except for the one or two that were used by the studio employees. However, as you might have anticipated this immediately led to the other problem, the CREATION
bug. Obviously after you delete many megabytes of data from a live registry you have substantial dead space in that registry ( 'holes' ) so now you should go an compact it following Microsoft's instructions to export the registry and re-create it in DOS. Yep, the other problem shows up depending on the export size and complexity. Note: Creative Labs did not cause this particular CREATION error, but they forced us to discover it by instigating the DOS registry creation step. No workaround for this existed yet in 1995-1996, but it did once OSR came out in late 1996 but it wasn't until early 1997 that someone figured out we could just version patch the REGEDIT.EXE from OSR ( 4.00.1111 with presumed improved memory management and other optimizations in DOS ) down-level to Win95 and it would successfully finish the REGEDIT /C
procedure when using large exports. This was obviously not sanctioned by Microsoft, and I do not recall them even addressing it. These were fun times indeed, and really the precedent was being set that Microsoft's expert user base would become the ultimate problem solvers since Microsoft was already getting comfortable letting broken things stay broken. We now know that the CREATION problem could have been avoided entirely had REGEDIT.EXE working in 32-bit Windows protected-mode simply allowed the ability to save a new registry to an arbitrary folder instead of only operating in DOS real-mode. In other words, just let REGEDIT export binary, simple! That LOADING problem when it occurs is a big deal because at that point the registry exists as the two files SYSTEM.DAT and USER.DAT and they will either be loaded successfully or fail to be loaded, it is make or break, and on a failure you DO NOT get a nice detailed explanation at all. In fact the displayed error message is almost always completely unrelated to the registry and will send the poor victim on a variety of wild goose chases.
Well, I decided to give it a go, one final time here in September 2012, revisiting the registry CREATION failure, and also the registry LOADING failure at bootstrap, both due to a large registry. Experimental Control:
I carefully designed this experiment methodology to use only one
of those possible variables ( registry size, complexity, available RAM, CPU model, speed, cache ). In this case I will make it crash solely due to size
. The proof is simply that this is a working registry and only volume will be changed with no added complexity, and no hardware alterations. So I needed a quick way to bloat the registry on a Win98se machine to force both the CREATION and LOADING bug. I finally settled on duplicating two working but already massive keys. I exported both HKLM
keys rooted at Software\Microsoft
, and placed them in a script. Then a quick search-replace the following ( before and after ) ...
Now all you need to do is import those two beasts ( or just drop the script into the open REGEDIT window ) and presto, registry mega-bloat! Reminder: at this point you are sitting in Windows 32-bit protected mode looking at a normal REGEDIT window. Within a few seconds there will be a flush of dirty data in RAM to disk. SYSTEM.DAT and USER.DAT just grew much larger but you wouldn't know there are any potential problems until you reboot. If you now simply export the current registry ( and I did ) it will show the new bloated structure. Here is the before and after ( NOTE: these numbers DO NOT represent any kind of useful comparison to anyone else's computer ) ...
;--- Original Registry Size
SYSTEM.DAT ... 12,738,592
USER.DAT ...... 3,686,432
Export.reg ... 21,075,255
;--- Registry Size after adding bloat
SYSTEM.DAT ... 15,347,744
USER.DAT ...... 3,829,792
Export.reg ... 25,804,411
Once again, by simply duplicating the Microsoft
key, no complexity is added, just a dummy key that will NOT be accessed by the bootstrap process. This means that a LOADING failure is solely confined to registry size
, and nothing else. BTW, if your system has more 'headroom' and needs more registry size bloat to crash, you can just create a 3rd or 4th key ( HKLM\Software\Microsoft3
... etc ). Rinse and repeat as needed.
Registry LOADING Error
. So first I simply do a nice reboot just as any unsuspecting user might. What could possibly go wrong! ( Really, just imagine Joe User
who just installed some giant SDK or something that added a massive chunk of useless data to an already large registry. The thing is
working just fine in 32-bit Windows. He shuts down and turns the computer on the next morning ... ) ...
The first thing to note is that error message says nothing about the registry. But I can absolutely guarantee that NO OTHER VARIABLES CHANGED except for registry size
. So why does it mention VFAT? My own theory is that it was simply due to running out of available memory at this particular step in the startup. Presumably with a different amount of RAM or larger registry size the error would have occurred earlier or later in the bootstrap process possibly resulting in an entirely different error message. I invite others with far more knowledge to explain it more accurately.
So we have established the critically important fact that this BSOD has NOTHING TO DO WITH VFAT or any other VXD. Obviously this is where the average Win9x victim begins his time wasting search for VFAT problems, replacing files and maybe even re-installing Windows. With a slightly different error message the time-wasting search will lead them down all manner of other wild goose chases. Just imagine the many hapless Win9x users stuck at this point. Very few of them likely had sufficient backups of the binary DATs or the knowledge of replacing them manually in DOS ( if on Win98 one could try SCANREG in DOS and quietly revert to an older DAT saved in RBxxx.cab, and now we know why there is such a thing as SCANREG in the first place ). Prior to SCANREG non-experts were simply out of luck, and I would suggest that even with it they still are since reverting to an old registry is not a real solution, especially if it contains older or different hardware and software keys. Only the well-prepared and 'expert' users are not sweating this BSOD, since they can simply drop into DOS and replace the two DATs, and all would be back to normal.
Registry CREATION Error
. So I reset the computer ( BTW, that BSOD is a hard lockup, CTL-ALT-DEL will not work here! Only a Power-Off cycle or pressing a physical RESET button attached to an ATX+ power supply will work ). As it booted I got into F8 Command Line DOS and CD'd over to my test directory where I have an ASCII Export of this bloated registry waiting for me ( this should be fun, NOT! ). So now we nicely ask the official Microsoft sanctioned REGEDIT to create a new set of DATs from the 25,804,411 byte
registry export. REGEDIT /C Export.reg
... and ... be patient ... really ... really ... REALLY ... patient! ...
Ah fun times again. That camera shot was at 11 minutes into the nightmare. But why explain when I can just show you the actual times that I checked ...
06 minutes = 12%
08 minutes = 13%
10 minutes = 17%
11 minutes = 18%
1.5 hours! = 26%
2.5 hours! = 28%
3.0 hours! = 30%
3.5 hours! = 31%
4.0 hours! = 32%
4.5 hours! = 33%
5.0 hours! = 33%
5.5 hours! = 34%
6.0 hours! = 35%
6.5 hours! = 36%
7.0 hours! = 36%
7.5 hours! = 37%
8.0 hours! = 38%
8.5 hours! = 38%
~12 hours! = Failed
Words cannot describe the feeling after one of those failures in the old days ( well there certainly are words for this, but they are NOT appropriate for this forum ). There must have been more than a few occasions where a sledgehammer met a Win9x computer. Most often I would let the machine go overnight, and hope that I didn't wake up to this ...
Never mind the fact that the "previous registry
" happens to be the one that blue screened with the bogus VFAT error.
UPDATE: In the comments below, Rudolph Loew reminded me to use SmartDrv in the DOS registry creation process. Talk about Doh! Trust me, I knew this 17 years ago, did it routinely, but I completely forgot about it when doing these tests. That disk cache absolutely did substantially speed up the process.
UPDATE-2: I have completed running this operation four more times. Test was conducted with both 512 MB and 1GB of RAM, once each with and without SmartDrv. Running this disk cache will effectively increase the speed of the operation six-fold, reducing the entire duration from 12-hours to 2 hours! Click the following button for the results ...
I had a variety of solutions back in the day. The most elaborate was to use a different computer, the fastest single-core that could be found and perform the operation there ( REGEDIT /C is simple binary output, it has no machine check or error testing ) and grab the new files if it was successful. Actually, this computer is clocked just shy of 3 GHz so there really isn't anywhere to go except up a few hundred MHz ( at least in the Intel universe ). There was no way to patch WinME REGEDIT ( that I am aware of ) to use two hives and backport it to Win98se as we had done in the past, even if it would somehow work on larger files. This sums up the pressing need for a way to CREATE a new registry in Windows. Hence the reason I created a kludge utilizing RegCompact we'll see later.
If someone wanted to pursue this further, perhaps to trigger some of the other possible variables instead of size ( e.g., registry complexity, available RAM, CPU model, speed, cache ), there are many possible ways to go about it. For testing memory amount
, obviously RAM can be simply pulled out ( NB: this computer was 512 MB ) and replaced with smaller amounts down to even 8 MB. Inversely, someone might try with more RAM. For complexity
one might edit the script to use multiple subkeys ( e.g., HKLM\Software\Microsoft\1\2\3\4\5\6\7\8\...\999\ ) or just research what happens in some versions of RealPlayer
that installed values
that were too long, among other mistakes. If the BIOS allows you could DISABLE the CPU L2 cache, or underclock the frequency to see if the registry can be loaded successfully. However, I will not likely be doing this myself because this a 17 year-old nightmare that I would prefer to leave alone. This was most likely the last time for me.
How Do You Protect Yourself?
Just keep a shortcut handy to a batch file that copies the two binary registry files to a folder on demand. Run it often. Definitely run it before installing some gigantic pig of an application or SDK that might push your registry past the point of no return. I always used one that added a DateTimeStamp as a suffix to the files and ran it often, piling up a collection of registries to rollback to if I ever needed it. Yeah, SCANREG can be set up in a similar fashion, but this is much faster and no CAB files are involved. This one has only one dependency, a 3rd party 16-bit DOS app called FDATE
that is still easily found ( link is in comments within the .BAT). Click to see it ...
ORIGINALLY POSTED: 2012-09-14. Okay, that's all for now. If anyone can add anything to this please leave details in the comments. Also, if you spot any errors, typos or otherwise please let me know. Thanks!
EDIT: typo, added registry backup BAT file, colors
EDIT: 2012-09-18 ... re-ran registry CREATION and added spreadsheet comparison
This post has been edited by CharlotteTheHarlot: 18 September 2012 - 08:52 PM