[Mordac85]

This is a topic split from another thread. In that thread, nivlacckw brought up a novel idea to adapt a physical to virtual (P2V) process to alter the HAL and add mass storage drivers to a system after an image has been laid down, but before mini-setup runs. The page he referenced is the Ultimate-P2V (http://www.rtfm-ed.co.uk/?page_id=174).

On the site, there is a reference to Notorious_bdg’s “HAL_Update” script (http://www.rtfm-ed.c.../HAL_Update.txt) which seems to detect and change the HAL using devcon. This only appears to be written for 4 HAL's but either the script could be completed to cover all of them or, depending on the licensing, the devcon source could be altered to create a utility that provides only this very defined function. Maybe it could also include a function to detect the mass storage controller and pull from a predefined driver folder to inject the correct drivers as well as updating the HAL?

He's making a similar inquiry on the BartPE forums.

nivlacckw, on Jul 11 2006, 11:41 PM, said:

Post in 911 CD forum to draw Pebuilder's community attention

http://www.911cd.net...showtopic=17752

So I just wanted to see if anyone can see problems with this process or does it seem to be a valid use of devcon? If you boot from a PE CD and make these types of changes will there be any problems when you boot?

[Scrapple]

This method is definately the best way to universal imaging as far as I can see.

How nice it is to: Just deploy any XP image (sysprepped or not, drivers or not) to any (virtual/real) machine and change some files/registry on the harddisk with a universal tool before you boot from it.

BTW, there are already 2 commercial tools doing this:
-Symantec Restore Anywhere
-Acronis Universal Restore

Their disadvantage is that they only handle massstorage and HAL, but don't handle pnpdrivers and they're add-ons to the company's own image-product. (Ghost, True Image)


BUT: You can also write such a tool yourself. It's not rocket science...


detect the HAL from the following key in BartPE:
HKLM\SYSTEM\CurrentControlSet\Enum\Root\ACPI_HAL\0000\HardwareID and if that does not exist
HKLM\SYSTEM\CurrentControlSet\Enum\Root\PCI_HAL\0000\HardwareID

If the HAL in the offline image is different from the one detected, then extract the right latest HAL + kernel on the offline image (usually SP2.CAB) and rename them to their proper names in C:\Windows\system32

load the C:\Windows\system32\config\system to a temp hive

empty the "Enum" branch in the loaded hive so it will be filled properly at boot.

detect the Processor vendor from BartPE with:
HKLM\HARDWARE\DESCRIPTION\System\CentralProcessor\0\VendorIdentifier
... if AMD then change the Startvalue of the intelppm service to 4 in the loaded hive, otherwise most AMD's will bluescreen.

So... that's pretty much the HAL detection/changing routine

That HAL script you mention in your link does not work when run from BartPE/WinPE, because it does't handle the registry in the offline image... it's also not always right on the HAL to be installed.



Now for the Mass Storage drivers it's more difficult. But basically comes down to: Detect the PNP-ID of the boot-device, find the correct .inf file on a driver distribution share mounted from BartPE and install the .sys, .cat files in the offline image. AND very important to import the proper entries in the CiriticalDeviceDatabase and Services section of the loaded hive so the pc will not get a bluescreen (7B error). To do this is too much to explain here in detail.


PNP-drivers is easy: detect the PNP-IDS of the hardware (also within BartPE) and find their corresponding .inf files (and .dll, .cat, .sys...etc...) in the extracted BTS driverpacks on a distribution share and throw them in the offile image in a folder i.e. C:\drivers.

If mini-setup or OOBE is set to run because sysprep was the last action taken before capturing the image (See certain: HKLM\System\Setup values) then,
Save the HKLM\System\Setup\Cmdline string value of the to i.e. C:\setup.cmd.
Change the value to "C:\mysetup.cmd"
copy over setupcopyoeminf.exe to C:\
Copy over/or create the C:\mysetup.cmd with the following lines:

C:\setupcopyoeminf C:\drivers
call C:\setup.cmd
del C:\*setup.cmd
del C:\setupcopyoeminf.exe

If sysprep was not used, then mysetup.cmd should be set to run from a runonce entry where also devcon.exe is being injected and a "devcon rescan" is being run from within mysetup.cmd


So...I'm probably forgetting some stuff, but that's basically it. A universal XP (image) changer. Let's make this a BartPE plugin called: "SysWrap" :-)

But hey, besides the obvious sys-admins/deployers, it could also be used for people changing motherboards/computer and wanting to keep their installation, and what about P2V, V2P, P2P, V2V.

For an imaging product I definately would use Vista's WAIK "ImageX" or script it's API "WIMGAPI.DLL"

This post has been edited by Scrapple: 12 July 2006 - 12:06 PM

[nivlacckw]

Some more external reference of this concept from Microsoft

http://download.microsoft.com/documents/uk...eploying_XP.ppt

[IcemanND]

actually there is a third comercial program that does this, the UIU (Universal Imaging Utility) from Binary Research the original makers of Ghost. It handles HAL, massstorage and includes a drivers database.

[IcemanND]

Quote

detect the HAL from the following key in BartPE:
HKLM\SYSTEM\CurrentControlSet\Enum\Root\ACPI_HAL\0000\HardwareID and if that does not exist
HKLM\SYSTEM\CurrentControlSet\Enum\Root\PCI_HAL\0000\HardwareID

Is this different between BartPE and WinPE? In BartPE the ACPU Uniprocessor comes up as
HKLM\SYSTEM\CurrentControlSet\Enum\ACPI_HAL\PNP0C08\0\HardwareID

I don't currently have access to my WinPE install, it's in my other office.

[Scrapple]

IcemanND, on Jul 13 2006, 06:27 PM, said:

actually there is a third comercial program that does this, the UIU (Universal Imaging Utility) from Binary Research the original makers of Ghost. It handles HAL, massstorage and includes a drivers database.

This UIU tool is too "old-fashioned" and too expensive... you have to run their tool (that runs sysprep) and after that recapture everytime a new driver is needed. Your image file swells due to their driverpack (which is also pretty limited compared to the BTS packs). It does not support RAID. You have to make two images if some of your computers are not ACPI capable. I've pretty much reverse engineered their method and it's not that difficult to make either.

The beauty of the "new method" that is describe above, is that:

- it's possible to use ANY (old) XP image you (once) captured
- do not depend upon sysprep to be run beforehand
- it can inject only the needed drivers, you don't have to put them in the image
- you can also update your boot/pnp-drivers without recapturing on a net-share
- can update HAL/kernel for ACPI <-> NON-ACPI (1 image is possible)
- it does not depend on a certain image product (granted: UIU isn't either)

[IcemanND]

You must have used thier last version not the new one, new one will do scsi, all HAL types, and at least for the machines I've done my testing on their driver database was more complete than BTS's.

Yes, is still uses sysprep, but for most of my images I do also.

If you want to edit the image make if fat32 and you can modify the image before deployment. At least to an extent, no you can't edit the registry.

I have not tried symantec restore anywhere, I'm hoping to get a copy soon to play with. I personally did not like acronis' method, but I also did not want to change imaging software either.

Not trying to argue. Just trying to get as much information out there for the poor souls who will stumble accross this thread. Finding some of this information has been a real pain in the but.

[Scrapple]

IcemanND, on Jul 13 2006, 07:05 PM, said:

Is this different between BartPE and WinPE? In BartPE the ACPU Uniprocessor comes up as
HKLM\SYSTEM\CurrentControlSet\Enum\ACPI_HAL\PNP0C08\0\HardwareID

I don't currently have access to my WinPE install, it's in my other office.

you looked under the wrong key... so don't forget the sub-branch "Root" in the key I mentioned:

HKLM\SYSTEM\CurrentControlSet\Enum\Root\ACPI_HAL\0000\HardwareID

or for non-acpi

HKLM\SYSTEM\CurrentControlSet\Enum\Root\PCI_HAL\0000\HardwareID

I think it is the same for WinPE, but it's definately ok in BartPE.

BTW I know another extremely novel way to detect the HAL... To let windows detect it itself on first boot... just like unattended does it in text-mode. If you guys are interested in this kind of stuff I'll share it with you.

[IcemanND]

Quote

BTW I know another extremely novel way to detect the HAL... To let windows detect it itself on first boot... just like unattended does it in text-mode. If you guys are interested in this kind of stuff I'll share it with you.

I would like to see that. If you'd be so kind. Thanks.

[Scrapple]

IcemanND, on Jul 13 2006, 08:32 PM, said:

You must have used thier last version not the new one, new one will do scsi, all HAL types, and at least for the machines I've done my testing on their driver database was more complete than BTS's.

Yes, is still uses sysprep, but for most of my images I do also.

If you want to edit the image make if fat32 and you can modify the image before deployment. At least to an extent, no you can't edit the registry.

I have not tried symantec restore anywhere, I'm hoping to get a copy soon to play with. I personally did not like acronis' method, but I also did not want to change imaging software either.

Not trying to argue. Just trying to get as much information out there for the poor souls who will stumble accross this thread. Finding some of this information has been a real pain in the but.

of course we're not argueing. It's a highly interesting subject to me. So I'd like to exchange a lot of info to find the most perfect way to do imaging. And yeah, I definately agree that finding and learning all this stuff on the net is very time-consuming and goes pretty deep into the system sometimes.

I'm gonna look into the new version of UIU as soon as I have the time to research it. What version is it? The last version I looked into is that of june 06. I hope they have a better one now.

My driver-folder on the share is over a GB extracted and contains BTS driverpacks + extra diverpacks like HID, TV, modem, monitor etc... I'm very curious if the new UIU can top that...

About a Fat32 image... I know with ghost explorer i could edit that, but I'm using ImageX which isn't a sector based image, but file based, so it even allows you to keep the previous data and/or OS on the partition/disk after you deploy the new image. It also includes a filter, so you can mount and edit the image file (even script it!) It's kinda like a command line winrar.

[Scrapple]

IcemanND, on Jul 13 2006, 09:40 PM, said:

Quote

BTW I know another extremely novel way to detect the HAL... To let windows detect it itself on first boot... just like unattended does it in text-mode. If you guys are interested in this kind of stuff I'll share it with you.

I would like to see that. If you'd be so kind. Thanks.

Ok, here's how you do it:

Get the longhorn beta 1 "NTLDR" file and put it in the root of your xp image

add an option to the line in boot.ini, like:
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP Professional" /noexecute=optin /fastdetect /detecthal

then create a file named dtecthal.inf in C:\windows\inf that consists of the following

[Version]
signature="$Windows NT$"
DriverVer=07/01/2001

[hal]
MPS_MP		 = halmps.dll
MPS_UP		 = halapic.dll
E_ISA_UP	   = halstnd.dll
ACPIPIC_UP	 = halacpi.dll
ACPIAPIC_UP	= halaacpi.dll
ACPIAPIC_MP	= halmacpi.dll


[ACPIOptions]
ACPIEnable = 2
ACPIBiosDate = 01,01,1999

extract all HALs to system32 and rename them to the filenames above.
extract all kernels to system32 folder


Basically what MS did is incorporate the HAL detection of the text-mode (unattended XP) in the NT loader of Longhorn. That was still the time in which they had the many kind of HALs we know from XP.

The vista Beta's do NOT use/need this method, because the bootloader is different and they have only 1 universal HAL file (which BTW can't be used in XP...)

Hope anyone finds this useful.

[IcemanND]

The version of UIU I was using that would do all of the HALs and SCSI was from March/April timeframe this year.

Sounds like they have gone backwards since my testing earlier this year. Sad if they have, the version I had worked very well for what we wanted to do.

I don't think that work will let me use ntldr from vista beta 1. And I suppose the detecthall switch is not in XP. So I guess I'll be stuck with the slightly more dificult methods.

[Mordac85]

Scrapple, on Jul 13 2006, 05:14 PM, said:

extract all kernels to system32 folder

What do you mean extract all kernels? I thought there was only one so could you be more specific?

And I take it the Longhorn Beta 2 that's out there now won't cut it, right?

This is really an interesting topic for me. I know there are several commercial solutions out there, but I can't see paying the money for something that can be done just as easily w/a script or utility. Plus, I'm always intrigued by finding out how it works under the hood.

[IcemanND]

is there someplace that all of the switches for setup.exe are, I'm referring to the one which is run on first boot after sysprep is run.

-newsetup
-mini

are the two I know of off the top of my head, are there others that might do things that would help this effort?

[Bilou_Gateux]

I'm currently working of this kind of solution:

1/ I'm starting with a RestoreCD Plus from HP and a 2nd CD which contains an OEM Windows XP Professional OS sysprep'ed in factory mode.

2/ I boot the 1st CD, let the restore process copy required files and then ask for the 2nd CD (OS) and restore it, reboot the computer a first time to let HP tool checks hardware and configuring drivers and applications and when computer reboot, boot from WinPE and backup the HDD with a cloning tool to a VMWare virtual machine.

3/ inside the virtual disk (.vmdk) which i can mount with Ken Kato VDK driver on my host computer as drive, i can do all the changes required (inject drivers, configuring sysprep.inf oobeinfo.ini and winbom.ini, make appropriate changes to registry hives).

4/ and then restoring the virtual machine disk to a physical computer.

Some limitations: my OS is OEM preactivated and i can't use the image on different brand of computers.
I have some problems with oobeinfo.ini to make the first boot fully unattended.

[Scrapple]

IcemanND, on Jul 13 2006, 11:36 PM, said:

The version of UIU I was using that would do all of the HALs and SCSI was from March/April timeframe this year.

Sounds like they have gone backwards since my testing earlier this year. Sad if they have, the version I had worked very well for what we wanted to do.

I don't think that work will let me use ntldr from vista beta 1. And I suppose the detecthall switch is not in XP. So I guess I'll be stuck with the slightly more dificult methods.

About UIU: You really give it too much credit. Granted, they do a fine job on most systems... possibly all pc's in your company, but they will NOT detect just any Hal (or have just any driver issue resolved for just any system, which I know you haven't stated)

If you read the documentation of UIU (even that of march/april) you can read under the topic troubleshooting that when you've captured an ACPI image and deploy it to an non-acpi system, it will crash on boot. So you would still need 2 images with UIU, like I said before. UIU is therefore not really "universal", which they like to claim.

Other things they admit they cannot do is:
- deploy to RAID
- capture from SCSI and deploy to IDE, Pata/Sata

They also have missing/incorrect mainboard drivers sometimes, which crashes the deploy pc. And driver bangs (missing drivers) for bleeding edge hardware is also an issue.


About the longhorn NTLDR: You could use it only once and swap back the XP one on second boot once the HAL is properly detected and installed. If even that is out of the question, then use the method of WinPE reg HAL detect/swapped described in my long post. If you're not allowed to HAL swap at work (because it DOES break MS support!), then be prepared to make and manage multiple images... and also throw the word "universal" out the window, because it is simply not applicable anymore.

indeed, the "/detecthal" switch only works on Longhorn beta 1 NTLDR. XP's NTLDR is unaware of it.

Mordac85, on Jul 14 2006, 01:16 AM, said:

Scrapple, on Jul 13 2006, 05:14 PM, said:

extract all kernels to system32 folder

What do you mean extract all kernels? I thought there was only one so could you be more specific?

And I take it the Longhorn Beta 2 that's out there now won't cut it, right?

This is really an interesting topic for me. I know there are several commercial solutions out there, but I can't see paying the money for something that can be done just as easily w/a script or utility. Plus, I'm always intrigued by finding out how it works under the hood.

well, I mean kernels like: ntkrnlmp.exe, ntkrnlpa.exe, ntkrnlup.exe and ntkrpamp.exe

Yes, sadly Vista Beta 2 is of no use for us in this HAL swap thing.

This post has been edited by Scrapple: 14 July 2006 - 10:37 AM

[Scrapple]

IcemanND, on Jul 14 2006, 01:31 AM, said:

is there someplace that all of the switches for setup.exe are, I'm referring to the one which is run on first boot after sysprep is run.

-newsetup
-mini

are the two I know of off the top of my head, are there others that might do things that would help this effort?

Well factory and audit mode exist, of which factory can process winbom.ini (like winpe), but even if some secret setup.exe parameter existed... it wouldn't be of great use for switching hals, because XP has to be almost completely boot up, before it kicks setup.exe. If you had captured an image with a hal that was incompatible with the deployed system... it would've never even made it that far into the boot-process and would just reboot/blue screen before it could detect the HAL... and THAT is exactly why UIU has a big limitation and detecting/swapping the HAL with WinPE/BartPE is the way to go.

BTW, msoobe.exe is usually the program that's run with the cmdline reg-entry on a new system... most OEMs use it to enter the WinOOBE mode many of you are familiar with.

[IcemanND]

As far as the UIU, I built my test image on an ACPI system and deployed it to the Standard HAL, ACPI uni/multiprocessor HAL, ACPI without any problems. These machines ranged from a Dell GXA to the current Lenovo M52 desktop, there were a few drivers missing on the newest machines but all it took was a phone call and a days wait and they gave me an update to their driver pack.

The driver issue I've had with BTS's packs also so it's not a complete solution for drivers either.

I did not try going from SCSI to PATA/SATA but I did go the other way without issue.

The only driver issue they could not solve was for the TPM on the thinkpad systems which was not properly signed by the manufacturer.

I don't think that HAL swapping would be an issue but using beta compinents usually is.

[Mordac85]

Scrapple, on Jul 14 2006, 12:42 PM, said:

well, I mean kernels like: ntkrnlmp.exe, ntkrnlpa.exe, ntkrnlup.exe and ntkrpamp.exe

I knew about the various HAL's being used to create hal.dll, but I never thought they did the same thing for the kernel. Are there any references, that you know of off the top of your head, for which one is used in a particular instance?

So we'd have to include placing the correct kernel as well as the HAL? Doesn't seem like a major deal, unless there are other supporting files that follow the same concept. Thanks for the tip.

I think the premise of making this switch via a script, or coded utility, is possible. Granted MS won't touch it, but I never bother with that avenue anyway cause it usually takes way too long to get a resolution. I think I have enough to get started, now I just need to find the bandwidth.

[Scrapple]

IcemanND, on Jul 14 2006, 06:30 PM, said:

As far as the UIU, I built my test image on an ACPI system and deployed it to the Standard HAL, ACPI uni/multiprocessor HAL, ACPI without any problems. These machines ranged from a Dell GXA to the current Lenovo M52 desktop, there were a few drivers missing on the newest machines but all it took was a phone call and a days wait and they gave me an update to their driver pack.

The driver issue I've had with BTS's packs also so it's not a complete solution for drivers either.

I did not try going from SCSI to PATA/SATA but I did go the other way without issue.

The only driver issue they could not solve was for the TPM on the thinkpad systems which was not properly signed by the manufacturer.

I don't think that HAL swapping would be an issue but using beta compinents usually is.

Sure, it's possible to check the "Install Standard HAL" checkbox in UIU before you capture the image. It will install perfectly on most any system, but with very limited Advanced Power Capabilities on ACPI machines afterwards and lower performance for Hyperthreading machines.

But... I have the feeling you're gonna tell me you deployed a real ACPI image to a non-acpi machine and suffer no APC disadvantages. :-) If you succeeded in doing that... that's really "wonderful?" and pretty rare scenario... but it's really not something UIU anticipated on or even supports, let alone MS. Because HAL/Kernel wise it would be the most undesirable situation you could boot a pc under... HAL-wise, that is. It will drastically affect the way some drivers communicate with the hardware. I would be very surprised if it would even run stable.

I very much agree on the BTS packs... they are not the be-all-end-all solution. It needs updating and sometimes tuning too... but you can do it yourself...offline... so you don't need to include/update them IN the image, which is a big plus.

Yes, PATA->SCSI works flawless with UIU. They state/support that. I wouldn't expect any different.

About the thinkpad:Exotic unsigned drivers (usually laptops) will always be a pain in the a**... I've had my experiences...:-)

I wouldn't use the longhorn NTLDR in a heavy production environment too... but it works and is a pretty novell way of doing HAL detection/installation, so that's why I shared it with you deploy geeks... :-)

How much do you pay for the UIU license per seat if I may ask? I thought it was something like $19 dollars or $10 bucks high-bulk. For some that is cheap... other might find it very expensive... the same goes for Acronis and Symatec's solutions.

A tip: You could also write a tool like UIU yourself... check ta.exe (from XP Embedded) for hal detection and rundll32 or devcon for hal installation. And sysprep with -noreboot switch to change the HKLM\system\setup\cmdline reg-entry to your own hal-routine after sysprep has run.