Formatting an external drive using different interfaces

[Dave-H]

Don't worry @bphlpt I'm sure @jaclaz will publish all the information once we've finished experimenting!

@jaclaz, everything seems to be generally working fine, but observations below.

on Windows 8.x (or more generally in Vista and later) prompting for UAC when needed

I'm getting the UAC prompt on the eSATA connected machine, but not on the USB connected machine.

on Windows XP not prompting for UAC

I'm still getting the prompt on XP on both machines.

and YES and yes would do as well

Thanks for that, I'm sure people would have been caught out by it!

[jaclaz]

I'm getting the UAC prompt on the eSATA connected machine, but not on the USB connected machine.

Which should mean that the UAC prompt is *needed* on eSATA and NOT *needed* on USB, I can see no issues here,

but there must be some communication issues:

I'm still getting the prompt on XP on both machines.

I can't say anything (and I have to rely on your reports) about the elevate.exe behaviour on Windows 8.1, but this makes no sense, i.e. it is conflicting with what you already reported and with my almost ten years long experience running dsfi on XP, this line::

dsfi \\.\%SecondDriveLetter% 0 0 %Source% || elevate.exe -c -w dsfi \\.\%SecondDriveLetter% 0 0

 

Run the dsfi and IF IT FAILS (i.e. it reports an error) THEN run it through elevate.exe.

I can assure you that dsfi on Windows XP does NOT fail (provided that the arguments supplied to the dsfi command are correct), can you please double and triple check that you edited correctly the batch and that you are running the edited batch?

 

Try again changing the line to:

dsfi \\.\%SecondDriveLetter% 0 0 %Source% || PAUSE&elevate.exe -c -w dsfi \\.\%SecondDriveLetter% 0 0

 

On XP, you should get this message:

OK, written 4096 bytes at offset 0

(which means that the first command did succeed and NOT be prompted for UAC - because the prompt with UAC only happens because the "plain" command failed and it is re-run through elevate.exe).

On Windows 8.1 you should see the same on USB while on eSATA you should see the "press any key to continue" before being prompted by UAC elevation.

 

jaclaz

 

[Dave-H]

OK, I decided I was probably getting into a bit of a mess here, so I went right back to the start, deleted all the partitions and started again!

 

The prompt is not now coming up on XP at all, only on 8.1 when connected via the eSATA interface.

I'm wondering if this might be because the USB connected computer treats the drive as a removable one which doesn't have such high security requirements?

 

One thing I have noticed is that if I try now to edit switcher.cmd with my normal Notepad substitute, NoteTab, it tells me it may be a binary file and converts some characters to apostrophes. This has only recently started happening, and might have caused some corruption.

I'm now making sure that I only edit it with the old Windows Notepad!

 

All seems good now!

Edited January 16, 2015 by Dave-H

[dencorso]

When you open "My Computer" in Win 7 it shows the fixed and removable media separatedly, so it's easy to know whether the OS deems some HDD as fixed or removable... I presume 8.x+ does the same, so it's easy to be positive regarding how the OS classifies some HDD. So:

Are you positive 8.1 is seeing eSata as fixed (= internal) and USB as removable? Can you provide screenshots of "My Computer" showing it? It'd great as an evidence for later times, your set-up, in this case, is very uncommon, just because dual-interface enclosures are pretty uncommon, in fact, and that probably won't change, because USB 3.0 tends to kill both eSata and FireWare or, at the very least, render them yet more uncommon than they already are...

[jaclaz]

Good.

I believe the difference is as said earlier:

http://www.msfn.org/board/topic/173265-formatting-an-external-drive-using-different-interfaces/?p=1092545

 not so much between "Fixed" and "Removable" (as both hard disks will be sensed as the "Fixed" type, i.e. devices that can be partitioned normally, unlike USB sticks) but rather "internal" vs. "external", i.e. suitable to hold a page file or not, we have the paradox that you can probably put a pagefile on the disk when it is eSATA connected but not when it is USB connected (unless you add Diikmod as a filter driver).

 

The good (little) news are that I was a bit too pessimistic regarding the "wasted" space due to the overlapped FAT12 volume, by choosing some "smart" values I can "squeeze" some 2,800,000 available bytes out of a volume sized 3,145,728 bytes :.

 

Now a few open questions. on which your data and opinions are welcome.

This 3 Mb volume (which may also make the disk bootable, at least when connected through the interface that provides 512 bytes/sector) is IMHO more than enough for hosting on it *any* kind of loader/bootmgr, but I believe that the effect of a "plain" BCDBOOT.EXE command in Vista, 7 or 8.x is to produce a stupid amount of files, including fonts and largely useless "localization" files, or if you prefer, besides BOOTMGR the \boot\ directory has grown to some 20 or more Mb.

Can you check the size of your installed Windows 8.x \boot\ directory?

Would it be useful to make the first volume grow to 16 or 32 Mb?

Or maybe even bigger, switching when needed to FAT 16?

I guess that it would be acceptable to lose a few tens of Mb out of a disk that is at least 1 Tb, and since I don't like "one size fits all" solutions (basically as they never fit correctly anyone) maybe it is a good idea if I produce two "base images", one 3 Mb and one the needed size for people that want to have a "full" \boot\ directory.

Or do you think I should make a batch to allow the final user to also create the "base image" of (within limits) arbitrary sizes? 

 

jaclaz

[dencorso]

I believe the difference is as said earlier:

http://www.msfn.org/board/topic/173265-formatting-an-external-drive-using-different-interfaces/?p=1092545

 not so much between "Fixed" and "Removable" (as both hard disks will be sensed as the "Fixed" type, i.e. devices that can be partitioned normally, unlike USB sticks) but rather "internal" vs. "external", i.e. suitable to hold a page file or not, we have the paradox that you can probably put a pagefile on the disk when it is eSATA connected but not when it is USB connected (unless you add Diikmod as a filter driver).

 

Yes... No. My lovely Kingston DataTraveler Workspace 64GB, which is a common pendrive (although much maligned by the likes of you, who insist in saying the poor thing is a transvestite SSD, just because it has a decent SF-2241 SandForce controller), not only is seen as fixed by XP SP3, but also XP SP3 will gladly put the pagefile (and the hiberfil too, when hibernation is enabled) without DIskMod, and actually boot from it, from the BIOS (or using PloP, if required), when connected as "USB 2.1" to an USB 2.0 port. I don't have any machine capable of booting directly from USB 3.0, but I do belive it should behave the same, if connected instead to and USB 3.0 port. And, moreover, Dave-H's dual-interface enclosure has a USB 2.0 connection (not a USB 3.0), AFAIK, so the parallel applies perfectly. Therefore "no": USB cannot be internal for me and external for Dave-H at the same time. 

[Dave-H]

I've checked and the drives are appearing on both machines as "local disks", although when they first appear on both machines the pop-up appears in the top right hand corner for both of them saying "click or tap to select what happens with removable drives", if that means anything!

 

The Windows 8.1 "boot" folder on both machines is 17.8 MB in size.

[jaclaz]

@dencorso

Well, then you should send your transvestite SSD lovely fastish and pricey USB stick to Dave-H to let him ruin test it  with my batches  or, alternatively, you could install 8.1 to see how the stupid Windows 8.x behaves with it.

It is very possible that something changed in 8.x when compared to Windows 7, possibly connected/related directly or indirectly with the Windows to go (or *whatever*).

 

Your stick is for all windows knows, a USB connected SATA SSD, basically because (according to this source):

http://www.thessdreview.com/our-reviews/kingston-datatraveler-workspace-32gb-windows-to-go-usb-3-flash-drive-review/

1. it contains a  SSD (Sandforce SATA) controller
2. it contains a (Genesys) USB to SATA bridge 
3. it is USB connected

I thought we had agreed to disagree....

http://reboot.pro/topic/19827-sandisk-extreme-pro-usb-30-flash-drive/

 

@Dave-H

Good , so 20-24 Mb space should be enough.

 

jaclaz

[dencorso]

@dencorso

 

[...]

 

I thought we had agreed to disagree....

http://reboot.pro/topic/19827-sandisk-extreme-pro-usb-30-flash-drive/

 

Ooops!... Yes! we did and I still abide by that agrrement.

 

I thought however that our agreement wouldn't prevent offering an example of a USB-connected "internal" disk. 

If, in fact, it does, do please consider my example officially retracted.

[NoelC]

It is very possible that something changed in 8.x when compared to Windows 7

 

This brought to mind a thought...

 

If an infinite number of monkeys were placed in front of an infinite number of keyboards outfitted only with Delete keys, sooner or later one would delete just the right functionality so that the operating system would still function, but would be a little more irritating to use.

 

-Noel

[jaclaz]

Ok, I have checked several possible approaches, of them only two passed the tests (aiming to have something reproducible and extendable/modifiable without size constraints, queer behaviours or more generally producing less compatible results).

The partitioning schemes consist in either a Primary partition with a small FAT12 volume + the "rest of the disk" in a (huge) NTFS logical volume inside extended (which I cleverly called "PriLog") or in two logical volumes inside extended, the first being the small FAT12 volume and the second the huge NTFS volume (which I expectedly called TwoLog).

The second one did not make it after a pass of Occam's Razor as it involved a double-double EPBR (too complex and more prone to errors) while offering NO advantages (if not that of using only one MBR partition table entry, something not really an issue for the specific use of a dual USB/eSATA connection of an external disk) whilst the PriLog may offer "direct" bootability (at least on 512 bytes/sector interface) and when booting the switch may be offered by - say - a simple, small grub4dos batch.

There is a brief ReadMe.txt in the attached archive that should be READ (and READ CAREFULLY) before using the batches, which are however HEAVILY commented, so that it should be not difficult to understand the behaviour/functioning of the thingy.

Use of them is of course experimental, and even more obviously it is potentially dangerous , though I tried my best to add as many checks as possible (may be even too many of them).

They seem to work fine in my XP on a (simulated/virtual) disk mounted as either 4kb or 512 bytes/sector device, but I need feedback on how it behaves on Vista (and later).

I am adding also a simple Excel spreadsheet from which it may be easier to understand the scheme, basically we make a (fake) Extended Partition only one sector in size that magically contains either the whole disk (on 512 bytes/sector device) or only the logical (NTFS) huge volume (on 4kb/sector device, at least this is what Windows Disk Manager thinks .

jaclaz

mkPriLog.zip

Pri_TwoLog.zip

[Dave-H]

Thanks jaclaz.

I've collected all the necessary files into a folder and I'll be experimenting with this tomorrow!

The contents of your spreadsheet mean absolutely nothing to me I'm afraid, way over my head!

I'll let you know how it goes.

[Dave-H]

Hi again jaclaz!

Been trying this out using Windows 8.1 and the USB interface, which is the correct one for 4k sectors I hope.

 

You say in the readme "To run the mkdualmode.cmd, you need to have already deployed the prilog.cmd image to the target disk".

I assume that "mkdualmode.cmd" is the same file as "mkdualdisk.cmd" which is what you've actually called it.

 

The other thing is that I think you need to tell people how to actually do that.

I ran the command "C:\MkPriLog\dsfi \\.\ Physicaldrive1 0 0 PriLog.img", which seemed to work, but I had to look back in this thread to remind me how to do it!

 

I now have my drive reported in Disk Management as two partitions, a 256 MB FAT one, and a 931.26 GB one which I've formatted as a primary partition.

There are also two drives in "My Computer", a 31.6 MB one, and a 931 GB one.

 

I'm now a bit stuck, as the readme says to now copy dsfi.exe, as4kbNTFS.bss, as512NTFS.bss, elevate.exe, and Switcher.cmd, presumably to the 31.6 MB drive.

Where exactly are the two *.bss files?

Should they have be included from the start or should they have been generated, and if so where?

 

Cheers, Dave.

[dencorso]

<Somewhat Off-Topic:>

 

I actually think that's right out or order, selling on a hard drive with someone else's data still on it without re-formatting it, or at least deleting the partitions.

I will be contacting the person i bought it from to tell them this.

There was nothing sinister on it I'm glad to say, but even so I think this it's wrong on principle!

 

What reply did you get? Were you able to contact the original owner, too?

[jaclaz]

Yep, I must have changed names while developing the batches , the correct file name is "mkdualdisk.cmd" of course, I'll edit/correct the readme.txt.

 

When you run mkprilog.cmd, at the end it should display:

The image should have been created as Prilog.img, now you need to
manually dd it to the destination disk, a suitable command would be:
dsfi.exe \\.\PhysicalDrive[n] 0 0 Prilog.img

Make sure that you use the RIGHT Disk Number [n], do check in Disk Manager.
The disk can be mounted indifferently through the 4kb or the 512
bytes/sector interface to deploy the image.
Once the image has been deployed you may need to disconnect and reconnect the
disk to have the FAT12 volume be recognized and mounted.
Then you can go to Disk Manager and create a Primary partition, NTFS formatted
extending on all the available space on disk (or on just a portion of it).
It is "better" to use a Vista (or later) OS to create the NTFS partition
The partition would be better if Megabyte aligned.
(or do it manually or use a suitable third-party tool) as the XP will align it
on head/cylinder boundaries, whilst a MB-aligned partition will be faster on
slow buses such as USB 2.0
The partition and the NTFS filesystem MUST be created when the disk is connected as
a 4kb sectored device, this is NOT negotiable.
Once the second partition has been created, and I repeat, this MUST be done
with the device connected as 4kb sectored, you can run the mkdualdisk.cmd
Which will modify the disk and create the needed EPBR's.

 

 

Now , check the drive letter assigned in Disk Manager to the 256 MB FAT (or assign to it one), you will see how that that same drive letter in "My Computer" is assigned to the 31.6 MB FAT partition (the size of the FAT partition comes from the choice you made in mkprilog.cmd, where you are asked to provide a size for the partition between 1 and 32 Mb).

The batch create however an image with only one partition, the first one, primary (on the left in Disk Manager) which is a (smallish) FAT(12)  one and the second one (on the right in Disk Manager) which is a (largish) NTFS partition that you create manually in DIsk Manager, which is later trasformed by the riunning of mkdualdisk.cmd into a Logical Volume inside Extended so guess to which partition I refer with:

Once the mkdualmode.cmd has been run successfully, you need to copy to the first (FAT12) primary partition ...

 

 

I know that the choice of using dark blue for primary partition and of light blue for logical volumes and a tiny light green border around the Extended partition is not the best choice in the world to distinguish something visually, but you should sue MS for this .

 

I cannot refer to the partition as having a given size, as it depends on the choice you make when running mkprilog, and the size of the "other" partition of course depends from the size of the disk and/or from the size of the partition you created manually in Disk Manager, I thought, that order (first), filesystem used (FAT12) and nature of partition (primary) was clear enough, but I am open to any suggestion to make this more clear. 

 

The .bss files should have been generated in the same folder where mkdualmode.cmd was run, i.e. C:\MkPriLog\ in your case, and where should be also the dsfi.exe, elevate.exe and Switcher.cmd, maybe I could call this "tool folder" to better make it clear?

 

The file as4kbNTFS.bss and as512NTFS.bss are created by "mkdualdisk.cmd" when it displays:

 

Creating a copy of the 4kb bytes/sector NTFS bootsector
And modifying it to become the 512 bytes/sector NTFS bootsector

 

 

I may add a couple more lines f text so that it becomes:

 

 

Creating a copy of the 4kb bytes/sector NTFS bootsector and saving it as C:\MkPriLog\as4kbNTFS.bss
And modifying it to become the 512 bytes/sector NTFS bootsector, saving it as C:\MkPriLog\as512NTFS.bss

 

 

jaclaz

Edited January 23, 2015 by jaclaz