[wfc569]

hi jaclaz,

I used DMDE to seach the USB_Full. img from DSFO running result and was able to see the directory tree of my USB stick. I try to recover one file as example but ran into some situation here. I have attached the screen shot for your review. The Excel file was recovered but didn't seem to be able to open normally. My initial guess is that this file is damaged and can not be recovered. However, I still have some tiny hope deep down to seek expert opinion and hope there is a way to get the file back in tack. Really appreciate your input on this issue....

Attached File(s)

•  DMDE recovery sample.doc (94K)
  Number of downloads: 2

[jaclaz]

wfc569, on 15 October 2012 - 09:34 AM, said:

hi jaclaz,

I used DMDE to seach the USB_Full. img from DSFO running result and was able to see the directory tree of my USB stick. I try to recover one file as example but ran into some situation here. I have attached the screen shot for your review. The Excel file was recovered but didn't seem to be able to open normally. My initial guess is that this file is damaged and can not be recovered. However, I still have some tiny hope deep down to seek expert opinion and hope there is a way to get the file back in tack. Really appreciate your input on this issue....

Well, the good news are that seemingly DMDE managed to either find a "sound" or "almost sound" second copy of the FAT or more likely some "good" directory entries.
Of course this does not means that *all* files will be recoverable (or that they will be recovered without further processing.

Trying one file is not a "senceful" way to proceed, in the sense that you cannot get any conclusion from a single specimen, for all we know you might have been "unlucky" (please read as "very lucky" ) and that particular file may be the ONLY one damaged....

As a matter of fact, due to the hypothized "wrap around issue", the most "recent" files are those that are mre likely to be damaged.

In this cases - besides the fact that some knowledge of the way filesystems (and "recovery tools") work - the approach is usually a "negative" one, once you have recovered all possible files, you do a map of the actual clusters used by them, then see if "the rest" do contain bits and pieces of the "still missing" ones and see if you can "re-assemble them" or at least get the data from them.

That particular Excel file is particularly big, depending on the way it has been originally copied/saved to the stick, it is likely to be fragmented.
It seemingly starts on a "early" cluster, that could mean that it have been saved there when it was smaller and later updated (growing in size and possibly "wrapping around").

There are tools (I don't think that Freeware ones exist, though) capable of (sometimes) recovering the contents from a damaged Excel (.xls format) files, they actually get the values (and not the formulas) and need anyway some manual work to "fill the gaps".

However I see that the DMDE is on sector 16 which cannot possibly be an actual root directory, since it is identical to sector 0, these are evidently "parts" of the root written "in the wrong place" due to the issue.

Let's do another thing (I still miss the info about the original capacity of the stick).

jaclaz, on 13 October 2012 - 05:29 AM, said:

Roughly, and on "Gb size" of the filesystem (FAT32), there are 2 sectors of FAT per MB.
So, if the stick was "labeled" as 8 Gb, the beginning of 2nd copy of the FAT tables was likely around sector 16000, if it was 16 Gb, around sector 32000 and if 32 GB around sector 64000, which means that (in the hypothesis that you were actually very near the real physical size of the stick and what happened was a "wrap-around" of the 19 Mb), if the stick was 32 Gb, the second copy should be "intact", if 16 Gb it is "likely", if it was 8Gb there are very little or no chances.

If it was 8 Gb, extract from the image first 2*16000+2000=34000*512=17408000 bytes from the "whole" image, zip the result, upload to a free hosting site and PM me the link.
You can do the math allright in case of other sizes.

jaclaz

[wfc569]

Hi Jaclaz,

I also included the DMDE run stats here. Secondly, I have to apologize not getting you the information you have been asking.... because I was not sure I understood what you meant in the first first place. Now, I gotta ask this silly question on how to extract part of the file from while image file result in "USB_full.img". My USB stick is 2 GB so I figure i need to get 2 x 4,000 + 2,000 = 10,000 x 512 = 5,120,000 bytes from the "USB_full.img" file. thanks

Attached File(s)

•  DMDE Stats.doc (70K)
  Number of downloads: 2

[jaclaz]

wfc569, on 15 October 2012 - 10:53 AM, said:

Hi Jaclaz,

I also included the DMDE run stats here. Secondly, I have to apologize not getting you the information you have been asking.... because I was not sure I understood what you meant in the first first place. Now, I gotta ask this silly question on how to extract part of the file from while image file result in "USB_full.img". My USB stick is 2 GB so I figure i need to get 2 x 4,000 + 2,000 = 10,000 x 512 = 5,120,000 bytes from the "USB_full.img" file. thanks

Well, then most of what I have seen in my crystal ball was wroong after all.

Seriously, I was making the hypothesis that you had a "fake" USB stick, i.e. one bought as being (you choose) either 4, 8, 16, 32 or 64 Gb BUT being actually a 2 Gb stick "tricked" into showing as a bigger capacity device.
If you bought that stick as being a 2 Gb one and it is actually a 2 Gb one (dsfo produced a 1817706496 bytes file, which could be more or less "right") the reason of the filesystem corruption may lie "somewhere else".

Yes, once I get the "larger" snippet, the "DMDE stat" data you posted might be useful.

Do you remember if you used that stick "as is" (or re-formatted it with the standard XP/whatever "standard" format command) or if you ever ran on it such tools as "HP USB disk utility" or any similar software that creates a MBR and partition table on it?

Now, how you could have gotten the 51200 bytes?
http://www.msfn.org/...ost__p__1014548

jaclaz, on 12 October 2012 - 11:52 AM, said:

then use the dsfo to extract first 100 sectors (supposing the "full" stick image is C:\mystick.img

dsfo C:\mystick.img 0 51200 C:\USB_100.img

compress C:\USB_100.img in a .zip file and attach it to your next post.

if you use the same approach writing 5120000 instead of 51200 (and calling the resulting file C:\USB_10000.img) what would be the result?

Besides posting the above file (if I will need a bigger one I will tell you), can you try running ChipGenius on the stick and post results:
http://reboot.pro/4661/
(just to understand what is "inside" the stick)


jaclaz

[wfc569]

pls bear with me.... I think i got it by doing the following

dsfo \\.\physicaldrive 0 5120000 c:\dsfok\usb_10000.img

I never had a file uploaded to a free host site so i need to create one. will post the link here once i have that set up

thanks

[wfc569]

Here is the info from ChipGenius:

Description: [F:]USB Mass Storage Device(Kingston DataTraveler II+)
Device Type: Mass Storage Device

Protocal Version: USB 2.00
Current Speed: High Speed
Max Current: 200mA

USB Device ID: VID = 0951 PID = 1601
Serial Number: 5B6B1A291319

Device Vendor: Kingston
Device Name: DataTraveler II+
Device Revision: 0100

Manufacturer: Kingston
Product Model: DataTraveler II+
Product Revision: PMAP

Chip Vendor: Phison
Chip Part-Number: PS2251-54(PS2154) - F/W 03.00.10 [2005-09-06]
Flash ID Code: ECD35195 - Samsung [SLC-2K]

[wfc569]

I bought this 2 GB Kingston USB stick many many years ago, maybe 2001 or so. I am not a serious data end user so I thought its enough for me to back up some data. I don't remember how I format the USB stick to be honest. I got it working when I purchased it from Amazon. My home PC has been running NTFS format all these yrs until recently I had to re-build my PC. I vaguely remember that the Window XP ask my if I want to use NTFS or NTFS (express or something like that). After I got my PC rebuilt, I didn't use my USB stick right away until last week when I needed to do some work related from home. When I put my stick into the USB drive, that's when i notice I can not access the data inside the USB stick.

I hope this brief operational history may shade some light what I did wrong.....

[wfc569]

do you have a free hosting web site to suggest? I tried Mozy, but it is taking forever to do the 1st back up....

[jaclaz]

wfc569, on 15 October 2012 - 11:38 AM, said:

pls bear with me.... I think i got it by doing the following

dsfo \\.\physicaldrive 0 5120000 c:\dsfok\usb_10000.img

NO. (meaning that if you used \\.\physicaldrive1 then you got it (maybe).
I mean if the stick is still functional, the idea of making an image is that you use the image (of copies of it) and leave the stick alone.
But i'ts Ok as well in this case, since you don't have issues mounting/accessing the device.
Since you do have an image (the C:\dsfok\USB_full.img), you could have used it as source for dsfo instead of the actual stick/physicaldrive.

The stick sounds a lot like being bought in 2005 or 2006:

Quote

F/W 03.00.10 [2005-09-06]

in 2001 a 2 Gb stick probably didn't actually exist (or it was at a "crazy price" ), a common sizes in 2003 were 128/256/512 Mb, a 1 Gb sick was "huge", see history of Netac for year 2001:
http://www.netac.com...ac-Histroy.aspx
or date first available here:
http://www.amazon.co...B/dp/B000AV14M2

You did nothing "wrong" , the stupid stick *somehow* had an issue, you are perfectly innocent, sometime it is just the actual stick "inside" /controller or flash memory), sometimes a "quirk" in the mounting/unmounting/writing to it of the OS, sometime is not using the "Safely remove" tray icon or however not properly flushing the write cache....

jaclaz

P.S.: Try Zshare:
http://www2.zshare.ma/

This post has been edited by jaclaz: 15 October 2012 - 12:21 PM

[wfc569]

hi Jaclaz,

I have posted the file via the link below for that 2 GB

http://www.filefacto...p/5nwpeox3z7az/

May I trouble you be a bit more specific what larger DMDE stat data you are looking? I will try to get that information over for your diagnostics.


Yes, once I get the "larger" snippet, the "DMDE stat" data you posted might be useful.


thanks

[jaclaz]

wfc569, on 15 October 2012 - 01:13 PM, said:

hi Jaclaz,

I have posted the file via the link below for that 2 GB

http://www.filefacto...p/5nwpeox3z7az/

May I trouble you be a bit more specific what larger DMDE stat data you are looking? I will try to get that information over for your diagnostics.


Yes, once I get the "larger" snippet, the "DMDE stat" data you posted might be useful.


thanks

Got it.
I'll have a look.

Sorry, misunderstanding, the "stats" you already posted are useful to examine the usb_10000.img (larger snippet) they were/are unuseful with just the usb_100.img (smaller snippet).

jaclaz

[jaclaz]

I have some good news .

The thingy was originally formatted as FAT16 (not as FAT32) with a cluster size of 64 sectors/32 Kb.
What actually happened remains a mistery, both copies of the FAT tables are lost forever, but I managed to re-build a bootsector (with *valid* data) and two empty FATs that are enough to let DMDE (or TESTDISK) see the WHOLE root directory (and hopefully link to the contents of the subdirectory).
Still you won't be able to recover undamaged any file that was fragmented, and further recovery (in any case a partial one) can be possible only operating manually (a longish and "dirty" - in the sense of frustrating - job).

Instructions:

• you have an image of the stick "as is" in C:\dsfok\usb_full.img
  
• make a COPY of it as c:\dsfok\usb_full_fixed.img
  
• unzip the attached to c:\dsfok\ (C:\dsfok\usb_500_patch.img)
  
• open a command prompt and in it do:
  

  dsfi c:\dsfok\usb_full_fixed.img 0 0 C:\dsfok\usb_500_patch.img
  

  
  
• use DMDE to access the image c:\dsfok\usb_full_fixed.img
  
• alternatively you can use TESTDISK

When using testdisk:
Open a command prompt and navigate to the directory where you have testdisk, actual executable should be testdisk_win.exe, issue:

testdisk_win.exe /log c:\dsfok\usb_full_fixed.img

then (in the testdisk UI):

• Proceed
  
• None
  
• Advanced
  
• Boot
  
• List

(then use "c" to attempt copying files)
OR:

• Proceed
  
• None
  
• Analyse
  
• Quick Search
  
• P

(then use "c" to attempt copying files)

jaclaz

Attached File(s)

•  usb_500_patch.zip (7.28K)
  Number of downloads: 2

[wfc569]

Many thx, Jaclaz.... I will spend some good amount of time going through the instruction carefully and see if I get get to the final step. I'll let you know the process from my end. Again, many thanks for all the suggestions...

[wfc569]

Hi Jaclaz,

I have attached some screen shots with some comments and questions for you on the instruction you posted earlier. Hope these screen shots will yield some suggestions how to proceed for the next step.

thanks

Attached File(s)

•  Fixed DMDE Stats.doc (229K)
  Number of downloads: 2

[jaclaz]

Quote

*** For those marked with “X”, does that mean those files are corrupted and can not be retrieved?

Yes, and No, those are deleted files, these cannot normally be recovered or - as a matter of fact - only the "last written of them" may.

if you watch accurately on the right, you will see how most of them "share" the same cluster beginning address.

Quote

When I tried to recover “Yahoo v Impulse System.xls” file from DMDE, I got the following message when trying to open it in Excel.

Yes, I understand this, the file is (was) evidently fragmented.

Quote

Is there a way to fix “selected” file(s)?

There is rarely a definite answer when we are talking of data recovery, maybe yes/maybe no.

Quote

Maybe it will require very tedious manual operation here…..

Yes and no, it depends by such a big number of factors that it is really difficult whether one of the (Commercial) specialised tools might be able to do some magic.
In DMDE, if you check the entry for “Yahoo v Impulse System.xls” you will see how it begins (maybe) on cluster 371 (filesize 8909312 243.09.2012 16:36), BUT if you scroll down a bit, you will see that you have an entry for a (deleted) “Yahoo v Impulse System.xls~RF1c4f8a1.TMP” that begins on cluster 331 (filesize 8909312 243.09.2012 16:36).
Since we have clusters sized 32,768 bytes, a file 8909312/32768=271,89.. will occupy 272 clusters and thus it is evident a "superimposing".
If you go:
Tools->Update Custer maps
Tools->Cluster Map
then you start "navigating" the "bottom window", (with cursor keys) you will see how clusters from 331 to 370 are not allocated, (and thus may contain the "beginning" of the excel file), if you continue (right cursor arrow) up to vcn 271 and go beyond it over vcn 272, you see that the filename changes, BUT if you go "backwards" (left cursor arrow) you will notice how the "same" area is now linked to "other files".

Quote

If there is a way of doing that, do you recommend I should seek a professional assistance in the US?

It depends on how much you value those data.
Without any actual guarantee of recovering the whole file (or even anything more than "bits and pieces") we are talking of hundreds of US$, since there are some (I presume) earlier versions of that file - and depending on the actual contents of the file - in some case is possible to "combine" data or as said earlier extract values and re-use formulas from an older copy.


jaclaz

[wfc569]

Tools->Update Custer maps
Tools->Cluster Map
then you start "navigating" the "bottom window", (with cursor keys) you will see how clusters from 331 to 370 are not allocated, (and thus may contain the "beginning" of the excel file), if you continue (right cursor arrow) up to vcn 271 and go beyond it over vcn 272, you see that the filename changes, BUT if you go "backwards" (left cursor arrow) you will notice how the "same" area is now linked to "other files".


*********************************************************************************************************************************************************

OK, i followed your instruction and went through the cluster map for those lines regarding the “Yahoo v Impulse System.xls” file. Like you said, this file if fragmented. Please correct me if I am wrong.... if I have a "good" USB stick, the cluster map will show consecutive sequence for each file and cluster # will match to file attributes in the upper panel. Currently, my cluster map is just like a scramble egg.... correct?

I like to continue diving into this topic of manual recovery of a file..... Since DMDE provides the cluster map for a file, can I use the cursor arrow to locate other "fragmented" - “Yahoo v Impulse System.xls” piece and perform something like "cut and paste" so I can glue them in the same cluster area in order to fix the fragmentation issue? If my approach is not a good idea, would you recommend a way to piece back fragmented file together?

thanks

[jaclaz]

wfc569, on 16 October 2012 - 03:27 PM, said:

OK, i followed your instruction and went through the cluster map for those lines regarding the “Yahoo v Impulse System.xls” file. Like you said, this file if fragmented. Please correct me if I am wrong.... if I have a "good" USB stick, the cluster map will show consecutive sequence for each file and cluster # will match to file attributes in the upper panel. Currently, my cluster map is just like a scramble egg.... correct?

Yes (meaning No ) you are wrong.
There is nothing particularly "bad" in a file being fragmented (actually it is quite normal that a file, expecially a "growing one" becomes fragmented over time or - more simply - the deletion of files creates "empty spots" here and there that are later filled by a single larger file that then becomes fragmented.
On a healthy filesystem this is "normal" and there are no issues whatsoever because the exact position (and sequence) of clusters where the file is stored are recorded in the FAT table(s).
The issue here is that the FAT tables of that stick - for whatever reason - have been overwritten/are invalid.
The "patch" I sent you is made of an adequate bootsector and of two empty FAT tables assembled together with the (valid) ROOT directory already existing on the stick.
This allows for "normal" listing of the files/dirs, but the DMDE (or any other program) has only the info to where the files starts - and since the FAT tables are empty it assumes that the file is contiguous, i.e. it "virtually maps" enough clusters to contain the file size starting from it's starting address.
If you prefer, all filesystems after some use do become "scrambled eggs", but you normally have a slow motion video of the scrambling that you can virtualy play backwards to make again the eggs "as they were".

wfc569, on 16 October 2012 - 03:27 PM, said:

I like to continue diving into this topic of manual recovery of a file..... Since DMDE provides the cluster map for a file, can I use the cursor arrow to locate other "fragmented" - “Yahoo v Impulse System.xls” piece and perform something like "cut and paste" so I can glue them in the same cluster area in order to fix the fragmentation issue?

No.
Sorry but you have not yet grasped the above illustrated concept, the cluster map that DMDE provides is a "virtual one" or, if you prefer it represents the FAT tables how they should have been if ...

wfc569, on 16 October 2012 - 03:27 PM, said:

If my approach is not a good idea, would you recommend a way to piece back fragmented file together?

The idea is not bad, only (and simply) it doesn't work like that.

Imagine that you have a toy shop.
A mad employee opens (say) 100 boxes of (different) puzzles and puts the contents all together in a bag.
Your mission, should you accept it is to re-compose all puzzles in order to put back all the right pieces of the puzzles in the right box, then put together the pieces of the single puzzle you are interested in.
And you don't know if the mad employee (being mad) did not hide, destroy or paint "white" a certain number of pieces, both of the puzzle you want to rebuild and/or of the other puzzles.

So, you have two questions that only have very vague answers:
Can it be done? Yes or no, it depends.
How long will it take? Cannot say, it greatly depends on the original contents of the file and of the contents of all the other files, to continue in the puzzles example if the one you want to put together depicts a wiinter panorama oof a city and all the other 99 represent beaches, sea views, reproductions of still life paintings, etc. it will be easier, if all 100 represent winter panoramas, it woud be really tough.

The puzzles examples are good because they introduce the concept of "art", data recovery is as I see it a (very marginal ) form of "art", let's say painting, in due time you can learn allright the techniques used in it, but if you are not an artist, the resuts will not be good to the eye.

More seriously when it comes to manual recovery, it is "artisan work", you need experience and (deep) knowledge of the matter but you also grow your own "style" and abilities, when it comes to it, it is perfectly possible that one expert can make nothing of it (or take "forever" in doing it) while another one may put together "the pieces" quickly and successfully, there is simply no way to know in advance.

jaclaz

This post has been edited by jaclaz: 17 October 2012 - 03:07 AM

[wfc569]

Many thanks for this thorough explanation. In my mind, when I encounter an issue, I like understand it and try to solve it along the way if I have to learn along the side. In the beginning of this process, i totally had no idea about data recovery, slowly but surely, I start getting some idea how data works in the computer world.

Although I don't have a computer science degree under my belt, I am interested in learning how to do this data recovery technique if you think this is something can be taught over the world wide web. If, for whatever the reason, you think this is beyond the purpose of this forum, I totally understand and respect that. Still, I really appreciate all the explanation and instruction received from you through out this exercise.


thanks

[jaclaz]

wfc569, on 17 October 2012 - 06:45 AM, said:

Many thanks for this thorough explanation. In my mind, when I encounter an issue, I like understand it and try to solve it along the way if I have to learn along the side. In the beginning of this process, i totally had no idea about data recovery, slowly but surely, I start getting some idea how data works in the computer world.

Good, this is the "right" approach.

wfc569, on 17 October 2012 - 06:45 AM, said:

Although I don't have a computer science degree under my belt, I am interested in learning how to do this data recovery technique if you think this is something can be taught over the world wide web. If, for whatever the reason, you think this is beyond the purpose of this forum, I totally understand and respect that. Still, I really appreciate all the explanation and instruction received from you through out this exercise.

There is no need whatever for a computer science degree, as a matter of fact I have personally met "IT specialists" that deemed tasks similar to this one "impossible", yet *somehow* they were later proved to be perfectly possible .

In your case the issue is that there is no way to know ab initio IF it is possible or if the result, if possible, would be satisfactory.

I see it more a matter of evaluating the costs/benefits of such a task, after all it is an Excel spreadsheet of which - at least from the directory structure I saw - you seemingly have earlier versions (and I presume that a copy of such earlier version is still avaialble "somewhere" - like in your home or office PC) so you have to evaluate:

• the time that will take you to re-update the spreadsheet (from an earlier version)
  
• the time that it would take you to re-create it from scratch
  
• the time that it would take you to learn how to (hopefully) recover that file
  
• the time that it will take you to actually analyze and (hopefully) recover that file
  
• the value you atttribute to your time (and thus the price that you may be wanting to pay a profesional for it)

Of course I cannot make an "online course" on the matter (provided that I have the capabilities/knowledge to make one ), as said - unlike other much simpler cases seen (and often but not always solved) - through the forum, you are in a "difficult" situation with a huge number of "bifurcations", and exploring all the possible paths, with the aggravation of "not-real-time" comminication is not a viable possibility.

I can give you the "basics" (of which some I already gave you) and suggest you the (hopefully) "right" "generic" approach and possibly a few links to read, but not much else.

jaclaz

[wfc569]

thx for the candid suggestion!! I understand my case is probably on the complicated side so my hope to get my data back is not high....

If not too much of trouble from your end, I really appreciate for any pointers you can share on learning how to do data recovery. I don't plan to re-format my USB stick any time soon and have a new one ordered so I can keep this damaged one. Maybe use it as a learning opportunity.