Jump to content

Welcome to MSFN Forum
Register now to gain access to all of our features. Once registered and logged in, you will be able to create topics, post replies to existing threads, give reputation to your fellow members, get your own private messenger, post status updates, manage your profile and so much more. This message will be removed once you have signed in.
Login to Account Create an Account



Photo

Copy only files that have newer version?

- - - - -

  • Please log in to reply
30 replies to this topic

#1
tomasz86

tomasz86

    www.windows2000.tk

  • Member
  • PipPipPipPipPipPipPipPip
  • 2,525 posts
  • Joined 27-November 10
  • OS:none specified
  • Country: Country Flag
By using xcopy with /d swich it's possible to copy only files that are newer.

I wonder if it is possible to copy only files whose version is newer even though their date of creation may be older.

Is there any program than can do it from a command line?

Edited by tomasz86, 08 June 2011 - 12:26 AM.

post-47483-1123010975.png



How to remove advertisement from MSFN

#2
allen2

allen2

    Not really Newbie

  • Member
  • PipPipPipPipPipPipPip
  • 1,814 posts
  • Joined 13-January 06
No most tools out there doesn't check the version. You'll need to script it using filever.exe (from xp sp2 support tools) or do a vbs a explained there.

#3
tomasz86

tomasz86

    www.windows2000.tk

  • Member
  • PipPipPipPipPipPipPipPip
  • 2,525 posts
  • Joined 27-November 10
  • OS:none specified
  • Country: Country Flag
Thank you :)

I guess filever.exe from Win2k support tools is also OK ;) Actually I compared w2k filever.exe and w2k3 filever.exe and the only difference is that there is no "/R" switch in the older one.

I found a better tool :)

Edited by tomasz86, 10 June 2011 - 08:52 AM.

post-47483-1123010975.png


#4
CoffeeFiend

CoffeeFiend

    Coffee Aficionado

  • Super Moderator
  • 5,399 posts
  • Joined 14-July 04
  • OS:Windows 7 x64
  • Country: Country Flag
It would be pretty easy to make a xcopy-like program that checks version using C# (only copies files with newer versions, or makes a list of them or whatever). Not sure if that's any help as you're seemingly using win2k...
Coffee: \ˈkȯ-fē, ˈkä-\. noun. Heaven in a cup. Life's only treasure. The meaning of life. Kaffee ist wunderbar. C8H10N4O2 FTW.

#5
tomasz86

tomasz86

    www.windows2000.tk

  • Member
  • PipPipPipPipPipPipPipPip
  • 2,525 posts
  • Joined 27-November 10
  • OS:none specified
  • Country: Country Flag
Would such program not work under Win2k?

post-47483-1123010975.png


#6
CoffeeFiend

CoffeeFiend

    Coffee Aficionado

  • Super Moderator
  • 5,399 posts
  • Joined 14-July 04
  • OS:Windows 7 x64
  • Country: Country Flag

Would such program not work under Win2k?

It depends on which version of the .NET framework one uses. The newer versions have some really nice and useful additions like LINQ, but those frameworks don't run on Win2k. It's still fairly simple to write using an older version of the .NET framework (v2). It's just more work to do so.

There are a few things one would have to think of ahead though, like what to do when one or both files don't have an embedded "version" (just blindly overwrite? overwrite if timestamp is newer? just warn and copy nothing? ...) You can do just about anything. You could log to any file format (like lists of newer/older/unchanged/overwritten files and such), you could export a csv file with the differences and so on (you could include file hashes too). In fact, it's more work thinking about it (requirements, things that might occur and how to handle them, etc) than actually writing it.

Seriously, it's not that much work. Getting the list of all files (including all subfolders) using Directory.GetFiles method takes a single line of code, getting a file version using FileVersionInfo.GetVersionInfo is another simple one-liner. Then you just have to iterate through the list of files using a plain "foreach" statement and comparing the versions (copying files as necessary, using the File.Copy method -- another trivial one-liner)
Coffee: \ˈkȯ-fē, ˈkä-\. noun. Heaven in a cup. Life's only treasure. The meaning of life. Kaffee ist wunderbar. C8H10N4O2 FTW.

#7
tomasz86

tomasz86

    www.windows2000.tk

  • Member
  • PipPipPipPipPipPipPipPip
  • 2,525 posts
  • Joined 27-November 10
  • OS:none specified
  • Country: Country Flag
I have .NET Framework 1.1, 2.0 and 3.5 installed... but I don't have the skills to write such a program :o

post-47483-1123010975.png


#8
CoffeeFiend

CoffeeFiend

    Coffee Aficionado

  • Super Moderator
  • 5,399 posts
  • Joined 14-July 04
  • OS:Windows 7 x64
  • Country: Country Flag

I don't have the skills to write such a program :o

I can take care of that ;)

You'd just have to confirm the desired behavior. Like:
-if the file in "source" doesn't exist in "destination" then just copy it anyway?
-if the file in "source" doesn't have a "version" embedded in it (like say, a picture), then we copy it to "destination" only if the timestamp is newer?
-any other particular "rules" you'd want it to obey?

It's just those little things we have to figure out beforehand (requirements). Then it would only take a few minutes to write such a program.
Coffee: \ˈkȯ-fē, ˈkä-\. noun. Heaven in a cup. Life's only treasure. The meaning of life. Kaffee ist wunderbar. C8H10N4O2 FTW.

#9
dencorso

dencorso

    Iuvat plus qui nihil obstat

  • Supervisor
  • 6,019 posts
  • Joined 07-April 07
  • OS:98SE
  • Country: Country Flag

Donator

One caveat: both versions the hexadecimal and the text string File Version should be checked because they are not always the same...
MS filever.exe v. 5.2.3754.0 returns the hex file version. But in the properties tab one has the hex displayed above, and the text one below, when one selects File Version. Among the Win XP files ole32.dll is a simple example of this fact, but ole2.dll is even better.
The text File Version is part of VS_FIXEDFILEINFO, which is a member of VS_VERSION_INFO. Now, the hexadecimal File Info is itself another a member of VS_VERSION_INFO but it resides inside a VarFileInfo structure, IIRR.

#10
jaclaz

jaclaz

    The Finder

  • Developer
  • 14,660 posts
  • Joined 23-July 04
  • OS:none specified
  • Country: Country Flag
I would use fvertest and avoid .Net :ph34r:

Here:
http://www.westmesatech.com/wast.html

@dencorso
I am failing to see the practical implications.
Do you mean that there are executables around with SAME "hex" version BUT different "text" version and that the latter may be newer than the first AND that it is a reliable way to get the newer file? :unsure:

jaclaz

Edited by jaclaz, 15 November 2011 - 01:40 PM.


#11
CoffeeFiend

CoffeeFiend

    Coffee Aficionado

  • Super Moderator
  • 5,399 posts
  • Joined 14-July 04
  • OS:Windows 7 x64
  • Country: Country Flag

I would use fvertest

That's just yet another tool that reports the version, hardly an automatic "xcopy replacement with built-in version comparison".
Coffee: \ˈkȯ-fē, ˈkä-\. noun. Heaven in a cup. Life's only treasure. The meaning of life. Kaffee ist wunderbar. C8H10N4O2 FTW.

#12
tomasz86

tomasz86

    www.windows2000.tk

  • Member
  • PipPipPipPipPipPipPipPip
  • 2,525 posts
  • Joined 27-November 10
  • OS:none specified
  • Country: Country Flag
This is the script where xcopy is used (this part is taken from HFSLIP). It's explained in details in this topic.

Spoiler


The process is like this:
1. update is unpacked to a folder TEMP\HF
2. files from TEMP/HF are copied to HFMER using xcopy
3. TEMP/HF is deleted
4. another update unpacked to TEMP/HF
5. files copied to HFMER (xcopy)
...
repeated for all updates.

If files from the second update are older than those which are already present in HFMER, they are not copied. The problem is that some files have newer version but their date is older. Actually, in case of win2k there are just a few such updates but still...

To answer your questions:
1. Yes, I would like to have it fully automated.
2. I don't know :o but I don't think there will be any pictures, just system files (dll, exe, inf, etc.).
3. No "special" rules come to my mind ;)

Edited by tomasz86, 30 June 2011 - 06:22 AM.

post-47483-1123010975.png


#13
jaclaz

jaclaz

    The Finder

  • Developer
  • 14,660 posts
  • Joined 23-July 04
  • OS:none specified
  • Country: Country Flag

That's just yet another tool that reports the version, hardly an automatic "xcopy replacement with built-in version comparison".

Sure :).

It takes only a few lines of batch to implement the file version check, and as well only a few lines of code in C or in C# "standalone", if using .Net, it will take a few lines + a considerable amount of ".Net", with all the consequent .Net install and versioning problems on 2K.

At it's basics:
@ECHO OFF
:: V_copy.cmd - small batch sketch
:: to copy executables and dll's depending on their version
SETLOCAL ENABLEEXTENSIONS
SETLOCAL ENABLEDELAYEDEXPANSION
SET SOURCE=%~dpnx1
IF "%2"=="" GOTO :ERROR2
SET TARGET=%~dpnx2
SET TARGETDIR=%~dp2

IF "%TARGET:~-1,1%"=="\" SET TARGET=%~dp2%~nx1
IF "%~x1"=="" GOTO :ERROR1
IF NOT EXIST %SOURCE% GOTO :ERROR1
IF NOT EXIST %TARGETDIR%\NUL CALL :ERROR2


FOR /F "tokens=1 delims=[]" %%A IN ('fvertest.exe %source%') DO (
SET V_source=%%A
ECHO %source% version is %%A
FOR /F "tokens=1 delims=[]" %%B IN ('fvertest.exe %target%') DO (
SET V_target=%%A
ECHO %target% version is %%B
fvertest.exe -v %%B %target%
REM Insert here checks based on Errorlevel and "UNecho/modify" the following
ECHO Errolevel is !ERRORLEVEL!
ECHO COPY /B %SOURCE% %TARGET%
)
)
SET V_

GOTO :EOF

:ERROR1
ECHO.
ECHO SOURCE %source% is missing
GOTO :EOF

:ERROR2
ECHO.
ECHO TARGET %target% is missing
GOTO :EOF

Please note how the above is V_copy.cmd and NOT V_xcopy.cmd. ;)

jaclaz

#14
dencorso

dencorso

    Iuvat plus qui nihil obstat

  • Supervisor
  • 6,019 posts
  • Joined 07-April 07
  • OS:98SE
  • Country: Country Flag

Donator

@dencorso
I am failing to see the practical implications.
Do you mean that there are executables around with SAME "hex" version BUT different "text" version and that the latter may be newer than the first AND that it is a reliable way to get the newer file? :unsure:

No. Not by any means! That would be too easy... :D
I mean that there are executables around with SAME "hex" version BUT different "text" version, *and* that also are executables around with SAME "text" version BUT different "hex" version. The different version, whichever of the two types it is, can be used to decide which is the actual newer version, but there are cases in which the actual higher version number is the older one. PE Timestamps may also be used to help sort this conundrum, in the case of Win32 executables (= PE executables). But no solution is reliable, if by that you mean works for 100% of the cases... :ph34r:
If you follow the links here, you'll see some examples. This is a noteworthy excerpt:

In a nutshell, changing oleaut32.dll from v. 2.40.4522.0 to v. 2.40.4519.0, despite all the apearances, *is an upgrade*, not a downgrade! HTH


@tomasz86: getver is reliable under Win9x/ME but unreliable under XP (where it gives the version of the file that is in memory, when you try to find the version of any file that has the same name of a loaded dll or exe file). So take care. Filever, by contrary, never lies.

#15
CoffeeFiend

CoffeeFiend

    Coffee Aficionado

  • Super Moderator
  • 5,399 posts
  • Joined 14-July 04
  • OS:Windows 7 x64
  • Country: Country Flag

It takes only a few lines of batch to implement the file version check

You're just copying the one file there (where you have to manually pass in both full paths to a single file), that's still quite a long shot from being an "automated drop in replacement for xcopy with version checking" (where you pass in 2 paths and it does everything automatically). If you can do the whole thing in batch then more power to you. There's just no way I'm going to waste any time trying to write that (find all files incl subdirs, doing all the version checking including handling all the special cases like dencorso mentioned) using batch files (I avoid using batch as much as possible for my own sanity). Feel free to share a batchfile that automates the complete job, tomasz86 would surely appreciate. I just don't have that kind of time to waste.
Coffee: \ˈkȯ-fē, ˈkä-\. noun. Heaven in a cup. Life's only treasure. The meaning of life. Kaffee ist wunderbar. C8H10N4O2 FTW.

#16
dencorso

dencorso

    Iuvat plus qui nihil obstat

  • Supervisor
  • 6,019 posts
  • Joined 07-April 07
  • OS:98SE
  • Country: Country Flag

Donator

@Coffee: FileVersionInfo.GetVersionInfo can be the begining of a solution...
A quick glance at FileVersionInfo Class (System.Diagnostics) shows that:
1) The Hexadecimal File Version is FileVersionInfo.FileMajorPart & "." & FileVersionInfo.FileMinorPart & "." & FileVersionInfo.FileBuildPart & "." & FileVersionInfo.FilePrivatePart;
2) The Hexadecimal Product Version is FileVersionInfo.ProductMajorPart & "." & FileVersionInfo.ProductMinorPart & "." & FileVersionInfo.ProductBuildPart & "." & FileVersionInfo.ProductPrivatePart;
3) The Text File Version is FileVersionInfo.FileVersion;
4) The Text Product Version is FileVersionInfo.ProductVersion;
5) The Text Special Build is FileVersionInfo.SpecialBuild.
The latter 3 values (the text strings) may not exist or be void. Post Win9x/ME files usually don't have a Special Build, for instance.
So, the only thing not present in FileVersionInfo Class is the PE Timestamp, which can be read directly from the file, since it's at a fixed position in the PE header, when the file is a PE executable.

However, using Hex and Text File Versions and PE Timestamps should be enough for most cases.

If both File Versions and PE Timestamp are identical, the file accessed first could be copied, but either will do.
If both File Versions are identical, the file with the highest PE Timestamp should be copied.
If only one File Version differs, the highest of them should be copied.
If both File Versions differ, but each file has a consistent pair of File Versions, the highest of them should be copied (this condition is tricky because it requires parsing the text File Version string).
Else, throw an error and skip the pair of files: human intervention is needed to decide this pair, but others may be solved.

This logic should work for most files, while letting humans tackle the most interesting ones.

This is just my 5¢, of course. :D

PS:

This is what I get from my XP kernel32.dll (a PE exe):
Hex File Version: 5.1.2600.5781
Text File Version: 5.1.2600.5781 (xpsp_sp3_gdr.090321-1317)

and This is what I get from my XP shell.dll (a NE exe):
Hex File Version: 3.10.0.103
Text File Version: 3.10

both are different, yet both are consistent.

#17
CoffeeFiend

CoffeeFiend

    Coffee Aficionado

  • Super Moderator
  • 5,399 posts
  • Joined 14-July 04
  • OS:Windows 7 x64
  • Country: Country Flag
Great list dencorso :)

I was thinking of it as a xcopy that only copies (overwrites destination) with the newer files, as in:

If hex version of src file is newer then copy src file to dst folder; else
If text version of src file is newer then copy src file to dst folder; else
If both versions are the same and the PE timestamp of src file is newer then copy src file to dst folder
else do nothing (there's no indication that it's newer)

Please point out potential problems with this method. I will adapt it accordingly.

Text file version shouldn't really be a problem. Unless you've got some where the same file (from 2 updates) which have very different formats. Parsing it seems fairly simple too:
-truncate the text at the first space encountered (if any) to remove the part with "(operating system version here)" tacked on at the end
-convert the said text to a Version object which will do the heavy lifting (parsing, comparing) for us
Coffee: \ˈkȯ-fē, ˈkä-\. noun. Heaven in a cup. Life's only treasure. The meaning of life. Kaffee ist wunderbar. C8H10N4O2 FTW.

#18
dencorso

dencorso

    Iuvat plus qui nihil obstat

  • Supervisor
  • 6,019 posts
  • Joined 07-April 07
  • OS:98SE
  • Country: Country Flag

Donator

I think your algorithm will do fine. However I was thinking something a little more complex:

Usage: WhateverProgName source1 source2 destination logplace

Where:
source1 and source2 are fully qualified directory names for the sources;
destination is a fully qualified directory name for the resulting merge (might even be created if not existing already);
logplace is a fully qualified filename for the log file (where success -- and which of the files was the one copied -- or error is logged, for each file copied).

Advantage: both sources are kept intact.

But this causes the following changes to your algorithm:

If file is an orphan, then copy orphan to dst and log action else {
If hex version of either src file is newer then copy that src file to dst folder and log action; else
If text version of either src file is newer then copy that src file to dst folder and log action; else
If both versions are the same and the PE timestamp of either src file is newer then copy that src file to dst folder and log action
else since all tests are the same copy file from source1 and log action}

with orphans being files that exist just in one of the src dirs.

Now, the files may not have a text file version at all (very unusual) so an if exist test should be done for the text file version. Also the file may not be a PE, so an if PE test must be done for the PE Timestamp.

As for the strange things that may happen in the text file version string, it always begins with a version number, that may be of the following formats:
1.20
4.10.1998
or the usual 7.61.3456.7654
so, by truncating and considering these 3 formats almost all problems are covered. The only one that remains can be solved by searching the already truncated string for commas and replacing them by dots, if they're found. The Adobe Flash executables are the most common example where commas are used instead of the standard dots.

Also of interest:

The version number is stored as four 16-bit words, each part of it may have value 0 to 65535. So the lowest version number is 0.0.0.0 and the highest 65535.65535.65535.65535.

By looking at the Version object I see that now the version number may use four 32-bit words instead.

#19
CoffeeFiend

CoffeeFiend

    Coffee Aficionado

  • Super Moderator
  • 5,399 posts
  • Joined 14-July 04
  • OS:Windows 7 x64
  • Country: Country Flag

I think your algorithm will do fine. However I was thinking something a little more complex:

Usage: WhateverProgName source1 source2 destination logplace

I did think of that possibility. But that forces you to look for files in either sources that don't exist in the other (a bit of extra work yet again -- we just keep adding "an extra 5 lines here or there" with every special case to handle :lol: ). I thought that if he wanted to leave both original folders untouched, then it was simple enough to copy one to the "destination" directory ahead of time, then use the tool to merge the other into that.

As far as logging goes, the sky's the limit. You could even output a CSV file with a list of each file present (or not) in both source folders, along with their hex/text versions, PE timestamp, MD5/SHA1 hashes, etc -- and of course which action was taken. Although that's perhaps closer to reporting. One could go a bit overboard with it and use log4net, with different "outputs" (like straight to screen or text files) and its different levels (and XML config file, etc). Or writing separate file lists to disk of what files were newer, unique, identical, etc. I was just planning on doing basic console output for now (and just for the "special" cases, no point in flooding it with a line for each file IMO). But I guess what really matters is what tomasz86 wants/needs.

Now, the files may not have a text file version at all (very unusual) so an if exist test should be done for the text file version. Also the file may not be a PE, so an if PE test must be done for the PE Timestamp.

Indeed. There's always lots of little things like that to handle (like guard clauses in case source/destination folders don't exist)

As for the strange things that may happen in the text file version string, it always begins with a version number, that may be of the following formats:
1.20
4.10.1998
or the usual 7.61.3456.7654
so, by truncating and considering these 3 formats almost all problems are covered.

I was mainly worried that the formats might have changed between different versions of the same file but that's highly unlikely. Either ways, the Version class could still compare say, 4.10.1998 to 7.61.3456.7654 no problem if that ever happened.

The only one that remains can be solved by searching the already truncated string for commas and replacing them by dots, if they're found.

Yes. I've seen this when doing a quick check too. That's easily fixed.
Coffee: \ˈkȯ-fē, ˈkä-\. noun. Heaven in a cup. Life's only treasure. The meaning of life. Kaffee ist wunderbar. C8H10N4O2 FTW.

#20
tomasz86

tomasz86

    www.windows2000.tk

  • Member
  • PipPipPipPipPipPipPipPip
  • 2,525 posts
  • Joined 27-November 10
  • OS:none specified
  • Country: Country Flag
Wow, you've been having a pretty intesive discussion here :w00t: Thanks once again for such an interest in this topic!

@tomasz86: getver is reliable under Win9x/ME but unreliable under XP (where it gives the version of the file that is in memory, when you try to find the version of any file that has the same name of a loaded dll or exe file). So take care. Filever, by contrary, never lies.

Thanks for pointing this out. I wasn't aware that getver doesn't work under NT5.x. I've just done a test and can say that your words were right.

Edited by tomasz86, 01 July 2011 - 07:07 AM.

post-47483-1123010975.png


#21
dencorso

dencorso

    Iuvat plus qui nihil obstat

  • Supervisor
  • 6,019 posts
  • Joined 07-April 07
  • OS:98SE
  • Country: Country Flag

Donator

@tomasz86: Just for the sake of clarity: I understand you did reproduce the problem I described in Win 2k, is that right? That means it probably has the bug throughout all the NT-family OSes, Vista and 7 included.

@Coffee: OK. Point taken. So, if we stick to the simpler "Usage: WhateverProgName src dst" format, we can use a better algorithm:

if PE get the PETimestamp;
If hex version of src file is newer (and if PE, if PETimestamp(src) is >= PETimestamp(dst)) then copy src file to dst folder; else
If hex version of src file is newer (and if PE, if PETimestamp(src) is < PETimestamp(dst)) then throw error and skip file; else
If exist text version and text version of src file is newer (and if PE, if PETimestamp(src) is >= PETimestamp(dst)) then copy src file to dst folder; else
If exist text version and text version of src file is newer (and if PE, if PETimestamp(src) is < PETimestamp(dst)) then throw error and skip file; else
If both versions are the same and if PE, the PE timestamp of src file is newer then copy src file to dst folder; else
else just skip file.

This catches and points out the blorky cases that may happen as, for instance, the infamous oleaut32.dll v. 2.40.4522.0 (see post #14, above).

#22
CoffeeFiend

CoffeeFiend

    Coffee Aficionado

  • Super Moderator
  • 5,399 posts
  • Joined 14-July 04
  • OS:Windows 7 x64
  • Country: Country Flag
My current code should work. I just have to test it first. But it's a major PITA to hex edit dozens of files to have all possible combinations of newer/older pairs of everything (or finding non-PE files or those without a text version), so I can test every single corner case... That will take WAY more time than writing the tool did :wacko:
Coffee: \ˈkȯ-fē, ˈkä-\. noun. Heaven in a cup. Life's only treasure. The meaning of life. Kaffee ist wunderbar. C8H10N4O2 FTW.

#23
tomasz86

tomasz86

    www.windows2000.tk

  • Member
  • PipPipPipPipPipPipPipPip
  • 2,525 posts
  • Joined 27-November 10
  • OS:none specified
  • Country: Country Flag

@tomasz86: Just for the sake of clarity: I understand you did reproduce the problem I described in Win 2k, is that right? That means it probably has the bug throughout all the NT-family OSes, Vista and 7 included.

Yes, this is what I meant ;) I did a test under W2K and the reported file version was wrong.

post-47483-1123010975.png


#24
dencorso

dencorso

    Iuvat plus qui nihil obstat

  • Supervisor
  • 6,019 posts
  • Joined 07-April 07
  • OS:98SE
  • Country: Country Flag

Donator

@tomasz86: As I said before, reliable [Hex] File Versions are obtained using FILEVER, findable inside this package.
It runs OK on all Windows versions from 95 up. Petr was the first to report it, IIRR.

@Coffee: Since you're at it, and use 7 x64, this is a good opportunity for you to test n7Epsilon's PEChecksum v. 1.4 and tell us how well does it work. :angel

#25
tomasz86

tomasz86

    www.windows2000.tk

  • Member
  • PipPipPipPipPipPipPipPip
  • 2,525 posts
  • Joined 27-November 10
  • OS:none specified
  • Country: Country Flag

@tomasz86: As I said before, reliable [Hex] File Versions are obtained using FILEVER, findable inside this package.
It runs OK on all Windows versions from 95 up. Petr was the first to report it, IIRR

Yes, I had already found it before in Win2k Resource Kit. I just liked getver more for simplicity but now only filever is left.

post-47483-1123010975.png





0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users