Jump to content

Welcome to MSFN Forum
Register now to gain access to all of our features. Once registered and logged in, you will be able to create topics, post replies to existing threads, give reputation to your fellow members, get your own private messenger, post status updates, manage your profile and so much more. This message will be removed once you have signed in.
Login to Account Create an Account



Photo

How to merge two text files?

- - - - -

  • Please log in to reply
237 replies to this topic

#1
tomasz86

tomasz86

    www.windows2000.tk

  • Member
  • PipPipPipPipPipPipPipPip
  • 2,525 posts
  • Joined 27-November 10
  • OS:none specified
  • Country: Country Flag
There are two files:

1.txt
[SourceFileInfo]
clusapi.dll=B95AC82B54FE4359C3453264F848509A,0005000008931AA8,55568

2.txt
[SourceFileInfo]
clusnet.sys=A0610690266ED57A2D04EA5D7EC8084C,0005000008931AA8,67760

If I do "copy 1.txt+2.txt 3.txt" I get this:
[SourceFileInfo]
clusapi.dll=B95AC82B54FE4359C3453264F848509A,0005000008931AA8,55568
[SourceFileInfo]
clusnet.sys=A0610690266ED57A2D04EA5D7EC8084C,0005000008931AA8,67760

but I would like to get something similar to this:
[SourceFileInfo]
clusapi.dll=B95AC82B54FE4359C3453264F848509A,0005000008931AA8,55568
clusnet.sys=A0610690266ED57A2D04EA5D7EC8084C,0005000008931AA8,67760

Is it possible?

post-47483-1123010975.png



How to remove advertisement from MSFN

#2
GrofLuigi

GrofLuigi

    GroupPolicy Tattoo Artist

  • Member
  • PipPipPipPipPipPip
  • 1,362 posts
  • Joined 21-April 05
  • OS:none specified
  • Country: Country Flag
sort 3.txt /o 4.txt

One liner (but needs fixing by someone more knoledgable):

copy 1.txt+2.txt | sort /o 3.txt

GL

#3
dencorso

dencorso

    Iuvat plus qui nihil obstat

  • Supervisor
  • 6,017 posts
  • Joined 07-April 07
  • OS:98SE
  • Country: Country Flag

Donator

Beyond Compare is the way to go... it's not free, but it's worth the cost.

#4
allen2

allen2

    Not really Newbie

  • Member
  • PipPipPipPipPipPipPip
  • 1,814 posts
  • Joined 13-January 06
Or using unix tools:
grep -vi "\[SourceFileInfo\]" 1.txt  >>2.txt
"\" are need to escape the "[" used with grep for regular expression.

#5
tomasz86

tomasz86

    www.windows2000.tk

  • Member
  • PipPipPipPipPipPipPipPip
  • 2,525 posts
  • Joined 27-November 10
  • OS:none specified
  • Country: Country Flag
Thank you very much :) I'll check them for sure. I'm especially interested in this unix one as it could be used in a batch script.

By the way, could anyone help me with this script?

MD HFMER TEMP2
DIR/B/A-D/OGN/ON HF\*.EXE>HF.TXT
SET HF=
FOR /F %%I IN (HF.TXT) DO (SET HF=%%I&IF DEFINED HF CALL :HFEXTRACT)
DEL/Q/S HF.TXT

REM ======================TYPE 1 HOTFIXES=================================================
:HFEXTRACT
TITLE %T1% - Processing %HF%
ECHO %HF%
MD TEMP&START/WAIT HF\%HF% /Q /X:TEMP
XCOPY/DEHRY TEMP HFMER
MOVE TEMP\UPDATE\update.inf HFMER\UPDATE\%HF%.inf
MOVE TEMP\UPDATE\update_w2k.inf HFMER\UPDATE\%HF%.inf
MOVE TEMP\UPDATE\update.ver TEMP2\%HF%.ver
DEL/Q/S HFMER\UPDATE\update*.inf HFMER\UPDATE\update.ver
RD/Q/S TEMP
CLS
REM =====================================================================================

COPY/B TEMP2\*.ver HFMER\UPDATE\update.ver
SORT HFMER\UPDATE\update.ver /O HFMER\UPDATE\update.ver
RD/Q/S TEMP2

This is a mix of strings taken from HFSLIP and my own but the problem is that I need the last three lines to go after the rest of the script is finished. It work correctly if I remove the last line but the folder TEMP2 is not deleted then.

Edited by tomasz86, 08 June 2011 - 04:23 AM.

post-47483-1123010975.png


#6
tomasz86

tomasz86

    www.windows2000.tk

  • Member
  • PipPipPipPipPipPipPipPip
  • 2,525 posts
  • Joined 27-November 10
  • OS:none specified
  • Country: Country Flag
I've managed to overcome it:
IF EXIST HF\*.EXE (
	MD HFMER TEMP2
	DIR/B/A-D/OGN/ON HF\*.EXE>HF.TXT
	SET HF=
	FOR /F %%I IN (HF.TXT) DO (SET HF=%%I&IF DEFINED HF CALL :HFEXTRACT)
	DEL/Q/S HF.TXT
	CALL :HFCS
)
REM ======================TYPE 1 HOTFIXES=================================================
:HFEXTRACT
TITLE %T1% - Processing %HF%
ECHO %HF%
MD TEMP&START/WAIT HF\%HF% /Q /X:TEMP
XCOPY/DEHRY TEMP HFMER
MOVE TEMP\UPDATE\update.inf HFMER\UPDATE\%HF%.inf
MOVE TEMP\UPDATE\update_w2k.inf HFMER\UPDATE\%HF%.inf
MOVE TEMP\UPDATE\update.ver TEMP2\%HF%.ver
DEL/Q/S HFMER\UPDATE\update*.inf HFMER\UPDATE\update.ver
RD/Q/S TEMP
IF NOT EXIST HF.TXT (
	COPY/B TEMP2\*.ver HFMER\UPDATE\update.ver
	SORT HFMER\UPDATE\update.ver /O HFMER\UPDATE\update.ver
	RD/Q/S TEMP2
)
REM ======================================================================================

I have a question about this Unix tool called grep.

When I have only two files:

1.ver
[SourceFileInfo]
basesrv.dll=7F87C84D34813197A2360CEA800A7464,0005000008931B27,46352
cmd.exe=7705AED861C7FDBD919E771A1B42B5AA,0005000008931AA8,236304
2.ver
[SourceFileInfo]
clusapi.dll=B95AC82B54FE4359C3453264F848509A,0005000008931AA8,55568
clusnet.sys=A0610690266ED57A2D04EA5D7EC8084C,0005000008931AA8,67760

after "grep -vi "\[SourceFileInfo\]" 1.ver >>2.ver"
[SourceFileInfo]
clusapi.dll=B95AC82B54FE4359C3453264F848509A,0005000008931AA8,55568
clusnet.sys=A0610690266ED57A2D04EA5D7EC8084C,0005000008931AA8,67760
basesrv.dll=7F87C84D34813197A2360CEA800A7464,0005000008931B27,46352
cmd.exe=7705AED861C7FDBD919E771A1B42B5AA,0005000008931AA8,236304

Everything is OK. What about more than two files? Is it possible to use grep with multiple files?

Also an another question (2 questions ;)): Can the list be sorted alphabetically? Is there a way to remove duplicates?

Edited by tomasz86, 08 June 2011 - 08:09 AM.

post-47483-1123010975.png


#7
allen2

allen2

    Not really Newbie

  • Member
  • PipPipPipPipPipPipPip
  • 1,814 posts
  • Joined 13-January 06
Yes grep can work on multiple files and no it can't sort or remove duplicates unless you do the scripting for it .

#8
tomasz86

tomasz86

    www.windows2000.tk

  • Member
  • PipPipPipPipPipPipPipPip
  • 2,525 posts
  • Joined 27-November 10
  • OS:none specified
  • Country: Country Flag
How should I use this command
grep -vi "\[SourceFileInfo\]" 1.txt  >>2.txt
when there are multiple files?

I tried something like this:
grep -vi "\[SourceFileInfo\]" *.txt  >>3.txt
but it doesn't work. Also tried doing
copy 1.txt+2.txt 3.txt
grep -vi "\[SourceFileInfo\]" 3.txt  >>3.txt
but after doing this there is no longer [SourceFileInfo] in the final file.

Edited by tomasz86, 09 June 2011 - 12:09 AM.

post-47483-1123010975.png


#9
allen2

allen2

    Not really Newbie

  • Member
  • PipPipPipPipPipPipPip
  • 1,814 posts
  • Joined 13-January 06
grep -vih "\[SourceFileInfo\]" 1.txt 2.txt >>3.txt
But then if you need sorting and finding duplicates to choose the most recent version of each file, you might want to use vbs or autoit as it would be a lot more easier to script.

Edited by allen2, 09 June 2011 - 12:36 AM.


#10
tomasz86

tomasz86

    www.windows2000.tk

  • Member
  • PipPipPipPipPipPipPipPip
  • 2,525 posts
  • Joined 27-November 10
  • OS:none specified
  • Country: Country Flag
I see. Any way to do it for a larger number of files automatically, ex. 50 text files in one directory?

post-47483-1123010975.png


#11
allen2

allen2

    Not really Newbie

  • Member
  • PipPipPipPipPipPipPip
  • 1,814 posts
  • Joined 13-January 06
Yes something like this should work (i added the sort part with the unix tool "sort.exe") :
set filenames=
for /f "delims= usebackq" %%i in (`dir /b %target%\*.inf`) do (set %filenames%=%filenames% %%i)
echo [SourceFileInfo] >result.txt
grep -vih "\[SourceFileInfo\]" %filenames% |sort -d >>result.txt
Due to batch variables limitations %filenames% can't have more than 2047 characters so it might not work with long filenames.

Edited by allen2, 09 June 2011 - 12:58 AM.


#12
tomasz86

tomasz86

    www.windows2000.tk

  • Member
  • PipPipPipPipPipPipPipPip
  • 2,525 posts
  • Joined 27-November 10
  • OS:none specified
  • Country: Country Flag
It's getting a little bit complicated for me as I'm just a beginner when it comes to scripting ;)

Anyway, I'll try to get the above script to work.

post-47483-1123010975.png


#13
jaclaz

jaclaz

    The Finder

  • Developer
  • 14,657 posts
  • Joined 23-July 04
  • OS:none specified
  • Country: Country Flag
@allen2
You don't actually *need* grep or any other "third party" program for such a simple task.

You can use either FIND /V "[" to exclude the line(s) containing square brackets or or use a FOR /F with a SKIP=1 (if that line is always the first one), or use FIND or FINDSTR to only get lines containing commas.

This could do:
@ECHO OFF
IF EXIST result.txt DEL result.txt
FOR /F %%A IN ('dir /b *.txt') DO (
FOR /F "skip=1 tokens=*" %%B IN (%%A) DO (
ECHO %%A -- %%B
ECHO %%B >>lines.txt
)
)
ECHO [SourceFileInfo] >result.txt
SORT lines.txt >>result.txt
DEL lines.txt
ECHO.
MORE result.txt

jaclaz

#14
allen2

allen2

    Not really Newbie

  • Member
  • PipPipPipPipPipPipPip
  • 1,814 posts
  • Joined 13-January 06
@jaclaz
I hate the dos find because i always had hard time getting it to do what i wanted ; and also, as i began scripting on unix , i always find first an algorithm translation with a batch adapted from some kind of unix shell scripting.
Anyway, i never said my script coding was the best or even good.
That said, i would do what tomasz86 is trying to do (extracting hotfix, getting files with the higher version in one .inf to create a service pack) with autoit (for the duplicate file part) but a batch script might be easy for you.

#15
tomasz86

tomasz86

    www.windows2000.tk

  • Member
  • PipPipPipPipPipPipPipPip
  • 2,525 posts
  • Joined 27-November 10
  • OS:none specified
  • Country: Country Flag
Thank you very much :thumbup

I changed it like this:
@ECHO OFF
FOR /F %%A IN ('dir /b *.ver') DO (
FOR /F "skip=1 tokens=*" %%B IN (%%A) DO (
ECHO %%A -- %%B
ECHO %%B >>TEMP2\update.txt
)
)
ECHO [SourceFileInfo] >HFMER\UPDATE\update.ver
SORT TEMP2\update.txt >>HFMER\UPDATE\update.ver
DEL TEMP2\update.txt
ECHO.
MORE HFMER\UPDATE\update.ver
but I can't set the folder for '*.ver' files to be 'TEMP2\*.ver'. How should I edit this line?
FOR /F %%A IN ('dir /b *.ver') DO (

Edited by tomasz86, 09 June 2011 - 01:59 AM.

post-47483-1123010975.png


#16
allen2

allen2

    Not really Newbie

  • Member
  • PipPipPipPipPipPipPip
  • 1,814 posts
  • Joined 13-January 06
@jaclaz
Also , i am not sure the "echo %%B >>lines.txt" line will work for every kind of character the line might contains: i am thinking about "%", ">" or "&" for example but perhaps the .inf files won't ever contains one of them.

Edited by allen2, 09 June 2011 - 02:16 AM.


#17
jaclaz

jaclaz

    The Finder

  • Developer
  • 14,657 posts
  • Joined 23-July 04
  • OS:none specified
  • Country: Country Flag
@allen2
I don't know either, I just provided what IMHO is the simplest solution that fulfills the OP requirements and works with the examples posted.
It is not meant in any way as a competition of the type "my batch is better than yours", it's simply a way to exchange ideas.

@tomasz86
Try with:
@ECHO OFF 
FOR /F %%A IN ('dir /b .\TEMP2\*.ver') DO ( 
FOR /F "skip=1 tokens=*" %%B IN (.\TEMP2\%%A) DO ( 
ECHO .\TEMP2\%%A -- %%B 
ECHO %%B >>.\TEMP2\update.txt 
) 
) 
ECHO [SourceFileInfo] >.\HFMER\UPDATE\update.ver 
SORT .\TEMP2\update.txt >>.\HFMER\UPDATE\update.ver 
DEL .\TEMP2\update.txt 
ECHO. 
MORE .\HFMER\UPDATE\update.ver

Or use %~dp0:
@ECHO OFF
FOR /F %%A IN ('dir /b %~dp0*.ver') DO ( 
FOR /F "skip=1 tokens=*" %%B IN (%~dp0%%A) DO ( 
ECHO .%~dp0%%A -- %%B 
ECHO %%B >>%~dp0update.txt 
) 
) 
ECHO [SourceFileInfo] >%~dp0HFMER\UPDATE\update.ver 
SORT .\TEMP2\update.txt >>%~dp0HFMER\UPDATE\update.ver 
DEL %~dp0TEMP2\update.txt 
ECHO. 
MORE %~dp0HFMER\UPDATE\update.ver

If I get it right what you need/want.

%0 is the actual command line invoked i.e. the name of the batch or parameter 0 (zero).
%~dp0 is the same variable expanded to only drive and path (with a trailing backslash)

jaclaz

#18
tomasz86

tomasz86

    www.windows2000.tk

  • Member
  • PipPipPipPipPipPipPipPip
  • 2,525 posts
  • Joined 27-November 10
  • OS:none specified
  • Country: Country Flag
It works :) Thank you.

FOR /F %%A IN ('dir /b .\TEMP2\*.ver') DO ( 
	FOR /F "skip=1 tokens=*" %%B IN (.\TEMP2\%%A) DO ( 
	ECHO .\TEMP2\%%A -- %%B 
	ECHO %%B >>.\TEMP2\update.txt 
	) 
	) 
	ECHO [SourceFileInfo] >.\HFMER\UPDATE\update.ver 
	SORT .\TEMP2\update.txt >>.\HFMER\UPDATE\update.ver  
	RD /Q/S TEMP2
	ECHO. 
	MORE .\HFMER\UPDATE\update.ver

Generally, (as allen2 already pointed out) what I want to do here is to make a script for merging hotfixes(updates) for Windows 2000/XP/2003. It's code-name is HFMER (Hotfix Merger).

I asked about copying files and checking their file versions in an another thread but it seems to be too complicated to do now. Until I learn how to write such a script I'll stick to xcopy as it's the same way as HFSLIP uses when slipstreaming hotfixes. It may not be ideal but still such cases when the older file has newer version are quite rare so it's not such a big problem.

Anyway, by using this script I'm able to almost do everything to prepare the folders, files and the update.ver file containing the information from all of the merged hotfixes. Of course there will be duplicates in it but not that many so I can just remove them manually. Update.ver is not used by HFSLIP anyway.

The main problem lies in combining update.inf files. What I'm thinking about now is this. It's just an example. Let's say I have two update.inf files. After doing

copy update1.inf+update2.inf update3.inf
I get

[Version]

    Signature                 = "$Windows NT$"
	
[SourceDisksFiles]

    ipsecmon.exe=1
	
[Version]

    Signature                 = "$Windows NT$"
	
[SourceDisksFiles]

    remotesp.tsp=1
It would really help me if I could get from it this:

[Version]

    Signature                 = "$Windows NT$"
    Signature                 = "$Windows NT$"
	
[SourceDisksFiles]

    ipsecmon.exe=1
    remotesp.tsp=1

I don't care for duplicates now but is there any program that would do such a sorting automatically?

Edited by tomasz86, 09 June 2011 - 07:09 AM.

post-47483-1123010975.png


#19
tomasz86

tomasz86

    www.windows2000.tk

  • Member
  • PipPipPipPipPipPipPipPip
  • 2,525 posts
  • Joined 27-November 10
  • OS:none specified
  • Country: Country Flag
By the way, I checked the Beyond Compare program recommended by dencorso but I found something else which, while quite similar to the above, is simpler and I find it more suitable for the task of merging the update.inf files (unless there is a tool that can do the task I mentioned in the previous post...).

http://kdiff3.sourceforge.net/

Posted Image

It's GPL. The other tool I had already known and have been using is WinMerge but WinMerge can compare only 2 files at once while Kdiff and Beyond Compare can compare 3 files. What I like the most about KDiff is its interface which is sooooo simple that comparing and merging files is very easy and can be done quicker then Beyond Compare for example (and without the need to memorise tens of keyboard shortuts ;)).

Edited by tomasz86, 09 June 2011 - 08:48 AM.

post-47483-1123010975.png


#20
Yzöwl

Yzöwl

    Wise Owl

  • Super Moderator
  • 4,557 posts
  • Joined 13-October 04
  • OS:Windows 7 x64
  • Country: Country Flag

Donator

What you require can be done in pure batch, and obviously with the use of third party utilities.

What happens if you have multiple files in which the content potentially overwrites content of another, how do you decide which line takes precedence when your script orders it? Are you sure that alphabetical will not mean overwriting something which should take precedence. If the file for instance works in linear form meaning line 20 will always be processed after line 19, what happens if line 21 only happens if line 19 is set to 0, but line 20 alphabetically changed that to 1 because line 22 onwards only worked if if the data was set to 1. I know its hard to explain but that's why it is generally easier to allow each individual file to run in sequence.

I could of course just provide a routine for you, (but then I could simply post it as my own project and take all the credit for it myself). It is after all close to what HFSLIP should have been, had its not become bloated with pointless additions and spoiled by poor/unmanageable scripting.

#21
jaclaz

jaclaz

    The Finder

  • Developer
  • 14,657 posts
  • Joined 23-July 04
  • OS:none specified
  • Country: Country Flag
There is a pre-made batch, here:
http://www.robvander.../readini_nt.txt
to parse .ini files (very similar if not identical to .inf/.sif files) that can possibly be adapted to your uses.

Read also these:
http://www.msfn.org/...crets-revealed/
http://reboot.pro/3960/

Also, there is a tool, fedit.exe , here:
http://www.msfn.org/...ic/48200-fedit/

that has the "once" switch that may come useful.

The "prepare.cmd" here contains some examples of the use of fedit:
http://www.msfn.org/...y/page__st__306


jaclaz

#22
dencorso

dencorso

    Iuvat plus qui nihil obstat

  • Supervisor
  • 6,017 posts
  • Joined 07-April 07
  • OS:98SE
  • Country: Country Flag

Donator

[Version]

    Signature                 = "$Windows NT$"
    Signature                 = "$Windows NT$"
	
[SourceDisksFiles]

    ipsecmon.exe=1
    remotesp.tsp=1

I don't care for duplicates now but is there any program that would do such a sorting automatically?

Even if dupes (duplicate lines) aren't you concern right now, you'll have to eliminate them, so, just for the record, the yanklines script I put together only removes exact dupes, no matter how many and where they are on the text file. It's fast and can handle really big files, because it was intended to be used with merged history logs.

#23
bphlpt

bphlpt

    MSFN Addict

  • Member
  • PipPipPipPipPipPipPip
  • 1,801 posts
  • Joined 12-May 07
  • OS:none specified
  • Country: Country Flag
The comments by Yzöwl, and your own tests re version numbers of files rather than just dates, probably deserve a little more consideration. After all, depending on one's settings, sometimes just downloading or copying a file can change the date of the file - and then what?

Cheers and Regards

Posted Image


#24
tomasz86

tomasz86

    www.windows2000.tk

  • Member
  • PipPipPipPipPipPipPipPip
  • 2,525 posts
  • Joined 27-November 10
  • OS:none specified
  • Country: Country Flag
Thank you for all your input. I really appreciate it :thumbup

1. As for file version checking, I know it would be the ultimate way to be sure that only newest files are copied but even now with xcopy it's not a big problem. I did a test and compared Win2k UR2 beta1 which I created 100% manually with the same hotfixes merged by this script. The results were identical. Dates may change when copying files but here they are taken from inside the hotfixes.

I'll show you some screen shots:

Posted Image

2. Removing duplicates automatically may be tough. For example, let's say in the merged update.ver we have:

agentdpv.dll=1673840A5FDD52DB9A60D8B61AAB0FCE,0002000000000D62,53008  
agentdpv.dll=3D8249606BED3456A36A1F3069BAE2A3,0002000000000D60,53008
As file size is the same, determining the correct file would require md5 checking... but as such instances are not many it's probably easier to just check it manually and remove the unnecessary lines ;)

As for the update.inf the problem with duplicates is that the same entry is used many times, ex:

[System32.Files]

    ipsecmon.exe,,,8
    oakley.dll,,,8
    polagent.dll,,,8
    polstore.dll,,,8
    rasmans.dll,,,8

[Cache.Files]

    ipsecmon.exe,,,8
    oakley.dll,,,8
    polagent.dll,,,8
    polstore.dll,,,8
    rasmans.dll,,,8
Removing duplicates would require checking only files under each heading ([...]) separately. What I was saying about not caring about such duplicates meant that even if the final update.inf looks like this:

[System32.Files]

    ipsecmon.exe,,,8
    ipsecmon.exe,,,8
    ipsecmon.exe,,,8
    polagent.dll,,,8
    polagent.dll,,,8

[Cache.Files]

    ipsecmon.exe,,,8
    ipsecmon.exe,,,8
    ipsecmon.exe,,,8
    polagent.dll,,,8
    polagent.dll,,,8
I could just use Notepad++ and sort each list manually which is quite easy to be done and would also remove the duplicates automatically.

Also, what about something like this?

sp3res.dll
sp3res.dll,,,8

3. Anyway, I'll have a look at all these scripts and try to use them. Thanks once again for all your help B)

Edited by tomasz86, 09 June 2011 - 10:20 PM.

post-47483-1123010975.png


#25
dencorso

dencorso

    Iuvat plus qui nihil obstat

  • Supervisor
  • 6,017 posts
  • Joined 07-April 07
  • OS:98SE
  • Country: Country Flag

Donator

Dates can change easily. PE Timestamps not so. Since, in Win 2k, most executables are PE executables, PE Timestamps and version numbers should be enough to decide most ambiguities. I posted more about that in older posts, and below are poiters to them.

But, when push comes to shove, you can use the PE Timestamp (see this, this, this and this) to sort things out, in many occasions (provided the files in question *are* PE executables (EXE, SYS, OCX, DLL, TLB usually)... although, in some cases it just adds to the confusion. There's nothing equivalent for NE, LE and plain DOS executables, I'm sorry to say.
Also of interest may be this old post, by Petr, on versioning.
HTH






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users