[gunsmokingman]

Could you not add something like this to your script, it ask do you want it case sensitive or not case sensitive?
This would save having 2 script posted.

  If MsgBox( _
  "Would you like to make the script be Case senistive?" & vbCrLf & _
  "Examble Case Insenistive A1b, a1B, A1B are not dublicates" & vbCrLf & _
  "and only one entry would be applied" & vbCrLf & _
  "Examble Case Senistive A1b, a1B, A1B are dublicates would" & vbCrLf & _
  "be added as separate entries" & vbCrLf & _
  "Yes to make RemoveLines.vbs case Insenistive" & vbCrLf & _
  "No to make RemoveLines.vbs case senistive",4132,"Yes No senistive") = 6 Then
   '-> Case Insensitive
    Dic.CompareMode = 1
   Else
   '-> Case Sensitive
    Dic.CompareMode = 0
   End if

[dencorso]

Of course. But I think it'd be even more intrusive than adding a command-line switch to control whether it's case sensitive or not. Myself, I see case-sensitivity in yanklines more as a bug than as a feature. But I grant it may be useful in some select cases. All in all, I think the two versions of the script, as proposed by jaclaz, and which I've adopted already, by far the best solution.

[tomasz86]

allen2, on 08 August 2011 - 11:54 PM, said:

You need :
- to comment the line

RunMeWithCScript()

- to comment all lines with wscript.echo (unless you want popup for each line).
- to launch the script with wscript.exe instead of cscript if you're launching it from another script.

I've finally managed to make the script fully silent. Thanks, allen2

This is what I did.

1) I commented the following lines:

WScript.Echo "usage: YankLines <input_filename> <output_filename>"
WScript.Echo("Record: " & strFirstField)
wscript.Echo "Total Records Writen:   " & objDict.count
wscript.Echo "Total Duplicates found: " & j

and then changed:

If scriptEngine = "WSCRIPT.EXE" Then

to

If scriptEngine = "CSCRIPT.EXE" Then

After doing so there's no visible output anymore.

This is a shrunken version of the whole script (all comments removed & case insensitive):

RunMeWithCScript()
If WScript.Arguments.Count <> 2 then 
WScript.Quit 
end If 
Const ForReading = 1, ForWriting = 2
Dim i, j 
Set objFSO = CreateObject("Scripting.FileSystemObject")
Set objInputFile = objFSO.OpenTextFile(WScript.Arguments.Item(0), ForReading)
Set objOutputFile = objFSO.OpenTextFile (WorkingDir & WScript.Arguments.Item(1), ForWriting, True)
Set objDict = CreateObject("Scripting.Dictionary")
objDict.CompareMode = 1
j = 0 
On Error Resume Next
While Not objInputFile.AtEndOfStream
arrinputRecord = split(objInputFile.Readline, "vbNewLine") 
strFirstField = arrinputRecord(0)                       
If objDict.Exists(strFirstField) then
j=j+1
Else
objDict.add strFirstField, strFirstField
End If
Wend
colKeys = objDict.Keys
For Each strKey in colKeys
objOutputFile.writeline objDict.Item(strKey)
Next
objInputFile.Close
objOutputFile.Close
Public Sub RunMeWithCScript()
Dim scriptEngine, engineFolder, Args, arg, scriptName, argString, scriptCommand
scriptEngine = Ucase(Mid(Wscript.FullName,InstrRev(Wscript.FullName,"\")+1))
engineFolder = Left(Wscript.FullName,InstrRev(Wscript.FullName,"\"))
argString = ""
If scriptEngine = "CSCRIPT.EXE" Then    
Dim Shell : Set Shell = CreateObject("Wscript.Shell")
Set Args = Wscript.Arguments
For each arg in Args
If instr(arg," ") > 0 Then arg = """" & arg & """"
argString = argString & " " & Arg
Next
scriptCommand = "cmd.exe /c " & engineFolder & "cscript.exe //U """ & Wscript.ScriptFullName & """" & argString
Shell.Run scriptCommand,,False
Wscript.Quit
Else
Exit Sub
End If
End Sub

This post has been edited by tomasz86: 20 July 2012 - 01:42 AM

[dencorso]

But... but... but if you wished to have it run with wscript.exe, which is the default mode, you could as well just have deleted the first line, and all lines from "Public Sub RunMeWithCScript()" to the end -- the result would basically be the same, in 20 lines less.

[tomasz86]

As I said, I know nothing about VBS scripting

You mean this, right?

If WScript.Arguments.Count <> 2 then 
WScript.Quit 
end If 
Const ForReading = 1, ForWriting = 2
Dim i, j 
Set objFSO = CreateObject("Scripting.FileSystemObject")
Set objInputFile = objFSO.OpenTextFile(WScript.Arguments.Item(0), ForReading)
Set objOutputFile = objFSO.OpenTextFile (WorkingDir & WScript.Arguments.Item(1), ForWriting, True)
Set objDict = CreateObject("Scripting.Dictionary")
objDict.CompareMode = 1
j = 0 
On Error Resume Next
While Not objInputFile.AtEndOfStream
arrinputRecord = split(objInputFile.Readline, "vbNewLine") 
strFirstField = arrinputRecord(0)                       
If objDict.Exists(strFirstField) then
j=j+1
Else
objDict.add strFirstField, strFirstField
End If
Wend
colKeys = objDict.Keys
For Each strKey in colKeys
objOutputFile.writeline objDict.Item(strKey)
Next
objInputFile.Close
objOutputFile.Close

[dencorso]

Sure!

[tomasz86]

Is there any simple way to modify yanklines to treat all lines with spaces/tabs at the beginning/end as equal (regardless of the number of spaces)?

I mean sth like this:

"abc.txt"
"abc.txt"      <-spaces / tabs
"abc.txt"                                   <-spaces / tabs
spaces->            "abc.txt"

This post has been edited by tomasz86: 26 July 2012 - 05:30 AM

[dencorso]

Not easily, no. This kind of thing is better done with sed. sed is unix, of course, but there are countless ports of sed for Win32. I like the cygwin version, but that needs also cygwin1.dll, so probably something based on mingw may suit you better. And sed is very powerful, so I do think you should invest the time needed to get familiar with it.

[tomasz86]

I guess I'll need to study sed later on but at the moment I've managed to get through it using the following method.


1) This is a real-life example:

AppPatch.Help.Files=10,"Help"<-no spaces
AppPatch.Help.Files=10,"Help"              <-spaces
Com.Files=11,com<-no spaces
Com.Files=11,com          <-spaces
ADMT.FIles=65622<-no spaces
ADMT.FIles=65622          <-spaces

2) The cmd script:

FOR /F %%I IN ('FINDSTR/V "," 1.inf') DO ECHO>>temp.txt %%I
FOR /F tokens^=1-2^ delims^=^", %%I IN ('FINDSTR ^"^"^" 1.inf') DO ECHO>>temp.txt %%I"%%J"
FOR /F "tokens=1,2 delims=, " %%I IN ('FINDSTR "," 1.inf ^| FINDSTR/V ^"^"^"') DO ECHO>>temp.txt %%I,%%J

3) Result:

ADMT.FIles=65622<-no spaces
ADMT.FIles=65622<-no spaces
AppPatch.Help.Files=10,"Help"<-no spaces
AppPatch.Help.Files=10,"Help"<-no spaces
Com.Files=11,com<-no spaces
Com.Files=11,com<-no spaces

[Yzöwl]

tomasz86, on 26 July 2012 - 07:43 PM, said:

I've managed to get through it using the following method.

FOR /F %%I IN ('FINDSTR/V "," 1.inf') DO ECHO>>temp.txt %%I
FOR /F tokens^=1-2^ delims^=^", %%I IN ('FINDSTR ^"^"^" 1.inf') DO ECHO>>temp.txt %%I"%%J"
FOR /F "tokens=1,2 delims=, " %%I IN ('FINDSTR "," 1.inf ^| FINDSTR/V ^"^"^"') DO ECHO>>temp.txt %%I,%%J

Not only does your 'fix' not show that your request of white space from the beginning of lines has been fulfilled you apear to have been a little carried away with a specific character, ^.

[tomasz86]

@Yzöwl

Well, the request was more general while here I need to deal with specific lines and at the moment I don't know of any better way then the one I used above.

[dencorso]

Yes, you do. By using sed (and perhaps unix2dos to correct the line terminators depending on the implementation of sed you choose) almost any modification one can conceieve, however complex, can be performed fast. I put yanklines together to remove dupes fast, in a file, which is very far from the things sed does well. But while you want to reformat individual lines, one by one, no other tool is needed since sed is very powerful and time-proven. sed can be used as a filter, just like FIND.

# delete leading whitespace (spaces, tabs) from front of each line
# aligns all text flush left
sed 's/^[ \t]*//'                    # see note on '\t' at end of file

<...>
USE OF '\t' IN SED SCRIPTS: For clarity in documentation, we have used
the expression '\t' to indicate a tab character (0x09) in the scripts.
However, most versions of sed do not recognize the '\t' abbreviation,
so when typing these scripts from the command line, you should press
the TAB key instead. '\t' is supported as a regular expression
metacharacter in awk, perl, and HHsed, sedmod, and GNU sed v3.02.80.

Reference: sed1liners

[allen2]

For the record autoit has a function to remove white space of any kind (tab included).

[Yzöwl]

tomasz86, on 27 July 2012 - 09:43 AM, said:

I need to deal with specific lines and at the moment I don't know of any better way then the one I used above.

Depending upon poison characters, and using a cmd file, simply using a CALL may be all you need.

FOR /F "DELIMS=" %%# IN (1.INF) DO CALL :SUB %%#
GOTO :EOF
:SUB
>>TEMP.TXT ECHO=%*

[tomasz86]

Yzöwl, on 27 July 2012 - 04:49 PM, said:

Depending upon poison characters, and using a cmd file, simply using a CALL may be all you need.

FOR /F "DELIMS=" %%# IN (1.INF) DO CALL :SUB %%#
GOTO :EOF
:SUB
>>TEMP.TXT ECHO=%*

You're right


There are sometimes comments present:

SystemRoot.Files=10             ; %windir% (replace if exist)

so this seems to work perfectly:

FOR /F "DELIMS=;" %%# IN (1.INF) DO CALL :SUB %%#
GOTO :EOF
:SUB
>>TEMP.TXT ECHO=%*

Thank you.

Edit: Actually it's still not perfect. This gets broken:

Source:

TSClientDefault.Files=65618,%TSCLIENTDIR%

Result:

TSClientDefault.Files=65618,

This post has been edited by tomasz86: 27 July 2012 - 10:07 PM

[tomasz86]

This is a "minified" version of the silent yanklines script which is created on the fly from commandline:

ECHO>yanklines.vbs ^If WScript.Arguments.Count ^<^> 2 then
ECHO>>yanklines.vbs WScript.Quit
ECHO>>yanklines.vbs end ^If
ECHO>>yanklines.vbs Const ForReading=1, ForWriting=2
ECHO>>yanklines.vbs Dim i,j
ECHO>>yanklines.vbs Set objFSO=CreateObject("Scripting.FileSystemObject")
ECHO>>yanklines.vbs Set objInputFile=objFSO.OpenTextFile(WScript.Arguments.Item(0),ForReading)
ECHO>>yanklines.vbs Set objOutputFile=objFSO.OpenTextFile (WorkingDir ^& WScript.Arguments.Item(1),ForWriting,True)
ECHO>>yanklines.vbs Set objDict=CreateObject("Scripting.Dictionary")
ECHO>>yanklines.vbs objDict.CompareMode=1
ECHO>>yanklines.vbs j=0
ECHO>>yanklines.vbs On Error Resume Next
ECHO>>yanklines.vbs While Not objInputFile.AtEndOfStream
ECHO>>yanklines.vbs arrinputRecord=split(objInputFile.Readline,"vbNewLine")
ECHO>>yanklines.vbs strFirstField=arrinputRecord(0)
ECHO>>yanklines.vbs ^If objDict.Exists(strFirstField) then
ECHO>>yanklines.vbs j=j+1
ECHO>>yanklines.vbs Else
ECHO>>yanklines.vbs objDict.add strFirstField,strFirstField
ECHO>>yanklines.vbs End ^If
ECHO>>yanklines.vbs Wend
ECHO>>yanklines.vbs colKeys=objDict.Keys
ECHO>>yanklines.vbs For Each strKey in colKeys
ECHO>>yanklines.vbs objOutputFile.writeline objDict.Item(strKey)
ECHO>>yanklines.vbs Next
ECHO>>yanklines.vbs objInputFile.Close
ECHO>>yanklines.vbs objOutputFile.Close

Maybe someone will fine it useful.

[jaclaz]

tomasz86, on 19 September 2012 - 11:09 AM, said:

This is a "minified" version of the silent yanklines script which is created on the fly from commandline:
....
Maybe someone will fine it useful.

IMHO this is - besides smaller - "plainer":
1.000 bytes instead of 1.356 BUT including 10 bytes for the "@ECHO OFF", so actually 366 bytes smaller or almost 27% smaller .

@ECHO OFF&(
ECHO ^If WScript.Arguments.Count ^<^> 2 then
ECHO WScript.Quit
ECHO end ^If
ECHO Const ForReading=1, ForWriting=2
ECHO Dim i,j
ECHO Set objFSO=CreateObject^("Scripting.FileSystemObject"^)
ECHO Set objInputFile=objFSO.OpenTextFile^(WScript.Arguments.Item^(0^),ForReading^)
ECHO Set objOutputFile=objFSO.OpenTextFile ^(WorkingDir ^& WScript.Arguments.Item^(1^),ForWriting,True^)
ECHO Set objDict=CreateObject^("Scripting.Dictionary"^)
ECHO objDict.CompareMode=1
ECHO j=0
ECHO On Error Resume Next
ECHO While Not objInputFile.AtEndOfStream
ECHO arrinputRecord=split^(objInputFile.Readline,"vbNewLine"^)
ECHO strFirstField=arrinputRecord^(0^)
ECHO ^If objDict.Exists^(strFirstField^) then
ECHO j=j+1
ECHO Else
ECHO objDict.add strFirstField,strFirstField
ECHO End ^If
ECHO Wend
ECHO colKeys=objDict.Keys
ECHO For Each strKey in colKeys
ECHO objOutputFile.writeline objDict.Item^(strKey^)
ECHO Next
ECHO objInputFile.Close
ECHO objOutputFile.Close
)>yanklines.vbs

jaclaz

[tomasz86]

Nice

I'm wondering if these lines are absolutely necessary:

1.

If WScript.Arguments.Count <> 2 then 
WScript.Quit 
end If 

2.

objInputFile.Close
objOutputFile.Close

The script seems to work without them too...

[bphlpt]

I believe (1) is very necessary, or or least the "< 2" part is, since I'm not sure what will happen, besides error, if you do not specify both the input and output file. If you have extra arguments, ie "> 2", they will just be ignored but it is an indication that something is screwed up in the program call.

AFAIK, (2) is not "necessary" since both files "should" be closed when the script exits, but it is best practice and I would always do it. You should always try to clean up after yourself, and not depend on what "should" happen. But that's just my opinion.

Cheers and Regards

This post has been edited by bphlpt: 19 September 2012 - 09:11 PM

[dencorso]

tomasz86, on 19 September 2012 - 02:46 PM, said:

I'm wondering if these lines are absolutely necessary:

2.

objInputFile.Close
objOutputFile.Close

The script seems to work without them too...

Removing these lines you'd be introducing a potential bug by relying on behavior which is not guaranteed.
And running a large number of times such a buggy script may lead to all kinds of hard to debug conditions.
Don't ever forget to close all open files before terminating.