[Pachilles]

I have a .txt file listing all possible names (about 400) to create folders with.
Simplified Example:

apple
orange
pear
white grape
purple grape
kiwi

I have a large number of files (about 30,000) with similar (but not very clean) names, that I need to organize into folders and also to cleanly rename.
Simplified Example:

apple pectin recipe #1.zip
Apple.Cores.Type3.#02.zip
apple.and.orange.comparison.001.doc
white grape juice 05 (10-5-2012) (Compressed).rar

They should be renamed and put into the correct folder(s):
Results Example:

C:\Apple\Apple Pectin Recipe #001.zip
C:\Apple\Apple Cores Type3 #002.zip
C:\Apple\Apple And Orange Comparison #001.doc
C:\Orange\Apple And Orange Comparison #001.doc
C:\White Grape\White Grape Juice #005 (10-5-2012).rar

I don't want the folders created until a file is found for it (an empty Kiwi folder would just be annoying).

Getting the files put into the folders is my primary problem, renaming is secondary since I have a 3rd party app that can help reduce my manual renaming time if necessary.

Two BATch files for this is highly preferable, since I can incorporate them into my other automation easier.

Does anyone have any suggestions?

[jumper]

I don't know of any batch commands that can process a file line-by-line, nor any that can rename files according to a script.

So I recommend using VBScript. The general algorithm would be something like:

Open fruit list
For each fruit Begin

Search for files of that fruit
If one or more create folder
For each file Begin
Clean up the filename
Move to folder
End

End

[jaclaz]

What happens if you run this batch from the "upper" directory containing the files?

@ECHO OFF
FOR /F %%A IN ('DIR /S /B *apple*.*') DO ECHO APPLE: ->%%A

jaclaz

[Pachilles]

jumper, on 09 October 2012 - 12:47 AM, said:

I don't know of any batch commands that can process a file line-by-line, nor any that can rename files according to a script.

So I recommend using VBScript. The general algorithm would be something like:

Open fruit list
For each fruit Begin

Search for files of that fruit
If one or more create folder
For each file Begin
Clean up the filename
Move to folder
End

End

I've used a FOR loop to process line by line before, but it was many years ago and don't remember how I did it.

I've been avoiding VBS, but I'll check it out.

jaclaz, on 09 October 2012 - 02:27 AM, said:

What happens if you run this batch from the "upper" directory containing the files?

@ECHO OFF
FOR /F %%A IN ('DIR /S /B *apple*.*') DO ECHO APPLE: ->%%A

jaclaz

Throwing some test files into a test folder and running this gave me a text file named "apple" (no extension), with "APPLE: -" in it.

I'm imagining a FOR loop to create an array storing each line of my text file, then using MD %%variable after DIR finds something, then moving it.

I'm not sure how I'd code that.

[jaclaz]

Pachilles, on 09 October 2012 - 09:50 AM, said:

Throwing some test files into a test folder and running this gave me a text file named "apple" (no extension), with "APPLE: -" in it.

I'm imagining a FOR loop to create an array storing each line of my text file, then using MD %%variable after DIR finds something, then moving it.

I'm not sure how I'd code that.

My bad , I forgot to escape the ">" with a caret, try with this:

@ECHO OFF
FOR /F %%A IN ('DIR /S /B *apple*.*') DO ECHO APPLE: -^>%%A

This one liner should list all files containing "apple" (case insensitive) in the name and display them on the screen.


jaclaz

[gunsmokingman]

Here is a VBS script that goes threw the parent folder and all of
the sub folders and list files. This will produce a text file with
the missing folder path, the move command. It will not make
any folders or move any files.

Quote

Example Of Output

 
 Missing Path      : D:\Test Sort\BingBarSetup
 Created Folder    : D:\Test Sort\BingBarSetup

 Fso.Move D:\Test Sort\BingBarSetup.EXE,D:\Test Sort\BingBarSetup\BingBarSetup.EXE


 Missing Path      : D:\Test Sort\BitTorrent-721
 Created Folder    : D:\Test Sort\BitTorrent-721

 Fso.Move D:\Test Sort\BitTorrent-7.2.1.exe,D:\Test Sort\BitTorrent-721\BitTorrent-7.2.1.exe

Sort_Folder.vbs

'-> Objects For Runtime
 Dim Act :Set Act = CreateObject("Wscript.Shell")
 Dim Fso :Set Fso = CreateObject("Scripting.FileSystemObject")
 Dim C1, Col, FPath, LPath, Ts, Tx
 LPath = "PATH_TO_FOLDER"
 Tx = Act.SpecialFolders("Desktop") & "\Results_OrginizeFolder.txt"
'-> Check To Make Sure Path Exits
  If Fso.FolderExists(LPath) Then
   Set Ts = Fso.CreateTextFile(Tx)
   Ts.WriteLine " Script Start Date : " & Date
   Ts.WriteLine " Script Start Time : " & Time
   Ts.WriteBlankLines 2
   Sort(Fso.GetFolder(LPath))
   Ts.Close 
   Act.Run(Tx),1,True 
   If MsgBox( _
    "Did You Want To Keep The Text Report?" & vbCrLf & _
    "Yes To Keep The File, No To Delete",4132, _
    "Keep Or Delete") = 7 Then Fso.DeleteFile(Tx),True
  Else
   MsgBox "Location Path Does Not Exists." & _
   "You Must Provide An Existing Path", 4128,"Error No Path"
  End If   
'->  Recursive To Go Threw Parent Folder And Sub Folder
   Function Sort(Folder)
'-> To Make The Results File Smaller For Testing
    If C1 = 10 Then Exit Function 
    C1 = C1 + 1
'-> List Files In Folder
    For Each Col In Folder.Files
     FPath = Fso.GetBaseName(Col.Path)
      If Not Fso.FolderExists(FPath) Then
      Ts.WriteLine " Missing Path      : " & FPath & vbCrLf & _
                   " Created Folder    : " & Fso.GetAbsolutePathName(FPath)
       Ts.WriteLine            
       Ts.WriteLine " Fso.Move " & Col.Path & "," & FPath & "\" & Col.Name 
      Else
       Ts.WriteLine " Confirm Path      : " & Col.Name
      End If 
     Ts.WriteBlankLines 2 
    Next
'-> Loop Threw All Sub Folders     
    For Each Obj In Folder.SubFolders
      Sort(Folder)
    Next
   End Function    

[iamtheky]

Here is an autoit solution. In my test:

1) The Folder list is a txt file named 'FolderList.txt' and is relative to the script
2) The Files are all in a folder named 'Files' (also relative to the script). The files can be scattered in sub folders beneath that folder.

Upon execution it will create a folder named 'Folders', and beneath, for all files that contain one of the strings from the list in their name, it will create the folder using the name from the list and copy the file to that destination.

RecFileListToArray UDF

#include<RecFileListToArray.au3>
#include<File.au3>

Dim $FolderArr

_FileReadToArray("Folderlist.txt" , $FolderArr)

$FilesArr = _RecFileListToArray(@ScriptDir & "\Files" , "*" , 1 , 1)

For $i = 1 to ubound($FilesArr) - 1
	For $k = 1 to ubound($FolderArr) - 1
		If stringinstr($FilesArr[$i] , $FolderArr[$k]) Then
		DirCreate (@ScriptDir & "\Folders\" & $FolderArr[$k])
		FileCopy(@ScriptDir & "\Files\" & $FilesArr[$i] , @ScriptDir & "\Folders\" & $FolderArr[$k])
		EndIf
	Next
Next

This post has been edited by iamtheky: 09 October 2012 - 02:43 PM

[Yzöwl]

Pachilles, on 08 October 2012 - 09:11 PM, said:

I have a .txt file listing all possible names (about 400) to create folders with.
Simplified Example:

apple
orange

I have a large number of files (about 30,000) with similar (but not very clean) names, that I need to organize into folders and also to cleanly rename.
Simplified Example:

apple.and.orange.comparison.001.doc

They should be renamed and put into the correct folder(s):
Results Example:

C:\Apple\Apple And Orange Comparison #001.doc
C:\Orange\Apple And Orange Comparison #001.doc

Just thought I'd highlight the above again as a reminder that

Your solution(s) need to bear the following in mind; to move 'apple.and.orange.comparison.001.doc' into \apple will make in no longer available to place in \orange! You'd problably need to use a copy method marking any successfully copied file for later deletion.

[Pachilles]

I'll definitely check into the above solutions/suggestions, but I'm getting pretty close with the following:

List.txt

Apple
Orange
Pear
White Grape
Purple Grape
Kiwi

Test.bat

FOR /F "delims=;" %%i in (List.txt) do (call Test-Sub.bat %%i) && (del *%%i*.*)

Test-Sub.bat

FOR /F "delims=;" %%j in ('DIR /B *%1*.*') do (if not exist .\%1 (md %1)) & (copy %%j .\%1\)

Test files:

apple pectin recipe #1.txt
apple.and.orange.comparison.001.txt
Apple.Cores.Type3.#02.txt
white grape juice 05 (10-5-2012) (Compressed).txt

The above results in:

In the CMD window:

C:\>FOR /F "delims=;" %i in (List.txt) do (call Test-Sub.bat %i )  && (del *%i*.* )

C:\>(call Test-Sub.bat Apple )  && (del *Apple*.* )

C:\>FOR /F "delims=;" %j in ('DIR /B *Apple*.*') do (if not exist .\Apple (md Apple ) )  & (copy %
j .\Apple\ )

C:\>(if not exist .\Apple (md Apple ) )  & (copy apple pectin recipe #1.txt .\Apple\ )
The syntax of the command is incorrect.

C:\>(if not exist .\Apple (md Apple ) )  & (copy apple.and.orange.comparison.001.txt .\Apple\ )
        1 file(s) copied.

C:\>(if not exist .\Apple (md Apple ) )  & (copy Apple.Cores.Type3.#02.txt .\Apple\ )
        1 file(s) copied.
C:\>(call Test-Sub.bat Orange )  && (del *Orange*.* )

C:\>FOR /F "delims=;" %j in ('DIR /B *Orange*.*') do (if not exist .\Orange (md Orange ) )  & (cop
y %j .\Orange\ )
File Not Found
C:\>(call Test-Sub.bat Pear )  && (del *Pear*.* )

C:\>FOR /F "delims=;" %j in ('DIR /B *Pear*.*') do (if not exist .\Pear (md Pear ) )  & (copy %j .
\Pear\ )
File Not Found
C:\>(call Test-Sub.bat White Grape )  && (del *White Grape*.* )

C:\>FOR /F "delims=;" %j in ('DIR /B *White*.*') do (if not exist .\White (md White ) )  & (copy %
j .\White\ )

C:\>(if not exist .\White (md White ) )  & (copy white grape juice 05 (10-5-2012) (Compressed).txt
 .\White\ )
The syntax of the command is incorrect.
C:\>(call Test-Sub.bat Purple Grape )  && (del *Purple Grape*.* )

C:\>FOR /F "delims=;" %j in ('DIR /B *Purple*.*') do (if not exist .\Purple (md Purple ) )  & (cop
y %j .\Purple\ )
File Not Found
C:\>(call Test-Sub.bat Kiwi )  && (del *Kiwi*.* )

C:\>FOR /F "delims=;" %j in ('DIR /B *Kiwi*.*') do (if not exist .\Kiwi (md Kiwi ) )  & (copy %j .
\Kiwi\ )
File Not Found

Resulting organized as:

C:\Apple
C:\Apple\apple.and.orange.comparison.001.txt
C:\Apple\Apple.Cores.Type3.#02.txt

C:\White

C:\white grape juice 05 (10-5-2012) (Compressed).txt

I've apparently forgotten a bit of my batch skills, and having syntax issues.

I played with quotes to fix my issues, but everything just got worse.
Also, I don't feel safe with using the del command where it's at (even after fixing the rest), but copy without an automated delete will overflow my HDD once I'm using the actual files (they total around 350 GB).
Some where along the way, I lost the getting an Orange folder

This post has been edited by Pachilles: 09 October 2012 - 03:46 PM

[allen2]

Yzöwl, on 09 October 2012 - 02:44 PM, said:

Just thought I'd highlight the above again as a reminder that

Your solution(s) need to bear the following in mind; to move 'apple.and.orange.comparison.001.doc' into \apple will make in no longer available to place in \orange! You'd problably need to use a copy method marking any successfully copied file for later deletion.

It would indeed need to mark it but , copying wouldn't be a good solution as it would slow with big files and take too much space. To resolve this, i would use depending on the filesystem shortcut creation or hardlink.
Also as i understood first, you needed to create folder automatically so the name of the files you need to process might perhaps be parsed to create the list of folder and decide which file will go where and if it will need to go in many folders. But from the filenames provided, it may not be possible unless you decide that it is the first word of the name of the file and word separator can be either space or dot (but then which word would need to be parsed to get the second like in the case of apple.and.orange.comparison.001.txt). Also fruits with space in their name wouldn't be parsed properly.

[Pachilles]

allen2, on 09 October 2012 - 11:40 PM, said:

Yzöwl, on 09 October 2012 - 02:44 PM, said:

Just thought I'd highlight the above again as a reminder that

Your solution(s) need to bear the following in mind; to move 'apple.and.orange.comparison.001.doc' into \apple will make in no longer available to place in \orange! You'd problably need to use a copy method marking any successfully copied file for later deletion.

It would indeed need to mark it but , copying wouldn't be a good solution as it would slow with big files and take too much space. To resolve this, i would use depending on the filesystem shortcut creation or hardlink.
Also as i understood first, you needed to create folder automatically so the name of the files you need to process might perhaps be parsed to create the list of folder and decide which file will go where and if it will need to go in many folders. But from the filenames provided, it may not be possible unless you decide that it is the first word of the name of the file and word separator can be either space or dot (but then which word would need to be parsed to get the second like in the case of apple.and.orange.comparison.001.txt). Also fruits with space in their name wouldn't be parsed properly.

The folder names being listed in the txt file is the search criteria searching the file names for all words in a given folder name should do it. Getting IFs, ANDs, ORs, proper quote usage, and deleting is the real trick here. Holding off the deletion of a file with "and" in it, until completion... maybe. Possibly a scan/commit process.

[jaclaz]

The Issues might come with some special characters in the filenames.
This is a "compacted" version of the two batches you made.
Hopefully putting double quotes around %%j will do the trick.
In theory having moved the deletion to the subroutine and having linked it to the copy command should avoid the accidental deletion of a non-copied file.

@ECHO OFF
FOR /F "delims=;" %%i in (List.txt) do (
call testsub %%i
) 
GOTO :EOF

:testsub
FOR /F "delims=;" %%j in ('DIR /B *%1*.*') do (
if not exist .\%1 md %1
copy "%%j" .\%1\ &&del "%%j"
)
GOTO :EOF

jaclaz

This post has been edited by jaclaz: 10 October 2012 - 05:13 AM

[Pachilles]

jaclaz, on 10 October 2012 - 05:11 AM, said:

The Issues might come with some special characters in the filenames.
This is a "compacted" version of the two batches you made.
Hopefully putting double quotes around %%j will do the trick.
In theory having moved the deletion to the subroutine and having linked it to the copy command should avoid the accidental deletion of a non-copied file.

@ECHO OFF
FOR /F "delims=;" %%i in (List.txt) do (
call testsub %%i
) 
GOTO :EOF

:testsub
FOR /F "delims=;" %%j in ('DIR /B *%1*.*') do (
if not exist .\%1 md %1
copy "%%j" .\%1\ &&del "%%j"
)
GOTO :EOF

jaclaz

Thanks for reminding me that I can 1. subroutine within one bat file, and 2. multi-line within (). Much prettier! :-)

I did get a set of quotes in that managed to give me a "White Grape" folder instead of just White, but the "white grape" file didn't copy. I'll give your cleaner version a try. I think the location of quotes may be better.

The del next to the copy matches my original "move" command, that I tried, but I pulled it out because the "...and orange..." file never made it to the Orange folder. I haven't found a better place to delete, though.

I'm thinking that either I make it not delete any file with "and" in it, until after the script is finished, or I do a scan/commit process. If I scan and sort into a text file that decides where all the files are going, and then run again for the actual copy/del...

Any ideas?



EDIT:
Your quote location made moving files with spaces in them to move, and my original location allowed for 2-word folders.
Having both worked beautifully!

@ECHO OFF

FOR /F "delims=;" %%i in (List.txt) do (
call :testsub "%%i"
) 
GOTO :End

:testsub
FOR /F "delims=;" %%j in ('DIR /B *%1*.*') do (
if not exist .\%1 md %1
copy "%%j" .\%1\
)
GOTO :EOF

:End

Now once I find a good location for the DEL, I'm all set... If I can't... I might have to rethink the handling of those multi-fruit files.

This post has been edited by Pachilles: 10 October 2012 - 02:25 PM

[jumper]

Since there will be some copying or linking, perhaps rather than doing any move/copy/deleting, consider always creating a link:

• The original files would never be touched. You would always be able to start over from scratch--even after you thought you had a "final" solution.
  
• The links could be created in a compressed folder to prevent them from consuming as much space as the originals
  
• They could be renamed independently from the original or each other ( "Apple and Oranges..." -> "Oranges and Apples...") in the respective target folders.

[Pachilles]

jumper, on 10 October 2012 - 03:06 PM, said:

Since there will be some copying or linking, perhaps rather than doing any move/copy/deleting, consider always creating a link:

• The original files would never be touched. You would always be able to start over from scratch--even after you thought you had a "final" solution.
  
• The links could be created in a compressed folder to prevent them from consuming as much space as the originals
  
• They could be renamed independently from the original or each other ( "Apple and Oranges..." -> "Oranges and Apples...") in the respective target folders.

I really like that concept, and in other situations that would be perfect (already thinking of some possibilities). The problem with THIS one is that this is part of the cleanup and organizing before allowing a database to run an automated scrub of everything internal to each file. I've already had to scratch everything and start over. Getting this external automation working takes a huge load off my manual workload... especially when considering the continuous flow of more files being added.

BTW, I've run this in a sub-group of the actual files, and I've found a few additional scenarios.
Add these files to see:
Pineapples - The first cut.txt
The Big Apple (New York).txt

Since Pineapple includes "apple", it's moved incorrectly. Putting a space in front of the word check fixes it... but messes with everything else.
The Big Apple... causes issues if the word checked isn't at the beginning.

Here's the fixed code that allows these as well (also added remarks to beautify):

@ECHO OFF

REM ******************  Loop subroutine for each item in List.txt
FOR /F "delims=;" %%i in (List.txt) do (
call :testsub "%%i"
) 
GOTO :End


REM ******************  Subroutine(s)

REM ******************  Search the filenames for an instance of the list item anywhere (with a space in front of it)
:testsub
FOR /F "delims=;" %%j in ('DIR /B *" "%1*.*') do (
IF NOT EXIST .\%1 md %1
MOVE "%%j" .\%1\
)

REM ******************  Check the beginning of the filename for the list item
FOR /F "delims=;" %%j in ('DIR /B %1*.*') do (
IF NOT EXIST .\%1 md %1
MOVE "%%j" .\%1\
)

GOTO :EOF

:End

Also of note... List.txt needs to be arranged in a way that puts the more important folders to be hit first. For instance, if I include "grape" in the list as a catch all for any grapes not accounted for (red grape, seedless, etc), it should be put under the more specific grape types. This is, of course, not an issue if the files are not moved or deleted as in my code above.


Thanks for the help everyone.
I think I'll end up using this version. Maybe at another time, I'll create a script to search the folders and copy any multi-'fruit' files needed... then append it to this script.

[jaclaz]

Still you don't need the :End label, the implied :EOF is good enough.

jaclaz