dictionaries[v0.1].py

A python script to help mange dictionaries/word-lists

  1. Remove dups - Scans each file, and simply removes any duplicate words.
  2. Remove dups Sorts 0-Z - Scans each file, removes any duplicate words, and sorts the result alphabetically with any numbers at the top.
  3. Remove dups Higher the dups, higher the list - Scans each file, removes any duplicate words, and sorts the list by placing words which are duplicated more at the top of the list.
  4. Remove dups Higher the dups, higher the list sorts the rest 0-Z - Scans each file, removes any duplicate words, and sorts the list by placing words which are duplicated more at the top of the list then sort the rest alphabetically with any numbers at the top.
  5. Merge - Doesn't remove anything, just simply combines files together.

Once complete, a new file is created called "output.txt" with the new dictionary.

Links

Download Script (dictionaries[v0.1].py): *coming soon*

Commands

1
2
3
4
5
6
python dictionaries.py

python dictionaries.py [File1] [File2] [File3]
python dictionaries.py word.lst dictionary.dic passphrase.txt

python dictionaries.py dictionaries/*

Example

Raw (18 total words. 2x Grape, 5x Carrot, 3x Melon, 1x #number, 2x multi words)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
Apple
Banana
Grape
Carrot
Carrot
Cherry
Garlic
Grape
Carrot
Carrot
Kiwi fruit
Carrot
Leak
2 Lemon
Melon
Melon
Mushroom
Melon

1. Remove duplicates

1
2
3
4
5
6
7
8
9
10
11
Apple
Banana
Grape
Carrot
Cherry
Garlic
Kiwi fruit
Leak
2 Lemon
Melon
Mushroom

2. Remove duplicates, then sorted (0-Z)

1
2
3
4
5
6
7
8
9
10
11
2 Lemon
Apple
Banana
Carrot
Cherry
Garlic
Grape
Kiwi fruit
Leak
Melon
Mushroom

3. Remove duplicates, then sorted (descending duplicataion frequecney)

1
2
3
4
5
6
7
8
9
10
11
Carrot
Grape
Melon
Apple
Banana
Cherry
Garlic
Kiwi fruit
Leak
2 Lemon
Mushroom

4. Remove duplicates, then sorted (descending duplicataion frequecney, then 0-Z)

1
2
3
4
5
6
7
8
9
10
11
Carrot
Grape
Melon
2 Lemon
Apple
Banana
Cherry
Garlic
Kiwi fruit
Leak
Mushroom

Notes

  • Merge does not work... yet! (Option 5 in the menu)
  • Created a backup before using this!
  • Buggy as hell!

Blog Post: https://blog.g0tmi1k.com/2010/06/dictionariesv01py/