vendredi 20 mars 2015

Looking to increase the speed of my bash script


The title pretty much says it all, I have a short bash script that processes gigs and gigs of data. I am looking for any improvements to make it faster. This is my very first bash script so please be gentle ;) I am really only concerned about the while loop, the rest of it is fine I think, its the while loop where the real work is done and could use the most enhancement.



#!/bin/bash

# This script will clean a WPA wordlist
# It will read every line of the given file
# Remove all whitespace except for newlines
# Delete the line if it is less than 8 chars or greater than 63
# It will then exit with time of execution


IFS=$'\n' # make newlines the only separator



startTime=$(date) # start time of execution
fileToClean=$1 # this is the file we will be sanatizing, 1st cmd line arg
deletedlines=0 # number of lines that did not meet WPA PSK critera
validlines=0 # number of lines that were valid PSKs and added to file


if [ -z $fileToClean ]; then #No file specified
echo ""
echo "No file specified!"
echo ""
exit -1
fi

if [ ! -f $fileToClean ]; then #File does not exist
echo ""
echo "File not found!"
echo ""
exit -1
fi

#By this point I am assuming I entered a valid file and will begin cleaning
echo ""
echo 'Cleaning word list: ' $1
echo "Start Time: " $startTime
echo ""


while read line; do #read every line in file and save to var line

trimmedline=$(echo $line | xargs -0) # use the xargs command to trim
# white space and save trimmed line
# into the new var trimmedline

if [ ${#trimmedline} -ge 8 ] && [ ${#trimmedline} -le 63 ]; then # if trimmedline length >= 8 && <= 63
echo $trimmedline >> $outputfile
((validlines++))
continue
fi

((deletedlines++))


done <$fileToClean


utime="$( TIMEFORMAT='%lU';time ( ls ) 2>&1 1>/dev/null )" # this stores the executio time in the var utime

echo "Processing completed, it took" $utime "and $deletedlines were deleted."
echo $validlines "were added to the output file "$outputfile" as they were valid PSKs"
echo ""




Aucun commentaire:

Enregistrer un commentaire