First up:

Find & replace filenames

This finds file that match Somethinghtml.html and renames to Something.html (HTTP Track users will recognize these two examples as fixes to common naming problems when scraping a site.) Note you can change the mask of files searched by changing the find mask at the end of the line.

<code>

perl -p -i -e ’s/(.*)html\.html/\1.html/g;’ `find ./ -name ‘*.html’`

</code>

Then Find & Replace text inside of files:

This finds things that have been incorrectly prefixed with dev.mistcat.com and removes that prefix so that this 2112.js file can load from a remote domain correctly.  Note you can change the find mask at the beginning to restrict or un-restrict your searched files for text replacement.

<code>

find . -name ‘*.html’ | perl -pi -e ’s/http:\/\/dev\.mistcat\.com\/(www\.dwin1\.com\/2112\.js)/http:\/\/\
1/g;’

</code>

<code>

ps -ylC apache2 –sort:rss | awk ‘{x += $8;y += 1} END {print "Apache Memory Usage (MB): "x/1024; pr
int "Average Process Size (MB): "x/(y*1024)}’

</code>

So the command ‘PS’ is going to give us process information for any process containing the apache2 text, I’m going to sort it by the physical/resident memory that process is taking up.  I then feed that data into Awk and on Ubuntu, the 8th column in is the memory info.  I then total that up in my x variable and total the number of processes in my y variable and then print those out nicely out to the terminal.  (Incidentally you could use this in a cron job to create a sort of very ghetto/basic apache historical tracking if you wanted to see apache memory usage over time)

So as I understand it, your MaxClients setting in apache should follow this formula:

MaxClients ≈ (RAM – size_all_other_processes)/(size_apache_process)

So with my handy script you now have the size_apache_process variable, you probably know your boxes total ram, (‘free –m’ will tell you if you don’t know) and I’ve been estimating ‘size_all_other_processes’ as about 20% of total ram.  It occurs to me that I could probably write a script to total up the size of all other processes as well, but I’ll have to investigate that a little bit…  Anyhow it’s pretty quick to figure out a good MaxClients setting using this script!

Find in files cheap and dirty unix style!
find . -type f| xargs grep 'some_string'

Determine the total page size of a webpage including graphics and what not:
wget -r -nv --level=0 --ignore-tags=a --delete-after https://www.domain.com/whatever