Blog Data Gathering Code

These two pieces of code gather data about blogs. To use them, you need to have curl, awk, and wc installed and using Bash as your shell. Create a folder called onblogs. Then create a file in that folder called bloglist with the URLs of the blogs you want data on. Change to the directory above onblogs and run the code. The first one will figure out how many comments are on the front page of the blog. The second one will figure out how many posts are on the front page of that blog.

rm onblogs/comments; for blog in `cat onblogs/bloglist`; do curl $blog 2> /dev/null | grep "comments</a>" | cut -b 213- | awk '{s+=$1} END {print "There are " s " comments on the blog " f "."}' f=$blog >> onblogs/comments; done; cat onblogs/comments

rm onblogs/posts; for blog in `cat onblogs/bloglist`; do curl $blog 2> /dev/null | grep '<h3 class="post-title">' | wc -l | awk '{print "There are " $1 " posts on the blog " f "."}' f=$blog >> onblogs/posts; done; cat onblogs/posts