WGET - SPIDEDR

wget --spider --debug -e robots=off -r -p http://jcode.me 2>&1 \
        | egrep -A 1 '(^HEAD|^Referer:|^Remote file does not)' > ~/wget.log
Magnificent Mongoose