The web get feature “wget” enables a user to grab the entire content of the website. There may be situations where your may have difficulty in getting the source code from your software vendor, or may not have access details to certain location in which case “Wget” comes in handy. This feature is available for win32 platform as well. Work around for this is for the user to install cygwin to run the “wget” command

$ cd /tmp
$ mkdir sitename.com
$ cd sitename.com
$ wget -r -H -k -Dsitename.com, www.sitename.com sitenamey.com

Switches used …

-r switch for recursively handling files
-H switch to “span hosts” as in some sites there may be links to other other domains
-D switch to indicate from which domains we will gather the files. It is ideal to use this with -H switch
-k switch to indicate that the links from the site refer to local copies and not to the original internet location

To get a mirror copy

$ wget -r http://somesite.com

Use it with –convert-links to make offline copies with local links

$ wget –convert-links -r http://somesite.com

To save the files with .html extension

$ wget –html-extension -r http://somesite.com

Now you will have all the files downloaded to sitenamey.com.

Where to get WGET?

http://wget.addictivecode.org/FrequentlyAskedQuestions?action=show&redirect=Faq#download

Some sites may block the user agent used by Wget in order to avoid heavy trafficking of their bandwidth. You could inturn use the user agent bot from yahoo or google or msn to grab a local copy.

“wget” can be used with various settings to grab site content which includes options like setting http-keep-alive (persistent connections), cookies, post-data, https settings and ftp settings. Check the manual for more detailed help on this topic.

Another contender to “wget” is a tool called “HTTrack” which is free as well. It has a version for Windows users as well. Check http://www.httrack.com/ to download copy of this tool.

First, download openSSL source
$ cd /tmp
$ wget http://www.openssl.org/source/openssl-0.9.8k.tar.gz

To get MD5 checksum file
$ wget http://www.openssl.org/source/openssl-0.9.8k.tar.gz.md5 (MD5 checksum)

Execute the following commands to verify if the download matches the MD5 checksum
$ cat openssl-0.9.8k.tar.md5
e555c6d58d276aec7fdc53363e338ab3
$ md5sum openssl-0.9.8k.tar.gz
e555c6d58d276aec7fdc53363e338ab3  openssl-0.9.8k.tar.gz

Another way to check this is to edit “openssl-0.9.8k.tar.md5″ and have the entries as follows
e555c6d58d276aec7fdc53363e338ab3  openssl-0.9.8k.tar.gz
(Separation between the checksum and program name is 2 spaces)

Now execute,
$ md5sum -c openssl-0.9.8k.tar.md5
openssl-0.9.8k.tar.md5

To get SHA1 checksum file
$ wget http://www.openssl.org/source/openssl-0.9.8k.tar.gz.sha1 (SHA15 checksum)

Execute the following commands to verify if the download matches the SHA1 checksum
$ cat openssl-0.9.8k.tar.sha1
$ sha1sum openssl-0.9.8k.tar.gz

To get PGP checksum file
$ wget http://www.openssl.org/source/openssl-0.9.8k.tar.gz.asc (PGP)

Then go to openssl.org website and check for the link from where you could download the PGP key. Create a file called openssl.pgp from that key and then execute

$ gpg –import openssl.pgp
gpg: /root/.gnupg/trustdb.gpg: trustdb created
gpg: key F295C759: public key “Dr Stephen Henson <shenson@drh-consultancy.co.uk>” imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)
gpg: no ultimately trusted keys found

$ gpg –verify openssl-0.9.8k.tar.gz.asc
gpg: Signature made Wed 25 Mar 2009 09:13:54 AM EDT using RSA key ID F295C759
gpg: Good signature from “Dr Stephen Henson <shenson@drh-consultancy.co.uk>”
gpg:                 aka “Dr S N Henson <shenson@drh-consultancy.demon.co.uk>”
gpg:                 aka “Dr Stephen Henson <stephen.henson@opennetworksecurity.com>”
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: D0 5D 8C 61 6E 27 E6 60  41 EC B1 B8 D5 7E E5 97

Check for the message indicating that the signature is good. For self-signed certificates there may be warning messages.

It is always advisable to download the checksum and the source from two different sources in order to avoid a hacker in the middle. For programs which require authenticity, it is always ideal to go with pgp signatures as it shows who owns the signature adding to the awareness of the person who installs the software.