Warning: Undefined variable $html in /usr/share/nginx/html/lib/plugins/tabinclude/helper.php on line 240

HTTP Headers

Curl Basics
Website Cloaking

Flag	Description
-I, –head	Fetch the HTTP-header only
-v, –verbose	Fetching is more talkative. Used for debugging
-s, –silent	Silent or Quiet (Mute). Don't show progress meter or error messages
-i, –include	Header plus content
-X , –request	Specify a customer request when communicating with HTTP web server
-L,	Follow the redirect

Troubleshooting a website. You can replace http://website.com with http://localhost if you wish to test the domain on the local machine

curl -LIX GET 12.34.56.78 -H 'Host: example.com'

The following command is used to test a https domain. -k will skip the certificate check

curl -LIkX GET https://website.com

Testing a vhost on the localhost

curl -LIvX GET localhost -H 'Host: vhost.com'

Body String Match

curl -sLX GET www.lukeslinux.co.uk | grep -i lukeslinux

curl -isX GET lukeslinux.co.uk | grep Maintenance

Output:

<title>Days Out With Kids | Maintenance</title>
                                    <h3>Sorry, Site Under Maintenance</h3>

Example without the s flag

curl -LiX GET https://tickets.daysoutwithkids.co.uk/longleat | grep Maintenance

Output:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  9885  100  9885    0     0   280k      0 --:--:-- --:--:-- --:--:-- 9653k
        <title>Days Out With Kids | Maintenance</title>
                                    <h3>Sorry, Site Under Maintenance</h3>

Curling localhost and ignoring ssl If the website has an ssl certificate you can also apply the 'k' option to bypass the certificate check. e.g:

curl -ikL localhost -H 'https://website.com' | grep body_match_string

Time

time for i in {1..10}; do curl -IsX GET http://example.co.uk | grep 'HTTP/1.1'; done

Using Lynx
Note: This is not as accurate as curl Testing to see if a website returns a certain string. Body match strings are often used in monitoring alerts

lynx -dump http://lukeslinuxlessons.co.uk | grep 'string match'

Time to First Byte

Make sure you replace x.x.x.x with the server IP and Host: Example.com with the website

curl -sko /dev/null -w "\ncontent_type=%{content_type}\nhttp_code=%{http_code}\nsize_download=%{size_download}\nsize_header=%{size_header}\nsize_request=%{size_request}\nspeed_download=%{speed_download}\ntime_connect=%{time_connect}\ntime_namelookup=%{time_namelookup}\ntime_pretransfer=%{time_pretransfer}\ntime_redirect=%{time_redirect}\ntime_starttransfer=%{time_starttransfer}\ntime_total=%{time_total}\nurl_effective=%{url_effective}\n\n" -H "Host: example.com" http://x.x.x.x

HTTP AUTHENTICATION

curl --user luke:password http://lukeslinux.co.uk/secret

HTTP Headers

Time to First Byte

Website Cloaking