Web Security : Automating with LibWWWPerl - Writing a Basic Perl Script to Fetch a Page, Programmatically Changing Parameters

- Free product key for windows 10
- Free Product Key for Microsoft office 365
- Malwarebytes Premium 3.7.1 Serial Keys (LifeTime) 2019

1. Writing a Basic Perl Script to Fetch a Page

1.1. Problem

For basic testing, or as a basis for something larger, you want a Perl script that fetches a page from an application and stores the response in a Perl data structure you can use. This is a basic GET request.

1.2. Solution

This is what LibWWWPerl (LWP) is all about. You need to install the following Perl modules :

  • LWP

  • HTTP::Request

Example 1 shows a basic script that issues a request for a page and checks the return value. If the return code is successful, it prints the response contents to standard output. If the return code indicates failure, just the return code and error message are printed.

Example 1. Basic Perl script to fetch a page
use LWP::UserAgent;
use HTTP::Request::Common qw(GET);

$UA   = LWP::UserAgent->new();
$req  = HTTP::Request->new( GET => "" );
$resp = $UA->request($req);

# check for error. Print page if it's OK
if ( ( $resp->code() >= 200 ) && ( $resp->code() < 400 ) ) {
    print $resp->decoded_content;
} else {
    print "Error: " . $resp->status_line . "\n";

1.3. Discussion

This script is a fundamental building block for all kinds of basic web requests. 

There are many kinds of requests you might make. Example 8-1 shows a GET request. POST is the other common request type. Additional request types are defined in HTTP and are supported by LWP. They include PUT, DELETE, OPTIONS, and PROPFIND, among others. One interesting set of security tests would be to determine your application’s response to some of these less frequently used methods. You may be surprised to find that, instead of a simple “405 Method Not Allowed” response, you receive a response that a hacker can use, like an error 500 with debugging information.

Other Useful LWP Scripts

It’s worth noting here that, in true Perl style, “there’s more than one way to do it,” and in fact Example 8-1 is a bit redundant. There are pre-made scripts that come with the LWP library that do basic jobs like this. When you’re building a test case, you might be more interested in using one of these pre-built scripts unless you need some special behavior. So that you’re aware of them, here’s a brief list. Each has its own man page or online documentation for more detailed information.


Use lwp-download to simply fetch something using a GET request and store it to a file. Similar to curl , it takes the URL from the command line. Unlike curl (or lwp-request), it has no ability to do anything sophisticated like cookies, authentication, or following redirects.


If you want to download a local copy of a file, but only if you don’t have the latest version, lwp-mirror can do that. That’s really its purpose: to be like lwp-download, but to check the server for the modification date of the file and only download it if the file has been modified.


Perl’s answer to curl is lwp-request. It gives you many of the same options and controls that curl does: authentication, cookies, content-type, arbitrary headers, etc. It is not quite as powerful as curl, but it is a good midway between writing your own Perl program and using a very complicated curl invocation.


If you need a primitive spider, the lwp-rget tool can help. It will fetch a page, parse the page, find all the links, and then fetch any of the links that you want it to.

2. Programmatically Changing Parameters

2.1. Problem

You want to programmatically change the inputs on a GET request. This might be to get a range of possible values, or because you need to calculate some part of the value (like today’s date).

2.2. Solution

We assume we have some kind of website that has a search page. Frequently, search pages have a parameter to limit the maximum number of matches returned. In our example, we assume that max can be in the URL. The script in Example 2 changes the max parameter in the URL to a variety of interesting values.

Example 2. Basic Perl script to change parameters
use LWP::UserAgent;
use HTTP::Request::Common qw(GET);
use URI;

use constant MAINPAGE => '';

$UA = LWP::UserAgent->new();
$req = HTTP::Request->new( GET => MAINPAGE );

# This array says test 8-bits, 16-bits, and 32-bits
my @testSizes = ( 8, 16, 32 );

foreach $numBits (@testSizes) {
    my @boundaryValues = (
        ( 2**( $numBits - 1 ) - 1 ),
        ( 2**( $numBits - 1 ) ),
        ( 2**( $numBits - 1 ) + 1 ),
        ( 2**$numBits - 1 ),
        ( 2**$numBits ),
        ( 2**$numBits + 1 ),
    foreach $testValue (@boundaryValues) {
        my $url = URI->new(MAINPAGE);
            'term' => 'Mac',
            'max'  => $testValue

		# do the fetching of pages inside a loop, where we change the
		# parameter we're tinkering with each time.
        $resp = $UA->request($req);

        # Report any errors
        if ( ( $resp->code() < 200 ) || ( $resp->code() >= 400 ) ) {
            print resp->status_line . $req=>as_string();


2.3. Discussion

Example 2 performs boundary case testing around byte values. That is, it considers the powers of 2 that might be significant boundary cases. We know that 28 is 256, so if the application had only 1 byte for storing the max parameter, boundary values like 255, 256, and 257 should sniff that out. Notice how easily this could be extended to 64 bits by simply putting a 64 into the line with 8, 16, and 32.

Top 10
Free Mobile And Desktop Apps For Accessing Restricted Websites
TOYOTA CAMRY 2; 2.5 : Camry now more comely
KIA SORENTO 2.2CRDi : Fuel-sipping slugger
How To Setup, Password Protect & Encrypt Wireless Internet Connection
Emulate And Run iPad Apps On Windows, Mac OS X & Linux With iPadian
Backup & Restore Game Progress From Any Game With SaveGameProgress
Generate A Facebook Timeline Cover Using A Free App
New App for Women ‘Remix’ Offers Fashion Advice & Style Tips
SG50 Ferrari F12berlinetta : Prancing Horse for Lion City's 50th
- Messages forwarded by Outlook rule go nowhere
- Create and Deploy Windows 7 Image
- How do I check to see if my exchange 2003 is an open relay? (not using a open relay tester tool online, but on the console)
- Creating and using an unencrypted cookie in ASP.NET
- Directories
- Poor Performance on Sharepoint 2010 Server
- SBS 2008 ~ The e-mail alias already exists...
- Public to Private IP - DNS Changes
- Send Email from Winform application
- How to create a .mdb file from ms sql server database.......
programming4us programming4us