DBD::Oracle is usually a pain to install if you haven’t already done it a gazillion times. After that, it’s just an annoying itch. In this article I’ll cover installing DBD::Oracle using Oracle Instant Client, Ubuntu 9.04, and Perl 5.10. Since we’re using the Instant Client, you’ll need some Oracle DB you can connect to in order to do the testing. You can choose to skip testing altogether, but you might be into a surprise later.

DBI and DBD::Oracle

Installing the DBI is always easy – regardless if you’re running Windows or Linux. Just follow the steps.

1. Get root password if you don’t already have one: sudo passwd root
2. Switch to root: su -
3. Run CPAN: perl -MCPAN -e shell
4. Check if DBI is already installed: m DBI
5. If it’s not installed, install it: install DBI

That should do the trick for the DBI. The DBD::Oracle is a bit more complicated and we’ll just use CPAN to download it for us. The rest is manual.

1. Check for DBD::Oracle: m DBD::Oracle
2. Download it: get DBD::Oracle
3. Exit CPAN: q

Now we should have the DBD::Oracle distribution downloaded to our CPAN build directory. If you don’t know where to find it, look for .cpan dir under your root home or (if you started CPAN for the first time doing sudo from your main user, look for it under your main user’s home directory). We’ll leave that distro aside for a moment and work on the other pre-requisites.

Oracle Instant Client

Download the instant client packages you’ll need. I chose to download the RPMs and convert them to .deb files using alien. Oracle also provides .zip files if you don’t want to do it the alien way.

Oracle Instant Client Download Site i386
Oracle Instant Client Download Site AMD64 32- and 64-bit

Accept the license agreement by clicking the Accept radio button. Since the i386 and amd64 files have different names, check the bold words in the file names to know which ones to download (the names below are for the i386 platform). Also, if you don’t have a user_id for Oracle, you’ll be prompted to register one once you click the link. It’s free of charge.

• oracle-instantclient11.1-basic-xx.x.x.x.x-x.i386.rpm
• oracle-instantclient11.1-sqlplus-xx.x.x.x.x-x.i386.rpm
• oracle-instantclient11.1-devel-xx.x.x.x.x-x.i386.rpm

AMD64 has only zip files from what I could find, and they’re named a bit differently, too (remove the 32 for the 64-bit version:

• instantclient-basic-linuxAMD64-32-xx.x.x.x.x-yyyymmdd.zip
• instantclient-sqlplus-linuxAMD64-32-xx.x.x.x.x-yyyymmdd.zip
• instantclient-sdk-linuxAMD64-32-xx.x.x.x.x-yyyymmdd.zip

Note: Extract the zips into a directory called instantclient and skip to the Set Up your Environment Variables section if you’re using the AMD64 or i386 zip files.

Install alien and libaio

Now is the time to install alien, which is an application that converts rpm files into .deb format to be used with dpkg. Instant Client also requires libaio. Both can be installed through the Synaptic Package Manager. Just open it, look for alien, mark it for install and do the same for libaio and libaio-dev. Once they’re installed, you’re good to move on to install the Instant Client. Just don’t forget to exit Synaptic Package Manager, since we’ll be using dpkg and it won’t work if Synaptic is open.

Install Oracle Instant Client

First step to install Oracle Instant Client from rpm files is to convert them into .deb files. Do that with alien by running the following command (from a command line in the directory where you downloaded your RPMs):

$ sudo alien --scripts *.rpm

It takes a little while to run, so be patient. Once it’s complete, install the newly created .deb files:

$ dpkg -i *.deb

Set up your Environment Variables

The nasty thing about rpm files is that it’s not always easy to know where the files were installed. If you opted for the zip file approach, your install will most definitely be different. For RPM install, add this to your .bashrc file (swap xx.x for your oracle version):


export ORACLE_HOME=/usr/lib/oracle/xx.x/client
export PATH=$PATH:$ORACLE_HOME/bin
export LD_LIBRARY_PATH=$ORACLE_HOME/lib


Update: If you get an ELFCLASS64 error, try setting LD_LIBRARY_PATH to $ORACLE_HOME/lib32 instead.

Reload your .bashrc file:
$. ~/.bashrc

Note: by default, Oracle Instant Client doesn’t come with a tnsnames.ora file or the directory structure where it’s usually found. We’ll have to create that ourselves -

$ mkdir -p $ORACLE_HOME/network/admin; touch $ORACLE_HOME/network/admin/tnsnames.ora

Install DBD::Oracle

It’s time to finally install DBD::Oracle. Go to your CPAN build directory and cd into DBD-Oracle-*. As a user having the environment variable set from the previous section, run Makefile.PL portion.

$ perl Makefile.PL

There’s no need to set INC or LIB with the alien approach, but if you run into any issues, try giving Makefile.PL the path to your include dir for INC, and lib dir for LIB.

Next, run

$ make.

It will raise a few warnings, but unless it exits with an error, you should be OK.

The next logical step is to run make test. However, this will undoubtedly fail unless you have a valid entry in your tnsnames.ora file. If you don’t, you can skip this test and hope that everything works later on. Otherwise, update your tnsnames.ora file with a valid entry, and set one more environment variable before running the test:

$ export ORACLE_USERID="user/passwd@tns_entry_name"
$ make test

This is usually the hardest part to pass with total success. Many things can go wrong. In my case, the valid entry I use doesn’t have full grants to the user I log in as. This always triggers an error on the create/access sequences portion of the testing. I simply ignore it nowadays – make sure you analyze your test results thoroughly before choosing to ignore the errors as well.

Now that the testing is over, simply run
$ make install
as a super user and you’re all set!

Book suggestions:

Update2: Removed the prototype from the subroutine.

I’ve always had a problem with recursion. Not with the general theory that a function will call itself, etc – no, that’s easy. The hard part was when I had to deal with complex data structures in Perl (an array- or hashref containing a hash of arrays of hashes, a gazillion levels deep). Well, I guess anyone would have a hard time with that kind of data.

Anyway, in this post I don’t intend to get all complicated explaining all the kinds of recursions out there. If you want that, check this article at wikipedia. What I do want to do is help all of those who are in the situation I was in, by explaining in the simplest way possible how to deal with this scenario.

Let’s start with a need. I have a complex data structure that needs its spaces trimmed on both sides. But since I’m lazy, I’d like my subroutine to modify the data directly, and not return the modified value (pass by reference, not pass by value).

Here’s our data structure:

    my $data = [
                       {
                          key1 => '   trim me!   ',
                          key2 => '   trim me too!    ',
                       },
                       [
                          'some element to trim   ',
                          '    another one    ',
                       ],
                       '    a simple string needing trimming    ',
                  ];

$data explained: an array containing 3 elements: element 0 is a hashref of keys "key1" and "key2", element 1 is an arrayref of 2 elements. Element 3 is a simple string. All values have some extra spaces that need trimming (or so they say). We could use whatever number of levels and data types we want (except for anonymous subroutines, I guess – let’s not get too complicated).

Now, to trim all that, I want to be able to simply call trim() à la PHP.

     trim($data); # note the lack of the lvalue (lvalue = rvalue)

I also want it to accept simple arrays and hashes, and the references thereof: trim(@array); trim(@array); trim(%hash); trim(%hash); trim($string). After all, I never know what kind of data my colleagues will be working with. Better have it deal with everything.

The logic to do that is this: our subroutine will have to do the trimming (s///g) on scalars only. For that, it has to check if the data it received is a hash, array, etc, and if it is, iterate through each element and trim the value… but only if the element is not itself a hash, array, etc. Found it confusing? No problem, it really is.

In Perl, if I tried to remove the white space from element 0 of my $data variable, it wouldn’t work. The reason being is that if I printed $data->[0] onto the screen, I’d get a funny looking output, something like HASH(0x1004f5f0). That’s Perl’s way of saying that you have a HASH structure stored in memory position 0x1004f5f0. You can try to trim the spaces off of that string, but it won’t do you any good. The elements of your hash will still be untouched. That’s why you need to de-reference your data structures and dive into them.

To de-reference a structure is simple, just add a % in front of the variable if it’s a hashref, or an @ if it’s an array. But how do you know which is which? Use ref().

       print ref($data->[0]) . "n"; # HASH
       print ref($data->[1]) . "n"; # ARRAY
       print ref($data->[3]) . "n"; # empty string, which is false

ref() tells you what kind of data you are dealing with. It returns CODE if you have a closure or anonymous subroutine, but we’re not going there today.

So, now that we know how to identify the type of element we’re going to be working with, we can build our subroutine…

sub trim() {
	for my $param (@_) {
		if (ref($param) eq 'ARRAY') {
			for my $element (@{$param}) {
				trim($element);
			}
		}
		elsif (ref($param) eq 'HASH') {
			for my $val (values %{$param}) {
				trim($val);
			}
		}
		elsif (ref($param) eq 'CODE') {
			return;
		}
		else {
			$param =~ s/(^s+|s+$)//g;
		}
	}
}

trim() explained:

We’re working with passing elements by reference instead of by value. This means that the elements themselves will be modified – no need to return any data. The first thing we do is to iterate through all parameters passed to trim(). In a subroutine, parameters (in our case, variables) are populated into the special @_ array, allowing us to call trim($var1, $var2, $var3) if we want.

We iterate through all elements of @_ and verify if they are an Array. If they are, we iterate through each of their elements once, and call trim() again against them. That will handle as many nested arrays we want (or that your computer can handle). Now we have to make it deal with hashes. Same technique – use ref() to see if it’s a hash. If it is, then iterate through each of its key/pair elements. There are several ways to do that. I personally prefer calling keys to get the keys and use them to fetch the values of the hash. The value of the hash is passed to trim() for more validation. We also check to see if we received a sub { } (anonymous subroutine). In that case, we do nothing, just return.

Finally, after handling Arrays, Hashes and Anonymous subroutines, we can set up the actual trimming of the strings. We take the $_[$i] which is the parameter passed and remove the leading and trailing spaces with one neat substitution: ^s+ stands for leading spaces, s+$ stands for trailing spaces, and it’s all joined by the ( | ) (this or that). We only call it once because we’re using the global (g) modifier of the substitution s///g.

And that’s all there is to it!

A note on prototypes: This post generated a healthy discussion on prototypes. I had originally added the dollar prototype to the trim() subroutine, but that was forcing it to accept only scalars (strings and references), and not working with normal hashes and arrays. Thanks to everyone who participated in the discussion.

This post is intended for those who, like me, aren’t MQSeries gurus and can’t make much of the documentation of the MQSeries module in CPAN. I hope it serves you well.

Very VERY Brief introduction to MQSeries

MQ Series (recently renamed Websphere MQ) is a messaging software developed by IBM. There is a client and a server piece to it.

MQ Messages are transmitted between servers by means of queues. These queues are managed by Queue Managers. Queues can be configured to receive messages (input or request) and/or return messages (output or reply).

The installation of MQ series is beyond the scope of this post, but it’s important to know that if you need to communicate with a queue manager installed in a local machine, you have to use one set of APIs (the server APIs), and if you want to talk to a remote queue manager, you’ll use the client APIs. This is important for later on.

A MQ message is composed of 2 main parts: the MQ Message Descriptor (MQMD) and the Data portion. The MQMD portion works as a header where several fields can be set to tell the server how to handle the message. The fields we will be using in this tutorial are MsgId, CorrelId, MsgType, Expiry, ReplyToQ, and ReplyToQMgr.

MsgId: Each message being sent into the server gets a unique Message ID. They never repeat, even when the server is restarted.

CorrelId: The Correlation Id is how a reply message can be associated to a request message.

MsgType: The kind of data you will be sending through the queues. We will be using a simple string format.

Expiry: The amount of time that the server should keep the message in a queue.

ReplyToQMgr: The name of the Queue Manager where the reply must be sent to.

ReplyToQ: The name of the queue where the reply must be sent to.

You will find more information about the fields and MQ Series in general at the MQSeries Application Programming Reference. I couldn’t find a link to the hardcopy at Amazon, but you can consider getting this one.

Before trying to put or get messages, you will probably need to set the MQSERVER environment variable (for both Unix and Windows – don’t know how Mac works), since the client usually requires it. The value for MQSERVER is “CHANNEL_NAME/TCP/HOST_IP(PORT)”. There can be variances according to the settings of the host you’ll connect to, but this is the most common scenario.

MQSeries CPAN modules

CPAN has a very handy set of modules which allow us to communicate with MQSeries. The main module, MQSeries, exports all the funny named subroutines that you need to use to send and receive messages. If you don’t have working knowledge with C/C++ using MQ Series, then don’t even bother trying to use it. The Object Oriented modules (by the same author) are much simpler for the unseasoned MQ Series explorer (like myself).

First off, you will need to install the modules in your computer. If you’re working on windows, read this article to get it done. It’s simpler on Unix, but both cases require that you have the MQ Series client installed. You can get it from here

When you install MQSeries module from CPAN, you also get MQSeries::QueueManager, MQSeries::Queue, MQSeries::Message and some other goodies. If you didn’t get any of those automatically, make sure you install them before you go on.

Writing your own MQSeries module

Here we’ll write a few methods which should make people’s lives easier when communicating with MQ Series. Now, depending on the way your application works, you will probably have to handle the creating and breaking up of the Data portion of the message, but again, that’s outside the scope of this post.

So let’s get started…

Start your module by giving it a package name. Remember to end the code with a true value. We’ll also import the modules we’ll need

package myMQModule;

use strict;
use MQSeries;
use MQSeries::QueueManager;
use MQSeries::Queue;
use MQSeries::Message;

1; # end with a true value

Now let’s lay out the basics of our module.

package myMQModule;

use strict;
use MQSeries;
use MQSeries::QueueManager;
use MQSeries::Queue;
use MQSeries::Message;

sub new() { # our constructor
}

sub openQueueMgr() { # open the Queue Manager
}

sub openQueue() { # opens the Queue
}

sub putRequest() { # puts the message
}

sub getReply() { # gets the reply using the correlId
} 

1; # end with a true value

You’ll probably notice that we haven’t added methods to close the queue and the queue manager. That’s ok, since the MQSeries* modules do that for us.

Let’s go on with our methods. Starting with new(). We’ll use the basic instantiation code. You will probably need to enhance this to suit your own needs, such as calling openQueueMgr() and openQueue() automatically. For now, we’ll KISS.

package myMQModule;

use strict;
use MQSeries;
use MQSeries::QueueManager;
use MQSeries::Queue;
use MQSeries::Message;
use Carp; # exports carp, confess, etc.

sub new() { # our constructor
    my $invocant = shift;
    my $class = ref($invocant) || $invocant; # Object or class name
    my $self = {};       # initiate our handy hashref
    bless($self,$class); # make it usable
    return $self;
}

sub openQueueMgr() { # open the Queue Manager
}

sub openQueue() { # opens the Queue
}

sub putRequest() { # puts the message
}

sub getReply() { # gets the reply using the correlId
} 

1; # end with a true value

Now we need to get the Queue Manager and the Queue open. In real life, you’ll probably have several queue managers and several queues. In our example, we’ll use 2 of each, one pair for outgoing traffic and another for incoming. Remember that this depends on your MQ Series configuration. These examples should simply allow you to get your toes wet in preparation before diving into your custom settings.

...
sub openQueueMgr() { # open the Queue Manager
    my $self = shift; # take the handy hashref
    my $qm_name    = shift; # the name of the queue manager
    my $qm_ip         = shift; # the IP to the queue manager
    my $qm_port      = shift; # the port it's listening to
    my $qm_channel = shift; # the channel configured in the QM
    my $type           = shift; # request or reply? we will need it later

    # some validation code here making sure that the QM name, ip,
    # port, and channel have been provided 

    # export the MQSERVER environment variable just in case your clients need it
    $ENV{MQSERVER} = "$qm_channel/TCP/$qm_ip($qm_port)";

    # Here we create a MQSeries::QueueManager object to connect to our QM
    # It takes, among other parameters, the QM name, AutoConnect, and
    # a hashref with the data we gave to $ENV{MQSERVER}
    my $connOpts = {
          ChannelName => $qm_channel,
          TransportType => 'TCP',
          ConnectionName => "$qm_ip($qm_port)",
          MaxMsgLength   => 16 * 1024 * 1024,
    };

    # Create MQSeries::QueueManager object
    my $qm = MQSeries::QueueManager->new(
        QueueManager => $qm_name,
        AutoConnect => 0, # we do not autoconnect now
                          # because we want to handle it better later
        ClientConn => $connOpts,
    );

    #  kick it off and see if it connects
    eval {
        $qm->Connect()
            || die(sprintf("Connect failed with CompCode: %s",
                               Reason %sn",$qm->CompCode(),$qm->Reason()));
    };
    if ($@) { # eval caught the die
       confess($@);
   }

   # got this far, so it obviously connected
   # now save that connection in a safe place
   $self->{uc($type)}->{QM_CONN} = $qm; # MQSeries::QueueManager object.
                                        # We need it to open the queue
   $self->{uc($type)}->{QM_NAME} = $qm_name; # note that we also saved the QM_NAME and it is all under the $type (REQUEST/REPLY)

}
 ...

The snippet above showed you how to connect to the queue manager. It’s pretty well commented, so we can go on with our next step, which is opening the queue.

...
sub openQueue() { # opens the Queue
    my $self       = shift;
    my $queue_name = shift;
    my $type       = shift; # again, we will need it for later.
                           # (REQUEST/REPLY)

    # set mode according to type
    my $mode = uc($type) eq 'REQUEST' ? 'output' : 'input';

    # open the queue directly this time
    # here we use the MQSeries::Queue module
    eval{
        $self->{uc($type)}->{Q_OBJ} =
           MQSeries::Queue->new(
               QueueManager  => $self->{uc($type)}->{QM_CONN}, # the QueueManager object from before
               Queue         => $queue_name,
               Mode         => $mode,
               AutoOpen         => 1, # open it directly
           )|| die("Could not open Queue $queue_name");
    };
    if ($@) {
        confess($@);
    }

    # made it this far, so we obviously got a connection
    # so let's save the queue name for later
    $self->{uc($type)}->{QUENAME} = $queue_name;
}
...

Now we have both our QueueManager and our Queue open. Since we’re dealing with one request pair and one reply pair, you’ll have to call it once for the request and once for the reply. You can also choose to save the connections in a different data structure to handle several queue managers and even more queues per queue manager. The only limit is your creativity. Right now, we’ll stick to the $self->{REQUEST} and $self->{REPLY} structures.

Continuing our layout, we now have to handle putting data into the queue.

...
sub putRequest() { # puts the message
    my $self = shift;
    my $message = shift; # the data we will be putting

    # We need a MQSeries::Message object to inject
    my $msg =
        MQSeries::Message->new(
             MsgDesc => { # this is the MQMD (header) portion
				  Format	=> MQFMT_STRING,
				  Expiry	=> 100, # in tenths of seconds
				  ReplyToQ => $self->{REPLY}->{QUENAME},
				  ReplyToQMgr => $self->{REPLY}->{QM_NAME},
       				},
            Data => $message,
        );

    # now that we have a message object to inject, do the injecting
    my $request = $self->{REQUEST}->{Q_OBJ};
    eval {
       $request->Put(Message=>$msg) || die(sprintf("Put failed with CompCode: %s, Reason: %sn",
                                                            $request->CompCode(),$request->Reason()));
    };
    if($@){
        confess($@);
    }

    # got this far, so Put was successful. We'll return the MsgId to be used
    # as CorrelId when getting the reply
    return $msg->MsgDesc('MsgId'); # comes from the injected MQSeries::Message object
}
...

If you try printing the MsgId, you’ll get some funny characters in the screen, since it’s pure binary data. Look up “perl bin to hex” on Google if you’d like to see some ways of converting and not getting gibberish.

Next comes the last but not least portion of our module – to get the reply message.

...
sub getReply() { # gets the reply using the correlId
    my $self      = shift;
    my $correlId = shift;

    # some sanity checking, since we can't do anything without $correlId
    confess("I need a correlId!!!n") unless $correlId;

    # create the Message object with the $correlId
    my $reply_msg =
            MQSeries::Message->new(
                 MsgDesc =>
				{
				    CorrelId	=> $correlId,
				},
            );

    my $reply_q = $self->{REPLY}->{Q_OBJ};
    eval {
       $reply_q->Get(
                     Message=> $reply_msg,
                     Wait => 100, # some interval before timing out
                ) || die(sprintf("Get message failed with CompCode: %s, Reason: %sn",
                                      $reply_q->CompCode(),$reply_q->Reason()));
    };
    if ($@){
        confess($@);
    }

    # got this far, so the Get was successful.
    # now we return the Data portion of the reply

    return $reply_msg->Data(); # from the Message object, not the Queue object!
}
...

And that’s all there is to the module! Now you can call it from your script, like so

#!/usr/bin/perl

use strict;
use lib 'directory containing myMQModule';
use myMQModule;

# create object
my $mq = myMQModule->new();

# open queueMgr1
my($qm_name,$qm_ip,$qm_port,$qm_channel,$type) = qw(QMREQ some_ip some_port CHANNEL1 request);
$mq->openQueueMgr($qm_name,$qm_ip,$qm_port,$qm_channel,$type);

#open queueMgr2
($qm_name,$qm_ip,$qm_port,$qm_channel,$type) = qw(QMREP some_ip2 some_port2 CHANNEL2 reply);
$mq->openQueueMgr($qm_name,$qm_ip,$qm_port,$qm_channel,$type);

# now for the queues
$mq->openQueue('QUEUE1', 'request');
$mq->openQueue('QUEUE2', 'reply');

# put some message
my $msgId = $mq->putRequest('this is a test message');
my $reply_msg = $mq->getReply($msgId);

print "We're done!";

I really hope this was helpful. Comments are always welcome.

Book suggestions:

Being the extensible and flexible language that it is, Perl provides us with some safeguards and helpers to assist in avoiding what happened to me (I wish I knew that back then). The first of which I’ll talk about is Pragmas.

Pragmas

Pragmas are special modules that come installed by default in every Perl distribution. They tell the interpreter of how it is supposed to act. To turn them on, all you have to do is call the special word use with the appropriate Pragma. To turn them off, call no and the Pragma in question. The most common and powerful Pragma is, in my opinion, strict (hence the name of this blog: use strict;#)).

Strict tells the Perl interpreter that all variables must be declared and tightens up security a notch. To use strict; you have to have at least working knowledge of lexical variables. It takes a while to getting used to at first, but once you’re hooked, you won’t know how you could possibly have written Perl programs without it before (I know I don’t).

#!/usr/bin/perl

$var = 1; # OK
use strict;
$var1 = 2; # compile time error

example.pl

In our example above, Perl will refuse to run, raising a compile time error like such:

Global symbol “$var1″ requires explicit package name at example.pl line 5.
Execution of example.pl aborted due to compilation errors.

That’s strict in effect. Notice that $var was not cited, since strict was only enforced below it. In order to bypass that error, we should have declared our variable with either my, our, or local – depending on the need. my is the most common. Look up “Packages, Namespaces, and Lexical scopes” for more on those 3 operators.

#!/usr/bin/perl

$var = 1; # OK
use strict;
my $var1 = 2; # OK

use strict; is so important that it is usually the second line of code in any decent Perl program – the first line being the shebang.

If you need to turn off strict for one reason or another, you can do so with the key word no.

#!/usr/bin/perl

use strict;
my $var = 1; # OK
no strict;
$var1 = 2; # also OK

When writing your Perl programs, it’s also good to turn on warnings and diagnostics. warnings will complain about possible problems such as useless uses of certain functions. diagnostics, on the other hand, will throw you a truckload of information regarding errors. It’s a good place to start when you’re stumped.

Perl Modules

Perl modules are pieces of code or packages that can be imported into your script with the keyword use in the same way as Pragmas. They can be Object Oriented, Procedural, or both. I will not discuss how to write a module in this post, but I will tell you where to find them for download and how to install them.

My all time favorite module is Data::Dumper. So I will use it in the following examples. The funny :: between Data and Dumper is kind of like a directory separator. The module Dumper resides in the directory Data, found in one of the paths configured in the Perl config files or the PERL5LIB environmental variable (which set the @INC array).

Data::Dumper comes installed by default, along with hundreds other modules that the developers deemed worthy. To test that your Perl distro has it, run the following command in a command line:

$ perl -MData::Dumper -e 'print "OKn"'

You’ll most definitely see the OK being printed on your screen. The command passes 2 parameters to the Perl interpreter: -M which tells it to load a module (in this case Data::Dumper – no spaces between -M and the module name, or you’ll get a “missing argument” error), and -e which tells it to execute a piece of code (we told it to print OK, but any valid piece of code would do).

Look at what would have happened if we tried to load a module that wasn’t installed:

$ perl -Maaaa -e 'print "OKn"'
Can’t locate aaaa.pm in @INC (@INC contains: /usr/lib/perl5/5.10/i686-cygwin /usr/lib/perl5/5.10 /usr/lib perl5/site_perl/5.10/i686-cygwin /usr/lib/perl5/site_perl/5.10 /usr/lib/perl5/vendor_perl/5.10/i686-cygwin /usr/lib/perl5/vendor_perl/5.10 /usr/lib/perl5/vendor_perl/5.10 /usr/lib/perl5/site_perl/5.8 /usr/lib/perl5/vendor_perl/5.8 .).
BEGIN failed–compilation aborted.

To install a module, you can do it the hard way or the easy way. Some modules are harder than others, so I’ll stick with the easier ones for now. Let’s start with the hard way.

First, you should know where to find your module. CPAN – The Comprehensive Perl Archive Network is the place to go to get your modules. It has a handy search engine which will search through almost 16000 modules at the time of this writing (5/2009). There you will find code ranging from the most trivial to the craziest needs. If there’s one thing that CPANs contributors don’t lack, it’s creativity (that’s a compliment).

Enter a keyword in the search engine and it will list all relative modules. Click on the link to the one that interests you most. You will be shown its documentation in POD (Plain Old Documentation) format. That’s the data you will most likely need to learn how to use your module. It should also be the most up-to-date information, since it’s kept by its authors.

At the top of the POD screen, you’ll see a breadcrumb set of links showing the Author’s name, the module’s distribution name and version, and the module name itself:

Ilya Martynov > Data-Dumper-2.121 > Data::Dumper

Click on the link for the module’s distribution name and version, to go to the distribution details screen. You will see lots of information about the module and its sub-modules, but what should concern you right now is the download link next to the release name. Click on it and download the module tarball.

Now this is where I halt and tell you that potential headaches lie ahead. There are basically 2 kinds of modules: PurePerl ones and C-based ones. PurePerl modules are just that – modules that are solely written in Perl. The vast majority of modules, however, are written in C with bindings special for Perl. Those usually have to be compiled and as such, need a C/C++ compiler and a make program. The good news is that most *nix systems come with those tools already installed or readily available. Windows systems, however, require that you install nmake and preferably Microsoft Visual C++. Check out my post about installing MQSeries module on Windows for more information on how to get Microsoft Visual C++

Back to installing the module…

Unpack your module in some directory where you have full access. I like to keep an untouched copy of the tarball, by reading the gunzipped contents and throwing it to the screen with the -c option, and piping it to tar with xvf - parameters:

gunzip -c tarball.tar.gz | tar -xvf -

You’ll end up with a directory having the name of your distribution. Go in there and read all README files you can find. Read the INSTALL files if any.

One of the files you’ll see in the directory is Makefile.PL. That’s the kickoff file for the installation. It takes the following optional parameters: PREFIX, LIB, and INC, and creates a makefile tailor made for your system. It’s important to set the PREFIX parameter if you want the module installed in a place other than the default. LIB and INC point to the C lib and include directories, respectively.

Now that you have your makefile, it’s just a matter of running make, make test, and finally make install. Depending on the module, you should have no issues whatsoever. Other more “sensitive” modules, however, often require hours of work and even some tweaking of the makefiles by hand. DBD::Oracle is by far the craziest module I’ve ever had to install. In some machines it’s a piece of cake, and others it manages to amuse with the amount of errors it pulls out of the hat. Anyway…

That was the hard way. Now for the easy way.

CPAN

If you’re finding it strange to see the CPAN title here when I’ve already talked about it above, don’t worry – I’m talking about the application CPAN and not the website.

Perl comes with the CPAN module installed, and most of the time it also creates a script in the bin directory called cpan. It’s an interactive shell that allows you to fetch information regarding authors and modules, and also allows you to install modules without having to go through the whole process of downloading the distribution, unpacking, etc.

Start up the CPAN application by calling

$ perl -MCPAN -e shell

If it’s the very first time you call it, you will be promped to answer a series of questions. Sticking to the default values is almost always OK. One of the first questions you will be asked is if you want CPAN to configure everything automatically. I recommend against it unless you know that the defaults are correct and will enable you to successfully install modules. And if you know that, then you probably already know how to use the CPAN application and this post has nothing new.

Once the questions have been answered, you will be presented with a cpan prompt. Type ? to know what options you have.

cpan[1]> ?
Display Information                                                (ver 1.9205)
 command  argument          description
 a,b,d,m  WORD or /REGEXP/  about authors, bundles, distributions, modules
 i        WORD or /REGEXP/  about any of the above
 ls       AUTHOR or GLOB    about files in the author's directory
    (with WORD being a module, bundle or author name or a distribution
    name of the form AUTHOR/DISTRIBUTION)

Download, Test, Make, Install...
 get      download                     clean    make clean
 make     make (implies get)           look     open subshell in dist directory
 test     make test (implies make)     readme   display these README files
 install  make install (implies test)  perldoc  display POD documentation

Upgrade
 r        WORDs or /REGEXP/ or NONE    report updates for some/matching/all modu
les
 upgrade  WORDs or /REGEXP/ or NONE    upgrade some/matching/all modules

Pragmas
 force  CMD    try hard to do command  fforce CMD    try harder
 notest CMD    skip testing

Other
 h,?           display this menu       ! perl-code   eval a perl command
 o conf [opt]  set and query options   q             quit the cpan shell
 reload cpan   load CPAN.pm again      reload index  load newer indices
 autobundle    Snapshot                recent        latest CPAN uploads

Use o conf to display the parameters to which you answered all those questions. You can change them with o conf as well. If you didn’t enable auto-commit before, you will have to call o conf commit to save your changes for use in future sessions.

o conf http_proxy "http://some_proxy.com:80"

The m command allows you to fetch information regarding a certain module.

cpan[2]> m Data::Dumper
.... Possibly some data about fetching updated files from the internet here ....
Module id = Data::Dumper
    DESCRIPTION  Convert data structure into perl code
    CPAN_USERID  GSAR (Gurusamy Sarathy )
    CPAN_VERSION 2.121
    CPAN_FILE    I/IL/ILYAM/Data-Dumper-2.121.tar.gz
    DSLIP_STATUS SdpOp (standard,developer,perl,object-oriented,Standard-Perl)
    MANPAGE      Data::Dumper - stringified perl data structures, suitable for b
oth printing and C
    INST_FILE    /usr/lib/perl5/5.10/i686-cygwin/Data/Dumper.pm
    INST_VERSION 2.121_14

In the output above, CPAN tells us that we alreay have Data::Dumper version 2.121_14 installed. If you are not sure what is the exact name of a module, use the i command to fetch information using a regex:

cpan[3]> i /klingon/
Distribution    JALDHAR/DateTime-Event-Klingon-1.0.1.tar.gz
Distribution    PNE/Lingua-Klingon-Collate-1.03.tar.gz
Distribution    PNE/Lingua-Klingon-Recode-1.02.tar.gz
Distribution    PNE/Lingua-Klingon-Segment-1.03.tar.gz
Module    DateTime::Event::Klingon (JALDHAR/DateTime-Event-Klingon-1.0.1.tar.gz)

Module    Lingua::Klingon::Collate (PNE/Lingua-Klingon-Collate-1.03.tar.gz)
Module    Lingua::Klingon::Recode (PNE/Lingua-Klingon-Recode-1.02.tar.gz)
Module    Lingua::Klingon::Segment (PNE/Lingua-Klingon-Segment-1.03.tar.gz)
8 items found

Once you’re happy with the module name, you can check if it’s installed or not using m

cpan[4]> m Lingua::Klingon::Collate
Module id = Lingua::Klingon::Collate
    CPAN_USERID  PNE (Philip Newton
)
    CPAN_VERSION 1.03
    CPAN_FILE    P/PN/PNE/Lingua-Klingon-Collate-1.03.tar.gz
    INST_FILE    (not installed)

Install it with install command.

cpan[5] install Lingua::Klingon::Collate

Now, unless you already have module Test::Differences installed, Lingua::Klingon::Collate will fail with a dependency error. Not all modules are like that. Some are coded in a way that CPAN actually asks you if you want to follow and install dependencies automagically. Those are a cinch to install.

If something goes wrong, look at the output of the installation. Best case scenario, you’re just missing another module and it didn’t warn you about it. For example, after Lingua::Klingon::Collate failed with the warning that I should have Test::Differences installed, I tried to install that dependency directly. It also failed. When looking at the output on the screen, I see a bunch of lines like this:

t/regression..........Can't locate Text/Diff.pm in @INC (@INC contains: /home/vi
alves/.cpan/build/Test-Differences-0.4801-lI_xia/blib/lib /home/vialves/.cpan/bu
ild/Test-Differences-0.4801-lI_xia/blib/arch /usr/lib/perl5/5.10/i686-cygwin /us
r/lib/perl5/5.10 /usr/lib/perl5/site_perl/5.10/i686-cygwin /usr/lib/perl5/site_p
erl/5.10 /usr/lib/perl5/vendor_perl/5.10/i686-cygwin /usr/lib/perl5/vendor_perl/
5.10 /usr/lib/perl5/vendor_perl/5.10 /usr/lib/perl5/site_perl/5.8 /usr/lib/perl5
/vendor_perl/5.8 .) at /home/vialves/.cpan/build/Test-Differences-0.4801-lI_xia/
blib/lib/Test/Differences.pm line 213.

You’ll notice that the error is the same as when we did the $ perl -Maaaa -e 'print "OK"'. Module Text/Diff.pm (or namely Text::Diff) is not installed. So we now go on that quest of following dependencies by hand.

It so happens that Test::Differences tried to install Text::Diff, but Text::Diff failed during the test phase. Suppose I know that those failed tests are not important and won’t hinder the results of the rest (I don’t, but bare with me), I can force CPAN to disregard the test failures:

cpan[6]> force install Text::Diff
Running install for module 'Text::Diff'
Running make for R/RB/RBS/Text-Diff-0.35.tar.gz
  Has already been unwrapped into directory /home/vialves/.cpan/build/Text-Diff-
0.35-B8Qsuo
  Has already been made
Running make test
/usr/bin/perl.exe "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'b
lib/arch')" t/*.t
t/ext_format......ok
t/general.........Use of /g modifier is meaningless in split at t/general.t line
 129.
Use of /g modifier is meaningless in split at t/general.t line 130.
t/general.........ok
t/inputs..........ok
t/keygen..........ok
t/outputs.........1/8 No such file or directory at t/outputs.t line 12, 
line 6.
t/outputs......... Dubious, test returned 2 (wstat 512, 0x200)
 Failed 4/8 subtests
t/table...........ok

Test Summary Report
-------------------
t/outputs.t   (Wstat: 512 Tests: 4 Failed: 0)
  Non-zero exit status: 2
  Parse errors: Bad plan.  You planned 8 tests but ran 4.
Files=6, Tests=29,  1 wallclock secs ( 0.04 usr  0.04 sys +  0.59 cusr  0.26 csy
s =  0.93 CPU)
Result: FAIL
Failed 1/6 test programs. 0/29 subtests failed.
make: *** [test_dynamic] Error 255
  RBS/Text-Diff-0.35.tar.gz
  /usr/bin/make test -- NOT OK
//hint// to see the cpan-testers results for installing this module, try:
  reports RBS/Text-Diff-0.35.tar.gz
Running make install
Installing /usr/lib/perl5/site_perl/5.10/Text/Diff.pm
Installing /usr/lib/perl5/site_perl/5.10/Text/Diff/Table.pm
Writing /usr/lib/perl5/site_perl/5.10/i686-cygwin/auto/Text/Diff/.packlist
Appending installation info to /usr/lib/perl5/5.10/i686-cygwin/perllocal.pod
  RBS/Text-Diff-0.35.tar.gz
  /usr/bin/make install  -- OK
Failed during this command:
 RBS/Text-Diff-0.35.tar.gz                    : make_test FAILED but failure ign
ored because 'force' in effect

Now I can go on to installing Text::Differences and finally Lingua::Klingon::Collate.

cpan[7] install Test::Differences
.... some output here ....
Appending installation info to /usr/lib/perl5/5.10/i686-cygwin/perllocal.pod
  OVID/Test-Differences-0.4801.tar.gz
  /usr/bin/make install  -- OK

cpan[8] install Lingua::Klingon::Collate
Running install for module 'Lingua::Klingon::Collate'
Running Build for P/PN/PNE/Lingua-Klingon-Collate-1.03.tar.gz
  Has already been unwrapped into directory /home/vialves/.cpan/build/Lingua-Kli
ngon-Collate-1.03-MX7FZn
  -- No Build created, won't make
Running Build test
  Make had some problems, won't test
Running Build install
  Make had some problems, won't install

Ok, that happens. It’s because of the previous bad attempt. We can bypass that by telling CPAN to restart the build from scratch. To do so, it must first clean the build environment for that module.


cpan[9] clean Lingua::Klingon::Collate


That usually does the trick and enables you to run the install again, but if it doesn’t, you can force get Lingua::Klingon::Collate to get a fresh package.

cpan[27]> install Lingua::Klingon::Collate
Running install for module 'Lingua::Klingon::Collate'
Running Build for P/PN/PNE/Lingua-Klingon-Collate-1.03.tar.gz
  Has already been unwrapped into directory /home/vialves/.cpan/build/Lingua-Kli
ngon-Collate-1.03-ZA4a1s

  CPAN.pm: Going to build P/PN/PNE/Lingua-Klingon-Collate-1.03.tar.gz

Checking whether your kit is complete...
Looks good

Checking prerequisites...
Looks good

Creating new 'Build' script for 'Lingua-Klingon-Collate' version '1.03'
Copying lib/Lingua/Klingon/Collate.pm -> blib/lib/Lingua/Klingon/Collate.pm
Manifying blib/lib/Lingua/Klingon/Collate.pm -> blib/libdoc/Lingua.Klingon.Colla
te.3pm
HTMLifying blib/lib/Lingua/Klingon/Collate.pm -> blib/libhtml/site/lib/Lingua/Kl
ingon/Collate.html
./Build: blib/lib/Lingua/Klingon/Collate.pm: cannot resolve L in par
agraph 60.
./Build: blib/lib/Lingua/Klingon/Collate.pm: cannot resolve L in par
agraph 60.
  PNE/Lingua-Klingon-Collate-1.03.tar.gz
  ./Build -- OK
Running Build test
t/01_base...........ok
t/02_strcoll........ok
t/03_strxfrm........ok
t/04_strunxfrm......ok
t/05_list...........ok
All tests successful.
Files=5, Tests=87,  1 wallclock secs ( 0.05 usr  0.03 sys +  0.52 cusr  0.25 csy
s =  0.85 CPU)
Result: PASS
  PNE/Lingua-Klingon-Collate-1.03.tar.gz
  ./Build test -- OK
Running Build install
Prepending /home/vialves/.cpan/build/Lingua-Klingon-Collate-1.03-ZA4a1s/blib/arc
h /home/vialves/.cpan/build/Lingua-Klingon-Collate-1.03-ZA4a1s/blib/lib to PERL5
LIB for 'install'
Installing /usr/lib/perl5/site_perl/5.10/Lingua/Klingon/Collate.pm
Installing /usr/share/man/man3/Lingua.Klingon.Collate.3pm
Installing /usr/share/doc/perl-5.10.0/html/html3/site/lib/Lingua/Klingon/Collate
.html
Writing /usr/lib/perl5/site_perl/5.10/i686-cygwin/auto/Lingua/Klingon/Collate/.p
acklist
  PNE/Lingua-Klingon-Collate-1.03.tar.gz
  ./Build install  -- OK

Type q to exit the shell and that’s it! There are many other options when using CPAN, but what you’ve seen so far in this post should be enough to get you started. Just remember to use ? every now and then to see what powers are offered to you.

Note from UseStrict: Some of the examples in this tutorial were borrowed from Randall Schwartz’s Learning Perl. It’s a book that EVERY beginner Perl programmer should have. If you don’t have a hardcopy, please consider getting one. You can find it here: Learning Perl, 5th Edition

In this article, you will learn how to use basic I/O in Perl, learn about @ARGV, and become familiar with string formatting using printf.



STDIN

<STDIN> tells Perl to read from the standard input – usually the keyboard.

while (defined($_ = <STDIN>)) {
	print "I saw $_";  # echoes whatever is typed onto the screen.
                                # Quit with ^D or ^Z (depending on your system)
}

foreach (<STDIN>) {
        print "I saw $_"; # almost the same as above
}

The difference between the while and foreach loops above is that while executes its statements at every hit of the return key, while foreach slurps into memory all the input until eof (^D on Unixes) and only then executes its instructions.

It is important to note this difference if you don’t want to crash your machine. If your input comes from a webserver with a 400MB log file, you’re better off processing each line individually than slurping it all into memory.




Diamond Operator

Another option to read data input is through the diamond operator (aka angle operator). It is handy to make Perl programs function like Unix commands such as cat, sed, awk, sort, grep, lpr, etc.

From the command line:
program_name.pl <file1> <file2> <someglob*>

In program_name.pl

while (<>)  {
	chomp;
 	print "It was $_ that I saw!n";
}

If you fail to give the program any parameters or pass it a dash (-), it will read from STDIN. The diamond operator will open each file and paste its contents into the special $_ variable. Just like any filehandle (more on filehandles here), you can slurp the input into an array in list context:

while ($line = <>)  { # populate $line instead of 	$_
	chomp($line);
 	print "It was $line that I saw!n";
}
# or
@lines = <>;

for (@lines) {
     ... # do something with lines
}

If the diamond operator fails to open a passed file for reading, it will raise a warning and move on to the next file. It’s interesting to say that the diamond operator is quite intelligent: depending on how you call your script, it will not think that the parameters are files. For example:

cat some_file.txt | ./script.pl

The above example will work just as if you had sent it some_file.txt as a parameter.



@ARGV

@ARGV stores all the command line parameters passed to your script during run time. It is the array that the diamond operator looks at when doing its thing. You can manually manipulate it inside the script, as you would any other array, and the results will be as if you had passed your changes through the command line.



STDOUT

STDOUT is the default output channel for Perl’s print and printf functions.

print
print takes a list as parameters. The function returns 1 if successful and 0 otherwise. As we saw in our previous examples, the parentheses in the print function is not required. However, it may sometimes help you to get what you need.

Question1: what is the output for the snippet below?

       $result = print(2+3) * 4;
       print $result;

Check the bottom of this article for the answer.

If you want to print an array, as we saw in our Arrays and Lists post, you can get different results depending on which quotes (if any) you use.

print @colors;  # prints out 'redyellowblue'
print "@colors";  # prints out 'red yellow blue'
print '@colors';  # prints out '@colors' - no interpolation with single quotes, remember?

printf

printf allows you to have a better control of your output. It takes a format and the data to be formatted. Since printf is basically imported from C, and so many languages have adopted it, it accepts a wide range of popular patterns – even some deprecated ones:

%%	a percent sign
%c	a character with the given number
%s	a string
%d	a signed integer, in decimal
%u	an unsigned integer, in decimal
%o	an unsigned integer, in octal
%x	an unsigned integer, in hexadecimal
%e	a floating-point number, in scientific notation
%f	a floating-point number, in fixed decimal notation
%g	a floating-point number, in %e or %f notation

%X	like %x, but using upper-case letters
%E	like %e, but using an upper-case "E"
%G	like %g, but with an upper-case "E" (if applicable)
%b	an unsigned integer, in binary
%B	like %b, but using an upper-case "B" with the # flag
%p	a pointer (outputs the Perl value's address in hexadecimal)
%n	special: *stores* the number of characters output so far
        into the next variable in the parameter list

%i	a synonym for %d
%D	a synonym for %ld
%U	a synonym for %lu
%O	a synonym for %lo
%F	a synonym for %f

Each format can also take one or more of the following flags:

space	prefix positive number with a space
+	prefix positive number with a plus sign
-	left-justify within the field
0	use zeros, not spaces, to right-justify
#	ensure the leading "0" for any octal,
	prefix non-zero hexadecimal with "0x" or "0X",
	prefix non-zero binary with "0b" or "0B"

Some examples:

printf "%g %g %gn", 5/2, 51/17, 51 ** 17;	# 2.5 3 1.0683e+29
printf "in %d days!n", 17.85;				# in 17 days!
printf "%10sn", "wilma";					# '     wilma'
printf "%-15sn", "flintstone";				# 'flintstone     '
printf "%12fn", 6 * 7 + 2/3;					# '   42.666667'
printf "%12.3fn", 6 * 7 + 2/3;				# '      42.667'
printf "%12.0fn", 6 * 7 + 2/3;				#  '          43'
printf “Monthly interest: %.2f%%n", 5.25/12;	# Monthly interest:

Answer to Question1: if you guessed 54, then congrats! The print(2+3) sends 5 to the screen and returns 1, which gets multiplied by 4 and assigned into $result. The following print $result sends 4 to the screen. Since there are no newlines anywhere in the data being printed, both digits get placed in the same line on the screen.

« Control Structures | TOC | Subroutines »

Opening Files

To read or write files in Perl, you need to open a filehandle. Filehandles in Perl are yet another kind of identifier.
They act as convenient references (handles, if you will) between your program and the operating system about a particular file. They contain information about how the file was opened and how far along you are in reading (or writing) the file. They also contain user-definable attributes about how the file is to be read or written.

To open a new file on system you need to create the filehandle for this file using the command open

open(filehandle, pathname);

The filehandle is the identifier that will describe the file and the pathname – the full path of the file you trying to open. Typically it is represented by a constant, but when working with complex programs, it is best to use a scalar variable in order to safely pass it from one subroutine or method to another.

Example:

	open(PASS_FILE, "/etc/passwd"); # equivalent to open(PASS_FILE,"< /etc/passwd");
	# or
	open(my $pass_file, "/etc/passwd");

Note: it’s always a good idea to test if open() worked well:

	open(PASS_FILE,"/etc/passwd") || die("Can't open passwd: $!n"); # $! gives the system error message

Any modern version of Perl also accepts the 3 parameter notion, which is safer:

	open(PASS_FILE, "<" , "/etc/passwd");

When you done all with the file you need to close then using the command close

        close(PASS_FILE);

Reading Files

You can read from Perl’s filehandles in a couple of different ways. The most common method is to use the file input operator, also called the angle (or diamond) operator (<>). To read a filehandle, simply put the filehandle name inside the angle operator and assign the value to a variable:

	open(MYFILE, "myfile");
	$line = <MYFILE>;

The angle operator in a scalar context reads one line of input from the file. When called after the entire file has been read, the angle operator returns the value undef.

In array context, the whole file is read and stored in the array – one line per element;

Writing Files

To write data into a file you need to open the filehandle to write. The open command before is for reading only. To open for writing we need to use ‘>’ mode to create a new file or overwrite an existing one and ‘>>’ mode to append an existing file or create a new one.

	open(NEWFILE, ">newfile"); # overwrites or creates newfile
	open(MYFILE, ">>myfile"); # appends data to myfile

After open a file to write you can use the print command with the filehandle:

	print NEWFILE "this goes into newfilen"; # note the lack of commas between print and filehandle

More on Modes

So far we’ve seen modes ‘<’, ‘>’, and ‘>>’. There are other modes we can use depending on our needs. Adding a ‘+’ before ‘>’ or ‘<’ (making ‘+>’ or ‘+<’) will grant us read AND write access to the file. ‘+>’, however, will truncate your file first.

Perl also allows you to use pipes on your filenames, print to anonymous temporary files, and much much more.
See perldoc’s open for more details.



Special filehandles

The Perl have some special filehandles that was always open this is for standard input, output and error
They are

• STDOUT – The standard output
• STDIN – The standart input
• STDERR – The error standard output

You can specify which is your default output handle (the one print sends data to when no filehandle is passed) with
the select() function:

	open(FH,"> myfile.txt);
	$old_fh = select(FH); # changes default output and saves original
	print "This goes to myfile.txtn";
	close(FH);
	select($old_fh);
	print "This goes to the screenn";

File test operators

Before you open a file, sometimes it’s nice to know whether the file exists, whether the file is really a directory, or whether opening the file will give a permission denied error. If you could examine the file’s metadata, you could get answers to these questions. For these situations, Perl provides the file test operators. The file test operators all have the following syntax

-X filehandle

OR

-X pathname

The valid operators for file tests are:

• -r Returns true if the file is readable
• -w Returns true if the file is writeable
• -e Returns true if the file exists
• -z Returns true if the file exists but is empty
• -s Returns size of the file in bytes if it exists
• -f Returns true if the file is a regular file rather than a directory
• -d Returns true if the file is a directory
• -T Returns true if the file appears to be a text file
• -B Returns true if the file appears to be a binary file
• -M Returns the age (in days) since the file was modified

Working with directories

The first step in obtaining directory information from your system is to create a directory handle. A directory handle is something like a filehandle, except that instead of a file’s contents, you read the contents of a directory through the directory handle. To open a directory handle, you use the opendir() function:

        opendir(dirhandle, pathname);

To get the content of the directory you need to use the readdir() function to get the next directory entry or the entire list of files, depending on context (scalar or list, respectively). It returns undef once you reach the end of your list.

	$next_file = readdir(dirhandle);
       # OR
	@files = readdir(dirhandle);

After you finish you need to close the directory using the function closedir()

	closedir(dirhandle);

A shortcut to this process is to use file globbing techniques:

	@all_files = <*>; # gets you the listing of all files in the current directory
	@shell_scripts = glob("*.sh"); # safer

Basic directory operations

If you need to change, create and remove directories you can use the functions chdir(), mkdir and rmdir using this syntax:

	chdir pathname; # to change remote dir
	mkdir pathname; # to create a directory
	rmdir pathname; # to remove the entire directory

Basic file operations

To remove files you need to use the unlink() function and to rename, use rename()

	unlink list_of_files;
	rename oldname, newname;

The stat Function

If you need to get all information about a file you need to use the stat() function.

It returns an array containing the following:

0 dev     Device number
1 ino     Inode number
2 mode    File's mode (permissions)
3 nlink   Number of links
4 uid     User ID (UID) of the owner
5 gid     Group ID (GID) of the owner
6 rdev    Special file info
7 size    Size of file in bytes
8 atime   Time of last access
9 mtime   Time of last modification
10 ctime  Inode change time
11 blksz  Disk block size
12 blocks Number of blocks in file

        ($dev,  $ino,   $mode,  $nlink, $uid,     $gid,   $rdev,
         $size, $atime, $mtime, $ctime, $blksize, $blocks) = stat(pathname);

« Subroutines | TOC | Some built-in functions for everyday use »

Control Structures are used to control the flow of a program. We are going to see programs that can iterate (loop) or make decisions (conditionals) based on the state of variables.

More interesting possibilities arise when we introduce control structures and looping. Perl supports lots of different kinds of control structures which tend to be like those in C, but are very similar to Pascal, too.




Conditionals

The simplest control structures are the if and unless statements. Its syntax basically follows the rule below:

if ( expression ) {
    instruction1;
    instruction2;
}

Value obtained from “expression”, determines whether it will be considered as true or false. Table below helps you to identify the meaning of values based on how Perl interprets expressions.

Everything in Perl is true, except:

• the strings “” (the empty string) and “0″ (the string containing only the character, 0), or any string expression that evaluates to either “” (the empty string) or “0″.
• any numeric expression that evaluates to a numeric 0.
• any value that is not defined (i.e., equivalent to undef).

A very simple example of a real “if statement” would be:

if ( $name ) {
    print "Non empty string";
}
else {
    print "Empty string";
}

Another example, to illustrate the syntax of the “if” statement:

if ( $money <= 0 ) {
    print "You are out of money";
}
elsif ( $money > 1000000 ) {
    print "You are a millionaire";
}
else {
    print "You are probably better than I am";
}

Blocks of code in Perl are enclosed by curly braces {   }. They are always required inside a control structure (except for cases where we use idiomatic ifs – see more below).

Two things to mention:

1 – elsif spelling;
2 – both elsif and else statements are optional.

Another conditional statement we need to mention is the unless. Its meaning is the opposite of an if statement. It will only execute a block of code if expression is NOT true.

Let’s look at an example:

if  ( $num == 10 ) { #Usual if statement
    print "Number is equal to 10";
} else {
    print "Number not equal to 10";
}

# Unless statement that will execute the block of code below
# if number's value is not 10 (same thing as the else statement above)

unless ( $num == 10 ) {
    print "Number not equal to 10";
}
else {
    print "Number is equal to 10";
}

Action based on statement test

You can test a single statement and perform a pre-defined action in case it is true or another one if statement is false.

For example:

    print "You are a man" if ($gender eq "M"); # parentheses are optional here, since only one expression is evaluated

Ternary or Trinary Operator

Like many other programming languages, Perl offers the ternary (aka trinary) operator “?:” which behaves just like if/elsif/else:

print "You are a " . (($gender eq "M") ? "man" : "woman") . "n";
# read: if $gender eq "M" then "man" else "woman"

# or perhaps
($birthday == $today) ? print "Happy birthday!" : print "Have a good day";

Mimmic elsif this way:

$month = (localtime)[4]; # fifth element of localtime in list context returns month, base 0;

print "The month is now " .
($month == 0) ? "January" :
($month == 1) ? "February" :
($month == 3) ? "March" :
                      "something other than Jan, Feb, or March";

Operators

Initially we will introduce the testing operators. They rely on a test being considered as true or false. Following you can see a few examples:

$a == $b                      # Is $a numerically equal to $b?
# Beware: Don't use the = operator.
$a != $b                       # Is $a numerically unequal to $b?
$a eq $b                       # Is $a string-equal to $b?
$a ne $b                       # Is $a string-unequal to $b?

Perl also supports the logical operators “OR” and “AND”. The “OR” operator is represented by the “||” characters while the “AND” is represented by “&&” characters.

Lets go through a few examples:

# If value of number is greater than 10 or lesser than 5
if ( ( $num > 10 ) || ( $num < 5 ) ) {
    ...
}

# If language is equal to "perl" and num is 10, or, if num is 0 (or has a "false" value)
if ( (( $language eq "perl" ) && ( $num == 10 )) || !( $num ) ) {
    ...
}

Loops

We have several statements to perform iterations using Perl. Perl supports: while, until, do ... while, for and foreach statements.

While

Performs the iteration while the given expression is true. Syntax:

while (expression) {
    instruction1;
    instruction2;
    ...
}

Until

It is the opposite of while. So take it as, in example:

while (!(expression)) {
    instruction1;
    instruction2;
}

#same as
until (expression) {
   instruction1;
   instruction2;
}

Do .. While

Its functionality is almost the same as for the normal “while” statement, but with this kind of control structure you guarantee that the block of code will be executed at least once.

do {
    instruction1;
    instruction2;
} while (expression);

For

It’s a C like style. Syntax:

for (initialize; expression; increment/decrement) {
    instruction1;
    instruction2;
    ...
}

• The initialize statement is basically used to initialize the variable that will be used on the test statement. It can also be used anywhere in the block. Declare it with my to keep it private to the for block;
• Expression follows the same rule for if or while. It will be tested, and if its return is true, block code will be executed;
• Increment/decrement is then executed

Real example:

for ($i = 0; $i < 10; $i++ ) {
    printf "Iteration number: %sn", $i;
}

Another good example (not using a numeric expression):

for ( @names = ('John', 'Paul', 'Roger', 'David'); @names; shift(@names) ) {
    print "$names[0]n";
}

# Or work on 2 expressions at the same time
for ($i = 0, $j = 10; $i<=10,$j>=0; $i++, $j--) {
    print "$i    $jn";
}

# Or kick off an infinite loop
for ( ; ; ) {
    instruction1;
    instruction2;
}

Foreach

It is another control structure that allows iteration, but here you can iterate over an array.

Following you can see a few examples of it:

@names = ('John', 'Paul', 'Roger', 'David');

#Passing through all names
foreach $name (@names) {
    printf "%sn", $name;
}

The scalar variable is optional to the foreach statement. So you can bypass it by doing:

@names = ('John', 'Paul', 'Roger', 'David');

#Passing through all names
foreach (@names) {
    printf "%sn", $_;
}

If you don’t specify a scalar variable to get each iteration, then the default scalar ($_) will be used. Note that for can also be used as a synonym to foreach – same exact functionality, but without the each portion of the word.

Loop flow control

We have a few special commands to help us controlling the flow under a loop statement. Here we go.

last

It is similar to the “break” statement used on C programming language (as well as in several others). Tells Perl to skip the current loop statement, or even, to skip a labeled loop statement. Take a look at the example below:

@names = ('fernando', 'smith', 'Mark', 'John');
foreach (@names) {
    last if $_ eq "John";
    print "Name is " . $_ . "n";
}

next

next has the same function as the “continue” loop statement used in C. Used to force your loop statement to go to the next iteration. In example:

@names = ('fernando', 'smith', 'Mark', 'John');
foreach (@names) {
    next if $_ eq "smith";
    print "Name is " . $_ . "n";
}

The example above will just force the “foreach” statement to read next iteration once “smith” is the value read from the “names” array.

If you need to work on multiple levels of loops, you can tell next and last which named block to work on:

LINE: while () {
	next LINE if /^#/;	# discard comments
	...
}

« Basic Regular Expressions | TOC | Basic I/O »

« Gettin’ jiggy wit it | TOC | Control Structures »

Subroutines are user-created functions that execute a block of code at any given place in your program. It is a best practice, however, to aggregate them all either at the beginning or the end the main program.

Subroutine declarations initiate with the key word “sub” . Conventionally, subroutine names are all lowercase characters

sub NAME (PROTOTYPE) BLOCK

print_hello; # subroutine can be executed/called before the actual block is created

sub print_hello {
      print "Hello worldn";
}

When we called print_hello we told Perl that we wanted the piece of code named print_hello to be executed. The result is a “Hello World” showing up on our screen. The only benefit we have from that snippet in its current form is that we won’t have to copy/paste the print statement all over our script if we want to repeat it. All we need to do is call print_hello;

Another, older, but still common form of calling it would be

&print_hello;

Parameters

As mentioned before, our previous example doesn’t do much by itself. The good news is that subroutines can take a list of parameters and do something based on what it receives. Let’s change our example for something a little better:

$me = "Vinny";

print_hello($me);

sub print_hello {
	$name = shift;
	print "Hello $namen";
}

# prints out "Hello Vinny"

Let’s take a closer look at that. During my subroutine call, I’m passing it a list of 1 element “Vinny”. That element is populated into the special array @_, which is then passed into the subroutine. Like $_, you don’t have to explicitly point out @_ when using certain array functions. In our example, we shifted the first element of @_ into our variable $name and printed it out.

It would have been possible to work directly against $_[0], yes, but doing so might be dangerous depending on what you’re doing. Here’s why: there are 2 ways of passing parameters into a subroutine.

The first and more common is pass-by-value. Pass-by-value is when we use the value of $_[0] into a variable of our own declared within the subroutine. Due to the lexical nature of subroutines ($name doesn’t exist outside of the subroutine in our example – we’ll see more about that later), the original variable ($me) is untouched.

The second and possibly dangerous way of doing it is by using the pass-by-reference method. In this case, we work against $_[0] directly, which is a reference to the variable passed. Any changes to $_[0] will affect the contents of $me. This is typically NOT what you want to do.

$me = "Vinny";

print_hello($me);
print "$men";

sub print_hello {
    print "Hello $_[0]n";
    $_[0] .= " Alves";
}

When passing more complex structures such as hashes and arrays, bear in mind that Perl flattens them out into a single array by default.

@myArray = qw(red green blue);

%myHash = ('apple'=>'fruit',  'dog' =>'pet');

print_hello(@myArray,%myHash);

sub print_hello {
	for $i (@_) {
		print "Got $in";
	}
}
# Prints :
# red
# green
# blue
# apple
# fruit
# dog
# pet

Again, that’s probably not what you wanted. To maintain the individuality and structure of your elements, you will need to pass a reference to each element as parameters, and then assign them to variables in your subroutine.

print_hello(@myArray,%myHash);

sub print_hello {

	$array_ref = shift;
	$hash_ref = shift;

	for $fruit (@$array_ref) {
		print "Got $fruitn";
	}

	print "A dog is a " . $hash_ref->{dog} . "n";
}

Returning Data

Subroutines are handy for returning some sort of data. By default, it returns 0 or 1 if the keyword return isn’t found – depending on the success or failure of the subroutine. Optionally, you can have it return a specific piece of data, such as a scalar, a list/array or reference to arrays, hashes, scalars, etc. The same rules we find in passing parameters apply to returning data.

#!/usr/bin/perl

$num1 = 10;
$num2 = 5;
$result = compare($num1,$num2);

print $result; # "is greater"

sub compare {
    return "is greater" if $_[0] > $_[1];
    return "is smaller" if $_[0] < $_[1];
    return "is equal"  if $_[0] == $_[1];
}

Here is another way to see the same results:

#!/usr/bin/perl

use strict;

$num1 = 10;
$num2 = 5;
$result = compare($num1,$num2);

print $result; # "is greater"

sub compare(@) {
	if ($_[0] > $_[1]) {
		"is greater"; # "return" operator not required
	}
	elsif($_[0] == $_[1]) {
		"is equal";
	}
	else {
		"is less than";
	}
}

As you can see, we didn't use the return operator in the example above. It is not required here because Perl will automatically return the value of the last statement evaluated. It's not a best practice to do it this way though. We strongly recommend that you keep your returns where they're supposed to be.

Prototypes

In order to bypass the flattening out of all the array and hash parameters into a single array, Perl allows us to specify prototypes for our subroutines. This tells our program if we're expecting an Array, or a Hash, or to treat the elements passed as individual references instead of flattening them out. It also lets us define which parameters are mandatory and which are optional.

There are ways of overriding (turning off) prototypes. By calling the subroutine in the "old" fashion (&my_proto; or &my_proto()) for instance, the checking will not be done. Calling subroutines as methods in Object Oriented Perl also turns it off.

The catch with prototypes is that you have to declare the subroutine BEFORE actually calling it. Look at the following example:

#!/usr/bin/perl -w

@array = qw(a b c );

my_proto(@array,'test');

sub my_proto(@) {
	print "@_n";
}
# Warns the following message:  main::my_proto() called too early to check prototype at ./test.pl line 5.
# and prints "a b c test"

In this example, the subroutine and its prototype are declared after the call. Perl warns that the call was too early to check prototype and treats the subroutine as if it didn't have any. The correct way of doing it would be this:

#!/usr/bin/perl -w

sub my_proto(@); # pre-declare the sub

@array = qw(a b c );
my_proto(@array,'test');

sub my_proto(@) {
	print "@_n";
}
# Compilation error:
# Too many arguments for main::my_proto at ./test.pl line 8, near "'test')"
# Execution of ./test.pl aborted due to compilation errors.

This time the script didn't even compile (remember that Perl is an interpreted AND compiled language). Prototype checking was in place and raised the error which aborted compilation.

You probably realized that the error message we got is the same as the ones we get when we fail to provide built-in functions the right parameters. That's exactly what prototyping does - it allows you to make your subroutines behave like built-in functions.

The perlsub section in perldoc shows the kinds of prototypes that we can use to mimick the built-in functions:

		Declared as		Called as

		sub mylink ($$)		mylink $old, $new
		sub myvec ($$$)		myvec $var, $offset, 1
		sub myindex ($$;$)	myindex &getstring, "substr"
		sub mysyswrite ($$$;$)	mysyswrite $buf, 0, length($buf) - $off, $off
		sub myreverse (@)	myreverse $a, $b, $c
		sub myjoin ($@)		myjoin ":", $a, $b, $c
		sub mypop (@)		mypop @array
		sub mysplice (@$$@)	mysplice @array, @array, 0, @pushme
		sub mykeys (%)		mykeys %{$hashref}
		sub myopen (*;$)	myopen HANDLE, $name
		sub mypipe (**)		mypipe READHANDLE, WRITEHANDLE
		sub mygrep (&@)		mygrep { /foo/ } $a, $b, $c
		sub myrand (;$)		myrand 42
		sub mytime ()		mytime

Notes:

• ';' is used to separate mandatory (to the left of ; ) from optional fields (to the right).
• @, $, %, etc. makes the sub require that exact element in the position in which it was declared.
• sub mysub([@$%]) allows you to call mysub with a $var, or @array, or %hash, etc.
• sub mysub($) when called with mysub(@array) will force the array to be handled in SCALAR context

Another catch of declaring subs with prototypes is that you will get a warning if you declare your sub with a given prototype first, and then again with the rest of the code and a different prototype. Let's look at a couple of examples to get a better idea of what I mean.

Example 1

#!/usr/bin/perl -w

sub mysub(@); # defined in array context

@array = qw(a b c);

mysub(@array);

sub mysub($) {  # defined in scalar context
	print "@_n";
}
# Prototype mismatch: sub main::mysub (@) vs ($) at ./test.pl line 15.
# prints out "a b c"

Example 2

#!/usr/bin/perl -w

sub mysub($) {  # defined in scalar context
	print "@_n";
}

@array = qw(a b c);

mysub(@array);

# works just fine
# prints out "3" due to the scalar context in which the sub is defined.

Lexical Scopes

Perl by default is a very loose language. It lets you get away with things that other languages would never dream of - like using global variables indiscriminately. It's all fine and good if you want to write a quick and dirty script that isn't more than a few lines long, but if you're trying to build a complex application such as a bulletin board system (Matt Wright's wwwboard was written in (horrible) perl), you will probably find yourself missing a few toes after all the shots you foot took.

A solution for this problem is to ALWAYS use strict. Strict is a pragmatic module (or just pragma) that enforces several security measures such as having to pre-declare your variables, and another large list of constraints. It should always be the second line of your program (being the shebang the first line).

One of the things that use strict; enforces the most is to have you make good usage of lexical scopes. Lexical scopes mean that you can't access a variable that was not pre-declared, or somehow imported into the current block of code.

#!/usr/bin/perl -w

$name = 'vinny';

myprint();

sub myprint() {
	print "$namen";
}
# main::myprint() called too early to check prototype at ./test.pl line 7.
# vinny

The example above accesses the value of the global variable $name from within the subroutine. That is not a very safe thing to do. If we turn on strict, we get an error when trying to do that. The error, however, is because we didn't declare $name with either my, local, or our.

#!/usr/bin/perl -w

use strict;

$name = 'vinny';

myprint();

sub myprint() {
	print "$namen";
}

# main::myprint() called too early to check prototype at ./test.pl line 7.
# Global symbol "$name" requires explicit package name at ./test.pl line 5.
# Global symbol "$name" requires explicit package name at ./test.pl line 11.
# Execution of ./test.pl aborted due to compilation errors.

Changing the script above to comply with strict, we are once again able to access the global $name, but we still don't want to do that, especially if we want to modify the value of $name only inside the subroutine.

#!/usr/bin/perl -w

use strict;

my $name = 'vinny'; # added my to comply with strict

myprint();

print "$namen";

sub myprint() {
	$name .= " Alves"; # changes the original variable instead of a private one
	print "$namen";
}

# main::myprint() called too early to check prototype at ./test.pl line 7.
# vinny Alves
# vinny Alves

So it's best if we play it safe and not work on global variables at all. This is especially the case if you one day plan on writing scripts to use with Apache/mod_perl.

#!/usr/bin/perl -w

use strict;

my $name = 'vinny'; # added my to comply with strict

myprint($name); # calls the sub

print "$namen"; # $name here still has the original value

sub myprint() {
	my $name = shift;
	$name .= " Alves";
	print "$namen";
}
# main::myprint() called too early to check prototype at ./test.pl line 7.
# vinny Alves
# vinny

Context and wantarray

Something that often catches Perl beginners off guard is the concept of context. Calling an @array in scalar context will return the number of elements, and not the elements themselves. We can mimick this behavior with wantarray. This is how it works:

#!/usr/bin/perl

use strict;

my @array1 = qw(a b c d e);
my @array2 = reverse_or_count(@array1);
my $count = reverse_or_count(@array1);

print "@array2n";
print "$count";

sub reverse_or_count(@){
	my @arr = @_;
	if (wantarray) {
		return reverse @arr;
	}
	else {
		return $#arr + 1; # last index of @arr plus 1
	}
}

Thanks to the wantarray operator returns true if the subroutine is being called in an array context, and false if not. With that kind of control, the sky is the limit .

Closures

Closures are usually thought of as very advanced topics, but they're not that bad at all. They're basically a way to work with lexical variables inside referenced subroutines. OK, I admit that that sounded terrible, but with a simple example we can clarify it.

#!/usr/bin/perl

use strict;

sub make_counter() {
	my $start = shift;
	return sub { $start++ }
}

my $from_ten = make_counter(10);
my $from_three = make_counter(3);

print $from_ten->();    # 10
print $from_ten->();    # 11
print $from_three->();  # 3
print $from_ten->();    # 12
print $from_three->();  # 4

This is how it works: Our sub make_counter takes the initial parameter into its own private $start variable and then returns an anonymous subroutine - the auto-incrementation of $start. So when we call make_counter(10) we are actually creating a reference to the auto-increment with the initial value of 10. The beauty of it is that the value isn't lost when it comes out of scope. It's saved in memory for the next time it's called. That's why calling $from_ten->() will increment on top the latest result.

What is even nicer is that we can create as many instances of that subroutine reference as we want. Calling $from_three->() will not impact the results of $from_ten->().

That is pretty much all there is to it. Use your creativity to do the rest.

« Basic I/O | TOC | File and directory tests and manipulation »