use strict ;#) » Perl http://usestrict.net Vinny's Technical Corner Wed, 14 Jul 2010 20:21:03 +0000 en hourly 1 http://wordpress.org/?v=abc Introduction to Ajax http://usestrict.net/2009/08/24/introduction-to-ajax/ http://usestrict.net/2009/08/24/introduction-to-ajax/#comments Mon, 24 Aug 2009 21:56:08 +0000 Vinny http://usestrict.net/?p=840 In this article, I provide an explanation of Ajax with a historical introduction. If you are eager to start seeing the code, please scroll down.

 

What’s Ajax?

 

Ajax stands for Asynchronous Javascript And XML. It’s a way to call back-end scripts asynchronously – that is, without impacting user experience/flow. Basically, you don't even see the cursor become an hourglass or whatever other "waiting/processing" icon your system uses. It's not a language, but a technique. As for back-end scripts, you can use whatever you feel more comfortable with: PHP, Perl, Java, JSP, Shell, C, etc. The way to choose which technology to use as back-end is not outside the scope of this article.

 

Historical approach for calling back-end apps

 

Throughout web-development history, the very first way used to achieve back-end processing followed by front-end display was to create a form and set the back-end script as the action to that form. Upon submission, the form fields would be sent to the back-end script as a series of special environment variables, which would then be handled by the programmed logic. The back-end output would be displayed on the screen (either the same page/frame or a different page/frame, depending on the target attribute of the form).

The catch 22 of this approach is that if you want to present another form after processing, that form must be produced by the back-end script, which leads to maintenance mayhem – to add or remove a field, you have to do so in both the original HTML and in the HTML of the back-end script.

 

Hidden IFRAMES and Javascript to the rescue

 

The natural evolution of the regular form submission was to avoid having to generate HTML from the back-end script. A technique that I eventually came up with (had never seen it before, but it was probably already out there) prior to adhering to Ajax was using a hidden IFRAME as target to a simple form containing an action_cd field and a data field. There would not be a regular submit button, but a set of input elements with onclick or onchange events and a regular (non-submit) button at the end.

In this approach, Javascript did all the work – when the user clicked the pseudo-submit button, an onclick event would read all the input fields and join them like a regular HTTP query (field1=val1&field2=val2…). The action_cd value would be dynamically evaluated and the back-end script would receive the data string and action code and process it. Javascript would perform the submission when, say, a value of a drop-down field was selected, and the output would be a series of dynamically generated Javascript commands inside the IFRAME. Those commands would manipulate the user’s form (e.g. Auto-populate a "state" drop-down after selecting a "country" value).

Although this approach successfully handled the no-longer-need to submit and reload a form, it still had 2 main downsides: generating and maintaining Javascript code is not a simple task (especially due to having to escape quotes), and the whole procedure was still synchronous – you could see the hourglass, browser process bars, clicking sounds (IE), etc. It's much better user experience, but still not the real deal, seamless/transparent back-end processing.

 

The XmlHttpRequest object

 

Javascript back-end calls are made possible thanks to the XmlHttpRequest object that most modern browsers now support. For IE 6 and older, it is done through an ActiveXObject.

 

Javascript code to initialize Ajax object

 


var xmlHttp; // global variable. It’s important for later on

function GetXmlHttpObject(){

      if (window.XMLHttpRequest){
        // code for IE7+, Firefox, Chrome, Opera, Safari
        return new XMLHttpRequest();
      }

      if (window.ActiveXObject){
        // code for IE6, IE5
        return new ActiveXObject("Microsoft.XMLHTTP");
      }
      return null;
}

It's important to have a global variable in which to assign the XMLHttpRequest object, since this technique has its shortcomings when dealing with passing variables around. We'll understand that better shortly. The function checks for the existence of window.XMLHttpRequest which is used in IE7 (finally, it complies to standards!), Firefox, and other non IE browsers, and checks for window.ActiveXObject existence for IE6 and 5. I’m not sure if this works on IE4 – it probably does not, but hopefully IE4 is already in extinction. If neither checks work, it returns null, which will be handled in the function calling the XMLHttpObject.

 

Making Synchronous Ajax calls (GET method)

 

Although the hype of Ajax is due to the "Asynchronosity" of the technique, it is also capable of making synchronous calls. Synchronous calls are important in cases where you don't want to allow the user to do something while the back-end is checking a previous action, but also don't want to have to submit the whole form.


function synch_call() {

      // initialize Ajax object
      var xmlHttp = GetXmlHttpObject(); 

      // Set the URL to your backend script with query vars
      var url = “/url/to/your/backend/script?with=var1&and=var2”;

      // Synchronous call
      xmlHttp.open("GET",url,false); // "false" means asynchronous = false
      xmlHttp.send(null);

      // the code above does the back-end call. Now we handle the response
      // this function continues further down in the tutorial

}

Ajax calls can be done using GET or POST methods. I recommend GET methods for small queries, POST for large ones. There used to be a limit of 255 characters on URL fields – not sure if a) this is still true; b) this could apply to these Ajax techniques.

The call to the back-end is done by 2 commands: xmlHttp.open(Method,url,async), and the xmlHttp.send(null) calls. For POST method, this is a little different, so we’ll stick to GET calls for the time being.

 

Handling back-end reply

 

Asynchronous Javascript And XML isn't the perfect name for Ajax. It should be more like AjaR, where R stands for Reply. The reason is because the reply can be either an XML structure or a plain text. We use xmlHttp.responseXML.documentElement if we’re handling XML, or a plain xmlHttp.responseText for plain text. Actually, we can use the latter for XML as well, especially when debugging the code (when you want to pop up the XML reply).

We use a try/catch block to capture any errors:

function synch_call() {

      // initialize Ajax object
      var xmlHttp = GetXmlHttpObject(); 

      // Set the URL to your backend script with query vars
      var url = “/url/to/your/backend/script?with=var1&and=var2”;

      // Synchronous call

      xmlHttp.open("GET",url,false); // “false” means asynchronous = false
      xmlHttp.send(null);

      // the code above does the back-end call. Now we handle the response
      try {            

            alert("Ajax Response: \n"+xmlHttp.responseText); // Just pop-up the reply
            return false;
      }
      catch(e) {
            alert(e); // if error, show it as a pop-up
            return false;
      }
}

 

To handle the reply as XML, you need to a) make sure your back-end script outputs an XML structure, and b), replace xmlHttp.reponseText for xmlHttp.responseXML.documentElement.

 

// Just the try/catch block changed        

       try {
              data = xmlHttp.responseXML.documentElement;
        }
        catch(e) {
               alert(e);
               return false;
        }

The data variable now holds a DOM object of the XML structure. To access its elements, we use DOM functions.

Example XML structure:

<OUT>
      <STATUS>Success</STATUS>
      <MSG type="some_type">This is a test message</MSG>
</OUT>

Accessing Example XML through DOM:


      // gets the Success text
      var msg_status = data.getElementsByTagName(‘STATUS’)[0].childNodes[0].nodeValue;

      // Assign MSG object to msg variable
      var msg = data.getElementsByTagName(‘MSG’)[0]; 

      // gets the attribute “type”
      var msg_type = msg.getAttribute(‘type’); 

      // gets the text inside the MSG node
      var msg_text = msg.childNodes[0].nodeValue;

 

Putting it all together

 

function synch_call() {

      // initialize Ajax object
      var xmlHttp = GetXmlHttpObject(); 

      // Set the URL to your backend script with query vars
      var url = “/url/to/your/backend/script?with=var1&and=var2”; 

      // Synchronous call
      xmlHttp.open("GET",url,false); // "false" means asynchronous = false
      xmlHttp.send(null);

      // the code above does the back-end call. Now we handle the response

      try {
            data = xmlHttp.responseXML.documentElement;
       }
       catch(e) {
             alert(e);
             return false;
       }

      // gets the Success text
      var msg_status = data.getElementsByTagName(‘STATUS’)[0].childNodes[0].nodeValue; 

      // Assign MSG object to msg variable
      var msg = data.getElementsByTagName(‘MSG’)[0]; 

      // gets the attribute “type”
      var msg_type = msg.getAttribute(‘type’); 

      // gets the text inside the MSG node
      var msg_text = msg.childNodes[0].nodeValue;
      alert("Got the following backend reply: " + msg_status + "\n" + msg + "\n" + msg_type + "\n" + msg_text);

      return true;
}

 

Making Asynchronous Ajax calls (GET method)

 

We saw on page 2 that what controls synchronicity of the Ajax call is the true/false value in the open() command. However, we also need changes in the way we handle the reply. We do that by setting onreadystatechange property on the xmlHttp object.

OnReadyStateChange should be a function that will check the readyState attribute of the xmlHttp object.

 

Understanding readyState

 

readyStates are the way Javascript controls what stage of the process the call is in. It ranges from 0 to 4, where 4 is the completed reply from the back-end script. We normally only care about readyState == 4. Here are the codes and meaning:

Value State
0 Uninitialized
1 Loading
2 Loaded
3 Interactive
4 Complete

 

Setting onreadystatechange

 

onreadystatechange can be assigned a anonymous function() { } block, or a named function. Just be careful with function parameters – to this day, I have not been able to pass any parameters to the functions set to onreadystatechange. Hence the need for some global variables in play.

Let's copy over our sync_call() function and make it Async_call()

function Asynch_call() {

      // initialize Ajax object
      var xmlHttp = GetXmlHttpObject(); 

      // Set the URL to your backend script with query vars
      var url = “/url/to/your/backend/script?with=var1&and=var2”;

      // onreadystatechange must be set BEFORE making the call

      // it takes the response handling code that we had in the sync_call() previously
      xmlHttp.onreadystatechange = function() {

            if (xmlHttp.readyState == 4) { // Complete

                  // Now we handle the response within the onreadystatechange
                  try {
                        data = xmlHttp.responseXML.documentElement;
                  }
                  catch(e) {
                        alert(e);
                        return false;
                  }

                  // gets the Success text
                  var msg_status = data.getElementsByTagName(‘STATUS’)[0].childNodes[0].nodeValue; 

                  // Assign MSG object to msg variable
                  var msg = data.getElementsByTagName(‘MSG’)[0]; 

                  // gets the attribute "type"
                  var msg_type = msg.getAttribute('type'); 

                  // gets the text inside the MSG node
                  var msg_text = msg.childNodes[0].nodeValue;

                  alert("Got the following backend reply: " + msg_status + "\n" + msg + "\n" + msg_type + "\n" + msg_text);
                  return true;
            }

      }; // don’t forget the ';' at the end of the function() {} block!!

      // Asynchronous call
      xmlHttp.open("GET",url,true); // 'true' means asynchronous = true
      xmlHttp.send(null);
}

 

As mentioned before, you can separate the anonymous function assignment into a named function. The good side of this is that your functions get more manageable, but the downside is that you might need global variables in order to share values between the main function and the onreadystatechange,/code> function.

 

function Asynch_call() {

      // initialize Ajax object
      var xmlHttp = GetXmlHttpObject(); 

      // Set the URL to your backend script with query vars
      var url = “/url/to/your/backend/script?with=var1&and=var2”; 

      // onreadystatechange must be set BEFORE making the call

      // it takes the response handling code that we had in the sync_call previously
      xmlHttp.onreadystatechange = handlerfunction; // See separate function below    

      // Asynchronous call
      xmlHttp.open("GET",url,true); // “true” means asynchronous = true
      xmlHttp.send(null);
}

function handlerfunction() {

      if (xmlHttp.readyState == 4) { // Complete – must be global xmlHttp variable

            // Now we handle the response within the onreadystatechange
            try {
                  data = xmlHttp.responseXML.documentElement;
            }
            catch(e) {
                  alert(e);
                  return false;
            }

             // gets the Success text
            var msg_status = data.getElementsByTagName(‘STATUS’)[0].childNodes[0].nodeValue; 

            // Assign MSG object to msg variable
            var msg = data.getElementsByTagName(‘MSG’)[0]; 

            // gets the attribute “type”
            var msg_type = msg.getAttribute(‘type’); 

            // gets the text inside the MSG node
            var msg_text = msg.childNodes[0].nodeValue;

            alert("Got the following back-end reply: " + msg_status + "\n" + msg + "\n" + msg_type + "\n" + msg_text);
            return true;

      }
}

 

Parsing forms to create the GET/POST strings

 

Since we are not submitting the form with the usual submit button, we have to handle building the queries ourselves. I came up with a function that mimics the way browsers handle function calls.

function build_post_string(frm) {

      var str;
      var poststr_array = [];

      if (!frm.id) {
            // assume it's a string. get the form object
            frm = document.getElementById(frm);
      }

      for (i=0;i<frm.elements.length;i++){

            var elem = frm.elements[i];

            if (!elem.id) {
                  // skip any fields that don't have IDs
                  continue;
            }

            if (elem.type == 'radio' || elem.type == 'checkbox') {

                  // only grab radio buttons and checkboxes that are checked
                  if (!elem.checked) {
                        continue;
                  }

            }            

            // get select values
            if (elem.nodeName.match(/SELECT/i) && elem.multiple) {

                  var sel_array = [];

                  for (var o=0;o<elem.options.length;o++) {

                        if (elem.options[o].selected) {
                              sel_array[sel_array.length] = elem.id+"="+elem.options[o].value;
                        }

                  }

                  var sel_str = sel_array.join('&');

                  // build key/value pairs for SELECTs
                  poststr_array[poststr_array.length] = sel_str;
            }
            else if (elem.nodeName.match(/SELECT/i)) {
                  poststr_array[poststr_array.length] = elem.id+'='+elem.options[elem.selectedIndex].value;
            }
            else {
                  // build key/value pairs for everything else
                  poststr_array[poststr_array.length] = elem.id+"="+elem.value;
            }

      }

      // build and return str
      str = poststr_array.join("&");
      return str;
}

This function takes a given form by ID and iterates through all of its elements. It is important that all fields you want in the query have an ID attribute. Otherwise they will be skipped. The return from this function can be used for both GET and POST requests.

 

Using POST method for Ajax calls

 

So far we saw that making GET calls is pretty straight-forward. POST calls are different in the way that they are not put in the HTTP Header, but rather in a separate packet. In order to do that, we need to set a few header variables:

function Asynch_call_with_post() {

      // initialize Ajax object
      var xmlHttp = GetXmlHttpObject(); 

      // Set the URL to your backend script with query vars
      var post_str = build_post_string(‘my_form’); 

      var url = “/url/to/your/backend/script?” + post_str;

      // onreadystatechange must be set BEFORE making the call
      // it takes the response handling code that we had in the sync_call previously
      xmlHttp.onreadystatechange = function() {

            if (xmlHttp.readyState == 4) { // Complete

                   // Now we handle the response within the onreadystatechange
                   try {
                         data = xmlHttp.responseXML.documentElement;
                    }
                    catch(e) {
                         alert(e);
                         return false;
                    }

                     // gets the Success text
                     var msg_status = data.getElementsByTagName(‘STATUS’)[0].childNodes[0].nodeValue; 

                      // Assign MSG object to msg variable
                     var msg = data.getElementsByTagName(‘MSG’)[0]; 

                     // gets the attribute “type”
                     var msg_type = msg.getAttribute(‘type’); 

                     // gets the text inside the MSG node
                     var msg_text = msg.childNodes[0].nodeValue;
                     alert("Got the following backend reply: " + msg_status + "\n" + msg + "\n" + msg_type + "\n" + msg_text);
                     return true;
               }

        }; // don't forget the ';' at the end of the function() {} block

        // Asynchronous call
        xmlHttp.open('POST',url, true); // Replace "GET" with "POST"

        // Set headers
        xmlHttp.setRequestHeader("Content-type", "application/x-www-form-urlencoded");
        xmlHttp.setRequestHeader("Content-length", url.length);
        xmlHttp.setRequestHeader("Connection", "close");

        // Send the packet. Note that it’s no longer null as parameter to send
        xmlHttp.send(url);
}

 

Showing/Hiding "Processing…" messages

 

Since Ajax in Asynchronous mode is 100% transparent to the user, it is necessary to handle our own messages asking the user to wait until processing finishes. The simplest way to do this is by using a DIV element with initial display property set as none, or visibility set as hidden (the difference is that visibility="hidden" makes the element still occupy its space in the page, whereas display="none" effectively removes the element from the page and the space that it occupied).

Typically, I include code to show the element in the Ajax function, and code to hide it again in the onreadystatechange function (where readyState == 4). 

        document.getElementById(‘wait_image’).display = ‘block’;
        document.getElementById(‘wait_image’).display = ‘none’;

 

Third-Party AJAX Libraries

 

There are several Ajax in a box libraries out there, but I am not comfortable with using them for company applications. The reason being is that Ajax is very powerful and can very well make calls to the maintainer's website to steal/capture sensitive information.

Google provides Java code that generates Ajax during build – that is a typical example of us not being able to control Ajax code generated.

 

Testing the back-end script

 

There are 2 simple ways of testing the back-end script. With PHP, you can access both GET and POST variables through the $_REQUEST array. Perl CGI doesn’t really care if it’s GET or POST. I don’t know how JSP can handle it. With this being said, I simply have the Ajax alert the URL variable and return false. I then copy the URL variable and paste it in another window to see the output.

I strongly recommend debugging with Firefox – it has a handy error console and also parses the output XML or complains if it’s not built right.

 

Keeping IE from crashing with the XMLs

 

I'm pretty sure that Microsoft IE developers have some satisfaction in making web developers go crazy. That's how I felt when I started working with XML and Ajax, and although my app was working fine in Firefox, it kept crashing bad in IE. After some googling, I found out that I needed to set some headers in order to keep IE from blowing up. I already had the "Content-type: text/xml" header in place, but it appears that IE needs some caching control and expiry headers as well.

This is what I use in PHP:

        header("Expires: Fri, 09 Jan 1981 05:00:00 GMT");
        header("Cache-Control: no-store, no-cache, must-revalidate");
        header("Cache-Control: post-check=0, pre-check=0", FALSE); // FALSE means add this Cache-Control line instead of replacing the previous one
        header("Pragma: no-cache");
        header("Content-type: text/xml");

In Perl, setting headers is simple: just print them before printing any content. When you're done printing all your headers, print an empty newline to indicate where the page content will go.

I hope you've enjoyed this tutorial. Feel free to paste comments regarding your experience and preferences.

]]> http://usestrict.net/2009/08/24/introduction-to-ajax/feed/ 0 Perl: Installing DBD::Oracle + Oracle Instant Client on Ubuntu 9.04 http://usestrict.net/2009/07/12/perl-installing-dbdoracle-on-ubuntu-904-and-oracle-instant-client/ http://usestrict.net/2009/07/12/perl-installing-dbdoracle-on-ubuntu-904-and-oracle-instant-client/#comments Sun, 12 Jul 2009 16:17:20 +0000 Vinny http://usestrict.net/?p=789 A couple of weeks ago I finally got a new laptop at work - which meant of course that I had to reinstall everything. Although we use Windows XP, there's one app that I have to run through Linux. The solution was Sun's VirtualBox running Ubuntu 9.04. When I tried to run the app, I realized that I still needed to install DBI and DBD::Oracle - which brings me to this article. Nothing better than a reinstall to generate article material :)

DBD::Oracle is usually a pain to install if you haven't already done it a gazillion times. After that, it's just an annoying itch. In this article I'll cover installing DBD::Oracle using Oracle Instant Client, Ubuntu 9.04, and Perl 5.10. Since we're using the Instant Client, you'll need some Oracle DB you can connect to in order to do the testing. You can choose to skip testing altogether, but you might be into a surprise later.

 

DBI and DBD::Oracle

Installing the DBI is always easy - regardless if you're running Windows or Linux. Just follow the steps.

  1. Get root password if you don't already have one: sudo passwd root
  2. Switch to root: su -
  3. Run CPAN: perl -MCPAN -e shell
  4. Check if DBI is already installed: m DBI
  5. If it's not installed, install it: install DBI

That should do the trick for the DBI. The DBD::Oracle is a bit more complicated and we'll just use CPAN to download it for us. The rest is manual.

  1. Check for DBD::Oracle: m DBD::Oracle
  2. Download it: get DBD::Oracle
  3. Exit CPAN: q

Now we should have the DBD::Oracle distribution downloaded to our CPAN build directory. If you don't know where to find it, look for .cpan dir under your root home or (if you started CPAN for the first time doing sudo from your main user, look for it under your main user's home directory). We'll leave that distro aside for a moment and work on the other pre-requisites.

 

Oracle Instant Client

Download the instant client packages you'll need. I chose to download the RPMs and convert them to .deb files using alien. Oracle also provides .zip files if you don't want to do it the alien way.

Oracle Instant Client Download Site i386
Oracle Instant Client Download Site AMD64 32- and 64-bit

Accept the license agreement by clicking the Accept radio button. Since the i386 and amd64 files have different names, check the bold words in the file names to know which ones to download (the names below are for the i386 platform). Also, if you don't have a user_id for Oracle, you'll be prompted to register one once you click the link. It's free of charge.

  • oracle-instantclient11.1-basic-xx.x.x.x.x-x.i386.rpm
  • oracle-instantclient11.1-sqlplus-xx.x.x.x.x-x.i386.rpm
  • oracle-instantclient11.1-devel-xx.x.x.x.x-x.i386.rpm

AMD64 has only zip files from what I could find, and they're named a bit differently, too (remove the 32 for the 64-bit version:

  • instantclient-basic-linuxAMD64-32-xx.x.x.x.x-yyyymmdd.zip
  • instantclient-sqlplus-linuxAMD64-32-xx.x.x.x.x-yyyymmdd.zip
  • instantclient-sdk-linuxAMD64-32-xx.x.x.x.x-yyyymmdd.zip

Note: Extract the zips into a directory called instantclient and skip to the Set Up your Environment Variables section if you're using the AMD64 or i386 zip files.

 

 

Install alien and libaio

Now is the time to install alien, which is an application that converts rpm files into .deb format to be used with dpkg. Instant Client also requires libaio. Both can be installed through the Synaptic Package Manager. Just open it, look for alien, mark it for install and do the same for libaio and libaio-dev. Once they're installed, you're good to move on to install the Instant Client. Just don't forget to exit Synaptic Package Manager, since we'll be using dpkg and it won't work if Synaptic is open.

 

Install Oracle Instant Client

First step to install Oracle Instant Client from rpm files is to convert them into .deb files. Do that with alien by running the following command (from a command line in the directory where you downloaded your RPMs):

$ sudo alien --scripts *.rpm

It takes a little while to run, so be patient. Once it's complete, install the newly created .deb files:

$ dpkg -i *.deb

 

Set up your Environment Variables

The nasty thing about rpm files is that it's not always easy to know where the files were installed. If you opted for the zip file approach, your install will most definitely be different. For RPM install, add this to your .bashrc file (swap xx.x for your oracle version):


export ORACLE_HOME=/usr/lib/oracle/xx.x/client
export PATH=$PATH:$ORACLE_HOME/bin
export LD_LIBRARY_PATH=$ORACLE_HOME/lib

Update: If you get an ELFCLASS64 error, try setting LD_LIBRARY_PATH to $ORACLE_HOME/lib32 instead.

Reload your .bashrc file:
$. ~/.bashrc

Note: by default, Oracle Instant Client doesn't come with a tnsnames.ora file or the directory structure where it's usually found. We'll have to create that ourselves -

$ mkdir -p $ORACLE_HOME/network/admin; touch $ORACLE_HOME/network/admin/tnsnames.ora

 

Install DBD::Oracle

It's time to finally install DBD::Oracle. Go to your CPAN build directory and cd into DBD-Oracle-*. As a user having the environment variable set from the previous section, run Makefile.PL portion.

$ perl Makefile.PL

There's no need to set INC or LIB with the alien approach, but if you run into any issues, try giving Makefile.PL the path to your include dir for INC, and lib dir for LIB.

Next, run

$ make.

It will raise a few warnings, but unless it exits with an error, you should be OK.

The next logical step is to run make test. However, this will undoubtedly fail unless you have a valid entry in your tnsnames.ora file. If you don't, you can skip this test and hope that everything works later on. Otherwise, update your tnsnames.ora file with a valid entry, and set one more environment variable before running the test:

$ export ORACLE_USERID="user/passwd@tns_entry_name"
$ make test

This is usually the hardest part to pass with total success. Many things can go wrong. In my case, the valid entry I use doesn't have full grants to the user I log in as. This always triggers an error on the create/access sequences portion of the testing. I simply ignore it nowadays - make sure you analyze your test results thoroughly before choosing to ignore the errors as well.

Now that the testing is over, simply run
$ make install
as a super user and you're all set!

]]> http://usestrict.net/2009/07/12/perl-installing-dbdoracle-on-ubuntu-904-and-oracle-instant-client/feed/ 44 New Tech Jobs section http://usestrict.net/2009/06/13/new-perl-jobs-section/ http://usestrict.net/2009/06/13/new-perl-jobs-section/#comments Sat, 13 Jun 2009 07:00:33 +0000 Vinny http://usestrict.net/?p=559 I've added a Tech Jobs page to the main pages section (right under the header) for those who are looking for jobs. Right now it only has the RSS from jobs.perl.org, but I intend to create a mash-up from several sites in the future.

Update: Added craigslist.com search for Perl jobs in Canada (Vancouver only). Next is to add some fields which allow folks to select keywords and places (Done! See update below).

Update: The Feed fetcher now works with craigslist.com worldwide. You can also use search keys for feeds that accept them.

Update (6/13/09): monster.com added with 192 countries (!!). Since some sources accept keywords and others don't, the form will now tell you so. Also, modified the form to allow blank keywords for some sources. The issue is that craigslist uses a totally separate query for blank keywords, and right now the fetcher doesn't handle alternate query strings per source. Total Sources: 4. Total feeds: 708.

Good luck job-hunting!

Vinny.

]]> http://usestrict.net/2009/06/13/new-perl-jobs-section/feed/ 0 Recursion with Perl and CDS http://usestrict.net/2009/06/10/recursion-with-perl-and-cds/ http://usestrict.net/2009/06/10/recursion-with-perl-and-cds/#comments Wed, 10 Jun 2009 22:16:10 +0000 Vinny http://usestrict.net/?p=720 Update: Changed subroutine to comply with Perl Best Practices

Update2: Removed the prototype from the subroutine.

I've always had a problem with recursion. Not with the general theory that a function will call itself, etc - no, that's easy. The hard part was when I had to deal with complex data structures in Perl (an array- or hashref containing a hash of arrays of hashes, a gazillion levels deep). Well, I guess anyone would have a hard time with that kind of data.

Anyway, in this post I don't intend to get all complicated explaining all the kinds of recursions out there. If you want that, check this article at wikipedia. What I do want to do is help all of those who are in the situation I was in, by explaining in the simplest way possible how to deal with this scenario.

Let's start with a need. I have a complex data structure that needs its spaces trimmed on both sides. But since I'm lazy, I'd like my subroutine to modify the data directly, and not return the modified value (pass by reference, not pass by value).

Here's our data structure:

 

    my $data = [
                       {
                          key1 => '   trim me!   ',
                          key2 => '   trim me too!    ',
                       },
                       [
                          'some element to trim   ',
                          '    another one    ',
                       ],
                       '    a simple string needing trimming    ',
                  ];

 

 

$data explained: an array containing 3 elements: element 0 is a hashref of keys "key1" and "key2", element 1 is an arrayref of 2 elements. Element 3 is a simple string. All values have some extra spaces that need trimming (or so they say). We could use whatever number of levels and data types we want (except for anonymous subroutines, I guess - let's not get too complicated).

Now, to trim all that, I want to be able to simply call trim() à la PHP.

 

     trim($data); # note the lack of the lvalue (lvalue = rvalue)

 

 

I also want it to accept simple arrays and hashes, and the references thereof: trim(@array); trim(\@array); trim(%hash); trim(\%hash); trim($string). After all, I never know what kind of data my colleagues will be working with. Better have it deal with everything.

The logic to do that is this: our subroutine will have to do the trimming (s///g) on scalars only. For that, it has to check if the data it received is a hash, array, etc, and if it is, iterate through each element and trim the value... but only if the element is not itself a hash, array, etc. Found it confusing? No problem, it really is.

In Perl, if I tried to remove the white space from element 0 of my $data variable, it wouldn't work. The reason being is that if I printed $data->[0] onto the screen, I'd get a funny looking output, something like HASH(0x1004f5f0). That's Perl's way of saying that you have a HASH structure stored in memory position 0x1004f5f0. You can try to trim the spaces off of that string, but it won't do you any good. The elements of your hash will still be untouched. That's why you need to de-reference your data structures and dive into them.

To de-reference a structure is simple, just add a % in front of the variable if it's a hashref, or an @ if it's an array. But how do you know which is which? Use ref().

 

       print ref($data->[0]) . "\n"; # HASH
       print ref($data->[1]) . "\n"; # ARRAY
       print ref($data->[3]) . "\n"; # empty string, which is false

 

 

ref() tells you what kind of data you are dealing with. It returns CODE if you have a closure or anonymous subroutine, but we're not going there today.

So, now that we know how to identify the type of element we're going to be working with, we can build our subroutine...

 

sub trim() {
	for my $param (@_) {
		if (ref($param) eq 'ARRAY') {
			for my $element (@{$param}) {
				trim($element);
			}
		}
		elsif (ref($param) eq 'HASH') {
			for my $val (values %{$param}) {
				trim($val);
			}
		}
		elsif (ref($param) eq 'CODE') {
			return;
		}
		else {
			$param =~ s/(^\s+|\s+$)//g;
		}
	}
}

 

 

trim() explained:

We're working with passing elements by reference instead of by value. This means that the elements themselves will be modified - no need to return any data. The first thing we do is to iterate through all parameters passed to trim(). In a subroutine, parameters (in our case, variables) are populated into the special @_ array, allowing us to call trim($var1, $var2, $var3) if we want.

We iterate through all elements of @_ and verify if they are an Array. If they are, we iterate through each of their elements once, and call trim() again against them. That will handle as many nested arrays we want (or that your computer can handle). Now we have to make it deal with hashes. Same technique - use ref() to see if it's a hash. If it is, then iterate through each of its key/pair elements. There are several ways to do that. I personally prefer calling keys to get the keys and use them to fetch the values of the hash. The value of the hash is passed to trim() for more validation. We also check to see if we received a sub { } (anonymous subroutine). In that case, we do nothing, just return.

Finally, after handling Arrays, Hashes and Anonymous subroutines, we can set up the actual trimming of the strings. We take the $_[$i] which is the parameter passed and remove the leading and trailing spaces with one neat substitution: ^s\+ stands for leading spaces, \s+$ stands for trailing spaces, and it's all joined by the ( | ) (this or that). We only call it once because we're using the global (g) modifier of the substitution s///g.

And that's all there is to it!

A note on prototypes: This post generated a healthy discussion on prototypes. I had originally added the dollar prototype to the trim() subroutine, but that was forcing it to accept only scalars (strings and references), and not working with normal hashes and arrays. Thanks to everyone who participated in the discussion.

 

 

]]> http://usestrict.net/2009/06/10/recursion-with-perl-and-cds/feed/ 14 Perl and MQSeries for the faint of heart http://usestrict.net/2009/05/28/perl-and-mqseries-for-the-faint-of-heart/ http://usestrict.net/2009/05/28/perl-and-mqseries-for-the-faint-of-heart/#comments Fri, 29 May 2009 02:54:31 +0000 Vinny http://usestrict.net/?p=502 Today I managed to finally get Perl to put and get messages to MQ Series. It's something that I've been wanting to do for quite some time, but didn't have the time or even MQ knowledge to do so.

This post is intended for those who, like me, aren't MQSeries gurus and can't make much of the documentation of the MQSeries module in CPAN. I hope it serves you well.

Very VERY Brief introduction to MQSeries

MQ Series (recently renamed Websphere MQ) is a messaging software developed by IBM. There is a client and a server piece to it.

MQ Messages are transmitted between servers by means of queues. These queues are managed by Queue Managers. Queues can be configured to receive messages (input or request) and/or return messages (output or reply).

The installation of MQ series is beyond the scope of this post, but it's important to know that if you need to communicate with a queue manager installed in a local machine, you have to use one set of APIs (the server APIs), and if you want to talk to a remote queue manager, you'll use the client APIs. This is important for later on.

A MQ message is composed of 2 main parts: the MQ Message Descriptor (MQMD) and the Data portion. The MQMD portion works as a header where several fields can be set to tell the server how to handle the message. The fields we will be using in this tutorial are MsgId, CorrelId, MsgType, Expiry, ReplyToQ, and ReplyToQMgr.

MsgId: Each message being sent into the server gets a unique Message ID. They never repeat, even when the server is restarted.

CorrelId: The Correlation Id is how a reply message can be associated to a request message.

MsgType: The kind of data you will be sending through the queues. We will be using a simple string format.

Expiry: The amount of time that the server should keep the message in a queue.

ReplyToQMgr: The name of the Queue Manager where the reply must be sent to.

ReplyToQ: The name of the queue where the reply must be sent to.

You will find more information about the fields and MQ Series in general at the MQSeries Application Programming Reference. I couldn't find a link to the hardcopy at Amazon, but you can consider getting this one.

Before trying to put or get messages, you will probably need to set the MQSERVER environment variable (for both Unix and Windows - don't know how Mac works), since the client usually requires it. The value for MQSERVER is "CHANNEL_NAME/TCP/HOST_IP(PORT)". There can be variances according to the settings of the host you'll connect to, but this is the most common scenario.

MQSeries CPAN modules

CPAN has a very handy set of modules which allow us to communicate with MQSeries. The main module, MQSeries, exports all the funny named subroutines that you need to use to send and receive messages. If you don't have working knowledge with C/C++ using MQ Series, then don't even bother trying to use it. The Object Oriented modules (by the same author) are much simpler for the unseasoned MQ Series explorer (like myself).

First off, you will need to install the modules in your computer. If you're working on windows, read this article to get it done. It's simpler on Unix, but both cases require that you have the MQ Series client installed. You can get it from here

When you install MQSeries module from CPAN, you also get MQSeries::QueueManager, MQSeries::Queue, MQSeries::Message and some other goodies. If you didn't get any of those automatically, make sure you install them before you go on.

Writing your own MQSeries module

Here we'll write a few methods which should make people's lives easier when communicating with MQ Series. Now, depending on the way your application works, you will probably have to handle the creating and breaking up of the Data portion of the message, but again, that's outside the scope of this post.

So let's get started...

Start your module by giving it a package name. Remember to end the code with a true value. We'll also import the modules we'll need

package myMQModule;

use strict;
use MQSeries;
use MQSeries::QueueManager;
use MQSeries::Queue;
use MQSeries::Message;

1; # end with a true value

Now let's lay out the basics of our module.

package myMQModule;

use strict;
use MQSeries;
use MQSeries::QueueManager;
use MQSeries::Queue;
use MQSeries::Message;

sub new() { # our constructor
}

sub openQueueMgr() { # open the Queue Manager
}

sub openQueue() { # opens the Queue
}

sub putRequest() { # puts the message
}

sub getReply() { # gets the reply using the correlId
} 

1; # end with a true value

You'll probably notice that we haven't added methods to close the queue and the queue manager. That's ok, since the MQSeries* modules do that for us.

Let's go on with our methods. Starting with new(). We'll use the basic instantiation code. You will probably need to enhance this to suit your own needs, such as calling openQueueMgr() and openQueue() automatically. For now, we'll KISS.

package myMQModule;

use strict;
use MQSeries;
use MQSeries::QueueManager;
use MQSeries::Queue;
use MQSeries::Message;
use Carp; # exports carp, confess, etc.

sub new() { # our constructor
    my $invocant = shift;
    my $class = ref($invocant) || $invocant; # Object or class name
    my $self = {};       # initiate our handy hashref
    bless($self,$class); # make it usable
    return $self;
}

sub openQueueMgr() { # open the Queue Manager
}

sub openQueue() { # opens the Queue
}

sub putRequest() { # puts the message
}

sub getReply() { # gets the reply using the correlId
} 

1; # end with a true value

Now we need to get the Queue Manager and the Queue open. In real life, you'll probably have several queue managers and several queues. In our example, we'll use 2 of each, one pair for outgoing traffic and another for incoming. Remember that this depends on your MQ Series configuration. These examples should simply allow you to get your toes wet in preparation before diving into your custom settings. 

...
sub openQueueMgr() { # open the Queue Manager
    my $self = shift; # take the handy hashref
    my $qm_name    = shift; # the name of the queue manager
    my $qm_ip         = shift; # the IP to the queue manager
    my $qm_port      = shift; # the port it's listening to
    my $qm_channel = shift; # the channel configured in the QM
    my $type           = shift; # request or reply? we will need it later

    # some validation code here making sure that the QM name, ip,
    # port, and channel have been provided 

    # export the MQSERVER environment variable just in case your clients need it
    $ENV{MQSERVER} = "$qm_channel/TCP/$qm_ip($qm_port)";

    # Here we create a MQSeries::QueueManager object to connect to our QM
    # It takes, among other parameters, the QM name, AutoConnect, and
    # a hashref with the data we gave to $ENV{MQSERVER}
    my $connOpts = {
          ChannelName => $qm_channel,
          TransportType => 'TCP',
          ConnectionName => "$qm_ip($qm_port)",
          MaxMsgLength   => 16 * 1024 * 1024,
    };

    # Create MQSeries::QueueManager object
    my $qm = MQSeries::QueueManager->new(
        QueueManager => $qm_name,
        AutoConnect => 0, # we do not autoconnect now
                          # because we want to handle it better later
        ClientConn => $connOpts,
    );

    #  kick it off and see if it connects
    eval {
        $qm->Connect()
            || die(sprintf("Connect failed with CompCode: %s",
                               Reason %s\n",$qm->CompCode(),$qm->Reason()));
    };
    if ($@) { # eval caught the die
       confess($@);
   }

   # got this far, so it obviously connected
   # now save that connection in a safe place
   $self->{uc($type)}->{QM_CONN} = $qm; # MQSeries::QueueManager object.
                                        # We need it to open the queue
   $self->{uc($type)}->{QM_NAME} = $qm_name; # note that we also saved the QM_NAME and it is all under the $type (REQUEST/REPLY)

}
 ...

The snippet above showed you how to connect to the queue manager. It's pretty well commented, so we can go on with our next step, which is opening the queue.

...
sub openQueue() { # opens the Queue
    my $self       = shift;
    my $queue_name = shift;
    my $type       = shift; # again, we will need it for later.
                           # (REQUEST/REPLY)

    # set mode according to type
    my $mode = uc($type) eq 'REQUEST' ? 'output' : 'input';

    # open the queue directly this time
    # here we use the MQSeries::Queue module
    eval{
        $self->{uc($type)}->{Q_OBJ} =
           MQSeries::Queue->new(
               QueueManager  => $self->{uc($type)}->{QM_CONN}, # the QueueManager object from before
               Queue         => $queue_name,
               Mode         => $mode,
               AutoOpen         => 1, # open it directly
           )|| die("Could not open Queue $queue_name");
    };
    if ($@) {
        confess($@);
    }

    # made it this far, so we obviously got a connection
    # so let's save the queue name for later
    $self->{uc($type)}->{QUENAME} = $queue_name;
}
...

Now we have both our QueueManager and our Queue open. Since we're dealing with one request pair and one reply pair, you'll have to call it once for the request and once for the reply. You can also choose to save the connections in a different data structure to handle several queue managers and even more queues per queue manager. The only limit is your creativity. Right now, we'll stick to the $self->{REQUEST} and $self->{REPLY} structures.

Continuing our layout, we now have to handle putting data into the queue.

...
sub putRequest() { # puts the message
    my $self = shift;
    my $message = shift; # the data we will be putting

    # We need a MQSeries::Message object to inject
    my $msg =
        MQSeries::Message->new(
             MsgDesc => { # this is the MQMD (header) portion
				  Format	=> MQFMT_STRING,
				  Expiry	=> 100, # in tenths of seconds
				  ReplyToQ => $self->{REPLY}->{QUENAME},
				  ReplyToQMgr => $self->{REPLY}->{QM_NAME},
       				},
            Data => $message,
        );

    # now that we have a message object to inject, do the injecting
    my $request = $self->{REQUEST}->{Q_OBJ};
    eval {
       $request->Put(Message=>$msg) || die(sprintf("Put failed with CompCode: %s, Reason: %s\n",
                                                            $request->CompCode(),$request->Reason()));
    };
    if($@){
        confess($@);
    }

    # got this far, so Put was successful. We'll return the MsgId to be used
    # as CorrelId when getting the reply
    return $msg->MsgDesc('MsgId'); # comes from the injected MQSeries::Message object
}
...

If you try printing the MsgId, you'll get some funny characters in the screen, since it's pure binary data. Look up "perl bin to hex" on Google if you'd like to see some ways of converting and not getting gibberish.

Next comes the last but not least portion of our module - to get the reply message.

...
sub getReply() { # gets the reply using the correlId
    my $self      = shift;
    my $correlId = shift;

    # some sanity checking, since we can't do anything without $correlId
    confess("I need a correlId!!!\n") unless $correlId;

    # create the Message object with the $correlId
    my $reply_msg =
            MQSeries::Message->new(
                 MsgDesc =>
				{
				    CorrelId	=> $correlId,
				},
            );

    my $reply_q = $self->{REPLY}->{Q_OBJ};
    eval {
       $reply_q->Get(
                     Message=> $reply_msg,
                     Wait => 100, # some interval before timing out
                ) || die(sprintf("Get message failed with CompCode: %s, Reason: %s\n",
                                      $reply_q->CompCode(),$reply_q->Reason()));
    };
    if ($@){
        confess($@);
    }

    # got this far, so the Get was successful.
    # now we return the Data portion of the reply

    return $reply_msg->Data(); # from the Message object, not the Queue object!
}
...

And that's all there is to the module! Now you can call it from your script, like so

#!/usr/bin/perl

use strict;
use lib 'directory containing myMQModule';
use myMQModule;

# create object
my $mq = myMQModule->new();

# open queueMgr1
my($qm_name,$qm_ip,$qm_port,$qm_channel,$type) = qw(QMREQ some_ip some_port CHANNEL1 request);
$mq->openQueueMgr($qm_name,$qm_ip,$qm_port,$qm_channel,$type);

#open queueMgr2
($qm_name,$qm_ip,$qm_port,$qm_channel,$type) = qw(QMREP some_ip2 some_port2 CHANNEL2 reply);
$mq->openQueueMgr($qm_name,$qm_ip,$qm_port,$qm_channel,$type);

# now for the queues
$mq->openQueue('QUEUE1', 'request');
$mq->openQueue('QUEUE2', 'reply');

# put some message
my $msgId = $mq->putRequest('this is a test message');
my $reply_msg = $mq->getReply($msgId);

print "We're done!";

I really hope this was helpful. Comments are always welcome.

]]> http://usestrict.net/2009/05/28/perl-and-mqseries-for-the-faint-of-heart/feed/ 12 Perl Crash Course: Pragmas, Perl Modules, and CPAN http://usestrict.net/2009/05/23/perl-crash-course-pragmas-perl-modules-and-cpan/ http://usestrict.net/2009/05/23/perl-crash-course-pragmas-perl-modules-and-cpan/#comments Sun, 24 May 2009 01:00:14 +0000 Vinny http://usestrict.net/?p=475 I always like to say that 90% of Perl is its modules. Back in 2000 when I was working as a junior Perl programmer I was asked to write a web application that, among other things, could send contact messages through email. Unfortunately, I never had anyone to really teach me the Path of Perl - I only relied on Learning Perl by Randall Schwartz, and whatever I could find on the net. I had a really hard time with that application, mainly because I didn't know about Perl modules, MySQL and SQL language. Had I been familiar with at least the Perl modules part, I wouldn't have had to spend 8 days and nights in the office (including my birthday). I didn't even know how to use strict; at the time! Keep reading if use strict; makes no sense to you.

Being the extensible and flexible language that it is, Perl provides us with some safeguards and helpers to assist in avoiding what happened to me (I wish I knew that back then). The first of which I'll talk about is Pragmas.

Pragmas

Pragmas are special modules that come installed by default in every Perl distribution. They tell the interpreter of how it is supposed to act. To turn them on, all you have to do is call the special word use with the appropriate Pragma. To turn them off, call no and the Pragma in question. The most common and powerful Pragma is, in my opinion, strict (hence the name of this blog: use strict;#)).

Strict tells the Perl interpreter that all variables must be declared and tightens up security a notch. To use strict; you have to have at least working knowledge of lexical variables. It takes a while to getting used to at first, but once you're hooked, you won't know how you could possibly have written Perl programs without it before (I know I don't).

#!/usr/bin/perl

$var = 1; # OK
use strict;
$var1 = 2; # compile time error

example.pl

In our example above, Perl will refuse to run, raising a compile time error like such:

Global symbol "$var1" requires explicit package name at example.pl line 5.
Execution of example.pl aborted due to compilation errors.

That's strict in effect. Notice that $var was not cited, since strict was only enforced below it. In order to bypass that error, we should have declared our variable with either my, our, or local - depending on the need. my is the most common. Look up "Packages, Namespaces, and Lexical scopes" for more on those 3 operators.

#!/usr/bin/perl

$var = 1; # OK
use strict;
my $var1 = 2; # OK

use strict; is so important that it is usually the second line of code in any decent Perl program - the first line being the shebang.

If you need to turn off strict for one reason or another, you can do so with the key word no.

#!/usr/bin/perl

use strict;
my $var = 1; # OK
no strict;
$var1 = 2; # also OK

When writing your Perl programs, it's also good to turn on warnings and diagnostics. warnings will complain about possible problems such as useless uses of certain functions. diagnostics, on the other hand, will throw you a truckload of information regarding errors. It's a good place to start when you're stumped.

Perl Modules

Perl modules are pieces of code or packages that can be imported into your script with the keyword use in the same way as Pragmas. They can be Object Oriented, Procedural, or both. I will not discuss how to write a module in this post, but I will tell you where to find them for download and how to install them.

My all time favorite module is Data::Dumper. So I will use it in the following examples. The funny :: between Data and Dumper is kind of like a directory separator. The module Dumper resides in the directory Data, found in one of the paths configured in the Perl config files or the PERL5LIB environmental variable (which set the @INC array).

Data::Dumper comes installed by default, along with hundreds other modules that the developers deemed worthy. To test that your Perl distro has it, run the following command in a command line:

$ perl -MData::Dumper -e 'print "OK\n"'

You'll most definitely see the OK being printed on your screen. The command passes 2 parameters to the Perl interpreter: -M which tells it to load a module (in this case Data::Dumper - no spaces between -M and the module name, or you'll get a "missing argument" error), and -e which tells it to execute a piece of code (we told it to print OK, but any valid piece of code would do).

Look at what would have happened if we tried to load a module that wasn't installed:

$ perl -Maaaa -e 'print "OK\n"'
Can't locate aaaa.pm in @INC (@INC contains: /usr/lib/perl5/5.10/i686-cygwin /usr/lib/perl5/5.10 /usr/lib perl5/site_perl/5.10/i686-cygwin /usr/lib/perl5/site_perl/5.10 /usr/lib/perl5/vendor_perl/5.10/i686-cygwin /usr/lib/perl5/vendor_perl/5.10 /usr/lib/perl5/vendor_perl/5.10 /usr/lib/perl5/site_perl/5.8 /usr/lib/perl5/vendor_perl/5.8 .).
BEGIN failed--compilation aborted.

To install a module, you can do it the hard way or the easy way. Some modules are harder than others, so I'll stick with the easier ones for now. Let's start with the hard way.

First, you should know where to find your module. CPAN - The Comprehensive Perl Archive Network is the place to go to get your modules. It has a handy search engine which will search through almost 16000 modules at the time of this writing (5/2009). There you will find code ranging from the most trivial to the craziest needs. If there's one thing that CPANs contributors don't lack, it's creativity (that's a compliment).

Enter a keyword in the search engine and it will list all relative modules. Click on the link to the one that interests you most. You will be shown its documentation in POD (Plain Old Documentation) format. That's the data you will most likely need to learn how to use your module. It should also be the most up-to-date information, since it's kept by its authors.

At the top of the POD screen, you'll see a breadcrumb set of links showing the Author's name, the module's distribution name and version, and the module name itself:

Ilya Martynov > Data-Dumper-2.121 > Data::Dumper

Click on the link for the module's distribution name and version, to go to the distribution details screen. You will see lots of information about the module and its sub-modules, but what should concern you right now is the download link next to the release name. Click on it and download the module tarball.

Now this is where I halt and tell you that potential headaches lie ahead. There are basically 2 kinds of modules: PurePerl ones and C-based ones. PurePerl modules are just that - modules that are solely written in Perl. The vast majority of modules, however, are written in C with bindings special for Perl. Those usually have to be compiled and as such, need a C/C++ compiler and a make program. The good news is that most *nix systems come with those tools already installed or readily available. Windows systems, however, require that you install nmake and preferably Microsoft Visual C++. Check out my post about installing MQSeries module on Windows for more information on how to get Microsoft Visual C++

Back to installing the module...

Unpack your module in some directory where you have full access. I like to keep an untouched copy of the tarball, by reading the gunzipped contents and throwing it to the screen with the -c option, and piping it to tar with xvf - parameters:

gunzip -c tarball.tar.gz | tar -xvf -

You'll end up with a directory having the name of your distribution. Go in there and read all README files you can find. Read the INSTALL files if any.

One of the files you'll see in the directory is Makefile.PL. That's the kickoff file for the installation. It takes the following optional parameters: PREFIX, LIB, and INC, and creates a makefile tailor made for your system. It's important to set the PREFIX parameter if you want the module installed in a place other than the default. LIB and INC point to the C lib and include directories, respectively.

Now that you have your makefile, it's just a matter of running make, make test, and finally make install. Depending on the module, you should have no issues whatsoever. Other more "sensitive" modules, however, often require hours of work and even some tweaking of the makefiles by hand. DBD::Oracle is by far the craziest module I've ever had to install. In some machines it's a piece of cake, and others it manages to amuse with the amount of errors it pulls out of the hat. Anyway...

That was the hard way. Now for the easy way.

CPAN

If you're finding it strange to see the CPAN title here when I've already talked about it above, don't worry - I'm talking about the application CPAN and not the website.

Perl comes with the CPAN module installed, and most of the time it also creates a script in the bin directory called cpan. It's an interactive shell that allows you to fetch information regarding authors and modules, and also allows you to install modules without having to go through the whole process of downloading the distribution, unpacking, etc.

Start up the CPAN application by calling

$ perl -MCPAN -e shell

If it's the very first time you call it, you will be promped to answer a series of questions. Sticking to the default values is almost always OK. One of the first questions you will be asked is if you want CPAN to configure everything automatically. I recommend against it unless you know that the defaults are correct and will enable you to successfully install modules. And if you know that, then you probably already know how to use the CPAN application and this post has nothing new. :)

Once the questions have been answered, you will be presented with a cpan prompt. Type ? to know what options you have.

cpan[1]> ?
Display Information                                                (ver 1.9205)
 command  argument          description
 a,b,d,m  WORD or /REGEXP/  about authors, bundles, distributions, modules
 i        WORD or /REGEXP/  about any of the above
 ls       AUTHOR or GLOB    about files in the author's directory
    (with WORD being a module, bundle or author name or a distribution
    name of the form AUTHOR/DISTRIBUTION)

Download, Test, Make, Install...
 get      download                     clean    make clean
 make     make (implies get)           look     open subshell in dist directory
 test     make test (implies make)     readme   display these README files
 install  make install (implies test)  perldoc  display POD documentation

Upgrade
 r        WORDs or /REGEXP/ or NONE    report updates for some/matching/all modu
les
 upgrade  WORDs or /REGEXP/ or NONE    upgrade some/matching/all modules

Pragmas
 force  CMD    try hard to do command  fforce CMD    try harder
 notest CMD    skip testing

Other
 h,?           display this menu       ! perl-code   eval a perl command
 o conf [opt]  set and query options   q             quit the cpan shell
 reload cpan   load CPAN.pm again      reload index  load newer indices
 autobundle    Snapshot                recent        latest CPAN uploads

Use o conf to display the parameters to which you answered all those questions. You can change them with o conf as well. If you didn't enable auto-commit before, you will have to call o conf commit to save your changes for use in future sessions.

o conf http_proxy "http://some_proxy.com:80"

The m command allows you to fetch information regarding a certain module.

cpan[2]> m Data::Dumper
.... Possibly some data about fetching updated files from the internet here ....
Module id = Data::Dumper
    DESCRIPTION  Convert data structure into perl code
    CPAN_USERID  GSAR (Gurusamy Sarathy )
    CPAN_VERSION 2.121
    CPAN_FILE    I/IL/ILYAM/Data-Dumper-2.121.tar.gz
    DSLIP_STATUS SdpOp (standard,developer,perl,object-oriented,Standard-Perl)
    MANPAGE      Data::Dumper - stringified perl data structures, suitable for b
oth printing and C
    INST_FILE    /usr/lib/perl5/5.10/i686-cygwin/Data/Dumper.pm
    INST_VERSION 2.121_14

In the output above, CPAN tells us that we alreay have Data::Dumper version 2.121_14 installed. If you are not sure what is the exact name of a module, use the i command to fetch information using a regex:

cpan[3]> i /klingon/
Distribution    JALDHAR/DateTime-Event-Klingon-1.0.1.tar.gz
Distribution    PNE/Lingua-Klingon-Collate-1.03.tar.gz
Distribution    PNE/Lingua-Klingon-Recode-1.02.tar.gz
Distribution    PNE/Lingua-Klingon-Segment-1.03.tar.gz
Module    DateTime::Event::Klingon (JALDHAR/DateTime-Event-Klingon-1.0.1.tar.gz)

Module    Lingua::Klingon::Collate (PNE/Lingua-Klingon-Collate-1.03.tar.gz)
Module    Lingua::Klingon::Recode (PNE/Lingua-Klingon-Recode-1.02.tar.gz)
Module    Lingua::Klingon::Segment (PNE/Lingua-Klingon-Segment-1.03.tar.gz)
8 items found

Once you're happy with the module name, you can check if it's installed or not using m

cpan[4]> m Lingua::Klingon::Collate
Module id = Lingua::Klingon::Collate
    CPAN_USERID  PNE (Philip Newton
)
    CPAN_VERSION 1.03
    CPAN_FILE    P/PN/PNE/Lingua-Klingon-Collate-1.03.tar.gz
    INST_FILE    (not installed)

Install it with install command.

cpan[5] install Lingua::Klingon::Collate

Now, unless you already have module Test::Differences installed, Lingua::Klingon::Collate will fail with a dependency error. Not all modules are like that. Some are coded in a way that CPAN actually asks you if you want to follow and install dependencies automagically. Those are a cinch to install.

If something goes wrong, look at the output of the installation. Best case scenario, you're just missing another module and it didn't warn you about it. For example, after Lingua::Klingon::Collate failed with the warning that I should have Test::Differences installed, I tried to install that dependency directly. It also failed. When looking at the output on the screen, I see a bunch of lines like this:

t/regression..........Can't locate Text/Diff.pm in @INC (@INC contains: /home/vi
alves/.cpan/build/Test-Differences-0.4801-lI_xia/blib/lib /home/vialves/.cpan/bu
ild/Test-Differences-0.4801-lI_xia/blib/arch /usr/lib/perl5/5.10/i686-cygwin /us
r/lib/perl5/5.10 /usr/lib/perl5/site_perl/5.10/i686-cygwin /usr/lib/perl5/site_p
erl/5.10 /usr/lib/perl5/vendor_perl/5.10/i686-cygwin /usr/lib/perl5/vendor_perl/
5.10 /usr/lib/perl5/vendor_perl/5.10 /usr/lib/perl5/site_perl/5.8 /usr/lib/perl5
/vendor_perl/5.8 .) at /home/vialves/.cpan/build/Test-Differences-0.4801-lI_xia/
blib/lib/Test/Differences.pm line 213.

You'll notice that the error is the same as when we did the $ perl -Maaaa -e 'print "OK"'. Module Text/Diff.pm (or namely Text::Diff) is not installed. So we now go on that quest of following dependencies by hand.

It so happens that Test::Differences tried to install Text::Diff, but Text::Diff failed during the test phase. Suppose I know that those failed tests are not important and won't hinder the results of the rest (I don't, but bare with me), I can force CPAN to disregard the test failures:

cpan[6]> force install Text::Diff
Running install for module 'Text::Diff'
Running make for R/RB/RBS/Text-Diff-0.35.tar.gz
  Has already been unwrapped into directory /home/vialves/.cpan/build/Text-Diff-
0.35-B8Qsuo
  Has already been made
Running make test
/usr/bin/perl.exe "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'b
lib/arch')" t/*.t
t/ext_format......ok
t/general.........Use of /g modifier is meaningless in split at t/general.t line
 129.
Use of /g modifier is meaningless in split at t/general.t line 130.
t/general.........ok
t/inputs..........ok
t/keygen..........ok
t/outputs.........1/8 No such file or directory at t/outputs.t line 12, 
line 6.
t/outputs......... Dubious, test returned 2 (wstat 512, 0x200)
 Failed 4/8 subtests
t/table...........ok

Test Summary Report
-------------------
t/outputs.t   (Wstat: 512 Tests: 4 Failed: 0)
  Non-zero exit status: 2
  Parse errors: Bad plan.  You planned 8 tests but ran 4.
Files=6, Tests=29,  1 wallclock secs ( 0.04 usr  0.04 sys +  0.59 cusr  0.26 csy
s =  0.93 CPU)
Result: FAIL
Failed 1/6 test programs. 0/29 subtests failed.
make: *** [test_dynamic] Error 255
  RBS/Text-Diff-0.35.tar.gz
  /usr/bin/make test -- NOT OK
//hint// to see the cpan-testers results for installing this module, try:
  reports RBS/Text-Diff-0.35.tar.gz
Running make install
Installing /usr/lib/perl5/site_perl/5.10/Text/Diff.pm
Installing /usr/lib/perl5/site_perl/5.10/Text/Diff/Table.pm
Writing /usr/lib/perl5/site_perl/5.10/i686-cygwin/auto/Text/Diff/.packlist
Appending installation info to /usr/lib/perl5/5.10/i686-cygwin/perllocal.pod
  RBS/Text-Diff-0.35.tar.gz
  /usr/bin/make install  -- OK
Failed during this command:
 RBS/Text-Diff-0.35.tar.gz                    : make_test FAILED but failure ign
ored because 'force' in effect

Now I can go on to installing Text::Differences and finally Lingua::Klingon::Collate.

cpan[7] install Test::Differences
.... some output here ....
Appending installation info to /usr/lib/perl5/5.10/i686-cygwin/perllocal.pod
  OVID/Test-Differences-0.4801.tar.gz
  /usr/bin/make install  -- OK

cpan[8] install Lingua::Klingon::Collate
Running install for module 'Lingua::Klingon::Collate'
Running Build for P/PN/PNE/Lingua-Klingon-Collate-1.03.tar.gz
  Has already been unwrapped into directory /home/vialves/.cpan/build/Lingua-Kli
ngon-Collate-1.03-MX7FZn
  -- No Build created, won't make
Running Build test
  Make had some problems, won't test
Running Build install
  Make had some problems, won't install

Ok, that happens. It's because of the previous bad attempt. We can bypass that by telling CPAN to restart the build from scratch. To do so, it must first clean the build environment for that module.


cpan[9] clean Lingua::Klingon::Collate

That usually does the trick and enables you to run the install again, but if it doesn't, you can force get Lingua::Klingon::Collate to get a fresh package.

cpan[27]> install Lingua::Klingon::Collate
Running install for module 'Lingua::Klingon::Collate'
Running Build for P/PN/PNE/Lingua-Klingon-Collate-1.03.tar.gz
  Has already been unwrapped into directory /home/vialves/.cpan/build/Lingua-Kli
ngon-Collate-1.03-ZA4a1s

  CPAN.pm: Going to build P/PN/PNE/Lingua-Klingon-Collate-1.03.tar.gz

Checking whether your kit is complete...
Looks good

Checking prerequisites...
Looks good

Creating new 'Build' script for 'Lingua-Klingon-Collate' version '1.03'
Copying lib/Lingua/Klingon/Collate.pm -> blib/lib/Lingua/Klingon/Collate.pm
Manifying blib/lib/Lingua/Klingon/Collate.pm -> blib/libdoc/Lingua.Klingon.Colla
te.3pm
HTMLifying blib/lib/Lingua/Klingon/Collate.pm -> blib/libhtml/site/lib/Lingua/Kl
ingon/Collate.html
./Build: blib/lib/Lingua/Klingon/Collate.pm: cannot resolve L in par
agraph 60.
./Build: blib/lib/Lingua/Klingon/Collate.pm: cannot resolve L in par
agraph 60.
  PNE/Lingua-Klingon-Collate-1.03.tar.gz
  ./Build -- OK
Running Build test
t/01_base...........ok
t/02_strcoll........ok
t/03_strxfrm........ok
t/04_strunxfrm......ok
t/05_list...........ok
All tests successful.
Files=5, Tests=87,  1 wallclock secs ( 0.05 usr  0.03 sys +  0.52 cusr  0.25 csy
s =  0.85 CPU)
Result: PASS
  PNE/Lingua-Klingon-Collate-1.03.tar.gz
  ./Build test -- OK
Running Build install
Prepending /home/vialves/.cpan/build/Lingua-Klingon-Collate-1.03-ZA4a1s/blib/arc
h /home/vialves/.cpan/build/Lingua-Klingon-Collate-1.03-ZA4a1s/blib/lib to PERL5
LIB for 'install'
Installing /usr/lib/perl5/site_perl/5.10/Lingua/Klingon/Collate.pm
Installing /usr/share/man/man3/Lingua.Klingon.Collate.3pm
Installing /usr/share/doc/perl-5.10.0/html/html3/site/lib/Lingua/Klingon/Collate
.html
Writing /usr/lib/perl5/site_perl/5.10/i686-cygwin/auto/Lingua/Klingon/Collate/.p
acklist
  PNE/Lingua-Klingon-Collate-1.03.tar.gz
  ./Build install  -- OK

Type q to exit the shell and that's it! There are many other options when using CPAN, but what you've seen so far in this post should be enough to get you started. Just remember to use ? every now and then to see what powers are offered to you.

]]> http://usestrict.net/2009/05/23/perl-crash-course-pragmas-perl-modules-and-cpan/feed/ 2 Perl Crash Course: Basic I/O http://usestrict.net/2009/04/16/perl-crash-course-basic-io/ http://usestrict.net/2009/04/16/perl-crash-course-basic-io/#comments Thu, 16 Apr 2009 14:08:39 +0000 Vinny http://usestrict.net/?p=443 author: Valeria Paixão
revision: André Batosti
enhancement/translation: Vinny Alves

Note from UseStrict: Some of the examples in this tutorial were borrowed from Randall Schwartz's Learning Perl. It's a book that EVERY beginner Perl programmer should have. If you don't have a hardcopy, please consider getting one. You can find it here: Learning Perl, 5th Edition

 

In this article, you will learn how to use basic I/O in Perl, learn about @ARGV, and become familiar with string formatting using printf.

STDIN

<STDIN> tells Perl to read from the standard input - usually the keyboard.

while (defined($_ = <STDIN>)) {
	print "I saw $_";  # echoes whatever is typed onto the screen.
                                # Quit with ^D or ^Z (depending on your system)
}

foreach (<STDIN>) {
        print "I saw $_"; # almost the same as above
}

The difference between the while and foreach loops above is that while executes its statements at every hit of the return key, while foreach slurps into memory all the input until eof (^D on Unixes) and only then executes its instructions.

It is important to note this difference if you don't want to crash your machine. If your input comes from a webserver with a 400MB log file, you're better off processing each line individually than slurping it all into memory.


Diamond Operator

Another option to read data input is through the diamond operator (aka angle operator). It is handy to make Perl programs function like Unix commands such as cat, sed, awk, sort, grep, lpr, etc.

From the command line:
program_name.pl <file1> <file2> <someglob*>

In program_name.pl

while (<>)  {
	chomp;
 	print "It was $_ that I saw!\n";
}

If you fail to give the program any parameters or pass it a dash (-), it will read from STDIN. The diamond operator will open each file and paste its contents into the special $_ variable. Just like any filehandle (more on filehandles here), you can slurp the input into an array in list context:

while ($line = <>)  { # populate $line instead of 	$_
	chomp($line);
 	print "It was $line that I saw!\n";
}
# or
@lines = <>;

for (@lines) {
     ... # do something with lines
}

If the diamond operator fails to open a passed file for reading, it will raise a warning and move on to the next file. It's interesting to say that the diamond operator is quite intelligent: depending on how you call your script, it will not think that the parameters are files. For example:

cat some_file.txt | ./script.pl

The above example will work just as if you had sent it some_file.txt as a parameter.

@ARGV

@ARGV stores all the command line parameters passed to your script during run time. It is the array that the diamond operator looks at when doing its thing. You can manually manipulate it inside the script, as you would any other array, and the results will be as if you had passed your changes through the command line.



STDOUT

STDOUT is the default output channel for Perl's print and printf functions.

print
print takes a list as parameters. The function returns 1 if successful and 0 otherwise. As we saw in our previous examples, the parentheses in the print function is not required. However, it may sometimes help you to get what you need.

Question1: what is the output for the snippet below?

       $result = print(2+3) * 4;
       print $result;

Check the bottom of this article for the answer.

If you want to print an array, as we saw in our Arrays and Lists post, you can get different results depending on which quotes (if any) you use.

print @colors;  # prints out 'redyellowblue'
print "@colors";  # prints out 'red yellow blue'
print '@colors';  # prints out '@colors' - no interpolation with single quotes, remember?

printf

printf allows you to have a better control of your output. It takes a format and the data to be formatted. Since printf is basically imported from C, and so many languages have adopted it, it accepts a wide range of popular patterns - even some deprecated ones:

%%	a percent sign
%c	a character with the given number
%s	a string
%d	a signed integer, in decimal
%u	an unsigned integer, in decimal
%o	an unsigned integer, in octal
%x	an unsigned integer, in hexadecimal
%e	a floating-point number, in scientific notation
%f	a floating-point number, in fixed decimal notation
%g	a floating-point number, in %e or %f notation

%X	like %x, but using upper-case letters
%E	like %e, but using an upper-case "E"
%G	like %g, but with an upper-case "E" (if applicable)
%b	an unsigned integer, in binary
%B	like %b, but using an upper-case "B" with the # flag
%p	a pointer (outputs the Perl value's address in hexadecimal)
%n	special: *stores* the number of characters output so far
        into the next variable in the parameter list

%i	a synonym for %d
%D	a synonym for %ld
%U	a synonym for %lu
%O	a synonym for %lo
%F	a synonym for %f

Each format can also take one or more of the following flags:

space	prefix positive number with a space
+	prefix positive number with a plus sign
-	left-justify within the field
0	use zeros, not spaces, to right-justify
#	ensure the leading "0" for any octal,
	prefix non-zero hexadecimal with "0x" or "0X",
	prefix non-zero binary with "0b" or "0B"

Some examples:

printf "%g %g %g\n", 5/2, 51/17, 51 ** 17;	# 2.5 3 1.0683e+29
printf "in %d days!\n", 17.85;				# in 17 days!
printf "%10s\n", "wilma";					# '     wilma'
printf "%-15s\n", "flintstone";				# 'flintstone     '
printf "%12f\n", 6 * 7 + 2/3;					# '   42.666667'
printf "%12.3f\n", 6 * 7 + 2/3;				# '      42.667'
printf "%12.0f\n", 6 * 7 + 2/3;				#  '          43'
printf “Monthly interest: %.2f%%\n", 5.25/12;	# Monthly interest:

Answer to Question1: if you guessed 54, then congrats! The print(2+3) sends 5 to the screen and returns 1, which gets multiplied by 4 and assigned into $result. The following print $result sends 4 to the screen. Since there are no newlines anywhere in the data being printed, both digits get placed in the same line on the screen.

« Control Structures | TOC | Subroutines »

]]> http://usestrict.net/2009/04/16/perl-crash-course-basic-io/feed/ 0 Perl Crash Course: File and Directory Tests and Manipulation http://usestrict.net/2009/04/15/perl-crash-course-file-and-directory-tests-and-manipulation/ http://usestrict.net/2009/04/15/perl-crash-course-file-and-directory-tests-and-manipulation/#comments Wed, 15 Apr 2009 15:15:55 +0000 Vinny http://usestrict.net/?p=429 by André Batosti
revision: Vinny alves

Opening Files

To read or write files in Perl, you need to open a filehandle. Filehandles in Perl are yet another kind of identifier.
They act as convenient references (handles, if you will) between your program and the operating system about a particular file. They contain information about how the file was opened and how far along you are in reading (or writing) the file. They also contain user-definable attributes about how the file is to be read or written.

To open a new file on system you need to create the filehandle for this file using the command open

open(filehandle, pathname);

The filehandle is the identifier that will describe the file and the pathname - the full path of the file you trying to open. Typically it is represented by a constant, but when working with complex programs, it is best to use a scalar variable in order to safely pass it from one subroutine or method to another.

Example:

	open(PASS_FILE, "/etc/passwd"); # equivalent to open(PASS_FILE,"< /etc/passwd");
	# or
	open(my $pass_file, "/etc/passwd");

Note: it's always a good idea to test if open() worked well:

	open(PASS_FILE,"/etc/passwd") || die("Can't open passwd: $!\n"); # $! gives the system error message

Any modern version of Perl also accepts the 3 parameter notion, which is safer:

	open(PASS_FILE, "<" , "/etc/passwd");

When you done all with the file you need to close then using the command close

        close(PASS_FILE);

Reading Files

You can read from Perl's filehandles in a couple of different ways. The most common method is to use the file input operator, also called the angle (or diamond) operator (<>). To read a filehandle, simply put the filehandle name inside the angle operator and assign the value to a variable:

	open(MYFILE, "myfile");
	$line = <MYFILE>;

The angle operator in a scalar context reads one line of input from the file. When called after the entire file has been read, the angle operator returns the value undef.

In array context, the whole file is read and stored in the array - one line per element;

Writing Files

To write data into a file you need to open the filehandle to write. The open command before is for reading only. To open for writing we need to use '>' mode to create a new file or overwrite an existing one and '>>' mode to append an existing file or create a new one.

	open(NEWFILE, ">newfile"); # overwrites or creates newfile
	open(MYFILE, ">>myfile"); # appends data to myfile

After open a file to write you can use the print command with the filehandle:

	print NEWFILE "this goes into newfile\n"; # note the lack of commas between print and filehandle

More on Modes

So far we've seen modes '<', '>', and '>>'. There are other modes we can use depending on our needs. Adding a '+' before '>' or '<' (making '+>' or '+<') will grant us read AND write access to the file. '+>', however, will truncate your file first.

Perl also allows you to use pipes on your filenames, print to anonymous temporary files, and much much more.
See perldoc's open for more details.

Special filehandles

The Perl have some special filehandles that was always open this is for standard input, output and error
They are

  • STDOUT - The standard output
  • STDIN - The standart input
  • STDERR - The error standard output

You can specify which is your default output handle (the one print sends data to when no filehandle is passed) with
the select() function:

	open(FH,"> myfile.txt);
	$old_fh = select(FH); # changes default output and saves original
	print "This goes to myfile.txt\n";
	close(FH);
	select($old_fh);
	print "This goes to the screen\n";





File test operators

Before you open a file, sometimes it's nice to know whether the file exists, whether the file is really a directory, or whether opening the file will give a permission denied error. If you could examine the file's metadata, you could get answers to these questions. For these situations, Perl provides the file test operators. The file test operators all have the following syntax

-X filehandle

OR

-X pathname

The valid operators for file tests are:

  • -r Returns true if the file is readable
  • -w Returns true if the file is writeable
  • -e Returns true if the file exists
  • -z Returns true if the file exists but is empty
  • -s Returns size of the file in bytes if it exists
  • -f Returns true if the file is a regular file rather than a directory
  • -d Returns true if the file is a directory
  • -T Returns true if the file appears to be a text file
  • -B Returns true if the file appears to be a binary file
  • -M Returns the age (in days) since the file was modified

Working with directories

The first step in obtaining directory information from your system is to create a directory handle. A directory handle is something like a filehandle, except that instead of a file's contents, you read the contents of a directory through the directory handle. To open a directory handle, you use the opendir() function:

        opendir(dirhandle, pathname);

To get the content of the directory you need to use the readdir() function to get the next directory entry or the entire list of files, depending on context (scalar or list, respectively). It returns undef once you reach the end of your list.

	$next_file = readdir(dirhandle);
       # OR
	@files = readdir(dirhandle);

After you finish you need to close the directory using the function closedir()

	closedir(dirhandle);

A shortcut to this process is to use file globbing techniques:

	@all_files = <*>; # gets you the listing of all files in the current directory
	@shell_scripts = glob("*.sh"); # safer

Basic directory operations

If you need to change, create and remove directories you can use the functions chdir(), mkdir and rmdir using this syntax:

	chdir pathname; # to change remote dir
	mkdir pathname; # to create a directory
	rmdir pathname; # to remove the entire directory

Basic file operations

To remove files you need to use the unlink() function and to rename, use rename() :) 

	unlink list_of_files;
	rename oldname, newname;

The stat Function

If you need to get all information about a file you need to use the stat() function.

It returns an array containing the following:

0 dev     Device number
1 ino     Inode number
2 mode    File's mode (permissions)
3 nlink   Number of links
4 uid     User ID (UID) of the owner
5 gid     Group ID (GID) of the owner
6 rdev    Special file info
7 size    Size of file in bytes
8 atime   Time of last access
9 mtime   Time of last modification
10 ctime  Inode change time
11 blksz  Disk block size
12 blocks Number of blocks in file

        ($dev,  $ino,   $mode,  $nlink, $uid,     $gid,   $rdev,
         $size, $atime, $mtime, $ctime, $blksize, $blocks) = stat(pathname);

« Subroutines | TOC | Some built-in functions for everyday use »

]]> http://usestrict.net/2009/04/15/perl-crash-course-file-and-directory-tests-and-manipulation/feed/ 0 Perl Crash Course: Control Structures http://usestrict.net/2009/04/14/perl-crash-course-control-structures/ http://usestrict.net/2009/04/14/perl-crash-course-control-structures/#comments Wed, 15 Apr 2009 03:06:22 +0000 Vinny http://usestrict.net/?p=409 by André Batosti
revision: Fernando Giorgetti and Vinny Alves

Control Structures are used to control the flow of a program. We are going to see programs that can iterate (loop) or make decisions (conditionals) based on the state of variables.

More interesting possibilities arise when we introduce control structures and looping. Perl supports lots of different kinds of control structures which tend to be like those in C, but are very similar to Pascal, too.


Conditionals

The simplest control structures are the if and unless statements. Its syntax basically follows the rule below:

if ( expression ) {
    instruction1;
    instruction2;
}

Value obtained from "expression", determines whether it will be considered as true or false. Table below helps you to identify the meaning of values based on how Perl interprets expressions.

Expression String/Number? Boolean value
0 number false
0.0 number false
0.0000 number false
"" or '' string false
"0" string false
"0.0" string true
undef N/A false
42 - (6 * 7) number false
"0.0" + 0.0 number false
"0E0" number true
0E0 number false
"foo" string true

Everything in Perl is true, except:

  • the strings "" (the empty string) and "0" (the string containing only the character, 0), or any string expression that evaluates to either "" (the empty string) or "0".
  • any numeric expression that evaluates to a numeric 0.
  • any value that is not defined (i.e., equivalent to undef).

A very simple example of a real "if statement" would be:

if ( $name ) {
    print "Non empty string";
}
else {
    print "Empty string";
}

Another example, to illustrate the syntax of the "if" statement:

if ( $money <= 0 ) {
    print "You are out of money";
}
elsif ( $money > 1000000 ) {
    print "You are a millionaire";
}
else {
    print "You are probably better than I am";
}

Blocks of code in Perl are enclosed by curly braces {   }. They are always required inside a control structure (except for cases where we use idiomatic ifs - see more below).

Two things to mention:

1 - elsif spelling;
2 - both elsif and else statements are optional.

Another conditional statement we need to mention is the unless. Its meaning is the opposite of an if statement. It will only execute a block of code if expression is NOT true.

Let's look at an example:

if  ( $num == 10 ) { #Usual if statement
    print "Number is equal to 10";
} else {
    print "Number not equal to 10";
}

# Unless statement that will execute the block of code below
# if number's value is not 10 (same thing as the else statement above)

unless ( $num == 10 ) {
    print "Number not equal to 10";
}
else {
    print "Number is equal to 10";
}

Action based on statement test

You can test a single statement and perform a pre-defined action in case it is true or another one if statement is false.

For example:

    print "You are a man" if ($gender eq "M"); # parentheses are optional here, since only one expression is evaluated





Ternary or Trinary Operator

Like many other programming languages, Perl offers the ternary (aka trinary) operator "?:" which behaves just like if/elsif/else: 

print "You are a " . (($gender eq "M") ? "man" : "woman") . "\n";
# read: if $gender eq "M" then "man" else "woman"

# or perhaps
($birthday == $today) ? print "Happy birthday!" : print "Have a good day";

Mimmic elsif this way:

$month = (localtime)[4]; # fifth element of localtime in list context returns month, base 0;

print "The month is now " .
($month == 0) ? "January" :
($month == 1) ? "February" :
($month == 3) ? "March" :
                      "something other than Jan, Feb, or March";

Operators

Initially we will introduce the testing operators. They rely on a test being considered as true or false. Following you can see a few examples:

$a == $b                      # Is $a numerically equal to $b?
# Beware: Don't use the = operator.
$a != $b                       # Is $a numerically unequal to $b?
$a eq $b                       # Is $a string-equal to $b?
$a ne $b                       # Is $a string-unequal to $b?

Perl also supports the logical operators "OR" and "AND". The "OR" operator is represented by the "||" characters while the "AND" is represented by "&&" characters.

Lets go through a few examples:

# If value of number is greater than 10 or lesser than 5
if ( ( $num > 10 ) || ( $num < 5 ) ) {
    ...
}

# If language is equal to "perl" and num is 10, or, if num is 0 (or has a "false" value)
if ( (( $language eq "perl" ) && ( $num == 10 )) || !( $num ) ) {
    ...
}

Loops

We have several statements to perform iterations using Perl. Perl supports: while, until, do ... while, for and foreach statements.

While

Performs the iteration while the given expression is true. Syntax:

while (expression) {
    instruction1;
    instruction2;
    ...
}

Until

It is the opposite of while. So take it as, in example:

while (!(expression)) {
    instruction1;
    instruction2;
}

#same as
until (expression) {
   instruction1;
   instruction2;
}

Do .. While

Its functionality is almost the same as for the normal "while" statement, but with this kind of control structure you guarantee that the block of code will be executed at least once.

do {
    instruction1;
    instruction2;
} while (expression);

For

It's a C like style. Syntax:

for (initialize; expression; increment/decrement) {
    instruction1;
    instruction2;
    ...
}
  • The initialize statement is basically used to initialize the variable that will be used on the test statement. It can also be used anywhere in the block. Declare it with my to keep it private to the for block;
  • Expression follows the same rule for if or while. It will be tested, and if its return is true, block code will be executed;
  • Increment/decrement is then executed

Real example:

for ($i = 0; $i < 10; $i++ ) {
    printf "Iteration number: %s\n", $i;
}

Another good example (not using a numeric expression):

for ( @names = ('John', 'Paul', 'Roger', 'David'); @names; shift(@names) ) {
    print "$names[0]\n";
}

# Or work on 2 expressions at the same time
for ($i = 0, $j = 10; $i<=10,$j>=0; $i++, $j--) {
    print "$i    $j\n";
}

# Or kick off an infinite loop
for ( ; ; ) {
    instruction1;
    instruction2;
}

Foreach

It is another control structure that allows iteration, but here you can iterate over an array.

Following you can see a few examples of it:

@names = ('John', 'Paul', 'Roger', 'David');

#Passing through all names
foreach $name (@names) {
    printf "%s\n", $name;
}

The scalar variable is optional to the foreach statement. So you can bypass it by doing:

@names = ('John', 'Paul', 'Roger', 'David');

#Passing through all names
foreach (@names) {
    printf "%s\n", $_;
}

If you don't specify a scalar variable to get each iteration, then the default scalar ($_) will be used. Note that for can also be used as a synonym to foreach - same exact functionality, but without the each portion of the word.

Loop flow control

We have a few special commands to help us controlling the flow under a loop statement. Here we go.

last

It is similar to the "break" statement used on C programming language (as well as in several others). Tells Perl to skip the current loop statement, or even, to skip a labeled loop statement. Take a look at the example below:

@names = ('fernando', 'smith', 'Mark', 'John');
foreach (@names) {
    last if $_ eq "John";
    print "Name is " . $_ . "\n";
}

next

next has the same function as the "continue" loop statement used in C. Used to force your loop statement to go to the next iteration. In example:

@names = ('fernando', 'smith', 'Mark', 'John');
foreach (@names) {
    next if $_ eq "smith";
    print "Name is " . $_ . "\n";
}

The example above will just force the "foreach" statement to read next iteration once "smith" is the value read from the "names" array.

If you need to work on multiple levels of loops, you can tell next and last which named block to work on:

LINE: while () {
	next LINE if /^#/;	# discard comments
	...
}

« Basic Regular Expressions | TOC | Basic I/O »

]]> http://usestrict.net/2009/04/14/perl-crash-course-control-structures/feed/ 0 Perl Crash Course: Basic Regular Expressions http://usestrict.net/2009/04/07/perl-crash-course-basic-regular-expressions/ http://usestrict.net/2009/04/07/perl-crash-course-basic-regular-expressions/#comments Tue, 07 Apr 2009 14:06:43 +0000 Vinny http://usestrict.net/?p=384 by Nadia Kozievich
revision: Vinny Alves

Introduction

A regular expression (or regex) is a simple, rather mindless way of matching a series of symbols to a pattern you have in mind. The origins of regular expressions lie in automata theory and formal language theory, both of which are part of theoretical computer science.

In computing, regular expressions provide a concise and flexible means for identifying strings of text of interest, such as particular characters, words, or patterns of characters. Regular expressions (abbreviated as regex or regexp, with plural forms regexes, regexps, or regexen) are written in a formal language that can be interpreted by a regular expression processor, a program that either serves as a parser generator or examines text and identifies parts that match the provided specification.

Regular expressions are used by many text editors, utilities, and programming languages to search and manipulate text based on patterns. For example, Perl, Ruby and Tcl have a powerful regular expression engine built directly into their syntax. Several utilities provided by Unix distributions including the editor ed and the filter grep where the first to popularize the concept of regular expressions.


What They Are

Regular expressions are a syntax, implemented in Perl and certain other environments, making it not only possible but easy to do some of the following:

#Complex string comparisons
$string =~ m/log2008/; # m before the first slash is the "match" operator, not required if using / / as delimiters

#Complex string selections
$string =~ m/log(date)txt/;
$date = $1;

#Complex string replacements
$string =~ tr/originaltext/newtext/; # tr before first slash is "translate" operator.

Perl's regular expression syntax is actually a derivation of the POSIX implementation, resulting in considerable similarities between the two. Let's start with a simple example of a Perl-based regular expression:

/food/

Notice that the string food is enclosed between two forward slashes. Just as with POSIX regular expressions, you can build a more complex string through the use of quantifiers:

/fo+/

This will match fo followed by one or more characters. Some potential matches include food, fool, and fo4. Here is another example of using a quantifier:

/fo{2,4}/

This matches f followed by two to four occurrences of o. Some potential matches include fool, fooool, and foosball.

Note that you can use just about anything you want as a delimiter. If using / /, then the leading m (like m/ /) is not required. Here are some more examples:

m| |;

m[ ];

m{ };

Doing String Comparisons

We start with string comparisons because they're the easiest, and yet most of what's contained here is applicable in selecting and replacing text.

Quantifiers

If you want to look for strings containing one or more instances of the letter p, strings containing at least two p's, or even strings with the letter p as their beginning or ending character. Here are several examples of these characters:

  • p+ matches any string containing at least one p.
  • p* matches any string containing zero or more p's.
  • p? matches any string containing zero or one p.
  • p{2} matches any string containing a sequence of two p's.
  • p{2,3} matches any string containing a sequence of two or three p's.
  • p{2,} matches any string containing a sequence of at least two p's.
  • p$ matches any string with p at the end of it.
  • ^p matches any string beginning with p.

Now for some examples:

$string =~ m/\s*rem/i;   # true text contains 0 or more spaces followed by rem or REM.
                                  # The trailing i specifies case insensitivity
$string =~ m/^\S{1,8}\.\S{0,3}/;   # check for DOS 8.3 filename.
                                                 # \S means non-space characters. More on that later.

Simple String Comparisons

The most basic string comparison is 

$string =~ m/log2008/;

The above returns true if string $string contains substring "log2008", false otherwise. If you want only those strings where the log2008 appears at the very beginning, you could write the following: $string =~ m/^log2008/;

Similarly, the $ operator indicates "end of string". If you wanted to find out if the sought text was the very last text in the string, you could write this: 

$string =~ m/log2008$/;

Now, if you want the comparison to be true only if $string contains log2008 and nothing but log2008, simply anchor it like this: 

$string =~ m/^log2008$/;

Now what if you want the comparison to be case insensitive? All you do is add the letter i after the ending delimiter: 

$string =~ m/^log2008$/i;


Using Simple "Wildcards" and "Repetitions"

Calling these "wildcards" may actually conflict with the theoretical grammar and syntax of Perl, but in fact is the most intuitive way to think of it, and will not lead to any coding mistakes.

.   Match any character (except newline)
\w  Match "word" character (alphanumeric plus "_")
\W  Match non-word character
\s  Match whitespace character (spaces, tabs, form-feeds, etc)
\S  Match non-whitespace character
\d  Match digit character
\D  Match non-digit character
\t  Match tab
\n  Match newline
\r  Match carriage return
\f  Match form-feed
\a  Match alarm (bell, beep, etc)
\e  Match escape
\b  Match word boundary
\B  Match non-boundary
\021  Match octal char ( in this case 21 octal)
\xf0  Match hex char ( in this case f0 hexidecimal)





Using Groups ( ) in Matching

Groups are regular expression characters surrounded by parentheses. They have two major uses:

  • To allow alternative phrases as in /(log2008|log2009|log2007)/i.
  • As a means of retrieving selected text in selection, translation and substitution.

Powerful regular expressions can be made with groups At its simplest, you can match either all lowercase or name case like this:

if ($string =~ m/(L|l)og (day|month).txt/){
        print "Found the log description!\n";
}

Detect all strings containing vowels 

if ($string =~ m/(A|E|I|O|U|a|e|i|o|u)/){
        print "Vowels!\n";
}

Detect if the line starts with any of three Brazilian presidents: 

if ($string =~ m/^(Lula|Itamar|Sarney)/i){
        print "$string\n"
};

Using Character Classes [ ]

Character classes have three main advantages:

Shorthand notation, as [AEIOUY] instead of (A|E|I|O|U|Y).

Character Ranges, such as [A-Z].

One to one mapping from one class to another, as in tr/[a-z]/[A-Z].

An uparrow (^) immediately following the opening square bracket means "Anything but these characters", and effectively negates the character class. For instance, to match anything that is not a vowel, do this:

if ($string =~ /[^AEIOUYaeiou]/){
        print "This string contains a non-vowel";
}

Contrast to this: 

if ($string !~ /[AEIOUYaeiou]/){
        print "This string contains no vowels at all";
}

Print all people whose name begins with A through E 

if ($string =~ m/^[A-E]/){
        print "$string\n";
}


Matching: Putting it All Together

Print everyone whose last name is Lula, Itamar or Sarney. Each element of list is first name, blank, last name, and possibly more blanks and more info after the last name. 

if ($string =~ m/^\S+\s+(Lula|Itamar|Sarney)/i){
        print "$string\n"
};

Print every line with a valid phone number. 

if ($string =~ m/[\)\s\-]\d{3}-\d{4}[\s\.\,\?]/){
          print "Phone line: $string\n";
}

Symbol Explanations

char	meaning
=~	find a string that matches
!~	find a string that doesn't match
^	beginning of string
$	end of string
.	any character except newline
*	match 0 or more times
+	match 1 or more times
?	match 0 or 1 times; or: shortest match
|	alternative
( )	grouping; “storing”
[ ]	set of characters
{ }	repetition modifier
\	escape character


Examples

expression	matches...
abc		abc (that exact character sequence, but anywhere in the string)
^abc		abc at the beginning of the string
abc$		abc at the end of the string
a|b		either of a and b
^abc|abc$	the string abc at the beginning or at the end of the string
ab{2,4}c	an a followed by two, three or four b’s followed by a c
ab{2,}c		an a followed by at least two b’s followed by a c
ab*c		an a followed by any number (zero or more) of b’s followed by a c
ab+c		an a followed by one or more b’s followed by a c
ab?c		an a followed by an optional b followed by a c; that is, either abc or ac
a.c		an a followed by any single character (not newline) followed by a c
a\.c		a.c exactly
[abc]		any one of a, b and c
[Aa]bc		either of Abc and abc
[abc]+		any (nonempty) string of a’s, b’s and c’s (such as a, abba, acbabcacaa)
[^abc]+		any (nonempty) string which does not contain any of a, b and c (such as defg)
\d\d		any two decimal digits, such as 42; same as \d{2}
\w+		a “word”: a nonempty sequence of alphanumeric characters and low lines (underscores), such as foo and 12bar8 and foo_1
100\s*mk	the strings 100 and mk optionally separated by any amount of white space (spaces, tabs, newlines)
abc\b		when followed by a word boundary (e.g. in abc! but not in abcd)
perl\B		when not followed by a word boundary (e.g. in perlert but not in perl stuff)





« Gettin' jiggy wit it | TOC | Control Structures »

]]> http://usestrict.net/2009/04/07/perl-crash-course-basic-regular-expressions/feed/ 3