PHP-ist: All about learning PHP, frameworks, tips and tricks

PHP4, PHP5, cakephp, codeigniter, code example (scriptlets / snipplets) and many easy and effective way to learn

PHP5: Screen scraping with DOM and XPath

Posted in php5 by PHP-ist on the March 6th, 2008

This tutorial is continuation from previous yahoo screen-scraping using PHP4 tutorial.
We will try different method using DOM and XPath which only supported in PHP5.

First, a bit knowledge of XPath is required. More about XPATH can be read on:
http://www.zvon.org/xxl/XPathTutorial/General/examples.html

Also there's small concern that using XPATH is a bit slower than pure DOM Traversal. Read Speed: DOM traversal vs. XPath in PHP 5
But i personally also think that XPath is neat and easier.

Let's start. First we diagnose document structure using Mozilla Firebug.
Try a very easy case, which is to grab the title "Top Movies":

Copy XPath using Firebug and get this query:

/html/body/center/table[8]/tbody/tr/td[5]/table[4]/tbody/tr/td/font/b

  1. Firefox automatically fix broken html structure, and it also add tbody tag. So, we need to remove this tag.
  2. Only grab first row of table. Change .../tr/td/font/b into .../tr[1]/td/font/b

Now we get our first XPath query:

/html/body/center/table[8]/tr/td[5]/table[4]/tr[1]/td/font

Next harder case is to grab contents.

XPath query from Firebug is:

/html/body/center/table[8]/tbody/tr/td[5]/table[4]/tbody/tr[2]/td[2]/a/font/b

  1. Same problem here. Firefox automatically fix broken html structure, and it also add tbody tag. Remove tbody tag from XPath query.
  2. Grab all row of table. Change .../tr[2]/td[2]/a/font/b into .../tr/td[2]/a/font/b

Final XPath query for content is:

/html/body/center/table[8]/tr/td[5]/table[4]/tr/td[2]/a/font/b

Now final step is to put all two XPath queries into few lines of code, and we're done:

PLAIN TEXT
PHP:
  1. <?php
  2.     error_reporting(E_ERROR);// | E_WARNING | E_PARSE);
  3.     include ('Snoopy.class.php');
  4.    
  5.     $snooper = new Snoopy();
  6.     if ($snooper->fetch('http://movies.yahoo.com/mv/boxoffice/')) {
  7.         $dom = new DomDocument();
  8.         $dom->loadHTML($snooper->results);
  9.        
  10.         $x = new DomXPath($dom);
  11.  
  12.         //  /html/body/center/table[8]/tbody/tr/td[5]/table[4]/tbody/tr/td/font/b
  13.         $nodes = $x->query('/html/body/center/table[8]/tr/td[5]/table[4]/tr[1]/td/font/b');
  14.         echo $nodes->item(0)->nodeValue, "<br/>\n"; //Top Movies
  15.  
  16.         //  /html/body/center/table[8]/tbody/tr/td[5]/table[4]/tbody/tr[2]/td[2]/a/font/b
  17.         $nodes = $x->query('/html/body/center/table[8]/tr/td[5]/table[4]/tr/td[2]/a/font/b');
  18.         foreach ($nodes as $node) {
  19.             echo $node->nodeValue, "<br/>\n";
  20.         }
  21.     }
  22. ?>

Tags: php5, screen scraping, code example

2 Comments

CakePHP: How to add Search Engine Friendly (SEF) URLs

Posted in cakephp by PHP-ist on the February 18th, 2008

How many times you wondered how great it would be if your URLs didn't look so much like:
···http://www.server.com/posts/view/1058
But more like:
···http://www.server.com/posts/view/my_first_post

Read the rest on:
http://bakery.cakephp.org/articles/view/adding-friendly-urls-to-the-cake-blog-tutorial

0 Comments

PHP4: How to Steal from Yahoo! (another Web Screen Scraper)

Posted in php4 by PHP-ist on the February 11th, 2008

If you've been wondering on how to scrape some information from your favorite website. This web screen scraping techniques has already there since dinosaur age. Here's an example on how to do it in PHP 4. Thanks to:

  1. Mozilla Firefox and Firebug for spying and taking overall bird view so we could design a strategy.
  2. The good old snoopy-php project which will do his works as http client, snipping and sucking the whole page.
  3. And also phphtmlparser that make dirty HTML parsing work a lot more easy, taking down the enemy element by element.

First, our target operation is to grab a list of currently hot box office movies from Yahoo! Movies.
Target located: http://movies.yahoo.com/mv/boxoffice/ ..... locked on!!!

We need to see the HTML layout of the page, using Firefox and Firebug
getting html layout for screen scraping yahoo movies

Place your sight on the right side of screenshot.
The area we want to steal started with Top Movies and ended with Top Cast/Crew'.
Here's part of the code you'll see later.

PLAIN TEXT
PHP:
  1. ...
  2.             incIf($step1, 0, $parser->iNodeValue == 'Top Movies');
  3. ...
  4.             incIf($step1, 1, $parser->iNodeValue == 'Top Cast/Crew');
  5. ...

Now look at the bottom part. The full element tree starting from the root html tag, and ended in b tag.
xpath for screen firefox and firebug scraping yahoo movies
Decide that b < font < a < td should enough to distinct and separate the element from others.
So we will use a variable $step2 for digging. If $step2 == 0 and current element is td, we set $step2 into 1. If $step2 == 1 and current element is b, increment it. This $step2 continues to dig further to font and until we reach the treasure box in b tag. Finally, print out what inside treasure box and go up again to the surface, set $step2 to 0.

Now it's time for the full code guys.
If you're too lazy to copy paste it, just download the source code of web scraping tutorial (PHP 4).

PLAIN TEXT
PHP:
  1. <?
  2.     // Let's hire both experts
  3.     include ('Snoopy.class.php');
  4.     include ('htmlparser.inc');
  5.  
  6.     // Move and dig deeper
  7.     function incIf(&$step_counter, $current_step, $condition) {
  8.         if (($step_counter == $current_step) && ($condition))
  9.             $step_counter = $current_step+1;
  10.     }
  11.  
  12.     // If it's deep enough, take it and leave;
  13.     function doIf(&$step_counter, $current_step, $condition) {
  14.         if (($step_counter == $current_step) && ($condition)) {
  15.             $step_counter = 0;
  16.             return true;
  17.         }
  18.         return false;
  19.     }
  20.  
  21.     // C'mon snoopy suck that page
  22.     $snooper = new Snoopy();
  23.     if ($snooper->fetch('http://movies.yahoo.com/mv/boxoffice/')) {
  24.         // Pass the page to HtmlParser, and let him do his work
  25.         $parser = new HtmlParser ($snooper->results);
  26.         $step1 = 0; $step2 = 0;
  27.         echo "TODAY's BOX OFFICE\r\n<br/>";
  28.         while ($parser->parse()) {
  29.             incIf($step1, 0, $parser->iNodeValue == 'Top Movies');
  30.             incIf($step1, 1, $parser->iNodeValue == 'Top Cast/Crew');
  31.             if ($step1 == 1) {
  32.                 incIf($step2, 0, $parser->iNodeName == 'TD');
  33.                 incIf($step2, 1, $parser->iNodeName == 'A');
  34.                 incIf($step2, 2, $parser->iNodeName == 'FONT');
  35.                 incIf($step2, 3, $parser->iNodeName == 'B');
  36.                 if (doIf($step2, 4, $parser->iNodeType == NODE_TYPE_TEXT))
  37.                     echo $parser->iNodeValue."\r\n<br/>";
  38.             }
  39.         }
  40.     }
  41. ?>

Tags: php4, screen scraping, code example

0 Comments

Cake PHP: Simple Contact Form Tutorial!

Posted in cakephp by PHP-ist on the February 8th, 2008

This guide provide a quick tutorial covering simple contact form for CakePHP v1.2, based on this post
Download code example here! CakePHP 1.2 Simple Contact Form.zip

1. Create the VIEW in cake/app/views/contact/contact.ctp

PLAIN TEXT
PHP:
  1. <?php if (isset($success)) { ?>
  2.     <p><?= $success?></p>
  3. <?php } else { ?>
  4.     <?php if (isset($error)) { ?>
  5.         <p><?= $error?></p>
  6.     <?}?>
  7.     <?php echo $form->create(null,array('action' => 'contact', 'enctype' => 'multipart/form-data')); ?>
  8.     <p>
  9.         <?php echo $form->label('Page.name','Name: ',null); ?>
  10.         <?php echo $form->text('Page.name', array('size' => '25'))?>
  11.     </p>
  12.     <p>
  13.         <?php echo $form->label('Page.email','Email: ',null); ?>
  14.         <?php echo $form->text('Page.email', array('size' => '25'))?>
  15.     </p>
  16.     <p>
  17.         <?php echo $form->label('Page.message','Message: ',null); ?>
  18.         <?php echo $form->textarea('Page.message', array('cols' => '40', 'rows' => '10'))?>
  19.     </p>
  20.     <p>
  21.         <?php echo $form->submit('Send email')?>
  22.     </p>
  23.     <p>
  24.         All fields are required!
  25.     </p>
  26.     <?php echo $form->end();?>
  27. <?php }?>

2. Create Email COMPONENT, cake/app/controllers/components/email.php

PLAIN TEXT
PHP:
  1. <?php
  2. class EmailComponent extends Object {
  3.     function email($email, $name, $message) {
  4.         $f_mail = "name@domain.com";
  5.         $f_name = "WebMaster domain.com";
  6.         $f_message = "\nName : $name"; // start building the message
  7.         $f_message .= "\n\nE-mail : $email";
  8.         $f_message .= "\n\nMessage \n : $message";
  9.         $headers = "From: www.domain.com <administrator@domain.com>\n";
  10.         $headers .= "X-Sender: <admin @domain.com>\n";
  11.         $headers .= "X-Mailer: PHP\n";
  12.         $headers .= "Reply-To:$f_mail\n\n";
  13.         $subject = "Message from your website domain.com";
  14.         mail ( "$f_mail", "$subject", "$f_message", $headers ); // send the email
  15.     }
  16.    
  17.     // this function checks to see if the email supplied by the visitor is a valid one
  18.     function validMail($email) {
  19.         $pattern = "^[-_.[:alnum:]]+@((([[:alnum:]]|[[:alnum:]][[:alnum:]-]*[[:alnum:]])\.)+(ad|ae|aero|af|ag|ai|al|am|an|ao|aq|ar|arpa|as|at|au|aw|az|ba|bb|bd|be|bf|bg|bh|bi|biz|bj|bm|bn|bo|br|bs|bt|bv|bw|by|bz|ca|cc|cd|cf|cg|ch|ci|ck|cl|cm|cn|co|com|coop|cr|cs|cu|cv|cx|cy|cz|de|dj|dk|dm|do|dz|ec|edu|ee|eg|eh|er|es|et|eu|fi|fj|fk|fm|fo|fr|ga|gb|gd|ge|gf|gh|gi|gl|gm|gn|gov|gp|gq|gr|gs|gt|gu|gw|gy|hk|hm|hn|hr|ht|hu|id|ie|il|in|info|int|io|iq|ir|is|it|jm|jo|jp|ke|kg|kh|ki|km|kn|kp|kr|kw|ky|kz|la|lb|lc|li|lk|lr|ls|lt|lu|lv|ly|ma|mc|md|mg|mh|mil|mk|ml|mm|mn|mo|mp|mq|mr|ms|mt|mu|museum|mv|mw|mx|my|mz|na|name|nc|ne|net|nf|ng|ni|nl|no|np|nr|nt|nu|nz|om|org|pa|pe|pf|pg|ph|pk|pl|pm|pn|pr|pro|ps|pt|pw|py|qa|re|ro|ru|rw|sa|sb|sc|sd|se|sg|sh|si|sj|sk|sl|sm|sn|so|sr|st|su|sv|sy|sz|tc|td|tf|tg|th|tj|tk|tm|tn|to|tp|tr|tt|tv|tw|tz|ua|ug|uk|um|us|uy|uz|va|vc|ve|vg|vi|vn|vu|wf|ws|ye|yt|yu|za|zm|zw)|(([0-9][0-9]?|[0-1][0-9][0-9]|[2][0-4][0-9]|[2][5][0-5])\.){3}([0-9][0-9]?|[0-1][0-9][0-9]|[2][0-4][0-9]|[2][5][0-5]))$";
  20.         // the regex for an email addres
  21.         if (eregi ( $pattern, $email ))
  22.             $ll = 1; else
  23.             $ll = 0;
  24.         return $ll;
  25.     }
  26. }
  27. ?>

3. Create the CONTROLLER, cake/app/controllers/contact_controller.php

PLAIN TEXT
PHP:
  1. <?php
  2. class ContactController extends AppController {
  3.     var $uses = array ( );
  4.     var $components = array ('Email' );
  5.     var $name = 'Contact';
  6.    
  7.     function contact() {
  8.         $this->pageTitle = 'Contact';
  9.         if (! empty ( $this->data )) {
  10.             $email = $this->data ['Page'] ['email'];
  11.             $name = $this->data ['Page'] ['name'];
  12.             $message = $this->data ['Page'] ['message'];
  13.             $error = 0;
  14.             // start filter vars
  15.             if (strlen ( trim ( $email ) ) <3) {
  16.                 $error = 1;
  17.             }
  18.             if (strlen ( trim ( $message ) ) <4) {
  19.                 $error = 1;
  20.             }
  21.             // verify email address
  22.             if ((strlen ( trim ( $email ) ) <3) || ($this->Email->validMail ( $email ) == 0)) {
  23.                 $error = 1;
  24.             }
  25.             if ($error == 0) {
  26.                 $this->Email->email ( $email, $name, $message ); //here is the email sent
  27.                 $this->set ( 'success', 'The message was sent <br />Thank you!' );
  28.                 // setting "succes" variable for displaying if email was sent.
  29.             } else {
  30.                 $this->set ( 'error', 'Please complete all fields' );
  31.                 // this is in case of error
  32.             }
  33.         } else {
  34.         }
  35.     }
  36. }
  37. ?>

FINALLY
After all that just test it in your favorite web browser http://localhost/cake/contact/contact

Tags: php4, cakephp, code example

0 Comments

CakePHP: How To Make Controller Without a Model?

Posted in cakephp by PHP-ist on the February 8th, 2008
PLAIN TEXT
PHP:
  1. class MyController extends AppController
  2. {
  3.     // var $uses = null; works too, BUT MIGHT INTRODUCE BUGS
  4.     var $uses = array();
  5.  
  6.     function index()
  7.     {
  8.     }
  9. }

From: http://cakebaker.42dh.com/

Tags: php4, cakephp, code example

0 Comments

CakePHP: How to Add Theme Support?

Posted in cakephp by PHP-ist on the February 7th, 2008

Now with CakePHP v1.2 (beta) we can easily add theme with its ThemeView class.
This guide about how to add theme into fresh CakePHP installation will be explained in very basic step-by-step . I learn it frrom here.
Updated: sample code provided

Download code example, extract inside CakePHP root.

First!
Create a file called app_controller.php inside cake\app\controllers.
Create CakePHP AppController
Edit the file and just with two lines of code, add theme support on your application:

PLAIN TEXT
PHP:
  1. class AppController extends Controller {
  2.     var $view = 'Theme';
  3.     var $theme = 'default';
  4. }

Second!
You need to put all your theme collections inside a folder called themed instead views. This folder doesn't exists yet, so go to cake\app\views and create new folder themed inside it.
Create themed folder inside view

Third!
Create another folder called default inside cake\app\views\themed, and place all your template files there.
This theme name is taken from

PLAIN TEXT
PHP:
  1. var $theme = 'default';

We can just create another folder and activate it very easily thanks to ThemeView class.

That's it now you have an application that support theme. Easy?! Yea, that was sooo Cake :)

Tags: php4, cakephp, code example

2 Comments

CakePHP: How To Make Model Without a Table?

Posted in cakephp by PHP-ist on the February 6th, 2008
PLAIN TEXT
PHP:
  1. class ModelWithoutTable extends AppModel
  2. {
  3. var $useTable = false;
  4. }

From: http://cakebaker.42dh.com/

Tags: php4, cakephp, code example

0 Comments

Cannot find your answer here?
Feel free to get in touch and ask PHP-ist anything, just anything :)