Here is my version of Derick’s bash script. It is tuned for building on a mac or BSD. You many need to fiddle around with the compile flags to get it work on your system. There are also some hard coded paths which will need altering.

#!/bin/sh

VERSION=$1
DEBUG=$2
ZTS=$3
THIRTYTWO=$4
POSTFIX=
EXTRA_FLAGS=

if (test "${DEBUG}" = "nodebug"); then
	POSTFIX="$POSTFIX-nodebug"
else
	EXTRA_FLAGS="$EXTRA_FLAGS --enable-debug"
fi

if (test "${ZTS}" = "zts"); then
	EXTRA_FLAGS="$EXTRA_FLAGS --enable-maintainer-zts"
	POSTFIX="$POSTFIX-zts"
fi

if (test "${THIRTYTWO}" = "32bit"); then
	export CFLAGS="-m32"
	POSTFIX="$POSTFIX-32bit"
fi

PHP_DIR=/Users/nathanielmchugh/nmdev/php/php-src/
cd $PHP_DIR

SCRUBBED=`echo $VERSION | sed 's/[0-9.]*//'`
if (test "${SCRUBBED}" = "dev"); then
	BRANCH=`echo PHP-${VERSION} | sed 's/dev//'`
	git checkout $BRANCH
else
	TAG=`echo php-${VERSION} `
	git checkout $TAG
fi

echo "Building ${VERSION}${POSTFIX} with ($EXTRA_FLAGS)"

make clean
rm -rf configure
./vcsclean
./buildconf --force

if (test "${THIRTYTWO}" = "32bit"); then
	OPTIONS="--disable-all"
else
	OPTIONS="--with-gd --with-jpeg-dir=/opt/local --with-png-dir=/opt/local --with-xpm-dir=/opt/local --with-freetype-dir=/opt/local \
		--with-mysql=mysqlnd --enable-bcmath --with-gmp=/opt/local --with-readline=/opt/local \
		--with-openssl --with-curl --without-esmtp \
		--with-mysqli=mysqlnd --enable-pcntl \
		--enable-memory-limit --with-mcrypt=/opt/local \
		--with-iconv=/opt/local --enable-wddx  \
		--enable-spl --enable-pdo --with-pdo-mysql --with-pdo-sqlite \
		--with-ctype --with-bz2 --enable-mbstring --with-mime-magic \
		--with-xmlrpc --with-zlib --disable-zend-memory-manager --with-esmtp \
		--with-xsl --enable-exif --enable-soap --enable-ftp"
fi

./configure \
--prefix=/usr/local/php/${VERSION}${POSTFIX} ${EXTRA_FLAGS} ${OPTIONS}

make -j 5
make install

Heres an example of using it to build the stable version of php5.4

$sudo ./php-build.sh 5.4.0
$pe 5.4.0
$php - v
PHP 5.4.0 (cli) (built: Apr 20 2012 12:37:52) (DEBUG)
Copyright (c) 1997-2012 The PHP Group
Zend Engine v2.4.0, Copyright (c) 1998-2012 Zend Technologies

The traditional MySQL Query Cache

The MySQL query cache is a really nice simple feature which helps speed up repeated queries of the same data. A certain portion of memory can be set aside to hold results sets in a look table against the query string. When a SELECT query comes in which the MySQL server has seen before it looks in the query cache and if it has seen the query before it doesn’t touch the table data or indexes instead it simply returns the result set it has in memory which is much faster.  This is particularly useful in web applications where repeated SELECT queries are very common.

Here’s a little script to demo the effect query cache can have

<?php
$mysqli = new mysqli('127.0.0.1', 'root', '', 'war_and_peace');
$mysqli->query('RESET QUERY CACHE');
$querySQL = "SELECT * FROM words WHERE word = 'sausage' ";
$time_start = microtime(true);
if ($results = $mysqli->query($querySQL)) {
 echo "Cold got {$results->num_rows} results in ";
 echo microtime(true) - $time_start,"s",PHP_EOL;
 $results->close();
}
$time_start = microtime(true);
if ($results = $mysqli->query($querySQL)) {
 echo "Server cache got {$results->num_rows} results in ";
 echo microtime(true) - $time_start,"s",PHP_EOL;
 $results->close();
}

Running this script against a completely un-normalised table with the 600 thousand odd words of war and peace (thanks Project Gutenberg) in it gives you

$ php MySQL_QueryCache.php
Cold got 5 results in 0.13019204139709s
Server cache got 5 results in 0.00035595893859863s

Why not put this in the client?

What would make it even faster is if the client did not need to connect to the server at all. If the client had a cache of all queries it performed it could save the time taken for the network round trip and help take load off the server by not even asking the question.

mysqlnd_qc attempts to do this.

There is, however, a major problem though, cache invalidation. In the traditional server based cache a very simple table based cache invalidation algorithm is used. If a query which alters the data in a table is detected all caches of data with that table in the result set are invalidated. This will even work for servers which are part of a replication cluster since the update sql will propagate from the master where the data was updated to the slaves and the slaves’ caches will then be invalidated.

But what if there are multiple clients? One client could update a table while another client still thinks its cache is still valid and so continues to serve it. The situation of two clients is actually relevant even if you don’t have multiple web servers talking to the same database server. It could occur by using something other than PHP and mysqlnd to connect the database. For instance using the command line mysql client (which uses libmysql) any updates made through this would not invalidate other clients query caches.

This problem is seemingly insoluble without some complex mechanism of feeding back to every client which caches are still valid. So the simple answer is to only cache for a few seconds and then only use the caching where slightly stale data is not hugely important.

Therefore by default after installing the extension with a quick

phpize
./configure
make
sudo make install

(you will need to be using mysqlnd obviously to be able to use it)

You will need to prefix all queries you want to cache with

“/*qc=on*/ SELECT * from some_table”

which for portability can be written as “/*” . MYSQLND_QC_ENABLE_SWITCH . “*/”;

e.g.

<?php
$mysqli = new mysqli('127.0.0.1', 'root', '', 'war_and_peace');
$mysqli->query('RESET QUERY CACHE');
$querySQL = "SELECT * FROM words WHERE word = 'sausage' ";
$time_start = microtime(true);
if ($results = $mysqli->query($querySQL)) {
	echo "Cold got {$results->num_rows} results in ";
 	echo microtime(true) - $time_start,"s",PHP_EOL;
    $results->close();
}
$querySQL = "SELECT * FROM words WHERE word = 'sausage' ";
$time_start = microtime(true);
if ($results = $mysqli->query($querySQL)) {
        echo "Server cache got {$results->num_rows} results in ";
        echo microtime(true) - $time_start,"s",PHP_EOL;
    $results->close();
}
$querySQL = "/*" . MYSQLND_QC_ENABLE_SWITCH . "*/";
$querySQL .= "SELECT * FROM words WHERE word = 'sausage' ";
$mysqli->query($querySQL);
$time_start = microtime(true);
if ($results = $mysqli->query($querySQL)) {
        echo "Client cache got {$results->num_rows} results in ";
        echo microtime(true) - $time_start,"s",PHP_EOL;
    $results->close();
}

Running this script will show an even better speed improvement.

$ php MySQL_QueryCache_mysqlnd_qc.php
Cold got 5 results in 0.1253068447113s
Server cache got 5 results in 0.00036311149597168s
Client cache got 5 results in 9.8943710327148E-5s

Win Win Win!

This is pretty impressive! This is using the default settings for the length of cache or time to live. The default setting is 30 seconds and it is set with the ini setting mysqlnd_qc.ttl. Of course after our 30 seconds are up the server cache is still there so we will still be able to make use of that. So in a way we can think of mysqlnd_qc as a way of protecting the db server from excessive repeated queries.

More Options

We are using the default handler which behaves in a similar way to the query cache in the mysql server. There are also other backends available such as APC, memcache and sqlite. Sqlite may well be an excellent choice even though using another db may seem a little weird, the reason is that the results sets are in the form of table data which fits well with sqlite. There is also the possibility of creating you own user handler which can use a custom invalidation algorithm there is an example at http://uk3.php.net/manual/en/mysqlnd-qc.set_user_handlers.php

Forking the PHP process requires the pcntl extension. This extension is limited to *nix operating systems. Once installed the key function is pcntl_fork(). The manual contains this excellent example of its use

$pid = pcntl_fork();
 if ($pid == -1) {
      die('could not fork');
 } else if ($pid) {
 // we are the parent
     pcntl_wait($status); //Protect against Zombie children
 } else {
      // we are the child
 }

if we insert a known delay in there and time it we can get an idea of whether we are running in parallel.

$pid = pcntl_fork();
 if ($pid) {
      sleep(10);
     pcntl_wait($status);
} else {
     sleep(10);
}

Mr-McHughs-MacBook-Pro:$ time php pcntl_fork.php
real	0m10.060s
user	0m0.020s
sys	0m0.025s

Excellent so it would appear we are running two PHP processes in parallel. Using a sleep() in this way we model an expensive and parallelisable function call for instance a heavy calculation or http request. By using a known delay instead of the real call we can work out how long it should take to execute and check our logic.

What about more than two processes

The first thing I tried was to put the previous code in a for loop with a couple of obvious alterations

for ($i=0; $i < 5; $i++) {
        $pid = pcntl_fork();
        if ($pid) {
                pcntl_wait($status);
        } else {
                echo 'starting child ',$i,PHP_EOL;
                sleep(10);
                die();
        }
}

The major alteration you will notice is that we exit or die after each child has done it’s thing. Otherwise each child will go through the remaining loops of the for loop and the script will run for a very long time. The other change is that we are no longer doing any work in the parent. There is no real reason for this other than we would always have to remember to add one to the the number of calls we were expecting to make. Even with these alterations this script has a major problem, one, that you get a good idea about if you watch it run.

Mr-McHughs-MacBook-Pro:$ time php pcntl_fork.php
starting child 0
starting child 1
starting child 2
starting child 3
starting child 4
real	0m50.113s
user	0m0.038s
sys	0m0.056s

Watching it in the terminal it becomes obvious that each child only gets started after the previous one has finished. The problem is the call to pcntl_wait($status) in the parent section. What this is doing is waiting for each child to end and hence stopping execution of the parent until it receives a signal to say a child has finished.

The solution is to put all calls to pcntl_wait in the parent but outside the loop that forks each process. The above script fixed is.

for ($i=0; $i < 5; $i++) {
        $pid = pcntl_fork();
       if ($pid) {
        } else {
                echo 'starting child ',$i,PHP_EOL;
                sleep(10);
                die();

        }
}
for ($i=0; $i <5; $i++) {
        pcntl_wait($status);
}

This way we run all 5 sleep commands in parallel

Mr-McHughs-MacBook-Pro:$ time php pcntl_fork.php
starting child 0
starting child 1
starting child 2
starting child 3
starting child 4

real	0m10.077s
user	0m0.037s
sys	0m0.056s

The neatest solution is to use an array to register the pid of each child we create and then check it to see if we have any open children.

<?php
$forks = array();
for ($i=0; $i < 5; $i++) {
        $forks[] = $pid = pcntl_fork();
        if ($pid == -1) {
                die('could not fork');
        } else if (0 === $pid) {
                echo 'starting child ',$i,PHP_EOL;
                sleep(10);
                die();

        }
}
do {
     pcntl_wait($status);
     array_pop($forks);
} while (count($forks) > 0);

The code is on github

To install just do

git clone git://github.com/natmchugh/PHP-Num-Processors.git
cd PHP-Num-Processors
phpize
./configure
make
sudo make install

Then add …

extension=num_procs.so

.. to your php.ini

It exports two functions

$available = num_processors_available();
var_dump($available); // int(2)
$configured = num_processors_configured();
var_dump($configured); //int(2)

The effect of parallelising the calculations was pretty dramatic by the way speed up execution by 50% at least. I intend to write a post on that sometime.

httpv://www.youtube.com/watch?v=8pAUIWYVXXM

There is another option. In PHP you probably know you can do requests with the streams extension and use functions like file_get_contents to retrieve a url contents.

<?php
$page = file_get_contents('http://fishtrap.co.uk');

Since PHP 5.0 you can also set a stream_context to include POST data. This can include files and other multipart form data.

I created a short script to try add a more intuitive and usable wrapper around the functions. To use it you just need to do:

<?php
include('StreamsHttpPost.php');
$request = new StreamsHttpPost('http://example.com');
$data = array(
'foo' => 'bar',
'baz' => 'bat',
);
$request->addFile('picture',  'path/to/file');
$page = $request->post($data);
var_dump($request->getResponseCode()); // int(302)

This will then send a post request as if the user had submitted a form with the form fields.

The code is here on github. Comments, pull requests and corrections welcome.

The generation of increasing digits of Π is in itself completely pointless but makes an interesting area of research for mathematicians and computer scientists with parallel advances in algorithm and hardware. PHP is possibly the least suitable language to implement these computationally intense calculations. However, it is possible for todays PHP scripts to quickly generate digits that took hours of computing time in the 1950′s and 1960′s.  Borwein’s algorithm is interesting for the increasing number of correct digits it gives with each iteration. According to wikipedia the algorithm is given by starting out with:

And then iterating

function borwein($precision){
	// set scale with suitable spare room
	bcscale($precision+5);

	//inital conditions
	$rootTwo = bcsqrt(2);
	$a = bcsub(6, bcmul(4, $rootTwo));
	$y = bcsub($rootTwo, 1);

	//number of interations needed
	$count = floor(log($precision)/log(4));
	for($k = 0; $k < $count; $k++){
		$sqrt = bcsqrt(bcsub(1,bcpow($y, 4)));
		$right = bcsqrt($sqrt);
		$top = bcsub(1, $right);
		$bottom = bcadd(1, $right);
		$y = bcdiv($top, $bottom);

		$onePlusY = bcadd(1, $y);
		$left = bcmul($a, bcpow($onePlusY, 4));
		$y2 = bcpow($y, 2);
		$power = bcpow(2, (2*$k+3));
		$expansion = bcadd($onePlusY, $y2);
		$right = bcmul($power, bcmul($y, $expansion));
		$a = bcsub($left, $right);
	}
return bcdiv(1, $a, $precision);
}
echo borwein(2037), PHP_EOL;

As you will see extensive use of factorials is made. I was using the bc extension for PHP which does not have a simple way of calculating factorials. I came up with the following class to try and calculate them efficiently from the nearest previously calculated. Using it I was able to cut the calculation time for 2037 digits in half.

<?php

class Factiorial {

	private static $_cache = array(1 => 1);

	public function calculate($n) {
		$nearest = $this->findNearestInCache($n);
		$answer = self::$_cache[$nearest];
		$diff = $n - $nearest;
		if ($diff > 0) {
			for ($i = $nearest; $i < $n; ++$i) {
				$answer = bcadd($answer, bcmul($answer,$i));
				self::$_cache[$i+1] = $answer;
			}
		} else {
			for ($i = $nearest; $i > $n; --$i) {
				$answer = bcdiv($answer,$i);
			}
		}
		return $answer;
	}

	private function findNearestInCache($n) {
		$keys = array_keys(self::$_cache);
		rsort($keys);
		$nearest = current($keys) ;
		$smallestDiff = abs(current($keys) - $n);
			foreach ($keys as $key) {
				if ($diff = abs($n - $key) < $smallestDiff) {
					$nearest = $key;
					$smallestDiff = $diff;
				}
			}
			return $nearest;
	}
}

This only really works because we are calculating successive factorials and will often be calculating the same value several times. Having arrays of several thousand items in the cache could be quite slow so I tried limiting the size of the cache array but the extra bookkeeping overhead proved to slow the script down.

The best way I’ve found of installing mysqlnd on ubuntu is to use dpkg to rebuild the .deb. Of course this will need to be done when ever you update the PHP package so is not recomended for anything other than experimental servers.

Quick step by step

Install everything needed to compile C code

sudo apt-get install build-essential debhelper

Install any special depandancies for building PHP such as bison

sudo apt-get build-dep php5

Install the PHP source for the current PHP package

sudo apt-get source php5

This will download the source code for your PHP version into the current directory

cd php5-5.3.2/

Then edit the file debian/patches/series or in older ubuntu releases debian/patches/order

sudo vim debian/patches/series

remove the lines
force_libmysqlclient_r.patch
and while we’re at it remove this line which removes the config flags from phpinfo() apparently done by debian to stop false bug reports but quite annoying
052-phpinfo_no_configure.patch

then edit the rules file

sudo vim debian/patches/rules

sudo vim debian/rules

search for
–with-mysql=shared, /usr \
replace with
–with-mysql=mysqlnd \

same with
–with-mysqli=shared, /usr \
replace with
–with-mysqli=mysqlnd \

make –enable-pdo=shared or –with-pdo=shared \

into –with-enable or –with-pdo

and
–with-pdo-mysql=shared, /usr \
into
–with-pdo-mysql=mysqlnd \

This will build mysql as a static module rather than load it dynamically this will save building each mysql extension separately with mysqlnd

stop the tests running by exporting an environment variable

export DEB_BUILD_OPTIONS=nocheck

Now build the package

sudo dpkg-buildpackage

Go and do something else for a little while.

Once done if you

cd ..
ls

You should see a big list of debs

dpkg the ones you need such as

sudo dpkg -i php5-common_5.3.2-1ubuntu4.9_amd64.deb
sudo dpkg -i libapache2-mod-php5_5.3.2-1ubuntu4.9_amd64.deb

bingo the output of phpinfo() and php -m should confirm you have mysqlnd installed