Friday, 21 June 2013

Uploading files with CURL in PHP the right way.

Uploading files with CURL in PHP use to involve prefixing the file path with an @ using a POST param.  But recently PHP has added curl_file_create() see: https://wiki.php.net/rfc/curl-file-upload

With this you can now upload a file as easily as:

$params = ['file_key' => curl_file_create('path/to/file')];
curl_setopt($ch, CURLOPT_POST,1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $params);


PHP takes another step in the right direction ;-)

Tuesday, 20 September 2011

MongoDb numeric field selection in PHP

If you are attempting to use numeric field selection with MongoCollection::find() and run into the error:

PHP Fatal error:  Uncaught exception 'MongoException' with message 'field names must be strings'

It is caused by PHP converting an array key that is defined as a string into an integer, so:

array('123' => 0);

becomes:

array(123 => 0);

Unfortunately mongo requires strings for all field names.  A quick workaround (found after browsing code, not yet documented) to the problem is to use an object instead of an array:

$fields = new stdClass;
$fields->{123} = 0;

Problem solved.

Wednesday, 26 May 2010

ExtJS - loading grid record into form with loadRecord() but getting the displayField instead of valueField submitted.


A long title I know, but today was one of those days that every programmer dreads.. a day with a small problem that takes more than 5minutes to figure out. This one took me a whole day! Reasons; partly because of a lack of documentation, partly me giving ExtJS to much magic credit and mostly me over-thinking the problem.

The only thing I could find online that really covered it, but not in a way that was at all obvious to me is here: http://www.extjs.com/forum/showthread.php?88614-Combo-Box-question&highlight=loadrecord+combo

If you have a grid and you load that grid with combo displayField values (from the store) and use loadRecord() to populate the form you will notice that it seems to work on the outside. However when you hit submit the displayField and not the valueField are submitted! I would have assumed ExtJS would have some magic that compares the displayField value with the valueField and updates it accordingly. Wrong, the valueField is set to whatever you set it to and ExtJS will do nothing to change that.

What you need to do is, add the valueField values to the grid as well as the displayField values but not including them in the column model. This way the grid is not cluttered with combo valueField values. But how do you then get that valueField to the form combos? You need to set the hiddenName value in the combos to point to the valueField in the grid and the name to the relevant displayField.

I'm going to stop myself as I think a simple clean code example will speak louder than my confusing ramblings, so here you go:

var form = new Ext.form.FormPanel({
title : 'Test form',
width : 300,
items : [{
xtype : 'combo',
fieldLabel : 'Gender',
name : 'gender_',
hiddenName : 'gender',
store : new Ext.data.ArrayStore({
fields : ['valueField', 'displayField'],
data : [[1, 'Male'] , [2, 'Female']]
}),
mode : 'local',
valueField : 'valueField',
displayField : 'displayField'
}]
});

var toolbar = new Ext.Toolbar({
items : ['->']
});

toolbar.addButton({
text : 'Clear',
handler: function(b, e)
{
form.getForm().reset();
}
});

toolbar.addButton({
text : 'Save',
handler: function(b, e)
{
var values = form.getForm().getFieldValues();
for(var field in form.getForm().getValues())
{
alert(field+':'+values[field]);
}
}
});

form.add(toolbar);

var grid = new Ext.grid.GridPanel({
title : 'test grid',
width: 600,
height: 300,
columns : [
{id: 'id', header : 'id', dataIndex : 'id'},
{header : 'Gender', dataIndex : 'gender_'}
],
store : new Ext.data.ArrayStore({
fields : [ 'id', 'gender_', 'gender' ],
data : [
[ 34, 'Male', 1],
[ 35, 'Female', 2]
]
}),
sm: new Ext.grid.RowSelectionModel({
singleSelect: true,
listeners: {
rowselect: function(sm, row, rec) {
form.getForm().loadRecord(rec);
}
}
})
});

var win = new Ext.Window({
title : 'test',
items : [{
xtype : 'panel',
layout : 'column',
items : [form, grid]
}]
});

win.show();

Notice the '_' added to the name value of the combo.

I hope that by documenting/explaining my problem it will help others avoid a wasted day as I have.

You may have noticed it has been over 2years since my last post.. well, starting your own business will do that to a blogger :D

Friday, 11 April 2008

Sponsor me! - Manchester 10k

I wouldn't normally post OT on my blog but this is for a good cause.

I'm running the Manchester 10k on the 18th of May and am trying to raise £200 for the Rainbow Family Trust (Francis House) who support and improve the quality of life for terminally ill children diagnosed with 'no cure'.

So dig deep and help me reach my target by going to this site and making a donation: http://www.justgiving.com/roseandrew

Many thanks!
Andrew

Tuesday, 5 February 2008

Roll your own N-server sandbox

Have you ever wanted to play around with(test ;) MySQL replication/clustering techniques, LVS/Apache load balancing etc but didn't have the hardware available and where smart enough not to use a production environment?

Well an easy way to be able to do this is by creating a sandbox environment using something like qemu, xen, vmware or UML which allow you to create virtual machines running inside a protected environment on your own desktop machine or whatever hardware you have spare, just be sure you have enough memory. This article will cover setting up a sandbox using UML.

My GNU distro of choice is Archlinux and this article will be based around it, but you should be able to take the information provided here and apply it relatively easily to which ever distro you use. Please also note that I am not going to go into depth about UML as there are plenty of resources just a search away.


Preparing the host.

So first things first we need to have a system capable of running UML, and luckily Archlinux is ready, all we need to do is install a few packages:

[root@ishell ~]# pacman -S user-mode-linux uml_utilities

Then setup our play area. In this example I am setting up a 4 server playground in roots home directory.

[root@ishell ~]# mkdir servers
[root@ishell ~]# cd servers
[root@ishell servers]# mkdir 1 2 3 4
[root@ishell servers]# ls
1 2 3 4
[root@ishell servers]#

Setup the host (the host is the machine "hosting" the UML instances) for networking the UMLs. There are all kinds of ways to do this but I use the uml_switch which is a virtual network switch that is a breeze to use. To allow the UMLs to communicate to and from the host and the outside world we need to create a tunnel and setup IP forwarding (We will start the actual uml_switch at the end in a startup script):

[root@ishell ~]# modprobe tun
[root@ishell ~]# tunctl -u root -t tap0
[root@ishell ~]# ifconfig tap0 192.168.0.100 up
[root@ishell ~]# iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
[root@ishell ~]# echo 1 > /proc/sys/net/ipv4/ip_forward


Setup of the UML filesystem

The UML instances require a file to hold their filesystem and a base GNU install. So we create one with dd. As ever salt to your tastes:

[root@ishell servers]# dd if=/dev/zero of=root.fs bs=1MB count=1000
[root@ishell servers]# mkfs.ext3 root.fs

Now we have formatted image we can loopback mount it ready for setting up the base install.

[root@ishell servers]# mount -o loop root.fs /mnt/cd/
[root@ishell servers]# mkdir -p /mnt/cd/var/lib/pacman

To install the base packages with Archlinux I wget the packages.txt from the main package download area which has a list of all base packages prefixed with base/. To install the base packages to the disk image which we have mounted to /mnt/cd we need to first initialize pacman with the -Sy --dbpath options then instruct pacman to install packages to our /mnt/cd root with the --dbpath and --root options. In this case I am also including apache, php, mysql and openssh.


[root@ishell servers]# wget ftp://ftp.archlinux.org/current/os/x86_64/packages.txt
[root@ishell servers]# pacman -Sy --dbpath /mnt/cd/var/lib/pacman
[root@ishell servers]# pacman -S --dbpath /mnt/cd/var/lib/pacman --root /mnt/cd/ \
`cat packages.txt | egrep '^base/' | sed 's|^.*/||;s/-[0-9]\+.*//' | grep -v kernel` \
apache mysql php openssh


Once we have the base install we need to modify some files. These changes apply to all servers we use this image for so it will save us some time if we make the changes now before making copies of the image.

  • add "/dev/ubd0 / ext3 defaults 0 0" to etc/fstab
  • edit /etc/hosts.deny and comment out ALL:ALL so we can gain ssh access
  • edit /etc/innittab and comment out all but the first virtual console
  • add sshd to daemons in rc.conf
  • change default gateway to 192.168.0.100 (or what you set the tunnel IP to)
  • chroot to /mnt/cd and change the root password other wise you will not be able to gain ssh access.

Now umount the image and copy to all servers:

[root@ishell servers]# umount /mnt/cd
[root@ishell servers]# cp root.fs 1/ &
[root@ishell servers]# cp root.fs 2/ &
[root@ishell servers]# cp root.fs 3/ &
[root@ishell servers]# cp root.fs 4/ &

Mount each root image in turn and make configuration changes to the eth0 ip address and hostname in rc.conf, these have to be unique for each server and are the IPs you will use to access the servers via ssh:

[root@ishell servers]# mount -o loop 1/root.fs /mnt/cd/
[root@ishell servers]# nano /mnt/cd/etc/rc.conf
[root@ishell servers]# umount /mnt/cd


And the finishing touch

Create a startup script (i.e. run.sh) and modify the mem parameter to suit your needs. ubd0 specifies the file that contains the filesystem the UML will use and con=null tell the UML not to spawn any consoles or direct any output from the kernel to the console as we will be using ssh as the main form of access, however you can change this if you wish.

The uml_switch is also started here and is passed the name of the tunnel device we setup earlier and told to fork off into a daemon.

#!/bin/bash
exec uml_switch -tap tap0 -daemon &
exec linux ubd0=1/root.fs con=null eth0=daemon mem=128M &
exec linux ubd0=2/root.fs con=null eth0=daemon mem=128M &
exec linux ubd0=3/root.fs con=null eth0=daemon mem=128M &
exec linux ubd0=4/root.fs con=null eth0=daemon mem=128M &


Final thoughts

After running the above script you should have 4 servers ready to go within a few minutes. Simply ssh to them as you would any other server and have fun!

Friday, 1 February 2008

PHP: Getting signals through to blocking socket_accept

Have you ever written a sockets server in PHP and wanted to use the blocking option but found you where unable to kill the process off cleanly and instead had to resort to kill -9 or even worse set blocking to false and set a delay in the main event loop?

Well there is a way to use blocking and be able to shutdown the process cleanly. socket_accept() blocking may ignore general signals, but it does not ignore pcntl alarms. An alarm fired every second or so will break you out of a blocking call to socket_accept() (with a warning) and allow your normal flow of logic to detect a shutdown signal i.e. sigterm.


Example code



<?php
declare(ticks = 1);

class test
{
private $shutdown = FALSE;

function sigTerm()
{
$this->shutdown = TRUE;
}

function sigAlarm()
{
pcntl_alarm(1);
}

public function doit()
{
$socket = socket_create(AF_INET, SOCK_STREAM, 0);
socket_bind($socket,'127.0.0.1', 5555);
socket_listen($socket);

socket_set_block($socket);

pcntl_signal(SIGTERM, array(&$this, "sigTerm"));
pcntl_signal(SIGALRM, array(&$this, "sigAlarm"));
pcntl_alarm(1);

while(1)
{
if($c = @socket_accept($socket))
{
echo "Got a connection\n";
socket_close($c);
}
else
{
echo "Alarm has broken us out of socket_accept blocking, ";

if($this->shutdown)
{
echo "and it is time to shutdown now Dave.\n";
return;
}

echo "but it is not time to shutdown yet Dave.\n";
}
}
}
}

$test = new test;
$test->doit();



And Finally

To test the code simply telnet 127.0.0.1 5555 to see it handling connections. Then when you're done a simple killall php should do the trick (just be sure you don't kill any other php process).

I should point out that socket_stream_accept() will respond to signals without the need for an alarm. So if you can, it may be worth going with the stream functions over the cardinal socket functions. (Edit: 04/02/2008) However the stream function response to signals seems a little erratic so I would use an alarm anyway.


UPDATE - 5th March 2008

I was stung by a nasty little oversight that you should know. When reading from a socket, any interrupt (read: alarm) will cause the read to abort. To solve this, after successfully establishing a connection with socket_accept() be sure to run pcntl_alarm(0) then socket_read() and then pcntl_alarm(1) again.

Wednesday, 30 January 2008

LAMP: The mailing lists you *SHOULD* be subscribed to.

The LAMP stack consists of some complicated software, and this software from time to time will develop faults and security flaws. How do you keep yourself informed? Hope that the issues crop up on Digg, Slashdot? Well the best way is to join the Announce lists for each of the LAMP stack components.

The Announce lists are used by the developers of the different components of the LAMP stack to keep users informed of important events, like when a security flaw emerges or a new version of there software is released, etc.


The lists

The first one is dependent on the GNU/Linux distribution your running, in my case it is Archlinux, so I've subscribed to:


Next in the LAMP stack is Apache, and you can find its Announce list here:


For MySQL:


And PHP:


And lastly, considering that SSH is the primary form of entry to my systems (and most likely yours) I am subscribed to the OpenSSL and OpenSSH mailing lists:



Final points

Now, subscribing to these lists in no way secures you from attack, but it should give you a fighting chance. When bugs / security issues do crop up you will need to take the necessary action, which should be as simple as upgrading the relevant software. But be prepared to build replacement packages from source if your GNU/Linux distribution is slow to release an updated package.