Laravel/eloquent isDirty() & getDirty()

Intro

I’ve done a quick test to exercise a new-to-me part of the Laravel (5.6) Eloquent API:

$model->isDirty();
$model->isDirty("field_name");
$model->isDirty(["field_name", "another_field_name"]);

$model->getDirty();

I want to know when ->isDirty() returns TRUE…

My model

I’ve got an Eloquent model for my contacts table:

namespace App\Contacts;
use Illuminate\Database\Eloquent\Model;

class Contact extends Model {
    // ...
}

Testing…

$contact = new Contact;
echo($contact->isDirty());
//  false

$contact->first_name = "Mek";

echo($contact->isDirty());
//  true

echo($contact->isDirty("last_name"));
//  false

echo($contact->isDirty(["first_name", "last_name"]));
//  true

$contact->save();
echo($contact->isDirty());
//  false

• isDirty() seems always to return boolean TRUE/FALSE regardless of what attributes you pass: whether you want to check globally (no attributes == any field), for changes to one field (string attribute == field name), or for changes to any number of fields (array attribute == array of field names)
• Works predictably, as expected, whether I use a model instance corresponding to a new record or an existing record.
• Also (extra test, not shown here): if I change a field to a different value, $contact->isDirty() returns TRUE; but if I “change” a field to the same value it had before, $contact->isDirty() returns FALSE – which is good: it’s not returning false positives.

PHP, MySQL, Table locks, Persistent Connections

So I’ve got:

• A MySQLtable of records that I want to process
• 3 x EC2 instances behind an ELB, each running the same PHP application called by
• Cron scheduler

I want the 3 identical EC2 instances for resilience: if one of them fails, there’ll still be 2 left running.

I want any given instance of the application to “claim” a row at a time for processing.

Because there are 3 instances, concurrency’s an issue: what if the following sequence of events happens?

1. Instance #1 reads the 1st unclaimed row from the table – ID 12345
2. Before instance #1 can mark that row as “claimed,” instance #2 reads the same, not-yet-claimed row, from the table (ID 12345).
3. Instance #1 marks row ID 12345 as “claimed”
4. Instance #2 marks row ID 12345 as “claimed”
5. Both instances go off to process the same record

My current answer to this is to use MySQL transactions and table locks. Effectively, each instance needs to attempt to run the following pseudo-SQL:

SET AUTOCOMMIT = 0;
LOCK TABLES my_table WRITE;

SELECT record_id FROM my_table
WHERE is_claimed = 0
ORDER BY create_datetime ASC
LIMIT 1;

[PHP notes my_table.record_id in variable]

UPDATE my_table
SET is_claimed = 1
WHERE record_id = $record_id;

COMMIT;

UNLOCK TABLES;

SET AUTOCOMMIT = 1;

This was working OK in itself. Then, I decided to knock the script over (by having PHP exit() after the SELECT query). I’d done a bit of reading that suggested MySQL would automatically unlock the table when the script finished.

BUT that didn’t happen.

The reason was, I’m running PHP as an Apache module, and I’d configured my application to use persistent MySQL database connections. IE, the database connection persists in between PHP script requests. Because of this configuration, the MySQL table remained locked, and – when I tried a query against table my_table in my desktop MySQL client, the query was hanging. MySQL recovered when I restarted Apache – which I guess has the effect of closing the database connection, which allows MySQL to release the locks.

So, when I re-configured my application not to use persistent DB connections, the problem went away: script failure results in the DB connection being closed, which results in MySQL releasing the table locks.

Amazon S3 SDK for PHP v2.0

I want to change my file upload code so that, instead of storing files on my server, they’re hosted in Amazon S3.

I’ve been playing with Undesigned‘s S3 class, but there seems to be a problem with it: when I specifyserver-side encryption of my files on S3, I get an error (watching tail -f /var/log/apache2/error.log):

S3::putObject(): [SignatureDoesNotMatch] The request signature we calculated does not

match the signature you provided. Check your key and signing method.

The error only appears when I request SSE, and the advice I’ve seen on Stack Overflow regarding this kind of error seems to be, use the Amazon AWS SDK for PHP. But there aren’t yet (early 2013) many tutorials around for a PHP SDK Version 2 solution, so I’m going to have to blunder around like an oaf until I work it out, leaving a turdtrail of crappy notes in my wake, purely selfishly because I know I’ll forget what I find out within a month.

Installation

Caveat

I’ve got an EC2 instance (Ubuntu 12.04) to play with. If you’re on shared hosting and/or don’t have command-line access to your server, I can’t help you installing what you might need to pursue this exact solution.

I’m not going to say “get Amazon EC2” because I know there are rumblings in the community about EC2 being wonderfully scalable, but not providing emphatically good value-for-money in terms of computing power – but having a free-tier AWS rig to play with is definitely teaching me some useful DevOps chops.

Composer

The SDK documentation makes multiple references to the PHP package manager Composer, so – while there are alternative solutions using PEAR or phar, I decided to install Composer, using these instructions…

In some directory (not important where):

curl -sS https://getcomposer.org/installer | php

curl -sS https://getcomposer.org/installer | php

sudo mv composer.phar /usr/local/bin/composer

That creates a command composer… and it worked for me, I can execute it (the file’s executable, and when I type composer at the command prompt, I see a list of options)

Tell Composer what libraries your project depends on

The project I’m working on uses the CodeIgniter framework, and 3rd-party class files are typically stored in the project’s application/libraries directory. Your project’s likely completely different – just work out the best file path for your code.

So my first attempt at installing the AWS SDK in my project is to create the file application/libraries/composer.json.

The file contains just a reference to the AWS SDK package – here’s its total contents:

{
    "require": {
        "aws/aws-sdk-php": "*"
    }
}

Actually install the packages

cd /path/to/application/libraries
composer install

OK – composer does some work, then reports in the terminal that it’s installed various packages (EG symfony/event-dispatcher, guzzle/guzzle, aws/aws-sdk-php) – no errors reported.

Tell PHP code about installed packages

The next step is to “require Composer’s autoloader”, says the AWS documentation. I haven’t read up on this properly – I’m guilty here of just installing Composer to achieve a specific goal, thinking “maybe that’ll come in handy again sometime”. Sorry. I expect Composer’s proper cool. Sorry.

At the command prompt, from my application/libraries directory, a ls -l command shows a new directory called “vendor”. In that, there’s a file “autoload.php”, and subdirectories with names like “aws” and “guzzle”. Looks good. So I predict that, to require my autoloader, it’s a case of writing a line in whatever files use the SDK, saying:

require_once('relative/path/to/application/libraries/vendor/autoload.php');

…And a quick test suggests that works fine.

Writing Code

So now for the actual challenge itself: code to upload a file to S3, optionally declaring that it should be server-side encrypted. This is where I was having trouble with the Undesigned S3 class (and I wasn’t alone) – I think maybe, in the S3 API, the option to specify at runtime whether a given file should be server-side encrypted is a relatively recent/late feature, and I don’t know how well-supported it is in the various libraries. That’s why I wanted to base my solution on the version 2 SDK.

I had a look at the documentation, but couldn’t see the relevant detail, so I dropped a post on the AWS Developer forum. Turns out I’d failed to work out the UI of the documentation, and just hadn’t clicked on the section heading to reveal the details. Patiently corrected by a guy called Michael at AWS, I found my way to the docs, and this time clicked on the paragraph and saw the goodness:

Accepts an associative array with the following keys (required keys are bold):

• ACL – (string) – The canned ACL to apply to the object.
• …ner ner probably very cool options here…
• Bucket – (string)
• …ner ner…
• Key – (string)
• …options ner…
• ServerSideEncryption – (string) – The Server-side encryption algorithm used when storing this object in S3.
• …

So now, I just needed to add the option to my code whenever it’s needed.

$s3 = S3Client::factory(array(
 'key' => $this->config->item('access_key_id', 's3'),
 'secret' => $this->config->item('access_key', 's3'),
 'region' => Region::IRELAND
));

$result = $s3->putObject(array(
 'Bucket' => $bucket,
 'Key' => $key,
 'SourceFile' => $file,
 'ServerSideEncryption' => 'AES256',
 'ACL' => CannedAcl::PRIVATE_ACCESS
));

Boom shaka-laka. I thought I should write it up to make sure I had the process recorded, because there are a number of steps there (EG Composer installation)… but it’s cool, it all works.

Plus, check out my funny band: http://www.youtube.com/watch?v=vZfzP88Fmlg

Amazon AWS: ClamAV

My web application needs to virus-check uploaded files. So I need to install ClamAV, schedule updates, and test virus scanning from PHP scripts.

sudo apt-get install clamav clamav-daemon clamav-freshclam

So that’s clamav installed. Next up – test it from the command line.

sudo clamscan test.pdf

That works, although it’s suspiciously slow.

Next – test from a PHP script:

1. Copy test.pdf into Apache’s DocumentRoot directory
2. Write a PHP script there, called scan.php
3. Must contain a (call to a) function virus_scan()…

    function virus_scan($path)
    {
        $command = 'clamscan ' . escapeshellarg($path);
        $out = '';
        $int = -1;

        exec($command, $out, $int);

        return $int == 0;
    }

Hooray.

Apache 2.2.x chroot pain #1: PHP datetime zones

I’m configuring a LAMP-ish (no MySQL) server on Amazon EC2 an for security reasons want Apache to run in a chroot jail. I’m running Ubuntu Server 12.10 and the current version of Apache (2.2.20-ish) has chroot capability built-in.

To configure it, so the tutorials said, all I needed to do was add a ChrootDir directive to /etc/apache2/apache2.conf. The idea was that Apache would start up, and just before it began serving requests, would set its chroot directory.

But my PHP wasn’t working, and in /var/log/apache2/error.log was a PHP error saying something like:

PHP Fatal error:  date(): Timezone database is corrupt – this should *never* happen! i…

The problem was, my calls to date functions in PHP required Apache to have access to timezone information recorded by default in /usr/share/zoneinfo/

Because of the chroot, Apache wasn’t able to find it.

In the end, I copied the contents of zoneinfo/ to /path/to/chroot_directory/usr/share/zoneinfo, so that when Apache goes out looking for its date/timezone information, it can follow the expected directory path DOWN FROM ITS chroot to the zoneinfo directory.

That feels hacky, though – I’m not sure what would happen if, for instance, an Ubuntu update changed the contents of /usr/shared/zoneinfo?

So I need to so some more research soon, to improve how it works.

I’m also a bit confused because it feels like maybe there’s some information that Apache does not process before it locks itself away in its chroot jail – maybe if it’s late-loaded information? Information not required until a PHP request is being processed?

Amazon AWS: Imagemagick, permissions

I’m exploring Amazon AWS (EC2 for web applications, RDS for MySQL databases). I’ve got a server configured, with my application’s DocumentRoot in /opt/my_application_name/www.

By the way – installing imagemagick on Ubuntu 12.10’s dead easy:

sudo apt-get install imagemagick

I’ve made Apache’s user & group (currently www-data:www-data) the owner of that directory and directories beneath it.

But I wanted to create previews of PDF files with imagemagick, and I was having permissions issues, from PHP, calling:

$command = "convert -alpha off -thumbnail 200  /path/to/from.pdf[0] /path/to/thumb.png";
$out = '';
$int = -1;
exec($command, $out, $int);
return $int == 0;

That’s because Apache processes are running as user www-data, but – I think – the processes created by calls to exec() are running as a different user (let’s name that account my_user).

The way I’ve got round it so far is to make my_user a member of the www-data group:

id -nG my_user
{Lists all groups of which my_user is a member}
sudo usermod -a -G www-data my_user
{Adds user my_user to group www-data

And then to recursively chmod the upload directory to mode 775 – meaning that any member of group www-data can write to the directory:

cd /opt/my_application_name
chmod -R 775 uploads/

I’ve got a feeling that’s a bit simplistic, going on suicidally dangerous, so I hope I’ll get chance to revise this post when I’ve got time to research a better way of doing it – but for now, that’s my notes on how to at least get the ImageMagick conversion working.