cron task logging
Some time ago I wrote about a magnificent plugin managing cron task runtime, sfTaskLoggerPlugin and I promised to provide some real world examples of the plugin's usage. The time has come, so let's go on!
archiving outdated products
I'll start with an easy functionality which is executed each day using a cron task. ProductManager class is just a high-level model class, which is separated from ORM business logic, since it has nothing to do with Doctrine directly. The archiveProducts method fetches all products that meet some specific criteria (in this case - products that are not active no more). The first lines use the ORM-level layer to fetch Doctrine records. Then, all products are again checked against another criteria, inside a loop, and if any pass, they're permanently archived. The result of this method is an array of archived product IDs - this is very important data for the task logger. Take a look at the following code:
class ProductManager
{
static public function archiveProducts()
{
$products = Doctrine::getTable('Product')
->findProductsQuery()
->execute();
$result = array();
foreach ($products as $product)
{
$availability_history = $product->getAvailabilityHistory()->getLast();
if ($availability_history->shouldBeArchived())
{
$product->archive();
$result[] = $product['id'];
}
}
return $result;
}
...
The model layer is done, now it's time to prepare the controller layer - and this is the main thing here. Let's start with implementing the command line symfony task. We create a lib/task/archiveProductsTask.class.php file:
<?php
require_once(dirname(__FILE__) . '/../../plugins/sfTaskLoggerPlugin/lib/task/sfBaseTaskLoggerTask.class.php');
class archiveProductsTask extends sfBaseTaskLoggerTask
{
}
We will put all necessary task stuff here. Note that our new custom task archiveProductsTask class extends the plugin sfBaseTaskLoggerTask class. And plugin sfBaseTaskLoggerTask class extends the base symfony sfBaseTask class (without sfTaskLoggerPlugin, a custom task would just directly extend sfBaseTask). Thanks to it, you may adapt the sfBaseTaskLoggerTask class to meet your needs, if you want some custom mechanisms to be available in all your custom tasks.
We define the constants that would be used to state the return code of a task run:
const ERROR_CODE_FAILURE = -1;
const ERROR_CODE_SUCCESS = 1;
Now let's define the symfony task standards - task name, namespace and descriptions, which are available in the command line:
protected function configure()
{
parent::configure();
$this->namespace = 'cron';
$this->name = 'archiveProducts';
$this->briefDescription = 'Mark products as archived';
$this->detailedDescription = <<<EOF
The [cron:archiveProducts|INFO] task finds all products that were
for a given amount of time and sets their status to archival. Call it
with:
[php symfony cron:archiveProducts|INFO].
EOF;
}
Having this class defined, you may run:
./symfony
in your project root directory to find, that a new command
cron:archiveProducts is now available. Simply, run:
./symfony cron:archiveProducts
And here comes the task logger part of the task implementation. Just to remind - each time the cron:archiveProducts is executed, a single task_logger record is inserted into the database. This record holds all important information about the task runtime, these are:
- task name
- task arguments
- task options
- records processed
- records not processed
- start time
- end time
- task is still running
- task has successfuly finished
- error code
- comment
and some more. Of course, you may modify it and do whatever you want (add new or remove existing information) - these are just the most basic information each task execution should log somewhere.
Let's implement the comment generator - you may either generate a plain text comment or even HTML code (if you want lists or links to be nicely displayed). You need to consider it yourself, since big amounts of HTML, generated every day or even more frequently, will use lots of disk space in MySQL, so that your database grows bigger and bigger each day. Think about that ;).
protected function generateComment($result)
{
$comment = '';
$count = count($result);
if ($count > 0)
{
$comment = 'Products changed into archival<ul>';
foreach ($result as $product)
{
$comment .= '<li>'.$product.'</li>';
}
$comment .= '</ul>';
}
$this->task->setCountProcessed($count);
return $comment;
}
And finally, let's implement the rest of task logger logic. The most important function here is the doProcess. Everything is done inside the try-catch block. First, the model layer is called to do the data processing stuff. Once the data is processed, we may interpret the results. We generate the comments (basing on the result array), we set the error code and set the is_ok field if no exceptions were raised. If any problem was encountered, is_ok has to be set to false, using setNOk() method:
protected function doProcess($arguments = array(), $options = array())
{
try
{
$result = ProductManager::archiveProducts();
$this->task->setComments($this->generateComment($result));
$this->task->setErrorCode(self::ERROR_CODE_SUCCESS);
$this->setOk();
} catch (Exception $e)
{
$this->task->setErrorCode(self::ERROR_CODE_FAILURE);
$this->setNOk($e);
}
}
Our new custom task class looks like this now:
require_once(dirname(__FILE__) . '/../../plugins/sfTaskLoggerPlugin/lib/task/sfBaseTaskLoggerTask.class.php');
class archiveProductsTask extends sfBaseTaskLoggerTask
{
const ERROR_CODE_FAILURE = -1;
const ERROR_CODE_SUCCESS = 1;
protected function configure()
{
parent::configure();
$this->namespace = 'cron';
$this->name = 'archiveProducts';
$this->briefDescription = 'Mark products as archived';
$this->detailedDescription = <<<EOF
The [cron:archiveProducts|INFO] task finds all products that were
for a given amount of time and sets their status to archival. Call it
with:
[php symfony cron:archiveProducts|INFO].
EOF;
}
protected function generateComment($result)
{
$comment = '';
$count = count($result);
if ($count > 0)
{
$comment = 'Produkty zmienione na archiwalne<ul>';
foreach ($result as $product)
{
$comment .= '<li>'.$product.'</li>';
}
$comment .= '</ul>';
}
$this->task->setCountProcessed($count);
return $comment;
}
protected function doProcess($arguments = array(), $options = array())
{
try
{
$result = ProductManager::archiveProducts();
$this->task->setComments($this->generateComment($result));
$this->task->setErrorCode(self::ERROR_CODE_SUCCESS);
$this->setOk();
} catch (Exception $e)
{
$this->task->setErrorCode(self::ERROR_CODE_FAILURE);
$this->setNOk($e);
}
}
}
one example is enough. Think about the new possibilities
I'm not going to give dozens of similar examples of code implementation, since the task logger plugin logic would be mostly the same.
If you have a big symfony 1.x application, I strognly encourage you to install the sfTaskLoggerPlugin and redefine some of your task, so that they would be logged each time they're executed. After some time take a look at your task logs - you'll see how many information you can get:
- analyse your task runtime performance
- analyse how many times task have failed
- analyse how many records are being processed
- analyse what time the task processes the most data and what time there is no processed data
- implement another mechanism to monitor all your task logs and alarm, when any task fails (check the
is_running against is_ok task_logger record fields)
and so on. Give it a try! ;)
