Tiny PHP Docker images

Docker has been around for a while now and I had my go at a few systems.
With some trial and error I worked up an intermediate knowledge of the inner workings of docker and how it feels to develop with it, from a developer and a devops perspective.

The DevOp in me wants to try out all the new tool, frameworks, scripts to automate as much as possible for my development, however the developer in me is very unhappy having to wait for hundreds of Megabytes (on larger projects it is in the GB range) of images to load and in the worst case I waited an hour for an image build to finish.

Hence I had to do something. I had to find a way to make those images smaller.
On one project we are using eZ Platform, which comes with a docker toolbox.

While developing with it, I don’t notice much, however when I send my project to be built on the CI server, I will always have to pull the base images provided which are quite large:

ezsystems/php 7.0-v1-dev 522.9 MB
ezsystems/php 7.0-v1 451.1 MB

This is a little bit of an overhead, so I searched in my scripts collection and put something together that will build me a stand alone php package, and called it packager which makes use of dockerize.

I can feed packager with a json file which describes which apt packages I want installed, which essential binaries I want and which applications I want to have packaged. Packager then uses an ubuntu:xenial image to execute those instructions, put all the files together and tars it out into the unix pipe.

Contents of package.json

{
"pre-commands": [],
"post-commands": [
"php -r \"copy('https://getcomposer.org/installer', 'composer-setup.php');\"",
"php composer-setup.php --install-dir=/tmp/build/usr/bin/ --filename=composer",
"php -r \"unlink('composer-setup.php');\""
],
"essentials": [
"env",
"bash",
"sh",
"dash"
],
"packages": [
"libcurl3",
"libedit-dev",
"libgd-dev",
"libmemcached-dev",
"libldb-dev",
"libxpm-dev",
"libxslt1-dev",
"libicu-dev",
"php",
"php-curl",
"php-fpm",
"php-gd",
"php-intl",
"php-memcached",
"php-mbstring",
"php-mysql",
"php-mysqlnd",
"php-pear",
"php-xml"
],
"bin": ["php", "php-fpm7.0"]
}
cat php.json \
| docker run \
--rm \
--name pckgr \
-ia stdin \
-a stdout \
-a stderr packager \
> build.tar

Then I define a Dockerfile for my image:

FROM busybox
ADD build.tar /

ENTRYPOINT ["php-fpm7.0", "-F"]

docker build -t local/php .

After I switched the PHP images the eZ application image was now only about 82MB small. The CI Server only needs to build the base image (which takes about 2 – 5 minutes to build) once a week and the application builds have cut down in time by about 6 minutes.
I have tried this with PHP and NGINX but failed to get erlang to run with it. Maybe some systems are not supposed to be run in isolation.
Check out my repository and feel free to contribute.

TwigSfHelper2ExtensionsBundle Release

It has been a while since I published some code online, however lately I had some time to look through my things and see what I can package as a plugin/bundle.

Since I have started development with Symfony2 rather than only Symfony 1.x I ran ahead and created my first bundle; the TwigSfHelper2ExtensionsBundle. Simply said this bundle is just a port of some of the Symfony 1.x text and date helpers into Twig filters.

Twig Filters included

  • distance_of_time_in_words: Displays two dates as “time ago”
  • time_ago_in_words: Same as distance_of_time_in_words but shows the difference of now
  • auto_link_text: Turns all urls and email addresses into clickable links. The +link+ parameter can limit what should be linked. Options are :all (default), :email_addresses, and :urls.
  • excerpt_text: Extracts an excerpt from the +text+ surrounding the +phrase+ with a number of characters on each side determined by +radius+. If the phrase isn’t found, nil is returned. Ex: “hello my world” | excerpt(“my”, 3) => “…lo my wo…”
  • simple_format_text: Returns +text+ transformed into html using very simple formatting rules Surrounds paragraphs with <tt>&lt;p&gt;</tt> tags, and converts line breaks into <tt>&lt;br /&gt;</tt> Two consecutive newlines(<tt>nn</tt>) are considered as a paragraph, one newline (<tt>n</tt>) is considered a linebreak, three or more consecutive newlines are turned into two newlines
  • strip_links_text: Turns all links into words, like “<a href=”something”>else</a>” to “else”.

Installation

Step 1: Add bundle to your deps file
[TwigSfHelper2ExtensionsBundle] 
git=http://github.com/mozzymoz/TwigSfHelper2ExtensionsBundle.git 
target=bundles/JustMozzy/TwigSfHelper2ExtensionsBundle
Step 2 Add bundle to your autoload.php and AppKernel:
'JustMozzy' => __DIR__.'/../vendor/bundles'
new JustMozzy\TwigSfHelper2ExtensionsBundle\TwigSfHelper2ExtensionsBundle(),

 

 

Working with CS Cart: Template Suggestions

It has been a little while since I posted my first “Working with CS Cart” entry. The previous post showed you how to change the footer of an existing CS Cart skin, something useful but not really complex like for example selecting different page templates depending on the page or controller you are viewing.

One of my clients had some very varying templates for different sections of the cart. If you are familiar with Drupal you know that the Drupal system looks automatically for specific template names depending on the currently requested URL based on specific criteria. Essentially this is exactly what I needed CS Cart to do, however I had to include a little hack in order to implement such a mechanism.

Suggestion Criteria

Usually the frame of a website design stays the same which would logically reside in the index.tpl, however the display of the actual content might be different for several sections/pages in the cart. For this we will have to modify the main.tpl file. Here you will see the default way of displaying the “frame” for content in CS Cart but what if we had a design that has a different structure for the home page and inner pages? Further more, what if one category has a different structure than the rest? Well now we are getting closer to build up our criteria. To make things short let me sum up the criteria I have setup for my client’s project:

  • Each controller (index, pages, products, categories etc.) may have a different structure
  • One or more pages may have a different structure
  • One or more categories may have a different structure
  • One or more products may have a different structure

Having those criteria defined gives us a great deal of flexibility to basically be able to control the content’s “frame” in any situation.

Smarty Modifier: Looking for a template

Unfortunately CS Cart nor Smarty have any built-in functionality to check whether or not a template exists from within a template, so I had to go in and quickly implement such a functionality. In order to keep things simple and not change any core code I had to think of a way to implement template suggestions from within a .tpl file which would require extending Smarty. Ideally an implementation of the required functionality would look something like this

{if $page_template|valid_template}
  {include file="`$page_template`"}
{else}
  {include file="default.tpl"}
{/if}

Looking through the Smarty documentation you will learn that this function is a Smarty modifier but in fact we are not modifying any values only running a quick check on the existence  of a template file. So we will simply create a new modifier for Smarty in order to do the check. Go to your CS Cart installation and create a file called modifier.valid_template.php in the /lib/templater/plugins open it up and copy and paste the following code into it

<?php
/**
 * Checks the presense of a template.
 * Usage example:
 * {if $page.include_file|valid_template}{include file=$page.include_file}{/if}
 */
 $GLOBALS['smarty'] = $smarty;
 function smarty_modifier_valid_template($template_name)
 {
   $smarty = $GLOBALS['smarty'];
   return($smarty->template_exists($template_name));
 }
?>

We are simply creating a wrapper for the Smarty template_exists function. Now whenever we use this modifier we can check if a template path is actually an existing template.

Putting it together

Now we have our function to check if a template exists and our criteria. It is time to put it all to work. First of all we will create the implementation of our first criteria “Each controller may have a different structure”. For this back up the main.tpl file before editing it. Once this is done simply rename the backup to “default.tpl” so that we have a fallback template in case no specific templates were found for any of the controllers. Now we will replace the content of main.tpl with this

{capture assign='page_template}main/page_{$controller}.tpl{/capture}
{if $page_template|valid_template}
  {include file="`$page_template`"}
{else}
  {include file="main/default.tpl"}
{/if}

It is as simple as these 6 lines to have custom layout for each of the controllers. Note that this snippet will look for the templates in the directory “main” which will have to be created manually and all page_<controller_name>.tpl files will reside in there.

Now you can create for example a page_index.tpl in order to differentiate the structure of the main.tpl for the homepage from the rest of the site. For any other controller simply create a file called ‘page_<controller_name>.tpl’ in your skin. If you are not sure about what controller is used for the section you want to change, simply enable debugging from the administration, navigate to the page you want to have changed and look up the “$controller” value in the debug window.

This change will only comply to our first criteria; we need more. For this we will need to create some more files and folders. I will document here how to implement suggestions for pages only but the same method can be applied to categories and products too. For better maintainability we create a new folder called “pages” in the “main” folder where all the page specific templates and a default.tpl fallback will reside. Also create a file called “page_pages.tpl” in the “main” folder. Here we will add the following code

{capture assign='page_template}main/pages/page_pages_{$page.page_id}.tpl{/capture}
{if $page_template|valid_template}
  {include file="`$page_template`"}
{else}
  {include file="main/pages/default.tpl"}
{/if}

Now we have setup a mechanism for CS Cart to look for main/pages/page_pages_<page_id>.tpl for page specific structures. Simply create folders and files for the products and categories controllers to do the same and you’re good to go.

Simple, ain’t it?

Predator: Intelligent Camera

Earlier this year PhD student Zdenek Kalal came up with an algorithm that can virtually track any object in a video feed. This can be used in a huge variety of applications but I think the video Zdenek prepared can explain things better to you. Predator might change a lot, looking back on one of my earlier posts you may get some ideas.

If you want to know more about the project visit Zdenek’s page

Working with CS Cart: The basics

As I announced earlier I will be writing in the next few weeks about CS Cart customization. In this post I will talk about the basics of the CS Cart structure and how you can start to make changes to existing Skins (by changes I mean structural, not design changes). I will assume at this point that you already downloaded and installed CS-Cart on your development environment and are ready to go on with the customization. In case you have not, check out http://www.cs-cart.com for more information on the different licenses and how to install the software.

Basic Structure

Looking at the directory structure you will notice the skins directory in your installation. This is where CS Cart stores the currently installed skins for your cart software. During the installation you were asked to choose a skin, if you haven’t chosen the basic skin, please do so from the administration section. Within the “basic” directory in the “skins” directory you will find three more entries, “admin”, “customer” and “mail”. These three directories refer to the three main sections of CS Cart skinning, one for the administration section (by default accessed by http://www.example.com/admin.php), the frontend of the cart (http://www.example.com/) and any from the system sent emails. Each of those directories has a number of files and sub-directories for you to play with, however you will later learn that you will probably not need to work with any of them, but rather create your own files to override existing files.

Most, if not all the files residing in these directories are .tpl files which are Smarty templates. If you are not familiar with Smarty I strongly suggest that you go over to their website and check out their documentation.

Template Hooks

I had to learn the hard way that most changes in CS Cart can be done using template hooks. The CS Cart documentation mentions template hooks briefly but doesn’t give very good examples or explanations on how to use them. What are template hooks you ask? According to CS Cart’s documentation,

Template hooks are parts of a template enclosed in the tags “{hook name=”section:hook_name”}{/hook}” that can be supplemented or completely redefined by any addon.

So how do we utilize these hooks? Let’s do this by example; open up the index.tpl file in the customer directory. In the bottom you will find a hook defined just before the body close tag. We will use this hook to add some static HTML to all pages in the store front. The CS Cart documentation says that hooks can be used by addons; does this mean we need to create an addon? Simply said, yes, however the guys over at CS Cart were kind enough to add an addon called “My Changes” which is essentially an empty addon used for customization purposes, so we won’t have to create one. This addon is by default enabled. Have again a look at the hook definition in the index.tpl, note down the section in the name attribute (“index”) and the hook name (“footer”). This information is enough for us to get things started.

In the addons directory under customer create now the folders “my_changes/hooks”. The hooks folder is the point where CS Cart will be looking for any hooks grouped by section, naming you will have to also create the sub-folder “index”. Your structure should look like this:

skins/
  basic/
    customer/
      addons/
        my_changes/
          hooks/
            index/

Before we go on and utilize the hook we will have to decide how we want to add the static HTML content. There are three types of utilization: pre, post and override. The pre hook will prepend any content to the defined hook and the post hook will append to the defined hook. This is very practical if for example the hook is enclosing any content already contained in the original template. An example for this is the index:main_content hook in the main.tpl template, but more to this file later on. The override hook will completely replace the predefined contents of the hook area. We currently do not know if other addons are utilizing the footer hook and we want to be sure that the static HTML is in the middle, meaning not prepended nor appended to the content, hence we will use the hook override. We will have to tell CS Cart somehow that we want to override the hook’s content, this is easily done by creating a file with the name “footer.override.tpl”. In general the filenames are as follows:

  • hook_name.pre.tpl – Add changes before content
  • hook_name.post.tpl – Add changes after content
  • hook_name.override.tpl – Replace content with changes

Simply open the footer.override.tpl and add any static HTML to it, like for example:

<p>
  I added this by reading this <a href="https://mozzymoz.wordpress.com/2011/10/26/working-with-cs-cart-basisc/" target="_blank">article</a>
</p>

Now simply go to your storefront, refresh and see the changes taking effect in the bottom of the page. In case you cannot see the changes, go to your administration and add ?cc to your query string in order to clear the cache.

This is all it takes to make changes to an existing skin. I do admit that this example is not really something that is useful, however you could play around with this code and see that you could for example easily add a Share This widget to all of your pages.

Working with CS Cart

In the past week I have been put to the task to implement a new skin for the CS Cart eCommerce Software. Usually the projects I work on are based on either Symfony framework, Drupal or WordPress; taking up this project on a new platform with a 2 week turn around was quite a challenge.

One might expect when purchasing a software which advertises itself as “a perfect platform for custom eCommerce requirements”, it would be thoroughly documented but I had to find out that this was not the case. Of course one can find answers around the Internet. However it is a tedious and time wasting task. I was left with having to dig through core code and debugging to somehow get a grasp on the internals. To CS Cart’s defense, the design template I received to integrate into CS Cart was extremely customized.

On this note I will start writing up a small series on development with CS Cart covering basic changes to existing skins, writing add-ons and creating your own skin.

Flex 3.0 meets Symfony meets PureMVC: sfFlexymfonyPlugin

I have just released the new sfFlexymfonyPlugin for Symfony.

The sfFlexymfonyPlugin provides means for a developer to easily integrate flex applications into Symfony. The undelying Felx application uses a specific framework aimed to give developers a Symfony-like feel.
Detailed documentation still to follow. Please for now try to have a look at the DefaultModule example in your application. The underlying architecture is based on PureMVC so please also have a look at their documentations. Soon I will update the documentation and provide a full tutorial on how to use the framework. Also browse through the whole source code (there is not much in there so you will have a nice overview of what’s going on).

Check it out here

New Symfony Plugin: sfFileTrunkPlugin

I have finally finished a new plugin for Symfony: sfFileTrunkPlugin

sfFileTrunk plugin

The sfFileTrunkPlugin provides a central point for file uploads and handling. Not only does it take care of all of your file uploads but also for content delivery easily implemented using partials and components.

Installation

This plugin requires sfThumbnailPlugin. If you want to make use of the components and actions of this plugin don’t forget to enable the sfFileTrunk module in your application settings.yml

Contents

This plugin contains several classes for easy file upload and devlivery.

Creating a file upload using sfFileTrunkPlugin

In order to make use of the plugin’s functionalities all you basically need to do is to create a file input widget in your form and pass the sfValidatedFileTrunk class to the sfValidatorFile.

See for example following form:

class TestForm extends BaseForm
{
    public function configure()
    {
        $this->setWidget('file', new sfWidgetFormInputFile());
        $this->setValidator('file', new sfValidatorFile(array(
            'validated_file_class' => 'sfValidatedFileTrunk', 
            'path' => FileTrunk::getPath()
        )));

        $this->getWidgetSchema()->setNameFormat('test[%s]');
    }
}

It as easy as that. Your action to handle the form could look like the following:

    public function executeIndex(sfWebRequest $request)
    {
        $this->form = new TestForm();

        if ($request->isMethod('post'))
        {
            $this->form->bind($request->getParameter('test'), $request->getFiles('test'));

            if ($this->form->isValid())
            {
                $file = $this->form->getValue('file');
                $file->save();
            }
        }
    }

That’s it. Everything else is already done in the background. The file is saved with a unique name and a reference is inserted in the database.

This was just a very simple example. The real use of the plugin comes through if you combine it with your model. Take the following schema for example:

propel:
  test_table:
    id:
    title:
      type: varchar
      size: 80
      required: true
    file_trunk_id:
      type: integer
      required: true
      foreignTable: file_trunk
      foreignReference: id

A reference to the file_trunk table is needed so that we can store the file_trunk_id in our table (this table could for example the images table for a gallery). Now all you need todo is to change the TestTable’s form a little bit like this:

class TestTableForm extends BaseTestTableForm
{
    public function configure()
    {
        $this->setWidget('file_trunk_file', new sfWidgetFormInputFile());
        $this->setValidator('file_trunk_file', new sfValidatorFile(array(
            'validated_file_class' => 'sfValidatedFileTrunk', 
            'path' => sfConfig::get('sf_upload_dir')
        )));

        unset($this->widgetSchema['file_trunk_id']);
        unset($this->validatorSchema['file_trunk_id']);
    }

    public function save($con = null)
    {

        $file = $this->getValue('file_trunk_file');
        $file->save();
        $this->getObject()->setFileTrunkId($file->getFileTrunk()->getId());
        return parent::save($con);
    }

}

And voila! You have now enabled file upload with reference to the file trunk in your model.

Content delivery with sfFileTrunkPlugin

To put out content is very easy. You can put out content with two components:

<?php include_component('sfFileTrunk', 'filetrunk_file', array('id' => 1)); ?>
<hr/>
<?php include_component('sfFileTrunk', 'filetrunk_image', array('id' => 1, 'width' => 100)); ?>

The filetrunk_image component supports thumbnailing of images. Minimum parameters to be passed so that it works is the width parameter. You can optionally add the height parameter. If you set width to 0 or leave them completely out the original image will be used.

NOTE: When not in the dev environment new thumbnail generation is limited to a 30 minutes interval. That means if you already generated a thumbnail of 30×30 you will have to wait another 30 minutes before you can generate a thumbnail with 40×40.

Symfony Plugin Install Script

This is a very short post.

Currently I am using Symfony 1.4.1 for my development but I have noticed that I could not install most if not all plug-ins through the PEAR packaging system. Since I am using Ubuntu I quickly wrote myself a short script to handle the download and unpacking of plug-ins that I want to install. When calling the script you just give it the URL to the .tgz package of the plug-in. Enjoy

#!/bin/sh
url=$1
#extract the package filename from the URL
package_name=$(echo $1 | sed 's/.*\///')
#extract the raw plug-in name (with version number)
plugin_raw=$(echo $package_name | sed 's/\(.*\?\)\.tgz/\1/')
#extract the actual name of the plug-in (without the version number)
plugin_name=$(echo $package_name | sed 's/\([^-]*\).*/\1/')

#go to the plug-in directory
cd plugins

#check if we already have downloaded the package. You may change the destination to anything you like.
#I store the packages in /usr/share
if [ -f /usr/share/$package_name ]
then
 sudo cp /usr/share/$package_name ./$package_name
else
 wget $1
 sudo cp ./$package_name /usr/share/$package_name
fi

# untar
tar zxpf ./$package_name
mv ./$plugin_raw ./$plugin_name
sudo rm ./$package_name

# remove package.xml file
if [ -f ./package.xml ]
then
 rm package.xml
fi

exit 0