Tiny PHP Docker images

Docker has been around for a while now and I had my go at a few systems.
With some trial and error I worked up an intermediate knowledge of the inner workings of docker and how it feels to develop with it, from a developer and a devops perspective.

The DevOp in me wants to try out all the new tool, frameworks, scripts to automate as much as possible for my development, however the developer in me is very unhappy having to wait for hundreds of Megabytes (on larger projects it is in the GB range) of images to load and in the worst case I waited an hour for an image build to finish.

Hence I had to do something. I had to find a way to make those images smaller.
On one project we are using eZ Platform, which comes with a docker toolbox.

While developing with it, I don’t notice much, however when I send my project to be built on the CI server, I will always have to pull the base images provided which are quite large:

ezsystems/php 7.0-v1-dev 522.9 MB
ezsystems/php 7.0-v1 451.1 MB

This is a little bit of an overhead, so I searched in my scripts collection and put something together that will build me a stand alone php package, and called it packager which makes use of dockerize.

I can feed packager with a json file which describes which apt packages I want installed, which essential binaries I want and which applications I want to have packaged. Packager then uses an ubuntu:xenial image to execute those instructions, put all the files together and tars it out into the unix pipe.

Contents of package.json

{
"pre-commands": [],
"post-commands": [
"php -r \"copy('https://getcomposer.org/installer', 'composer-setup.php');\"",
"php composer-setup.php --install-dir=/tmp/build/usr/bin/ --filename=composer",
"php -r \"unlink('composer-setup.php');\""
],
"essentials": [
"env",
"bash",
"sh",
"dash"
],
"packages": [
"libcurl3",
"libedit-dev",
"libgd-dev",
"libmemcached-dev",
"libldb-dev",
"libxpm-dev",
"libxslt1-dev",
"libicu-dev",
"php",
"php-curl",
"php-fpm",
"php-gd",
"php-intl",
"php-memcached",
"php-mbstring",
"php-mysql",
"php-mysqlnd",
"php-pear",
"php-xml"
],
"bin": ["php", "php-fpm7.0"]
}
cat php.json \
| docker run \
--rm \
--name pckgr \
-ia stdin \
-a stdout \
-a stderr packager \
> build.tar

Then I define a Dockerfile for my image:

FROM busybox
ADD build.tar /

ENTRYPOINT ["php-fpm7.0", "-F"]

docker build -t local/php .

After I switched the PHP images the eZ application image was now only about 82MB small. The CI Server only needs to build the base image (which takes about 2 – 5 minutes to build) once a week and the application builds have cut down in time by about 6 minutes.
I have tried this with PHP and NGINX but failed to get erlang to run with it. Maybe some systems are not supposed to be run in isolation.
Check out my repository and feel free to contribute.

Silence is Golden

Following the recent NSA incidents made me read a bit about secure internet communications and alternative networks to the world wide web. Stumbling upon a few mesh projects and brushing up on my trivial knowledge about cryptography made me excited about entering what could be the beginning of the end of the internet as we know it but I’ll leave that for another post.

What I want to talk about is the reaction we all or most of us had when we read the news. People are seemingly more scared and more aware about their internet privacy and took to the streets against their own governments.

Looking back on the events I was surprised at how outraged people were by  their governments “listening” . Surprised because since the birth of the internet intrusions have been a big problem.

Latest with Captain Zap’s hack on AT&T in 1981 it was clear that networks had to be secured from external intrusions the so called “hackers”. But intercepting communications didn’t just start 30 years ago, communication interception has been present ever since… well ever since a human being was able to communicate. If encrypted writings dating back 1900BC and codes have been cracked ever since, does it leave any hope for others not to put their nose where it does not belong?

Giving that humans had almost 4000 years experience in trying to know what the other is saying and trying to hide your thoughts it does not surprise me at all that there is basically no privacy on the internet, or telephone lines, or simple mail. Someone somewhere will always want to know what you are up to, might it be your grandmother, a bored hacker or your beloved big brother.

My guess is the people were shocked that the listener was this time someone they were supposed to trust. The government has now stepped into the shoes of the hacker, the person they have always warned us from but we all should have seen this coming 4000 years ago. If you don’t want someone to listen to what you are saying, then don’t say it.

TwigSfHelper2ExtensionsBundle Release

It has been a while since I published some code online, however lately I had some time to look through my things and see what I can package as a plugin/bundle.

Since I have started development with Symfony2 rather than only Symfony 1.x I ran ahead and created my first bundle; the TwigSfHelper2ExtensionsBundle. Simply said this bundle is just a port of some of the Symfony 1.x text and date helpers into Twig filters.

Twig Filters included

  • distance_of_time_in_words: Displays two dates as “time ago”
  • time_ago_in_words: Same as distance_of_time_in_words but shows the difference of now
  • auto_link_text: Turns all urls and email addresses into clickable links. The +link+ parameter can limit what should be linked. Options are :all (default), :email_addresses, and :urls.
  • excerpt_text: Extracts an excerpt from the +text+ surrounding the +phrase+ with a number of characters on each side determined by +radius+. If the phrase isn’t found, nil is returned. Ex: “hello my world” | excerpt(“my”, 3) => “…lo my wo…”
  • simple_format_text: Returns +text+ transformed into html using very simple formatting rules Surrounds paragraphs with <tt>&lt;p&gt;</tt> tags, and converts line breaks into <tt>&lt;br /&gt;</tt> Two consecutive newlines(<tt>nn</tt>) are considered as a paragraph, one newline (<tt>n</tt>) is considered a linebreak, three or more consecutive newlines are turned into two newlines
  • strip_links_text: Turns all links into words, like “<a href=”something”>else</a>” to “else”.

Installation

Step 1: Add bundle to your deps file
[TwigSfHelper2ExtensionsBundle] 
git=http://github.com/mozzymoz/TwigSfHelper2ExtensionsBundle.git 
target=bundles/JustMozzy/TwigSfHelper2ExtensionsBundle
Step 2 Add bundle to your autoload.php and AppKernel:
'JustMozzy' => __DIR__.'/../vendor/bundles'
new JustMozzy\TwigSfHelper2ExtensionsBundle\TwigSfHelper2ExtensionsBundle(),

 

 

Working with CS Cart: Template Suggestions

It has been a little while since I posted my first “Working with CS Cart” entry. The previous post showed you how to change the footer of an existing CS Cart skin, something useful but not really complex like for example selecting different page templates depending on the page or controller you are viewing.

One of my clients had some very varying templates for different sections of the cart. If you are familiar with Drupal you know that the Drupal system looks automatically for specific template names depending on the currently requested URL based on specific criteria. Essentially this is exactly what I needed CS Cart to do, however I had to include a little hack in order to implement such a mechanism.

Suggestion Criteria

Usually the frame of a website design stays the same which would logically reside in the index.tpl, however the display of the actual content might be different for several sections/pages in the cart. For this we will have to modify the main.tpl file. Here you will see the default way of displaying the “frame” for content in CS Cart but what if we had a design that has a different structure for the home page and inner pages? Further more, what if one category has a different structure than the rest? Well now we are getting closer to build up our criteria. To make things short let me sum up the criteria I have setup for my client’s project:

  • Each controller (index, pages, products, categories etc.) may have a different structure
  • One or more pages may have a different structure
  • One or more categories may have a different structure
  • One or more products may have a different structure

Having those criteria defined gives us a great deal of flexibility to basically be able to control the content’s “frame” in any situation.

Smarty Modifier: Looking for a template

Unfortunately CS Cart nor Smarty have any built-in functionality to check whether or not a template exists from within a template, so I had to go in and quickly implement such a functionality. In order to keep things simple and not change any core code I had to think of a way to implement template suggestions from within a .tpl file which would require extending Smarty. Ideally an implementation of the required functionality would look something like this

{if $page_template|valid_template}
  {include file="`$page_template`"}
{else}
  {include file="default.tpl"}
{/if}

Looking through the Smarty documentation you will learn that this function is a Smarty modifier but in fact we are not modifying any values only running a quick check on the existence  of a template file. So we will simply create a new modifier for Smarty in order to do the check. Go to your CS Cart installation and create a file called modifier.valid_template.php in the /lib/templater/plugins open it up and copy and paste the following code into it

<?php
/**
 * Checks the presense of a template.
 * Usage example:
 * {if $page.include_file|valid_template}{include file=$page.include_file}{/if}
 */
 $GLOBALS['smarty'] = $smarty;
 function smarty_modifier_valid_template($template_name)
 {
   $smarty = $GLOBALS['smarty'];
   return($smarty->template_exists($template_name));
 }
?>

We are simply creating a wrapper for the Smarty template_exists function. Now whenever we use this modifier we can check if a template path is actually an existing template.

Putting it together

Now we have our function to check if a template exists and our criteria. It is time to put it all to work. First of all we will create the implementation of our first criteria “Each controller may have a different structure”. For this back up the main.tpl file before editing it. Once this is done simply rename the backup to “default.tpl” so that we have a fallback template in case no specific templates were found for any of the controllers. Now we will replace the content of main.tpl with this

{capture assign='page_template}main/page_{$controller}.tpl{/capture}
{if $page_template|valid_template}
  {include file="`$page_template`"}
{else}
  {include file="main/default.tpl"}
{/if}

It is as simple as these 6 lines to have custom layout for each of the controllers. Note that this snippet will look for the templates in the directory “main” which will have to be created manually and all page_<controller_name>.tpl files will reside in there.

Now you can create for example a page_index.tpl in order to differentiate the structure of the main.tpl for the homepage from the rest of the site. For any other controller simply create a file called ‘page_<controller_name>.tpl’ in your skin. If you are not sure about what controller is used for the section you want to change, simply enable debugging from the administration, navigate to the page you want to have changed and look up the “$controller” value in the debug window.

This change will only comply to our first criteria; we need more. For this we will need to create some more files and folders. I will document here how to implement suggestions for pages only but the same method can be applied to categories and products too. For better maintainability we create a new folder called “pages” in the “main” folder where all the page specific templates and a default.tpl fallback will reside. Also create a file called “page_pages.tpl” in the “main” folder. Here we will add the following code

{capture assign='page_template}main/pages/page_pages_{$page.page_id}.tpl{/capture}
{if $page_template|valid_template}
  {include file="`$page_template`"}
{else}
  {include file="main/pages/default.tpl"}
{/if}

Now we have setup a mechanism for CS Cart to look for main/pages/page_pages_<page_id>.tpl for page specific structures. Simply create folders and files for the products and categories controllers to do the same and you’re good to go.

Simple, ain’t it?

Predator: Intelligent Camera

Earlier this year PhD student Zdenek Kalal came up with an algorithm that can virtually track any object in a video feed. This can be used in a huge variety of applications but I think the video Zdenek prepared can explain things better to you. Predator might change a lot, looking back on one of my earlier posts you may get some ideas.

If you want to know more about the project visit Zdenek’s page

Working with CS Cart: The basics

As I announced earlier I will be writing in the next few weeks about CS Cart customization. In this post I will talk about the basics of the CS Cart structure and how you can start to make changes to existing Skins (by changes I mean structural, not design changes). I will assume at this point that you already downloaded and installed CS-Cart on your development environment and are ready to go on with the customization. In case you have not, check out http://www.cs-cart.com for more information on the different licenses and how to install the software.

Basic Structure

Looking at the directory structure you will notice the skins directory in your installation. This is where CS Cart stores the currently installed skins for your cart software. During the installation you were asked to choose a skin, if you haven’t chosen the basic skin, please do so from the administration section. Within the “basic” directory in the “skins” directory you will find three more entries, “admin”, “customer” and “mail”. These three directories refer to the three main sections of CS Cart skinning, one for the administration section (by default accessed by http://www.example.com/admin.php), the frontend of the cart (http://www.example.com/) and any from the system sent emails. Each of those directories has a number of files and sub-directories for you to play with, however you will later learn that you will probably not need to work with any of them, but rather create your own files to override existing files.

Most, if not all the files residing in these directories are .tpl files which are Smarty templates. If you are not familiar with Smarty I strongly suggest that you go over to their website and check out their documentation.

Template Hooks

I had to learn the hard way that most changes in CS Cart can be done using template hooks. The CS Cart documentation mentions template hooks briefly but doesn’t give very good examples or explanations on how to use them. What are template hooks you ask? According to CS Cart’s documentation,

Template hooks are parts of a template enclosed in the tags “{hook name=”section:hook_name”}{/hook}” that can be supplemented or completely redefined by any addon.

So how do we utilize these hooks? Let’s do this by example; open up the index.tpl file in the customer directory. In the bottom you will find a hook defined just before the body close tag. We will use this hook to add some static HTML to all pages in the store front. The CS Cart documentation says that hooks can be used by addons; does this mean we need to create an addon? Simply said, yes, however the guys over at CS Cart were kind enough to add an addon called “My Changes” which is essentially an empty addon used for customization purposes, so we won’t have to create one. This addon is by default enabled. Have again a look at the hook definition in the index.tpl, note down the section in the name attribute (“index”) and the hook name (“footer”). This information is enough for us to get things started.

In the addons directory under customer create now the folders “my_changes/hooks”. The hooks folder is the point where CS Cart will be looking for any hooks grouped by section, naming you will have to also create the sub-folder “index”. Your structure should look like this:

skins/
  basic/
    customer/
      addons/
        my_changes/
          hooks/
            index/

Before we go on and utilize the hook we will have to decide how we want to add the static HTML content. There are three types of utilization: pre, post and override. The pre hook will prepend any content to the defined hook and the post hook will append to the defined hook. This is very practical if for example the hook is enclosing any content already contained in the original template. An example for this is the index:main_content hook in the main.tpl template, but more to this file later on. The override hook will completely replace the predefined contents of the hook area. We currently do not know if other addons are utilizing the footer hook and we want to be sure that the static HTML is in the middle, meaning not prepended nor appended to the content, hence we will use the hook override. We will have to tell CS Cart somehow that we want to override the hook’s content, this is easily done by creating a file with the name “footer.override.tpl”. In general the filenames are as follows:

  • hook_name.pre.tpl – Add changes before content
  • hook_name.post.tpl – Add changes after content
  • hook_name.override.tpl – Replace content with changes

Simply open the footer.override.tpl and add any static HTML to it, like for example:

<p>
  I added this by reading this <a href="https://mozzymoz.wordpress.com/2011/10/26/working-with-cs-cart-basisc/" target="_blank">article</a>
</p>

Now simply go to your storefront, refresh and see the changes taking effect in the bottom of the page. In case you cannot see the changes, go to your administration and add ?cc to your query string in order to clear the cache.

This is all it takes to make changes to an existing skin. I do admit that this example is not really something that is useful, however you could play around with this code and see that you could for example easily add a Share This widget to all of your pages.

Working with CS Cart

In the past week I have been put to the task to implement a new skin for the CS Cart eCommerce Software. Usually the projects I work on are based on either Symfony framework, Drupal or WordPress; taking up this project on a new platform with a 2 week turn around was quite a challenge.

One might expect when purchasing a software which advertises itself as “a perfect platform for custom eCommerce requirements”, it would be thoroughly documented but I had to find out that this was not the case. Of course one can find answers around the Internet. However it is a tedious and time wasting task. I was left with having to dig through core code and debugging to somehow get a grasp on the internals. To CS Cart’s defense, the design template I received to integrate into CS Cart was extremely customized.

On this note I will start writing up a small series on development with CS Cart covering basic changes to existing skins, writing add-ons and creating your own skin.

Few thoughts on intellectual property

The other day I watched a DVD movie and as usual one of those FBI warnings came up. Not really paying attention to the warning, another message came up which read “You are personally responsible for this disc and its content. This screener is digitally watermarked to identify you, the member. Do not loan [sic], rent, sell give away or otherwise transfer to any third party for any reason.”

This message really got me thinking; “personally responsible”. What do they mean by “personally responsible”? How can I be personally responsible about an item that is actually not my property? Yes I do understand that I would be held responsible if I illegally redistributed the DVD but the words got me thinking about who’s property the content of the DVD actually is? Should anyone actually “own” that content?
Movies, books, music, a lot of creative content makes a lot of money as we see day to day. Actors, directors, musicians, producers are among top earners and I totally understand that they want it to stay that way, so like minded “artists” got together and started to fight for the ownership of their “creations” but then who will own the creations of past artists? Of course this would fall into inheritance laws etc., but that is anyway not my point. Many artists are afraid that if they would not protect the ownership or copyright of their ideas they would not be able to make a living (or in other words a shit load of money), but if they lost that ownership and no one would have it, would that mean that they loose all their wealth and income?
In reality of course there are labels and big companies in the play but even they would not have to suffer from making their content freely accessible and redistributable.
It is hard to imagine free content bringing any type of income and logically only a few artists would be willing to publish their content for free as they have to eat also.

But is that really the only consequence?
You publish content for free and you don’t gain from that?

Let’s have a look at Wikipedia for example. Wikipedia is the single largest free online encyclopedia and is mantained by millions of people sharing and spreading information which is sold by other companies in expensive data DVDs or books. But how can all ths be maintained without taking a single penny for the information. Well ofcourse for one it is mostly maintained by volunteers but Wikipedia does employ a large amount of writers and IT people to keep the system up and running and to make sure that the community is always able to update information. How is all this financially maintained?
The answer is donations. Wikipedia’s and many other company’s main income are donations which make sure that the product is always available.

If they can do it, why can’t music or movie labels do it? I mean, these very companies will never be able to stop people from sharing art and ideas and the sharing of those is the only way to improve and spread them so why not simply give in and try to make a lot of money and on the side benefit the community?