PHP

CS Cart Devops and Deployment

September 24, 2014 by admin

I’ve been working with CS Cart quite a bit these past few months, and while it’s been painful (which is readily apparent to those PHP 5.5+ programmers who’ve had to work with it), I’m starting to get parts of it wrangled into place and I’d like to share some of those tips with you.

[NOTE: the version of CS Cart I’m working with has some significant internal data structure modifications to better align the product with our needs, but almost everything I discuss going forward should be applicable]

First, trying to get a proper devops development -> staging -> QA -> production environment has been a struggle. It’s very similar in deployment to WordPress in that most of the configuration is stored in a database table, and it’s hostname-based.

In a single developer environment, your devops flow is pretty simple – you’re developing locally (something like admin.domain.dev), hopefully in a Vagrant environment, and you may or may not be syncing your production database to your local dev environment, transmogrifying the domain. In fact, you don’t even have to include Github in the loop – you could push directly to the production server.

In our multiple-developer setup, each developer has his/her own local dev environment as they see fit. I personally use Vagrant to run Ubuntu, and I configure my Vagrant environment with the same Ansible scripts that I use to create/configure the staging, QA and production servers, so my local dev environment exactly matches what the code is actually going to be running on.

One caveat in our circumstance is that we feel the developers do not and should not have access to production data in either their local dev environments or on the staging server, both for security purposes as well as data leakage (hint: think about accidentally emailing to actual customers or accidentally adding/changing/deleting billing info).

So I develop and test locally, and when I’m ready to have others review my work, I push to Github. It’s important to have a git hook that lints your commit(s) before you push them up. Github has a Webhook to auto-POST to a script on our staging server which initiates a Github pull from the staging server. Some people would discourage an auto-pull but the staging server for us is a “throwaway” shared server that we use to allow others to test out functions and features, so we don’t mind if a commit breaks it.

[Since there’s only two developers, we’ve been doing most of our development into ‘master’, but you can also develop into a new branch, push that new branch through and (manually) switch to that branch on staging for testing]

The biggest thing that tripped me up in CS Cart development was remembering to “flip” any addons that have changed, and I wrote some scripts to automatically do that for me.

When we’re ready to test Github code against production data, we run a production sync script on the QA server that pulls down a snapshot of the production database (while changing the CSC hostname: qa.domain.com), and also the images directory, so the production product data will display properly.

If cost are an issue, it should be noted that you could eliminate the QA server and just “test” on staging with production data. We’ve found it handy to have staging as a “punching bag” where we can deploy and test with wild abandon, and QA being the server that mirrors production for the most realistic simulation of code and data, but your mileage may vary.

Once we feel the new code is tested out thoroughly on QA, we go ahead and pull over to production and test thoroughly as well. I would feel much better about this process if CS Cart had unit tests but unfortunately there aren’t any at this point.

This is our current devops flow for developing CS Cart code. I’m sure there are other ways to do it, but this works for now. There are certainly other changes that I’d like to make, including modifying the ‘refresh_db’ script to discard production customer data and perhaps swap it with dummy info.

One last thing I’d like to point out is that this same devops workflow could be used for WordPress code deployments, using the Search and Replace script in place of ‘refresh_db’.

Filed Under: Ecommerce, PHP, Wordpress

CS Cart git hook script to auto-flip

September 23, 2014 by admin

One of the biggest problems I’ve had in CS Cart development was remembering to disable and then re-enable any addons that had changes. I’ve created some scripts to help the devops development flow in CS Cart:

https://github.com/heybige/cscart_cli_scripts

‘cli_lib.php’ is a stripped down version of ‘admin.php’ that I use to bootstrap into CSC from the command line. ‘clear_cache.php’ should be self-explanatory – it just clears the cache from the command line.

HINT: if you’re having any kind of problem with CS Cart, always try clearing the cache. And you may consider doing a “hard reset” (rm -rf var/cache/* in your root CSC directory) because there are files that don’t get cleared out with a standard “clear cache” command.

So the magic starts with a git hook script ‘post-merge’ (.git/hooks/post-merge):

#!/bin/bash

export ROOT=/path/to/cscart/base

## HARD RESET
rm -rf $ROOT/var/cache/*

## RUN COMPOSER
cd $ROOT/app/lib
composer install

$ROOT/bin/flip_addon.sh

Pretty standard. It’s worth noting that you can add your own Composer packages in the existing composer.json file and use them throughout CSC. I’ll do a future post about how I added some custom logging to CS Cart (lack of file-based logging is another of my huge CSC pet peeves).

#!/bin/bash

export ROOT=/path/to/cscart

ADDONS=()

ADDON_LIST=`cd $ROOT; git diff "HEAD@{1}" --name-only | egrep 'app/addons/.+?/addon.xml'`

if [ ! -z "$ADDON_LIST" ]; then
    for file in $ADDON_LIST; do
        OIFS=$IFS
        IFS='/'
        arrItems=($file)
        ADDONS=("${ADDONS[@]}" ${arrItems[2]})
        IFS=$OIFS
    done
fi

ADDON_LIST=`cd $ROOT; git diff "HEAD@{1}" --name-only | grep 'var/themes_repository/basic/templates/addons/'`

if [ ! -z "$ADDON_LIST" ]; then
    for file in $ADDON_LIST; do
        OIFS=$IFS
        IFS='/'
        arrItems=($file)
        ADDONS=("${ADDONS[@]}" ${arrItems[5]})
        IFS=$OIFS
    done
fi

function join { local IFS="$1"; shift; echo "$*"; }

if [ ! -z "$ADDONS" ]; then

        if [ ${#ADDONS[@]} -gt 1 ]; then
                CHANGED=($(printf "%s\n" "${ADDONS[@]}" | sort -u))
                LIST=`join : "${CHANGED[@]}"`
        else
                LIST=${ADDONS[0]}
        fi

    echo "php $ROOT/bin/php_flip_addon.php $LIST"
        php $ROOT/bin/php_flip_addon.php $LIST
fi

Basically, this script just determines which addon files have changed, and builds a (unique) colon-separated list to pass to ‘php_flip_addon.php’

I won’t post ‘php_flip_addon.php’ but it takes the colon-separated addon list, starts building a stack of the addons that need to be “flipped”, taking into account any dependencies. It uninstalls all those addons in the proper order, then re-installs them in the reverse order. If there’s a problem uninstalling, it will immediately reverse the order to try and get back to the “known good” state.

Filed Under: Ecommerce, PHP, Scripts, Unix

Bash script to lint only changed PHP files in git before commit

June 7, 2014 by admin

I usually use Phing to manage my lint and unit tests, but I’m dealing with a rather large (existing) project, and Phing is taking too long to lint all the files in the project. I noticed that most of the suggested “only modified files” scripts used PHP, which is fine I guess but it seems like such a waste when a simple Bash script in .git/hooks/pre-commit can suffice:

#!/usr/bin/env bash

if [ "$(id -u)" == "0" ]; then
  echo "You cannot commit as root" 1>&2
  exit 1
fi

FILES=`git diff --cached --name-status --diff-filter=ACM | awk '{ if ($1 != "D") print $2 }' | grep -e \.php$`

for x in $FILES
do
  CMD="php -l $x"
  echo $CMD
  RES=`$CMD`
  if [ $? -gt 0 ]; then
    echo $RES
  fi
done

Filed Under: PHP, Scripts

Laravel 4 – non-standard username/passwd fields in user auth table

June 28, 2013 by admin

I’ve been doing a lot of development in Laravel 4 these days – just a great, great framework! But I’m learning it has certain expectations as far as naming conventions go. An example would be the way authentication is done via Eloquent (ORM). Eloquent’s default authentication fields are ‘username’ and ‘password’. If you want to have something different, you need to extend some functions in order to return the information you want:

// app/models/User.php

use Illuminate\Auth\UserInterface;
use Illuminate\Auth\Reminders\RemindableInterface;

class User extends Eloquent implements UserInterface, RemindableInterface {

protected $fillable = array('name','passwd','email','status','timezone','language','notify');
protected $hidden = array('passwd');

protected $table = "users_t";
protected $primaryKey = "uid";

public static $rules = array(
'name' => 'required',
'passwd' => 'required',
'email' => 'required'
);

public function getAuthIdentifier() {
return $this->getKey();
}

public function getAuthPassword() {
return $this->passwd;
}

public function getReminderEmail() {
return $this->email;
}

public static function validate($data) {
return Validator::make($data,static::$rules);
}
}

The key parts are ‘getAuthIdentifier()’ and ‘getAuthPassword()’ – if you notice, I use a non-standard table ‘users_t’ and I use ‘uid’ as the primary key instead of the expected ‘id’. getKey() picks up the $primaryKey variable.

Filed Under: PHP

PHPUnit fails silently – figuring out the problem

June 22, 2013 by admin

Quick post about PHPUnit – if you’re making changes on your existing tests and all of a sudden PHPUnit starts failing silently, the first thing to do is check the error code:

$ phpunit
$ echo $?
255
$

The error code ‘255’ indicates a parse problem, and it’s probably not displaying because you have ‘display_errors = false’ in your php.ini, but this is easily fixable! Just add the following code to your ‘phpunit.xml’ file:

And you should see what’s causing the problem.