CS Cart Devops and Deployment

I’ve been working with CS Cart quite a bit these past few months, and while it’s been painful (which is readily apparent to those PHP 5.5+ programmers who’ve had to work with it), I’m starting to get parts of it wrangled into place and I’d like to share some of those tips with you.

[NOTE: the version of CS Cart I’m working with has some significant internal data structure modifications to better align the product with our needs, but almost everything I discuss going forward should be applicable]

First, trying to get a proper devops development -> staging -> QA -> production environment has been a struggle. It’s very similar in deployment to WordPress in that most of the configuration is stored in a database table, and it’s hostname-based.

Single Developer Devops

In a single developer environment, your devops flow is pretty simple – you’re developing locally (something like admin.domain.dev), hopefully in a Vagrant environment, and you may or may not be syncing your production database to your local dev environment, transmogrifying the domain. In fact, you don’t even have to include Github in the loop – you could push directly to the production server.

Multiple Developer, Staging/QA/Production

In our multiple-developer setup, each developer has his/her own local dev environment as they see fit. I personally use Vagrant to run Ubuntu, and I configure my Vagrant environment with the same Ansible scripts that I use to create/configure the staging, QA and production servers, so my local dev environment exactly matches what the code is actually going to be running on.

One caveat in our circumstance is that we feel the developers do not and should not have access to production data in either their local dev environments or on the staging server, both for security purposes as well as data leakage (hint: think about accidentally emailing to actual customers or accidentally adding/changing/deleting billing info).

So I develop and test locally, and when I’m ready to have others review my work, I push to Github. It’s important to have a git hook that lints your commit(s) before you push them up. Github has a Webhook to auto-POST to a script on our staging server which initiates a Github pull from the staging server. Some people would discourage an auto-pull but the staging server for us is a “throwaway” shared server that we use to allow others to test out functions and features, so we don’t mind if a commit breaks it.

[Since there’s only two developers, we’ve been doing most of our development into ‘master’, but you can also develop into a new branch, push that new branch through and (manually) switch to that branch on staging for testing]

The biggest thing that tripped me up in CS Cart development was remembering to “flip” any addons that have changed, and I wrote some scripts to automatically do that for me.

When we’re ready to test Github code against production data, we run a production sync script on the QA server that pulls down a snapshot of the production database (while changing the CSC hostname: qa.domain.com), and also the images directory, so the production product data will display properly.

If cost are an issue, it should be noted that you could eliminate the QA server and just “test” on staging with production data. We’ve found it handy to have staging as a “punching bag” where we can deploy and test with wild abandon, and QA being the server that mirrors production for the most realistic simulation of code and data, but your mileage may vary.

Once we feel the new code is tested out thoroughly on QA, we go ahead and pull over to production and test thoroughly as well. I would feel much better about this process if CS Cart had unit tests but unfortunately there aren’t any at this point.

This is our current devops flow for developing CS Cart code. I’m sure there are other ways to do it, but this works for now. There are certainly other changes that I’d like to make, including modifying the ‘refresh_db’ script to discard production customer data and perhaps swap it with dummy info.

One last thing I’d like to point out is that this same devops workflow could be used for WordPress code deployments, using the Search and Replace script in place of ‘refresh_db’.

CS Cart git hook script to auto-flip

One of the biggest problems I’ve had in CS Cart development was remembering to disable and then re-enable any addons that had changes. I’ve created some scripts to help the devops development flow in CS Cart:

https://github.com/heybige/cscart_cli_scripts

‘cli_lib.php’ is a stripped down version of ‘admin.php’ that I use to bootstrap into CSC from the command line. ‘clear_cache.php’ should be self-explanatory – it just clears the cache from the command line.

HINT: if you’re having any kind of problem with CS Cart, always try clearing the cache. And you may consider doing a “hard reset” (rm -rf var/cache/* in your root CSC directory) because there are files that don’t get cleared out with a standard “clear cache” command.

So the magic starts with a git hook script ‘post-merge’ (.git/hooks/post-merge):

#!/bin/bash

export ROOT=/path/to/cscart/base

## HARD RESET
rm -rf $ROOT/var/cache/*

## RUN COMPOSER
cd $ROOT/app/lib
composer install

$ROOT/bin/flip_addon.sh

Pretty standard. It’s worth noting that you can add your own Composer packages in the existing composer.json file and use them throughout CSC. I’ll do a future post about how I added some custom logging to CS Cart (lack of file-based logging is another of my huge CSC pet peeves).

#!/bin/bash

export ROOT=/path/to/cscart

ADDONS=()

ADDON_LIST=`cd $ROOT; git diff "HEAD@{1}" --name-only | egrep 'app/addons/.+?/addon.xml'`

if [ ! -z "$ADDON_LIST" ]; then
    for file in $ADDON_LIST; do
        OIFS=$IFS
        IFS='/'
        arrItems=($file)
        ADDONS=("${ADDONS[@]}" ${arrItems[2]})
        IFS=$OIFS
    done
fi

ADDON_LIST=`cd $ROOT; git diff "HEAD@{1}" --name-only | grep 'var/themes_repository/basic/templates/addons/'`

if [ ! -z "$ADDON_LIST" ]; then
    for file in $ADDON_LIST; do
        OIFS=$IFS
        IFS='/'
        arrItems=($file)
        ADDONS=("${ADDONS[@]}" ${arrItems[5]})
        IFS=$OIFS
    done
fi

function join { local IFS="$1"; shift; echo "$*"; }

if [ ! -z "$ADDONS" ]; then

        if [ ${#ADDONS[@]} -gt 1 ]; then
                CHANGED=($(printf "%s\n" "${ADDONS[@]}" | sort -u))
                LIST=`join : "${CHANGED[@]}"`
        else
                LIST=${ADDONS[0]}
        fi

    echo "php $ROOT/bin/php_flip_addon.php $LIST"
        php $ROOT/bin/php_flip_addon.php $LIST
fi

Basically, this script just determines which addon files have changed, and builds a (unique) colon-separated list to pass to ‘php_flip_addon.php’

I won’t post ‘php_flip_addon.php’ but it takes the colon-separated addon list, starts building a stack of the addons that need to be “flipped”, taking into account any dependencies. It uninstalls all those addons in the proper order, then re-installs them in the reverse order. If there’s a problem uninstalling, it will immediately reverse the order to try and get back to the “known good” state.