Jenkins on Windows 7

Keith Casey did a talk a talk on Continuous Integration with Jenkins at Sunshine PHP. The first half of the talk was devoted to getting Jenkins installed on our laptops. In a room full of Mackbook Pros, I was one of the few Windows desktops. What follows is my experience with getting it working.

Keith talked about a lot of tools which help to check code quality including

  • Lint
  • Phing
  • CodeSniffer
  • PHPCPD
  • PHPMD
  • PHPLOC

Install PHP and Pear

  1. Download PHP and install it to c:\php. Add c:\php to your PATH variable.
  2. Download go-pear.phar and save it to c:\php. The information for that is here.
  3. Open an Administrative command prompt
  4. CD to c:\php and run php go-pear.phar. Press enter through all the prompts. Now you have Pear.
  5. Paste the entire giant blob of code into your Administrative command prompt
REM install phpunit
pear channel-update pear.php.net
pear upgrade-all
pear channel-discover pear.phpunit.de
pear channel-discover components.ez.no
pear channel-discover pear.symfony-project.com
pear update-channels

REM install phpqatools
pear config-set auto_discover 1
pear install pear.phpqatools.org/phpqatools

REM install phing
pear channel-discover pear.phing.info
pear install phing/phing-alpha

REM add PSR codesniffer
cd c:\php\pear\php\CodeSniffer\Standards
git clone https://github.com/klaussilveira/phpcs-psr PSR
phpcs --config-set default_standard PSR

Pear will park .bat files in c:\php for all the modules you installed. Notice that phing is installed as phing-alpha. This is only necessary until 2.7.0 is promoted to stable.

You also need to install Git for Windows.  Use MsyGit and pick “Git from Windows command prompt”.

Lastly, edit your php.ini and enable the extension php_xsl.dll

Install Jenkins

This was pretty straight forward. It’s a Java based project, and there is a Windows installer. Make sure to run the setup.exe, not the MSI.

Once Jenkins is installed, connect to http://localhost:8080/. If the Jenkins dashboard opens, it works!

The web2project example

Keith used one of his own projects, web2project, to demonstrate Jenkins. Getting the toolchain working on windows was a challenge.

  1. Login to the web console
  2. Click Manage Jenkins –> Manage Jenkins
  3. Under Updates, click available updates
  4. Install: Git, Github, PMD, Phing, CPD. A restart of the jenkins service may be necessary
  5. Go back to the dashboard
  6. Click New Job –> Build a freestyle project called web2project
  7. Under source code management, pick git and use the path http://github.com/web2project/web2project
  8. Under build pick a custom windows batch command and fill in: cd unit_tests & phing metrics
  9. Click save and build now
Advertisements

Git for Windows

Git is the popular source control tool which has become the darling of the internet. Unlike subversion, git keeps a local copy of the repository, allowing offline commits. This makes it a lot harder to do a full Windows port. Luckily, there is a package called msygit, which has all the dependencies bundled with it.

The default is to run git in a BASH prompt, but I can’t imagine why you would want to do that. If I’m running on the windows command prompt for other stuff, I don’t want to switch out of it into some hackish BASH prompt to run git. I just switch that option and let the good times roll.

Line endings are a little more complication. I use the default “check out Windows style, check in Unix style”, but I’m thinking to fine tune that a bit. Any good Windows text editor can handle Unix line breaks. If I’m doing a module for MediaWiki or Drupal, I might force it to stay in Unix mode. Conversely, if I’m doing something very windows-centric like ADSI or WMI scripts in PHP (or even VBS), then I’ll want it Windows style on the remote side. The idea is that if someone were to download a ZIP of a repo from git hub, they should be able to view that file using the native console tool (cat or type), and not have some crazy line breaks. More information about handling it is in the GitHub help article “Dealing with line endings“.

One other small thing is that on Windows systems with an NTLM proxy (looking at you TMG), you need to specify the proxy settings in the environment variable http_proxy and https_proxy. Setting it in http.proxy didn’t seem to do the job. I found a bunch of posts dealing with this, and having something to do with git not calling –proxy-ntlm on cURL and it being a hassle to override. I wimped out and used the environment variable.

Server side browser features with phpcaniuse

The Browser Capabilities Project is an effort to determine browser capabilities based on the User-Agent string. It’s a great project which has kept up to date with UA stings. The data is used to power the PHP function get_browser(), and is used by the phpbrowscap standalone project. The challenge with the project is that their features haven’t kept up with the rapid development of web technology.

The CanIUse.com project catalogs granular data about the capabilities of web browsers. They’ve done an excellent job of tracking emerging features like HTML5 canvas and web audio. They also make all of the data available as a json file on github. Thanks!

So what do you do when you have two different projects, with different goals but  some overlap? You create a mashup! Enter PHPCanIuse, which will let you check browser capabilities from the caniuse.com project, based on the decoded UA string from the browscap project.

JQuery and other javascript frameworks have client side feature detection, but there are a few scenarios where you might want to do it server side:

  • Embed a Flash video player or an HTML 5 video
  • Use native websockets or flash based (or, ew, timer based)
  • Send SVG as native markup, or pre-render it as a PNG

DISCLAIMER: Users can theoretically change their UA string, so it can’t be trusted 100%. It’s an advanced feature, so anyone who does, deserves the broken web experience they get.

Using phpcaniuse

phpcaniuse is a composer project. You’ll need the following require:

"robertlabrie/phpcaniuse": "dev-master"

Then as usual just composer install. The class is a single file, so if you hate composer, you can  just include CanIUse.php. Then just instantiate the object

$can = new phpcaniuse\CanIUse($browser,$data);

The $browser object can come from either get_browser() or phpbrowscap. CanIUse expects the object, not the associative array. $data is a string containing the contents of data.json from the caniuse.com project. Pass it in as a string, do not decode it.

The methods are pretty straight forward:

Function Description
check(list) Checks to see if the browser supports the specified list of features. Takes a string of a single feature, or an array of features. Returns the lowest level of support for the list.
featureList() Returns an associative array of all the features tracked by caniuse. The key is the key used to check a feature, and value is a friendly description.
featureGet(feature) Retuns the set of JSON data for the specified feature.
browserCan() Returns an associative array of all features and the status within the browser. Key is the feature, value is the status.
agentMapAdd(browscapName,caniuseName) Adds a mapping between a browscap name and a caniuse name.
browserSet(browser) Sets the browser object. Must come from get_browser() or phpcaniuse.
dataSet(data) Sets the json data from caniuse. Must be a string, not a json object or array.
dataGet() Returns the caniuse json data as an array.

phpcaniuse maps the browser name returned by browscap with the one used by caniuse. It’s currently done for Firefox, Internet Explorer, Opera, Safari and Chrome. Mobile browsers are not done, but if you can identify a mapping, please tell me.

The package includes a demo.php file which will tell you all about your browser.

Understanding the CanIUse.com data.json

The caniuse.com data.json file is broken up into several sections. I’ve laid it out in a table with my working notes, but the best way to understand it is print_r().

agents ie array of browsers – major ones listed here, many others supported
firefox
browser Firefox Seems to match the ->Browser property from browscap
versions …,”24″,”25″ array of browser versions where the index is used by caniuse
chrome
safari
opera
statuses status codes for features
rec Recommendation
pr Proposed Recommendation
cr Candidate Recommendation
wd Working Draft
other Other
unoff Unofficial / Note
cats categories for capabilities
updated date() the date the data was last updated
data
title A friendly title
description A good description
spec URL to specification
status Relates to the status codes above
stats Associative array of browsers, version feature
firefox Or any agent name
23 Vsersion 23
x Supported or not

The actual status codes are outlined below. They’re based on reverse engineering and might not be 100% accurate:

Stat Description
n Not supported
a Partially Supported
a x Partially Supported with prefix
p Polyfill required
y x Supported with prefix
y Supported
u Unknown

PHP, WMI and Microsoft NLB

I’ve been playing with my NLB cluster, and wanted to find a way to evict nodes which were failing some test. This functionality exists in most hardware load balancers, but isn’t something Microsoft NLB does natively. Controlling NLB nodes is done using WMI, and the classes are fairly well documented. I was also fishing for an excuse to try ReactPHP, which is an event driven non-blocking library for PHP.

The result was my MicrosoftNLB class and the simplewatcher.php script. I used wmi2php to generate the boiler plate, and write a wrapper class called NLBNode. There is still a lot of work to be done, but it’s achieved MVP for me. Simplewatcher.php which will get all the peers in an NLB cluster, challenge them for a specific URL, and evict any node which fails. This is good for taking a node out of load (rename nlb.php to nlb.oos), or for stopping a node if IIS stops responding. If a node fails at the network layer (ie hardware or OS failure), it will stop participating in the cluster automagically. It will not put a node back into load, you must do that manually.

 

This is also my first composer project, and you’ll need to use it to get ReactPHP installed. To use the node watcher, just edit the first few lines of examples\simplewatcher.php, and run it from the command line.

PHP detect browser name and version with phpbrowscap

This is an old problem: how to programatically detect the visitors browser and version. There are some very good reasons for wanting to do this:

  • Known compatibility issues
  • Customizing download or support links
  • Leveraging advanced features in newer browsers

Googling around for this, you always find the objection “someone can use a fake User-Agent string, so don’t bother”. That’s an obnoxious and unhelpful remark which should be deleted from forums. The vast majority of users do not adulterate their UA string, and those who do have to accept the broken experience that comes with it.

PHP has a native function called get_browser() which does the job. It relies on the php_browscap.ini file downloaded from the Browser Capabilities Project. To enable it you set a directive in the php.ini file and you’re off to the races. There is one catch: it can only be configured system wide. So what happens if you’re on shared hosting? Or if you want to keep your own version in git and not use the system version? The solution is phpbrowscap.

Before I get into browscap, there is one other thing tip I want to cover. What if you want this information client side? The Navigator.appName property appears promising, except that Mozilla, Chrome and Safari all return the exceptionally useless “Netscape” value. Only MSIE actually tells you who it is. You can use a server side snippet like this to expose the data client side:

echo "<script type=text/javascript>var Browser=" . json_encode($browser) . ";</script>\n";

This will work for you whether you got $browser from get_browser() or from phpbrowscap.

Phpbrowscap is fully compatible with get_browser() and is usually faster. It processes the browscap.ini file and writes it out to native PHP code so that subsequent loads don’t have to parse the ini. The simplest possible usage is:

require 'Browscap.php';
use phpbrowscap\Browscap;
$bc = new Browscap('/tmp');
$browser = $bc->getBrowser();

The default behaviour is to automatically download the latest version of the browscap.ini file and store it in the cache directory. If you have no access to write anywhere, and do not want auto-update, you need to do something like this:

$browscap = new phpbrowscap\Browscap(__DIR__);
$browscap->doAutoUpdate=false;
$browscap->updateMethod=false;

For this to work, you must have a copy of browscap.ini and cache.php as generated by phpbrowscap. What I did was run the above version on some development box and copy the two files from /tmp to my source directory, then added them to my git.

The documentation isn’t great, so I’ll try to help out a little:

Directive Properties
doAutoUpdate true – default, also false
localFile Path to local copy of browscap. Used for offline updating.
updateMethod Browscap::UPDATE_LOCAL – use a local file specified in localFileBrowscap::UPDATE_FOPEN – Uses the fopen url wrapper (use file_get_contents)Browscap::UPDATE_FSOCKOPEN – Uses the socket functions (fsockopen)

Browscap::UPDATE_CURL – Uses the cURL extension

null or false – do not attempt update

cacheFilename Override the default cache file name of cache.php. Useful if auto-update is off.
iniFilename Override the default ini file name of browscap.ini. Useful if auto-update is off.

There are also some useful helper functions:

  • autodetectProxySettings() – gets settings from the *_proxy environment variables
  • addProxySettings($server, $port = 3128, $wrapper = ‘http’, $username = null, $password = null) – pretty obvious. If you’re wondering, 3128 is used by cntlm.
  • clearProxySettings() – I guess you might want this
  • updateCache() – Rebuild the cache file from ini

Creating a new composer packageist package

Composer is a package manager for PHP, similar in purpose to NodePM for NodeJS. The getting started section is pretty good at pulling in other packages, but I had a hard time getting to “ok, I want to manage my project with composer”. I finally found a post from Junior Grossi which got me started. Enthusiasts will probably have more to say, but it comes down to 4 parts:

  1. Your source
  2. Composer.json
  3. The autoloader
  4. Required packages

Your source

Your source, and the autoloader go together, but I’ll discuss them separately. Basically your source has to be laid out to work with the PSR-0 autoloader. You can’t use your own auto-loader anymore. Your classes must all be namespaced, with the directory structure following the namespace structure. If you had a class like MicrosoftNLB\NLBNode you would use something like this:

your-working-directory
 |-src
 | \-MicrosoftNLB
 |   \-NLBNode.php
 |-examples
 \-test

Remember your source must be namesapced so NLBNode.php will start out like this:

<?php
namespace MicrosoftNLB;
class NLBNode extends MicrosoftNLB_Node
{

composer.json

So now you have your source laid out in a way which will appease composer. You need to create a composer.json file. Here is one:

{
    "name": "robertlabrie/microsoftnlb",
    "description": "Classes and tools for Microsoft NLB",
    "license": "GPL V2",
    "authors": [
        {
            "name": "Robert Labrie",
            "email": "robert.labrie@gmail.com"
        }
    ],
    "minimum-stability": "dev",
    "require": {
        "react/react": "0.3.*@dev"
    },
    "autoload": {
        "psr-0": {
            "MicrosoftNLB": "src/"
        },
        "classmap" :[ "src/MicrosoftNLB" ]
    }
}

Name, description, authors, all pretty straight forward. The minimum-stability setting is for your dependencies (see below).

The autoloader

This one tripped me up a bit. You need to use the composer auto-loader so that everyone’s packages can co-exist. Setup comes in the form of an array of namespace=path where path is the path where the namespace directory can be found. Junior’s directory layout which I shamelessly copied is what really made this make sense. So in the example above, the autoloader will search in src/CoolProject for classes. Remember your classes have to be namespaced or this thing won’t work.

Classmap

Or will it? The classmap section lets you bolt directories into PSR-0. Basically anything in here will be included for classes. The other case where you need to use a classmap is if classes have underscores in them. PSR-0 will replace underscores with slashes. Since my project uses boiler plate WMI classes, I had class names like MicrosoftNLB_Node, which was being turned into MicrosoftNLB/Node, which was then breaking. Thanks to Juan Treminio in #composer for that advise.

Required packages

This is also pretty straight forward and well explained at getcomposer.org.

Examples and testing

Your testing will probably use PHPUnit or similar. Examples can go in /examples, and will look something like this:

<?php
require_once __DIR__."../vendor/autoload.php";
$node = new MicrosoftNLB\NLBNode('localhost');

Including the autoloader from vendor directory lets composer do all the magic.

wmi2php

I’ve been playing with my NLB cluster and want to use WMI to control the nodes. The PHP COM library works great, and I’ve had some success. The properties of COM objects aren’t exposed to you, so if  you print_r($somevar), you get nothing. I’ve been using ScriptomaticV2 and the NLB WMI documentation to muddle through it, and I found I was doing a lot of copying and pasting. Well no more! I worked through the major methods in Scriptomatic and re-implemented them in PHP. Now I can export PHP class files for any WMI class.

Check it out in GitHub.

It should be noted that I’m not a huge fan of “getters and setters”, so this uses overrides and an array of valid properties. I’m planning to expand it to create more “native” classes.

So now I can:

wmi2php.php generate --namespace=root/CIMV2 --class=Win32_ComputerSystem

Enjoy.