Device Detection

by Bruce Szalwinski


The good folks that power Apache Mobile Filter use version 2 of the Device Repository from 51Degrees and currently have no plans for updating their software to use version 3.  Since we currently use the AMF handler to do device detection and since 51Degrees has announced the end of life for version 2, this provides an opportunity for us to write our own handler.  We attempted to do this in version 2 days, but there was no Perl API offered from 51Degrees and the C code was pretty shaky.  With version 3, the 51 Degrees folks now offer a Perl API that wraps around a much more robust C API.  With that, the stage is set to tackle writing our own Apache Handler.   I’ll use the Apache::Test module to help drive the development.  This article from last decade was very helpful in learning how to use this powerful module.  Full source code is available at DeviceDetection.


Analyze web traffic by a user specified set of device properties.


An Apache Handler allows for the customization of the default behavior of the web server.  We will write a handler that reads the user agent from the request, detects the device associated with the user agent, creates environment variables for each requested device property and writes the values to a log file.  Let’s get started.

Write tests first

To test our handler, we’ll send requests to an apache server, passing in various user agent strings and validating that we receive known device id values.

use strict;
use warnings FATAL => 'all';

use Apache::TestTrace;
use Apache::Test qw(plan ok have_lwp);
use Apache::TestRequest qw(GET);
use Apache::TestUtil qw(t_cmp);
use Apache2::Const qw(HTTP_OK);

use JSON;

plan tests => 6, have_lwp;

detect_device('unknown', '15364-5690-17190-18092');
 "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36",

sub detect_device {
  my ($user_agent, $device_id) = @_;

    reset => 1,
    agent => $user_agent

  my $response = GET '/cgi-bin/index.cgi';
  my $json = decode_json $response->content;

  debug "response", $response;

  ok defined($json->{_51D_ID}) eq 1;
  ok $json->{_51D_ID} eq $device_id;

Great, we have a unit test and it fails miserably because we don’t have a apache server.  We need an apache server that we can start, stop and configure for every test run.  Conveniently, the Apache::Test module provides a “whole, pristine and isolated” apache server at our disposal.  Cool, we have a server.  Next, the handler will push device properties into the environment via the subprocess environment table so we need a way to capture those values.  Line 28 shows how the unit tests make a call to a CGI script.  The CGI script will simply grab the variables pushed into the environment by the handler and return them to the test.


use CGI qw(:standard -no_xhtml -debug);
use JSON;

print header('application/json');

my %properties;

while ( my ($key, $value) = each(%ENV)) {
  if ( $key =~ /^_51D/) {
    $properties{$key} = $value;

print encode_json \%properties;

Ok, so we have a failing test, a web server, and a way of communicating between the two.  We have a little bit of wiring to do to let the server know about our CGI script as well as our handler.  By convention, Apache::Test will look for a file called t/conf/  This file contains configuration directives that will be added to httpd.conf before starting the server.  We’ll take this opportunity to configure the execution of our index.cgi test harness, configure our log format and setup our handler.

PerlSwitches -w

ScriptAlias /cgi-bin @ServerRoot@/cgi-bin
<Location /cgi-bin>
  SetHandler cgi-script
  Options +ExecCGI +Includes

LogFormat "%{_51D_ID}e|%{User-Agent}i" combined

PerlTransHandler +CDK::51DegreesFilter
PerlSetEnv DeviceRepository @ServerRoot@/data/51Degrees-Lite.dat
PerlSetEnv DevicePropertyList ScreenPixelsHeight,BatteryCapacity
PerlSetEnv DevicePrefix _51D

Man, when is this guy ever going to get around to writing some code?  Almost there. The Apache::TestRunPerl and Apache::TestMM modules combine together to provide all that is necessary to start, configure and stop Apache, as well as run all of the individual unit tests.  These get added into our Build.PL script.  The test action normally just executes tests.  We need to subclass this action so that we can start the server before the  tests executes and stop it when complete.   It would also be nice to produce Junit style output of the test results so that they can be published by the build server.

use Module::Build;
use ModPerl::MM ();
use Apache::TestMM qw(test clean);
use Apache::TestRunPerl ();
use IO::File;

my $class = Module::Build->subclass(
    class => 'CDK::Builder',
    code => q{
	sub ACTION_test {
	    my $self = shift;
	    $self->do_system('t/TEST -start-httpd');
	    $self->do_system('t/TEST -stop-httpd');

my $build = $class->new (
  module_name => 'CDK::51DegreesFilter',
  license => 'perl',
  test_file_exts => [qw(.t)],
  use_tap_harness => 1,
  tap_harness_args => {
    sources => {
      File => {
        extensions => ['.tap', '.txt'],
    formatter_class => 'TAP::Formatter::JUnit',
  build_requires => {
      'Module::Build' => '0.30',
      'TAP::Harness'  => '3.18',
  test_requires => {
      'Apache::Test' => 0,
  requires => {
      'mod_perl2' => 0,
      'FiftyOneDegrees::PatternV3' => 0,
      'JSON' => 0,
      'Apache2::Filter' => 0,
      'Apache2::RequestRec' => 0,
      'Apache2::RequestUtil' => 0,
      'Apache2::Log' => 0,
      'Apache2::Const' => 0,
      'APR::Table' => 0





Finally.  At this point, writing the handler is pretty anti climatic.  It reads the user agent from the header and passes it to the getMatch method from 51Degrees.  A set of device properties are returned as a JSON object.  Each requested property, defined by DevicePropertyList, is added to the environment, via subprocess_env().  The AMF handler used a caching mechanism to avoid detection costs for previously seen user agents.  The 51D folks said the new version was faster, so I wouldn’t need it.  Performance testing will prove this out.

sub handler {
  my $f = shift;

  my $user_agent=$f->headers_in->{'User-Agent'} || '';
  my $json = FiftyOneDegrees::PatternV3::getMatch($dataset, $user_agent);
  my %properties = %{ decode_json($json) };

  while ( my ($key, $value) = each(%properties) ) {
    my $dkey = uc("${prefix}_${key}");
    $f->subprocess_env($dkey => $value);

  return Apache2::Const::DECLINED;



To test performance, I setup Jmeter with 5 threads on a sandbox machine and looped over a set of 350K unique user agents.  The Jmeter instance made requests to apache running on a second sandbox machine with the new handler installed.  With 2,428,264 requests under its belt, the average response time is 10ms.  For v2, with caching, the average response time was 16ms.

How to Find Happiness In Your Job

By Kris Young (@thehybridform)

Are you happy with your job? Do you come into work, sit down, and then sigh with your head hung low? If so then you are not alone.

According to Forbes[1] 52% of Americans are not satisfied with their work. Some surveys place this number above 80%. This begs the question why? Do we study the wrong subject in school or apply for the wrong job? Maybe we followed the wrong advice from our high school guidance councilor.

Psychology Today [2] says we spend 90,000 hours of our lifetime working. Why not enjoy our time. The question is, “How do we find a job we love?”

Traits of a Job to Love

There are three areas that people who love their job have in common. These can be used as a measurement of job satisfaction:

  • Creativity
  • Impact
  • Control

The first two areas feed our natural human desire to belong. We are social beings and seek confirmation and a place in our community to achieve self worth. We do this by helping others, which in turn helps ourselves.

At the same time we wish for freedom, a way to disengage from the pack. The last area, Control, addresses this instinct.


Using one’s imagination or original ideas to express oneself is a human trait we all share. You can find creativity in all professions. Artists and musicians have obvious expressive qualities. How about IT professionals? Or bartenders? Absolutely!

Conjuring an internal thought to realization for an audience is the heart of creativity. Even if that audience is you. Be it a new algorithm or a business process. Think about what you do at work and how you can be creative. You may already be on the road to job satisfaction.


Knowing our creativity matters to other people is important. Who wants to toil away on a problem nobody cares about? An impact gives you validation of your hard work and results in a feedback loop, which in turn compels you continue creating. And on the loop goes.

Feedback can be in the form of pay raise, bonus, more autonomy, or peer recognition of a job well done. All feedback gives us a chance to adjust into the next loop. We will tend to veer towards creativity that gives us the most positive outcome and the biggest impact.


Living life on your terms is a very seductive prospect. Having the choice to do what you want when you want in you own time is a right we all share. This aspect of your job can be summed up as career capitol. The more career capitol you have the more control you will have in you daily life. It’s the freedom to express yourself and control your destiny.

You want to introduce a new process to the business or a new ingredient to a recipe? Can you leave early on Friday to pick the kids? If you feel you need permission to start a new creative endeavor then you need more career capitol in the bank. But how do you acquire this?

The Advice

At some point in your life you will hear someone say, “Follow your passion”. This could come form someone very successful or smart. A person you respect or wish to live up to. He or she seems legit; why not give the advice a try?

I’m here to tell you this is the worst advice to give or follow.

Sure there are people out there who have an inherit passion at a young age; those people are few and far between. Chances are you have no idea what you passion is anyway. And it will change overtime. If you don’t have a passion then you may feel left out or somehow not able to be happy with your work. This is demoralizing and contributes to the dissatisfaction with your job, a vicious cycle. FYI passion is a synonym for suffering and who want’s to suffer.

Better advice is to adopt a craftsmen mindset and become “So good they cannot ignore you.”[3] Having a craftsmen mindset is far more important than a passion mindset.

The Craftsmen Mindset

Achieving craftsmanship of an in demand and/or rare skill will present you with career capitol. Which goes a lot further in creating work you love.

This means working hard, learning as much as you can, and becoming the expert. Some people say the trick is to choose something you like, but that’s is not really important. You may not know what you like or where to start. If you do great! If not try something new. What ever you choose adopt a craftsmen mindset. But remember the more rare and in demand the skill, the faster you will accrue career capitol.

Adopting a craftsmen mindset will lead you to your passion. You will have creativity, impact, and control. You will find yourself creating the work you love to do. Your passion will find you.

Short Story

If Steve Jobs followed his passion we would not have the iPhones, iPads, or MacBooks. His passion was studying Zen Buddhism. Though a fluke meeting and noticing some people were excited about model-kit computers an empire was born.

He became passionate about technology and science. He became so good you could not ignore him.




Escaping Technical Debt

By Osman Shoukry (@oshoukry) & Kris Young (@thehybridform)

The Visit

On October 6th we had Michael Feathers (@mfeathers) author of Working Effectively With Legacy Code visit our facility.  The visit was two achieve two objectives.  The first was to give tech talks to our engineers about legacy code.  The second was to train selected key individuals in the organization.  Specifically, techniques and skills on how to deal with legacy code.

Mr. Feathers graciously agreed to give a recorded community talk about Escaping Technical Debt.


Key Takeaways

  • Tech debt

Technical debt is a metaphor for the amount of resistance in a system to change.  The larger the debt the higher the resistance.  When change is introduced, the time spent looking for where to make a change is one example of tech debt.  Another is when the system breaks in unexpected ways.

  • It takes a community to mitigate technical debt

Tech debt affects everybody, from engineers, to product owners to the CEO.  Mitigating tech debt requires everyone’s support and involvement.  As Engineers, we are responsible for how to mitigate technical debt.  The product owners should have input on tech debt mitigation effort.  The benefits of tech debt cleanup should be visible to everyone.  Don’t surprise your peers or management by the sudden change in productivity.  They will bring ideas to you to help pay down the debt in more enabling ways for the future… Involve them!

  • Don’t pay the dead

Code that doesn’t change, even if it is in production, doesn’t collect debt.  Dead debt is any part of the system that doesn’t change including bugs and poor design.  Many times engineers get wrapped up cleaning code that isn’t changing.  Resist the urge to refactor unchanging parts of the system.  If it isn’t changing it, it isn’t technical debt, it is dead debt, walk away.

  • Size your debt

Target the most complex changing code ordered by frequency of change.  Build a dashboard to show the amount of changing code by frequency.  These are the most expensive tech debt hot spots.  This should be the focus of the tech debt cleanup effort.

  • Seal the leak

Technical debt should be paid down immediately.  Simple code is easy to make more complex, and easy to make simple.  However, as the code becomes more complex, the balance tips in the favor of adding complexity than removing it.  No matter how complex the code is, it is always going to be easier to add complexity than remove it.  Inexperienced engineers, fail to see the initial complexity that is added to the system.  In turn, they follow the path of least resistance making the code more complex.

To seal the leak, first identify the most complex and frequently changing code.  Second, give explicit ownership for that code to the most seasoned engineers.  Third, let the seasoned engineers publish the eventual design objectives.  And finally, owners should review all changes to the code they own.

  • Slow down to go fast

Clean code takes time and disciplined effort.  Adequate time is needed to name classes, methods and variables.  Poorly named methods and classes will create confusion.  Confusion will lead to mixing roles and responsibilities.


Finally, low tech debt will yield high dividends with compound interest…

“Don’t do half assed, just do half”

%d bloggers like this: