Global Day Of Code Retreat 2013

By Osman Shoukry – Director, Software Engineering – Digital Advertising Platform @oshoukry.

Last November I attended the Software Craftsmanship North America (SCNA), the conference was very informative and I got introduced to some valuable techniques around developing craftsmanship within our company one of which is Code Retreat-ing.

What is Code Retreat?

Code retreat is a full day event where software engineers get together to deliberately practice their craft.  It is a forum to enable working outside of the normal comfort zone.  The reality is that most professional software engineers enter the professional world – wether self taught or having received formal education, without sufficient training into what is required to write great code.

Joining forces

Having returned from SCNA 2013, and gotten jazzed up with all that I saw, and excited to try out code retreats, I decided I’d organize one for my teams to try.  While doing the research on line to read more about it I stumbled on and saw that there is an international day of code retreat to be held on Dec 14th 2013.  I looked up Seattle events and found that Getty Images – a company in our building is sponsoring the event but it was all sold out.  I contacted the organizer (Andrew Parker @aparker42) and offered our Cobalt space.  We were all set and the event received additional sign up spots.

The Six Sessions

On the day of the event, Andrew and I decided to run the event in the Cobalt space bringing everyone together.  We had 42 crafts[men|women] that gave up their Saturday to attend this deliberate practice event.  We setup the tables to facilitate pairing with power and networking.  We started @8:30 with breakfast and socializing and then we got down to business.

photo 4

Session 1 – 9:15 am (Get familiar with the problem)

For the first session, we introduced the problem – Conway’s Game of Life, and let everyone work on it without any restrictions and constraints. We ran a count down timer with 45 min on the big screens.  With 5 minutes left, Andrew threw in a change to the problem, which was, if a cell was alive, then died then came back to life, it should now be considered a zombie and will never die again.  The purpose of this curve ball was to get everyone to think about how they would change their design.

After 45 min, it was “Delete your code, How did it go? and 5 min break”.  Many thought deleting was a very weird thing to do, however, there were a few that were relieved to see the code – aka the mess – get purged.

Some got quite far with the solution almost solving the problem fully, while others reported not much was accomplished because it took a while to understand how to approach the problem.

Everyone switched pairs and started again on a blank slate, people got to discover others who have languages they would like to learn to pair up with those individuals.  One engineer held an iPad up high as a sign, it read “I need Python”.

Session 2 – 10:15 am (Ping Pong TDD)

We noticed in session one there was a lot of domination of keyboard and mouse by one of the pairs, so we decided this would be a good next exercise.  The rule was to toggle back and forth, pair to write a test, the other makes it pass and adds a new test, returning back to the first pair.

This introduced a healthy dynamic and we saw a lot more collaboration, while walking around though, we noticed that some engineers kept the code from session 1.  This was interesting reinforcement that we – engineers, have a very hard time letting go of something that we worked so hard to put together.  And so we needed a new strategy.

At the end of this session, we requested that the right pair stand up when the left pair has successfully purged the code, and this practice carried on for all the remaining sessions.

We were planning to do a “Mute TDD ping pong” session after this one, where the pair aren’t allowed to verbally communicate but utilize only tests to drive the solution but opted against it.  We didn’t have enough mass in the group that were verse enough with TDD to make it effective.

Session 3 – 11:15 am (No Loops, No Conditionals & Methods < 4 lines)

This was probably one of the hardest sessions, many had a very hard time dealing with alive vs. dead cell state not utilizing boolean logic.  Not to mention lack of loops forces recursive solutions which aren’t readily available.  Again, there was different takes on this, some found the constraints insurmountable and others were able to make some progress, overall, it wasn’t easy, perhaps breaking the constraints apart into separate sessions next time may be more focused and provide a better learning experience.

Lunch Break 12:15 – 1:05

We had pre-ordered from Chipotle for most, and augmented the menu with sandwiches from Jimmy Johns.

Session 4 – 1:15 pm (Immutable)

For this session, everything had to be immutable, once you create an instance of something, you couldn’t modify it.  Most were able to cope with the constraint until it came to collections or array manipulation in languages like Java, how to add a new element since creation and appending must be on two steps.  We ended up modifying the restriction a bit for languages that make it unreasonable to allow adding a utility method that helps you with the appending collections / arrays as an exception to the rule.

As we were walking around we noticed two patterns:

Screen Shot 2013-12-18 at 1.34.44 PM


Screen Shot 2013-12-18 at 1.34.51 PM

Both of those looping techniques violate immutability because the loop variable is being mutated.  So the immutable restriction, meant for most no loops allowed unless you were creative enough to push the terminating condition to some method call return within the loop.  Feedback was received and welcomed by all, and alternate solutions materialized quickly.

Session 5 – 2:15 pm (No returns – Tell Don’t Ask)

This session is the exact opposite of the prior session, since all methods had to be declared with void return type, callbacks and other mechanism needed to be utilized for passing back values.  Of course there was the occasional “make it global variable” but all in all, the pairs found that the sessions provided a healthy and not frustrating challeng.  Some found it quite easy given their language of choice lent itself naturally to this style of programming (i.e. Java Script).  By this session, some folks were getting tired, we broke out some candy and snacks to help.  One pair was exhausted and decided to take a break this session.


Session 6 – 3:15 pm (Free form)

The final session, we gave the pairs the option to select any restriction to operate under and let them select.  A few were exhausted by then, but most persevered.

Goodies – 4:15 pm

We had two sets of things to give away that day.  Thanks to those sponsors for making everyone a winner.

  1. O’REILLY gave out one free ebook to everyone who attended a code retreat.
  2. C<>de School gave out “48 hours of Code School” to everyone who attended.
  3. Pragmatic Bookshelf gave out “25% off any published item” to everyone who attended.
  4. Solano Labs gave out “One month of free Solano Labs’ CI Service” to everyone who attended.
  5. PluralSight had some swag that we ruffled among the participants.


This code retreat session was quite valuable to everyone who participated, I think the biggest surprise to everyone who attended was the diversity of the possible solutions to such an easy to understand problem.

Craftsmanship is hard work, it takes a great deal of effort to communicate clearly the intent, and then proceed, and 45 min go by very quickly.

As a co-facilitator with Andrew Parker, I walked away understanding at a deeper level a few things.  I came face to face with understanding how truly amazing some of the engineers are and how desperate our craft is for forums to enable experimentation, quick learnings through failing safely.  Additionally, how much fun it is to be part of or facilitate a code retreat.

Many thanks to Andrew Parker @aparker42 for all his help organizing and facilitating the retreat, Michael Ibarra @bm2yogi for facilitating the initial event avenue – GettyImages.

Continuous Integration with Performance Certification and Results

By Murshid Mustafa – Development Manager – ADP Cobalt Performance Team


Often performance engineering teams face challenges fitting into an agile practice or methodology. As finding, debugging and fixing performance issues can take considerably longer time compared to functional problems. So it’s critical to expose the issues early in the development cycle. Early failure is cheap and easy to fix.

Typical hurts of a conventional model

  • Found performance issue(s) a week before the release date
  • Can not release business committed new feature
  • Changes to the product delivery date
  • Uncertainty in the release creates a culture of stress
  • Unhappy customers


In a typical software development cycle, performance certification fits in at the end. It gets delayed till the product is fully baked with unit and functional testing. This leaves all the performance bugs rolled up over time creating a snowball effect.

As we understand performance problem identification and resolution can take considerably longer time compared to a functional issue resolution. This practice in many cases causes late detection and resolution of the performance issues. Causing high risk to the releases.

Screen Shot 2014-02-05 at 11.54.23 PM

Why continuous performance integration and results?

  • Early detection of the performance problems
  • Decreasing the cycle time by resolving performance issues faster
  • Better release planning with peace of mind
  • Save time and resources
  • Increase business revenue targets
  • Happy customers


One of the biggest challenges in bringing performance certification early in the development cycle is product maturity or functional stability. So its true that we can’t fully evaluate peak product performance early while the product is under development. But we can break the entire performance suite into small pieces and design the tests to touch the functionality that is ready for certification. Stability of the system on stress and system reliability can be measured post-functional freeze.

Screen Shot 2014-02-05 at 11.53.38 PM

The approach can be different if we are developing a brand new product compare to releasing new functionality of an already marketed product. We at ADP/Cobalt are seeing the benefits of it almost every day.

Screen Shot 2014-02-05 at 11.55.18 PM

The biggest value that continuous integration brings is the availability of performance results right away. Having build-to-build comparison makes the life easy detecting issues in between the builds. The graphs above shows how easy it gets to detect the issues. This saves critical time debugging, identifying the root cause of the issue. It’s huge considering how difficult and time consuming it can get debugging a performance problem.

Be aware of the challenges

Achieving continuous integration with performance presents several challenges.

  • “System configuration and code dependencies are not in sync” This can be a hit to the productivity of the CI system. Always be on top of the code changes and new features making sure the system is configured properly to delivering expected output.
  • “Debugging or resolving issues in the code or in the system may require working with other teams e.g. development, IT, etc.” The resolution of these issues should be very timely. Make sure the stakeholders understand the needs and time constraints in order to have this system up and running at all times.
  • “System is not scalable” It’s possible over time with added features and functionality the system may deteriorate its scalability. Plan regular system maintenance and perform capacity planning studies. Make proper adjustments to the hardware and configuration to be able to run the system with proper performance and scalability.
  • “The results are varying run-to-run” Continuous integration without the accuracy of the results is meaningless. We have to run these tests in a controlled environment and inputs/outputs of all systems (configurations, network, load) should be understood correctly.
  • “Execution time of the tests is very long” Often performance tests suites takes long time to run; in order to generate results that gives proper average to compare. Also tests that gage performance, reliability and stability over time can be significantly longer. Choose your tests wisely, not every test has to be part of the CI. In order to have effective continuous integration, keep a good balance of tests and the number of build getting the daily coverage.


Key learnings

  • Continuous integration with performance certification has it’s cost
  • Partner with development and provide a self service model to get maximum benefits
  • Always maintain a controlled, clean room performance environment
  • Fixing performance bugs can be expensive on the clock so better start early
  • It requires a change in culture, make sure all organizations Dev, QA, RM, IT are enrolled with this philosophy
  • Yes it can be done
%d bloggers like this: