Disclaimer: I know personally and worked with a good portion of the conference organizers and talkers. I label them with an asterisk*.

The ShipItCon finally took place last Friday. I think it’s quite impressive, given the short amount of time since announcing it and being the first edition, that was so well organized. The venue was very good (and fairly unusual for a tech conference), and all the usual things that are easy to take as granted (food, space, projector, sound, etc) work like clockwork. Kudos to the organizers.

The conference was oriented towards releasing online services, with special emphasis on Continuous Integration/Delivery. I think that focusing a conference over this kind of topic is challenging, as talks need to be generic enough in terms of tools, but narrow enough that is useful. Conferences about a specific technology (like PyCon, RubyConf or Linux Con) are more focused by concept.

The following is some notes, ideas and follow up articles that I took. Obviously, there are biased over the kind of things I find more interesting. I’ll try to link the presentation slides if/once they’re available.

• The keynote by the Romero family was a great story and addressed a lot of specific points to the game industry (like the design challenges). It was also the exception in talking about shipping something other than a service, but a game (in Steam and iOS). I played a little GunMan Taco Track over the weekend!
  • “Ship a game while on a ship“. They released part of the game while on the Queen Elizabeth cruise, crossing the Atlantic.
• Release often and use feature toggles, detaching the code release and feature release. This is a point done in the Frederick Meyer talk that I heard recently in other places.
  • Friday night releases make me cringe, but it can make sense if the weekend is the lowest activity point of your customers.
  • Dependency trees grow to be more and more complex, to the point no one understands them anymore and only automated tools can plot them.
  • Challenges in treating data in CI. Use production data? A subset? Fake data? Redacted data? Performance analysis can be tricky.
  • Automate what you care about
• The need for early testing, including integration/system/performance, was the theme around Chloe Condon talk. Typically, a lot of testing will be performed at the “main branch” (after a feature is merged back) level that can be prepared in advance, giving better and faster feedback to developers. Test early, test often.
  • She presented Codefresh with seems an interesting Cloud CI tool aimed at working with containers.
• Lauri Apple talked about communication and how important READMEs and documentation are for projects, both internal and external. The WHAT to build is a key aspect that shouldn’t be overlooked.
  • READMEs should include a roadmap, as well as info about installation, run and configure the code.
  • This project offers help, review and advice for READMEs. I’ll definitively submit a  review for ffind (after I review it and polish it a little bit myself).
  • She talked about the Open Organization Maturity Model, a framework about how open organizations are.
  • A couple of projects in Zalando that catches my eye:
    • Patroni, an HA template for PostgreSQL
    • Zalenium, distribute a Selenium Grid over Docker to speed up Selenium tests.
    • External DNS, to help configure external DNS access (like AWS Route 53 or CloudFare) to Kubernetes cluster.
• If it hurts, do it more frequently. A great quote for Continuous Delivery and automated pipelines. Darin Egan talked about the mindfullness principes and how the status quo get challenges and driving change opposes inertia.
• The main point in Ingrid Epure‘s talk was the integration of security practices during the development process and the differences between academia and engineering practices.
  • Linters can play a part in enforcing security practices, as well as automating autoformatting to leave format differences out from the review process.
  • Standardizing the logs is also a great idea. Using Canonical Log Lines for Online Visibility. I talked before about the need to increasing logs and generate them during the development process.
• Eric Maxwell talked about the need to standardise the “upper levels” of the apps, mainly related to logging and metrics, and making applications (Modern Applications) more aware of their environment (choreography vs orquestration) and abstracted from the underlying infrastructure.
  • He presented habitat.sh, a tool aimed at working with these principles.
  • Packaging the application code and letting the tool to do the heavy lifting on the “plumbing“
• The pipeline in Intercom was discussed by Eugene Kenny, and the differences between “the ideal pipeline” and “the reality” of making dozens of deployments every day.
  • For example, fully test and deploy only the latest change in the pipeline, speeding deployments at the expense of fewer separations of changes.
  • Or allow locking the pipeline when things are broken.
  • Follow up article: Continuous Deployment at Instagram
• Observability is an indispensable property for online services: the ability to check what’s going on in production systems. Damien Marshall* had this concept of graphulsion that I can only share.

He gave some nice ideas on observability through the whole life cycle:

Development:

• Make reporting logs and metrics simple
• Account for the effort to do observability work
• Standardize what to report. The three most useful metrics are Request Rate, Error Rate and Duration per Request.

Deployment:

• Do capacity planning. Know approximately the limits of your system and calculate the utilization of the system (% of that limit)
• Ship the observability

Production:

• Make metrics easy to use
• Centralise dashboard views across different systems
• Good alerting is hard. Start and keep it simple.

• Riot Games uses custom generation of services to generate skeletons and standardise good practices and reduce development time. Adam Comeford talked about those practices and how they implemented them.
  • Thinking inside the container.
  • Docker-gc is a tool to reduce the size of image repos, as they tend to grow very fast very quickly.
• Jacopo Scrinzi talked about defining Infrastructure as Code, making the infrastructure changes through the same process as code (review, subjected to source control, etc). In particular using Terraform and Atlas (now Terraform Enterprise) to make automatic deployments, following CI practices for infrastructure.
  • Using modules in Terraform simplifies and standardises common systems.
• The last keynote was about Skypilot, an initiative inside Demonware to deploy a game fully using Docker containers over Marathon/Mesos , in the Cloud. It was given by Tom Shaw* and the game was last year’s release of Skylanders. As I’ve worked in Demonware, I know how big an undertaking is to prepare the launch of a game previously in dedicated hardware (and how much in underused to avoid risks), so this is a huge improvement.

As noted by the amount of notes I took, I found the conference very interesting and full of ideas that are worth following up. I really expect a ShipItCon 2018 full of great content.