Another year, another ShipItCon!

I’ve talked already on the previous instances of the conference. To keep things short, it’s an annual conference based in Dublin that works around the idea of releasing software.

Because it’s very open-ended topic that can be approached from lots of angles, and it’s something different from the more stereotypical “conference around a technology”, the talks are quite varied. The organisation always finds good speakers that talk about interesting concepts, from very different points of view, making it quite broad in scope. It’s full of, as the MC CK will say: “golden nuggets”

There’s always a general idea to shape the discussion around, and this year it was flow. But how that’s approached depends very much on the speaker, as we will see.

I always take lots of notes and reflect on the ideas that were presented.

Talks and Notes

DevOps Topologies 10 years on: what have we learned about silos, collaboration, and flow? by Matthew Skelton

Matthew Skelton, one of the authors of the book Team Topologies reflected over the concepts introduced on the book a decade ago. The idea of the DevOps topologies (which I didn’t know before attending to the talk) is quite interesting, describing different good and bad patterns (sorry, the word “anti-pattern” is quite strange to me) on how DevOps teams can work in relation to both Dev and Ops.

A very interesting point on idea of the topologies is the understanding that they are dynamic. Team structure is not a fixed thing, but it’s always a conversation where projects, initiatives and changes are always happening.

He talked about the concept of decoupling, and how that can help achieving high flow of value and improved efficiency. Decoupling is the opposed to coordination, allowing teams to work independently. But to achieve this, teams may need to change. Decoupling is actually very complicated, and it can be very hard to implement. One interesting tip was that making teams responsible for costs helps define boundaries, as they’ll need to impose limits on what cost is attributed to what team.

But decoupling is not the only thing that is required. Aligned independent teams is more difficult, and risk making them silos. As a counter-balance, it requires knowledge diffusion that flows from the teams to share lessons learn, create a cohesive culture and be sure that everyone is moving in the same direction in the organisation. Some of the elements that can help in that regard are things like internal conferences, lunch & learn, etc…

Organisation, Flow, and Architecture by Sam Newman

This was a very funny talk talking against ideas coming from Taylorism based on assembly lines of having “smart managers” telling what to do and “dumb workers” doing it. He introduced the idea of how hand-offs (tasks moving from one team to another) are terrible for flow and productivity, require a big deal of coordination and meetings. Team specialisation in software (e.g. a backend team, a database team, etc) tends to emphasise the necessity for handoffs of tasks from team to team.

He went back to the main ideas on the foundation of Agile (not on Agile as a consultant-driven idea), especially in relationship on self-organising, stable teams. Short term teams, in particular, are terrible as they are driven from outside (the objective that creates such short term team) and have all the incentives to create a big short-term mess of tech debt.

Data Driven Decisions to Improve Testing Flows by Heather Reid

The speaker described with examples different moments were capturing the right data was key to understand what was happening in production and make better decisions. She went over the concept of risk, in particular as related to testing, and how solid data can help understanding and limiting that risk to make more informed decisions that takes the right amount of risk.

Good testing is about finding problems

She also described how data should be used on day to day routine, not just when things are on fire, and critically analyse when more data is required, to collect it when possible.

She also went over some ideas for test teams, like close old bugs that are not critical (reopening them if they are important again) as a way to eliminate noise and improve morale. And also about how users doesn’t behave as we imagine and having solid data collected will showcase what they are actually doing, so changes can be addressed based on reality and not perceived problems.

Flow belongs in production by Charity Majors

She developed the idea that the main job for managers and leads is to create quick feedback loops and how performance is different from good engineering.

The main goal for that is to ship code swiftly and safely, which in itself is not purely a technical task, but a sociotechnical system (involving both people and technology interacting). A counterproductive realisation is that, when related to software, speed actually tend to lead to safety.

Important modern engineering practices are all about feedback loops, like engineers owning code in production, test in production, CD, feature flags or observability-driven deployment.

She spend a significant amount of time talking about observability and how “Observability 2.0”, with rich structured data flowing and clear single point of truth was going to enable better understanding on the behaviour of the system, and award curious developers with knowledge.

Establishing a culture of Observability at Phorest by Paweł Mason and Paul Daily

The speakers talked about the adoption in Phorest of observability tools (in this case, Honeycomb), and how , as they were gaining adoption, they were using the feedback that it provided as a way of improving their understanding of the system and reinforce it by better instrumenting the code. It was interesting to hear that, while the tool itself was not the total answer, it helped to shape the interaction to improve the practices.

As they introduced SLO (Service-Level Objective) as contracts between product and engineering, they worked with the idea of the Error Budget, the understanding that, based on an agreed SLO, you have some margin for error and how that can be monitored and used to control risk.

Go with the Flow by Vessy Tasheva

Vessy talked about the psychological aspects of Flow and how it’s an interaction between the internal individual forces of exploration and play (as opposed to reaction) and an external environment that provides a needed safety and trust to allow it (as opposed to fear).

She discussed the concept of “holding environment” (created by Winncott, a paediatrician in relation to mother-child bond) as one that’s both safe and uncomfortable enough to drive growth.

Flow-etry in Motion: Navigating Cascades of Threat Data by Claire Burns

Clarie commented on a (failed) migration of a system to Kubernetes and GCP. After analysis and some work, it was decided that it was not the right fit, and it pivoted from there.

Nonetheless, this project helped learning tools (like Kotlin, in this case), stretched the team to learn new stuff, even if ultimately was not the right fit for the project. Perhaps in the future? The main takeways, in her opinion, were that you need to set the ego aside, understand that no single tool work for every project or anyone and use the right tool for your use.

Panel discussion moderated by Damien Marshall, with Laura Nolan, Charity Majors, Ronan O’Dualing and Patrick Kua

The always interesting open discussion about the main topic of the conference, flow. I took some notes of ideas discussed, in an scatered format

• Losing hyper focus when moving to a managerial position or more senior positions, in general
  • Working with more ambiguous tasks
  • Difficult to get flow as Senior or Principal, as dealing with longer feedback loops
  • Protect your time, own your calendar, say no to things. Try to divide tasks into smaller blocks that can be finished in half a day, to set up realistic progress.
  • Avoid only providing feedback and reviewing, as creating is inherently more difficult than questioning, and that can make to loose touch and motivation
• A reference to this article by Will Larson: Migrations, the sole escalable fix to tech debt. When migrations are finished, they should be properly celebrated.
• Culture, which should allow for healthy disagreement. Companies should not be fun, but healthy.
  • Doing hard things brings up feelings
• The concept of “escaped interaction”, which means to perform a manual task outside of the process or tool, and requires context.

Harnessing Developer Insights to maximise flow by Mihai Paun and Alma Tarfa

They talked about two extremes, presenting two hypothetical examples in two sides of a spectrum to talk about how organisations tend to homeostasis, to be resistant to change and carry big inertia.

Some of the changes need to be driven slowly, but consistently, as small habits compound over time. They also talked how very important things are not quantifiable.

They described leaders as gardeners, not mechanics, as they need to nurture the proper environment and correct things constantly, not design something perfect and follow a template.

They talked about self-reflect as teams, and to ask and enable developers. As well as asking, what small thing I can commit to the next week (not a longer period of time) that will positively impact your environment?

From MLOps to AI systems by Jim Dowling

This talk was more technical and related to AI systems challenges and I personally found it a bit challenging to follow, as it was also quite quick. I learn some stuff that I didn’t know about production related challenges.

Jim talked about the challenges of bringing Machine Learning and AI systems into production. 48% of AI systems fail to get to a production environment.

There are big issues in the way that ML systems need to be architecture and the difference of skills and competences that are required in different areas, and how those areas interact to each other.

Finding personal flow by Patrick Kua

This talk was structured around three projects, in chronological order. The first one was about working on a side project to create a continuous integration system by setting a Perl script gluing code. This was more about individual flow, learning and ownership.

The second was to rework an email templating system with a team, on a tight deadline. This was more about playing around over team strengths, using Acceptance test-driven development (ATDD) and receive a design already lied out by an architect.

The third project was about a bigger project and the impact in trying to handle it all by a single person, and how that got to a lot of stress and required to stop for a few days. Once back, the responsibility was shared and people were made “feature leads” to get ownership of specific features, which lead to more engagement and less single point of decission.

He ended up commenting that flow is uneven, and you cannot wait for flow to find you. Also about flow as an spot between boredom and stress where you can grow.

Final thoughts

As it happened in previous years, I’m impressed with the quality of the talks and I was very happy to attend the conference. This year they sold out all the tickets, which is a sign that it’s been successful. Over the years I’ve been more and more convinced that the release is the heartbeat of a software company, and I always get back from the conference with ideas that I can try to apply in my work. In some cases a couple of years later.

So totally recommend to take a look if the release of software is something you’re interested in.