Dimensional Data

In almost any modern software infrastructure, there is inevitably some form of monitoring or logging. The launch of syslog for Unix systems in the 1980s established both the value of being able to audit and understand what is going on inside a system, as well as the architectural importance of separating that mechanism. However, despite the value and importance of this visibility into system behavior, too often monitoring and logging are treated as an afterthought. There are countless instances of systems emitting logs into a void, never being aggregated or analyzed for critical information. Or infrastructure where legacy monitoring systems were installed a decade ago and never updated to modern standards. Recently, shifts in the operational landscape have given rise to the concept of observability. Rather than expect engineers to form their own assumptions about how their application is performing from static measurements, observability enables them to see a holistic picture of their application behavior, and critically, how a user perceives performance. You’re invited! Join us on June 23rd for the GitLab 15 launch event with DevOps guru Gene Kim and several GitLab leaders. They’ll show you what they see for the future of DevOps and The One DevOps Platform. What is observability? To understand the value in observability, it’s helpful to first establish an understanding of what monitoring is, as well as what it does and does not provide in terms of information and context. At its core, monitoring is presenting the results of measurements of different values and outputs of a given system or software stack. Common metrics for measurement are things like CPU usage, RAM usage, and response time or latency. Classic logging systems are similar; a static piece of information about an event that occurred during system operation. Monitoring provides limited-context measurements that might indicate a larger issue with the system. Aggregation and correlation are possible using traditional monitoring tools, but typically require manual configuration and tuning to provide a holistic view. As the industry has advanced, the concept of what makes for effective monitoring has moved beyond static measurements of things like CPU usage. In its now-famous SRE book, Google emphasizes that you should focus on four key metrics, known as “Golden Signals“: Latency: The time it takes to fulfill a request Traffic: High-level measurement of overall demand Errors: The rate at which requests fail Saturation: Measurement of resource usage as a fraction of the whole; typically focuses on constrained resources While these metrics help home in on a better picture of overall system performance, they still require a non-trivial engineering investment to design, build, integrate, and configure a complete monitoring system. There is considerable effort involved in enumerating failure modes, and manually defining and associating the correct correlations in even simple cases can be time-consuming. In contrast, observability offers a much more intuitive and complete picture as a first-class feature: You don’t need to manually correlate disparate monitoring tooling. An aggregated monitoring dashboard is only as good as the last engineer that built it; conversely, an observability platform adapts itself to present critical information in the right context, automatically. This can even extend further left into the software development lifecycle (SDLC), with observability tooling providing important performance feedback during CI/CD runs, giving developers operational feedback about their code. Ultimately, observability provides more holistic debugging […]

Most of my daily conversations are focused on features and very deep technical concepts, which provide valuable and actionable insight. However, we miss the fact that tools and technology are leveraged to solve business challenges. When talking about features and technology, it’s very easy to see the possible financial gain when replacing different tools with a unified platform. But it’s missing all the improvement opportunities that will provide value at all the levels of a company from developers to executives. The reality is that we’re working in very complex systems, making it hard to see the forest from the trees. As an engineer, you’re focused on solving the next immediate problem that arises without taking a step back to reevaluate the system itself. In some cases, the problem itself is created by the design of our SDLC. As an executive, it’s difficult to balance the effort required to address the technical challenges with the pressure that comes from the business in this ever-increasing rhythm of change. My goal with this article is to provide a high-level map that contains the most important DevOps principles and a shortcut. I know this is a bold statement as there is a lot of literature on this topic but my approach will be different. First, I’m going to use the Three Ways as coined in The DevOps Handbook because those are the three foundational principles of DevOps as they were refined from Lean, the Toyota Production System, Theory of Constraints, Six Sigma, and System Thinking principles. Secondly, I’ll reference GitLab as the tool of choice because I think a good tool lets you focus on the work at hand, and GitLab does just that. You’re invited! Join us on June 23rd for the GitLab 15 launch event with DevOps guru Gene Kim and several GitLab leaders. They’ll show you what they see for the future of DevOps and The One DevOps Platform. Here is a short description of what the Three Ways are, what they’re about, and why you should care. First Way: Maximize Flow The First Way is all about making work/value flow better through the whole value stream (left to right), and to do that, we need to have a systems thinking approach and always look at the end-to-end result. In the case of IT, this means we optimize for speed from the moment we had the idea, to generating value with software running in production. We need to have a good understanding of the system to find potential bottlenecks and areas of improvement. Our improvements should always lead to better overall performance, be aware of the cases in which local enhancements lead to global degradation and avoid that. In this process, it is crucial to stop defects from passing downstream from one workflow stage to another. Why? Because defects generate waste (of time and resources). Second Way: Feedback loops The Second Way deals with feedback loops, amplifying and shortening feedback loops so that we get valuable insight into the work we’re doing. The feedback can be related to the code that’s written or the improvement initiatives. Feedback loops maximize flow from right to left of the value stream. Quick, strong feedback loops help build quality into the product and ensure that we’re not passing defects downstream. The quicker we do this […]

GitLab is focused on helping developers iterate faster and innovate more collaboratively – and that focus on enabling iteration extends to our own developer culture. As an organization, our CREDIT values are hardwired into our operations and culture. This empowers our development teams to work together – using our own product – to offer QA, feedback, and strategies that make everyone’s work stronger and help our organization iterate faster. We asked several engineers and engineering leaders at GitLab to tell us, in their own words, how our values come to life in our engineering organization and how that makes GitLab a unique place to be a developer. What attracts engineers to GitLab To start, we wanted to understand what attracted some of our current engineers and engineering leaders to join GitLab. You’re invited! Join us on June 23rd for the GitLab 15 launch event with DevOps guru Gene Kim and several GitLab leaders. They’ll show you what they see for the future of DevOps and The One DevOps Platform. “I was attracted to GitLab because I knew that I had the ability to make an impact. Being remote has shattered the walls between people and teams, so anybody can approach anybody. If something means something to you, you can really work on it. This culture of transparency and collaboration is really important to me.” – Sri Rangan, Fullstack Engineer, Incubation Engineering Team “People are attracted to the global diversity of the team and working asynchronously. I think we have a special working culture at GitLab. When you join, whether you’re the manager of multiple people or a manager of yourself, you work asynchronously regardless of where your teams are.” – Mek Stritti, VP, Quality “Before coming to GitLab, I was a frontend, backend, Android developer, data scientist, and machine learning engineer, among other things. But the thing about how I work is that I like to switch between those roles. And normally in companies, you can’t grow across all those roles. You need to grow as a specialist, not a generalist. But within the Incubation Engineering team, I get to do that.” – Eduardo Bonet, Fullstack Engineer, Incubation Engineering Team “The feedback that I quite often hear from engineers is just how strong the team is around them, and how collaborative the rest of the organization is. For my team in particular, a big part of their success is to be able to collaborate effectively with both the people that they work with and other teams. A lot of candidates are attracted to GitLab by the transparency value. Transparency is something that we really try to encourage, and it becomes a big mindset.” – Bartek Marnane, VP, Incubation Engineering How we ensure collaboration across the organization Beyond the aspects of GitLab that attracted many of our current engineers, it was clear that the culture they experienced during their time here ensured there was collaboration across various teams within our engineering organization. “We have an organization that supports each other. You propose a feature, you’re building something, and you can collaborate very easily across the globe, across departments with people in infrastructure and security. So when you’re building something it’s not all on you to ensure its stability and reliability and safety – the entire organization takes ownership of that.” – […]