Dimensional Data

A key to GitLab’s success is our vast community of advocates. Here at GitLab, we call these active contributors “GitLab Heroes.” Each hero contributes to GitLab in numerous ways, including elevating releases, sharing best practices, speaking at events, and more. Jean-Phillippe Baconnais is an active GitLab Hero, who hails from France. We applaud his contributions, including leading community engagement events. Baconnais shares his interest in Kubernetes and explains how to deploy and monitor an application in Kubernetes without leaving GitLab. Since 2007, I’ve been a developer. I’ve learned a lot of things about continuous integration, deployment, infrastructure, and monitoring. In both my professional and personal time, my favorite activity remains software development. After creating a new application with multiple components, I wanted to deploy it on Kubernetes, which has been really famous over the last few years. This allows me to experiment on this platform. This announces a lot of very funny things. I know some terms, I used them in production for five years. But as a user, Kubernetes Administration is not my “cup of tea” 😅. My first deployment in Kubernetes When I decided to deploy an application on Kubernetes, I wasn’t sure where to start until I saw, navigating in my project in GitLab, a menu called “Kubernetes.” I wanted to know what GitLab was hiding behind this. Does this feature link my project’s sources to a Kubernetes cluster? I used the credit offered by Google Cloud to discover and test this platform. Deploying my application on Kubernetes was easy. I wrote a blog post in 2019 describing how I do this, or rather, how GitLab helped me to create this link so easily. In this blog post I will explain further and talk about what’s changed since then. Behind the “Kubernetes” menu, GitLab helps you integrate Kubernetes into your project. You can create, from GitLab, a cluster on Google Cloud Platform (GCP), and Amazon Web Services (AWS). If you already have a cluster on this platform or anywhere else, you can connect to it. You just need to specify the cluster name, Kubernetes API UR, and certificate. GitLab is a DevOps platform and in the list of DevOps actions, there is the monitoring part. GitLab deploys an instance of Prometheus to get information about your cluster and facilitate the monitoring of your application. For example, you can see how many pods are deployed and their states in your environment. You can also view some charts and information about your cluster, like memory and CPU available. All these metrics are available by default without changing the application of your cluster. We can also read the logs directly in GitLab. For a developer, it’s great to have all this information on the same tool and this allows us to save time. A new way to integrate Kubernetes Everything I explained in the previous chapter doesn’t quite exist anymore. The release of GitLab 14.5 was the beginning of a revolution. The Kubernetes integration with certificates has limitations on security and many issues were created. GitLab teams worked on a new way to rely on your cluster. And in Version 14.5, the GitLab Agent for Kubernetes was released! GitLab Agent for Kubernetes GitLab Agent for Kubernetes is a new way to connect to your cluster. This solution is easy to […]

După doi ani de doar evenimente virtuale KubeCon, echipa de produse GitLab a fost încântată să participe și să se întâlnească cu colegi, parteneri și multe altele din industria noastră la KubeCon EU 2022, care a avut loc în Valencia, Spania. Am fost prezenți cu patru lideri de produse, un dezvoltator de software și un cercetător UX. Această postare rezumă principalele noastre concluzii de la conferință, o experiență care ne va afecta foile de parcurs. Vom discuta următoarele subiecte: Platforme interne și GitOps Managementul secretelor Integrarea infrastructurii WebAssembly aka WASM Au existat 32 de tipuri de subiecte și mai multe evenimente de 0 zile la KubeCon. Multe discuții s-au concentrat pe câteva instrumente. Multe proiecte Cloud Native Computing Foundation ( CNCF ) au avut întâlniri ale comunității în aceste zile. Unele discuții au fost date IRL, iar altele au fost difuzate virtual cu întrebări și răspunsuri în direct. Au fost o varietate de subiecte și abordări. Au existat multe discuții și despre diferitele aspecte ale managementului clusterelor. Cu toate acestea, am lăsat acest subiect în mod intenționat, deoarece la GitLab dorim să ne concentrăm pe dezvoltatorii de software și să oferim o platformă DevOps care să le susțină munca. Managementul clusterelor este la un pas de această focalizare. Totuși, am observat câteva modele remarcabile, așa cum sunt evidențiate de cele patru elemente ale listei noastre. Ești invitat! Alăturați-vă nouă pe 23 iunie pentru evenimentul de lansare a GitLab 15 cu guruul DevOps Gene Kim și câțiva lideri GitLab. Vă vor arăta ce văd pentru viitorul DevOps și The One DevOps Platform. Platforme interne și GitOps Companiile doresc ca dezvoltatorii lor să se concentreze pe activitatea lor de bază. Ei creează platforme interne pentru a ascunde complexitatea operațiunilor din Zilele 0-2 de inginerii lor software și permit în continuare mișcarea „deplasare la stânga” a DevOps. Aceste platforme presupun adesea sudarea mai multor scule. Multe discuții au prezentat modul în care echipa sau compania respectivă și-a abordat problema platformei și ce instrumente au folosit, și de multe ori se putea simți transpirația de 18 luni a unei întregi echipe de platformă care încearcă să găsească o soluție. Aceste platforme folosesc fie un model bazat pe push-sau pull-ul pentru implementări. Nu apare nicio abordare unică din cauza aplicațiilor vechi și a cerințelor diferite. Deși există o definiție a GitOps oferită de inițiativa OpenGitOps , mai mulți prezentatori și-au oferit propriile definiții, inclusiv a implementărilor bazate pe pull. Am desfășurat un sondaj la scară largă legat de secrete la KubeCon și am aflat că utilizatorii ar dori ajutor cu fluxul de lucru Pipeline Authoring . Pe lângă cablarea instrumentelor, industria încă caută o abordare unificată a multi-chiriei (s-ar putea să nu existe una), iar uneori integrarea proceselor de securitate pare prea dificilă. Cum ne afectează acest lucru foaia de parcurs? Există mult potențial în construirea unei platforme folosită ca punct de plecare pentru platformele interne. Imaginați-vă un „instrument” care scurtează timpul necesar pentru a crea o platformă internă la zile sau săptămâni în loc de un an întreg. Aceasta este viziunea GitLab a platformei The One DevOps. Drept urmare, nu planificăm nicio schimbare în direcția noastră. Vom continua să investim în direcția de implementare recent începută pentru a oferi toate elementele de bază ale unei platforme într-un singur instrument și căutăm deja în mod […]

After two years of only virtual KubeCon events, the GitLab product team was excited to participate in and meet colleagues, partners, and more from our industry at KubeCon EU 2022, held in Valencia, Spain. We were present with four product leaders, a software developer, and a UX researcher. This post summarizes our primary takeaways from the conference, an experience that will affect our roadmaps. We will discuss the following topics: Internal platforms and GitOps Secrets management Infrastructure integrations WebAssembly a.k.a. WASM There were 32 topic types and several 0-day events at KubeCon. Many talks focused on a few tools. Many Cloud Native Computing Foundation (CNCF) projects had their community meetings during these days. Some talks were given IRL, and others were broadcast virtually with live Q&A. There were a variety of topics and approaches. There were many talks about the various aspects of cluster management, too. However, we left this topic out on purpose because at GitLab we want to focus on the software developers and provide one DevOps platform to support their work. Cluster management is one step away from this focus. Still, we noticed some remarkable patterns as highlighted by the four elements of our list. You’re invited! Join us on June 23rd for the GitLab 15 launch event with DevOps guru Gene Kim and several GitLab leaders. They’ll show you what they see for the future of DevOps and The One DevOps Platform. Internal platforms and GitOps Companies want their developers to focus on their core business. They create internal platforms to hide the complexity of Day 0-2 operations from their software engineers and still allow the “shift left” movement of DevOps. These platforms often involve the welding of several tools. Many talks presented how the given team or company approached their platform problem and what tools they used, and one could often feel the 18-month sweat of a whole platform team trying to come up with a solution. These platforms use either a push- or pull-based model for deployments. No single approach is emerging due to legacy applications and different requirements. While there is a definition of GitOps provided by the OpenGitOps initiative, several presenters offered their own definitions, including of pull-based deployments. We fielded a large-scale survey related to secrets at KubeCon, and learned that users would like help with the Pipeline Authoring workflow. Besides the wiring of the tools, the industry is still looking for a unified approach to multi-tenancy (there might not be one), and sometimes integrating security processes seems overly challenging. How does this affect our roadmap? There is a lot of potential in building a platform used as the starting point for internal platforms. Imagine a “tool” that shortens the time required to create an internal platform to days or weeks instead of a whole year. This is the GitLab vision of The One DevOps platform. As a result, we don’t plan any changes in our direction. We will continue investing in the recently started Deployment direction to provide all the building blocks for a platform in a single tool and are already actively looking for integrated experiences across our offering. We’re working on a CI/CD Component Catalog that includes CI templates. This will support the Pipeline Authoring workflow. Secrets management One of the things that often came up in our […]

This blog post and linked pages contain information related to upcoming products, features, and functionality. It is important to note that the information presented is for informational purposes only. Please do not rely on this information for purchasing or planning purposes. As with all projects, the items mentioned in this blog post and linked pages are subject to change or delay. The development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc. Our SRE on call was getting paged daily that one of our SLIs was burning through our SLOs for the GitLab Pages service. It was intermittent and short-lived, but enough to cause user-facing impact which we weren’t comfortable with. This turned into alert fatigue because there wasn’t enough time for the SRE on call to investigate the issue and it wasn’t actionable since it recovered on its own. We decided to open up an investigation issue for these alerts. We had to find out what the issue was since we were showing 502 errors to our users and we needed a DRI that wasn’t on call to investigate. What is even going on? As an SRE at GitLab, you get to touch a lot of services that you didn’t build yourself and interact with system dependencies that you might have not touched before. There’s always detective work to do! When we looked at the GitLab Pages logs we found that it’s always returning ErrDomainDoesNotExist errors which result in a 502 error to our users. GitLab Pages sends a request to GitLab Workhorse, specifically the /api/v4/internal/pages route. GitLab Workhorse is a Go service in front of our Ruby on Rails monolith and it’s deployed as a sidecar inside of the webservice pod, which runs Ruby on Rails using the Puma web server. We used the internal IP to correlate the GitLab Pages requests with GitLab Workhorse containers. We looked at multiple requests and found that all the 502 requests had the following error attached to them: 502 Bad Gateway with dial tcp 127.0.0.1:8080: connect: connection refused. This means that GitLab Workhorse couldn’t connect to the Puma web server. So we needed to go another layer deeper. The Puma web server is what runs the Ruby on Rails monolith which has an internal API endpoint but Puma was never getting these requests since it wasn’t running. What this tells us is that Kubernetes kept our pod in the service even when Puma wasn’t responding, despite having readiness probes configured. Below is the request flow between GitLab Pages, GitLab Workhorse, and Puma/Webservice to try and make it more clear: Attempt 1: Red herring We shifted our focus on GitLab Workhorse and Puma to try and understand how GitLab Workhorse was returning 502 errors in the first place. We found some 502 Bad Gateway with dial tcp 127.0.0.1:8080: connect: connection refused errors during container startup time. How could this be? With the readiness probe, the pod shouldn’t be added to the Endpoint until all readiness probes pass. We later found out that it’s because of a polling mechanisim that we have for Geo which runs in the background, using a Goroutine in GitLab Workhorse, and pings Puma for Geo information. We don’t have Geo enabled on GitLab.com so we simply disabled it to reduce […]