[opensourcedex]

devlake is a free, open source data engineering & integration project written in Go and released under Apache-2.0. It has 3,139 GitHub stars, 811 forks and 54 open issues, and was last pushed 22 hours ago. On this registry it ranks #19 of 39 tracked projects in Data Engineering & Integration, with 5 head-to-head comparisons available. It gained 4 stars over the last 3 tracked days.

What is devlake?

What it is

Apache DevLake is an open-source dev data platform in the Apache ecosystem, written in Go and licensed under Apache-2.0. It ingests, analyzes, and visualizes fragmented data from DevOps tools to extract insights for engineering excellence, developer experience, and community growth. The platform lives in the data engineering and DevOps integration space, where development data is spread across tools such as GitHub, GitLab, Jenkins, Jira, Sonarqube, and more.

The concrete problem it solves is the fragmentation of software development life cycle data. Teams often need to understand development process data, but the relevant records sit in separate systems. DevLake brings these silos together so engineering leads, open-source software maintainers, and development teams can ask questions about their development process through connected sources and prebuilt dashboards.

Key capabilities

  • Ingests fragmented data from supported development tools, including GitHub, GitLab, Jenkins, Jira, and Sonarqube.
  • Analyzes and visualizes development data to support engineering excellence, developer experience, and community growth.
  • Implements metrics from DORA to scrum retrospectives through prebuilt dashboards that support common frameworks and goals.
  • Presents results through integrated dashboards powered by Grafana.
  • Extends to new data sources, metrics, and dashboards with a flexible framework for data collection and transformation.
  • Uses Blueprints in the DevLake Configuration UI to define data connections, data scope, transformation, and sync frequency.
  • Allows users to track the progress of a Blueprint after configuration.

Who uses it and how

  • Engineering leads use DevLake to make better sense of the development process and bring a data-driven approach to their practices.
  • Open-source software maintainers use DevLake to analyze community growth and developer experience from project data.
  • Development teams install the platform with Docker Compose or Helm when they need to run the data platform in their own environment.
  • Users create a Blueprint to connect DevOps tools, select the data scope, define transformation, and set sync frequency.
  • Terminal-focused users can deploy, configure, and monitor DevLake with the gh-devlake CLI extension, which supports local Docker and Azure deployments.

Getting started

Install Apache DevLake by following the step-by-step instructions for Docker Compose or Helm, or use the gh-devlake CLI extension to deploy, configure, and monitor it from the terminal with local Docker and Azure support.

When to use it — and when not to

Use Apache DevLake when a team needs an Apache-licensed, Go-based platform to unify fragmented DevOps data and produce DORA and retrospective dashboards. Avoid it when a managed hosted service is required, because the available deployment methods are Docker Compose, Helm, and CLI rather than a hosted option. The available facts do not list paid products it replaces, and they do not specify database, storage, or SMTP requirements; the registry entry also shows 54 open issues and no contributor count, so teams should verify support and operational dependencies before adoption.

project readme (upstream, from github) — read inline


Apache DevLake

PRs Welcome Dockerhub pulls unit-test Go Report Card Slack Twitter


🤔 What is Apache DevLake?

Apache DevLake is an open-source dev data platform that ingests, analyzes, and visualizes the fragmented data from DevOps tools to extract insights for engineering excellence, developer experience, and community growth.

Apache DevLake is used by Engineering Leads, Open Source Software Maintainers and development teams looking to make better sense of their development process and to bring a more data-driven approach to their own practices. You can ask Apache DevLake many questions regarding your development process. Just connect and query.

🎯 What can be accomplished with Apache DevLake?

  1. Your Dev Data lives in many silos and tools. DevLake brings them all together to give you a complete view of your Software Development Life Cycle (SDLC).
  2. From DORA to scrum retros, DevLake implements metrics effortlessly with prebuilt dashboards supporting common frameworks and goals.
  3. DevLake fits teams of all shapes and sizes, and can be readily extended to support new data sources, metrics, and dashboards, with a flexible framework for data collection and transformation.

👉 Live Demos

The main way you interact with DevLake is through the integrated dashboards powered by Grafana.

Live DORA Dashboard

Dashboards for Engineering Leads

Dashboards for OSS Maintainers

💪 Supported Data Sources

DevLake supports connections to many popular development tools, including GitHub, GitLab, Jenkins, Jira, Sonarqube and more. Here you can find all data sources supported by DevLake, their scopes, supported versions and more!

🚀 Getting Started

Installation

You can set up Apache DevLake by following our step-by-step instructions for either Docker Compose or Helm. Feel free to ask the community if you get stuck at any point.

🤓 Usage

Please see detailed usage instructions. Here's an overview on how to get started using DevLake.

1. Set up DevLake

Install using either Docker Compose or Helm.

2. Create a Blueprint

The DevLake Configuration UI will guide you through the process (a Blueprint) to define the data connections, data scope, transformation and sync frequency of the data you wish to collect.

3. Track the Blueprint's progress

You can track the progress of the Blueprint you have just set up.

4. View the pre-built dashboards

Once the first run of the Blueprint is completed, you can view the corresponding dashboards.

5. Customize the dashboards with SQL

If the pre-built dashboards are limited for your use cases, you can always customize or create your own metrics or dashboards with SQL.

Contributing

Please read the contribution guidelines before you make contribution. The following docs list the resources you might need to know after you decided to make contribution.

👩🏾‍💻 Contributing Code

If you plan to contribute code to Apache DevLake, we have instructions on how to get started with setting up your Development environemtn.

📄 Contributing Documentation

One of the best ways to get started contributing is by improving DevLake's documentation.

⌚ Roadmap

  • Roadmap: Detailed roadmaps for DevLake.

💙 Community

Message us on Slack

📄 License

This project is licensed under Apache License 2.0 - see the LICENSE file for details.

Frequently asked questions

Is devlake free to use?

devlake is open source under the Apache-2.0 licence. There is no licence fee and no seat count — you can self-host it or, where the project offers one, pay a vendor for a managed version instead.

What does devlake do?

Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering

What is devlake written in?

devlake is primarily written in Go. Its source is publicly available at https://github.com/apache/devlake, and it has 3,139 GitHub stars.