drasi-platform: Open Source Data Engineering & Integration

What is drasi-platform?

What it is

Drasi is an open-source data processing platform for change-data-capture and change-detection workflows. It is a C# Apache-2.0 project in the CNCF Sandbox ecosystem, and the README describes it as a platform that detects changes in data and takes immediate action. The project targets real-time processing without the overhead of traditional data methods, and it is presented as a solution for tracking system logs and change feeds, evaluating relevance, and initiating reactions.

The concrete problem it addresses is the need to notice and respond to data changes without repeatedly querying sources or copying data to a central data lake. Drasi uses Sources to monitor logs and feeds, Continuous Queries to evaluate changes, and Reactions to trigger responses when query result sets update. This design supports real-time data integration and automated responses inside software systems, especially when changes must produce timely actions.

Key capabilities

Drasi connects to data repositories through Sources that monitor logs and feeds for changing data.
It evaluates incoming changes with Continuous Queries written in the Cypher Query Language.
It updates query result sets when changes match criteria and conditions defined in the queries.
It triggers context-aware Reactions from updates to those result sets.
It avoids the need to copy data to a central data lake or repeatedly query data sources.
It supports workflows that combine multiple Sources, such as order status changes and driver availability.
It includes a Debug Reaction and tooling for development and testing.

Who uses it and how

Teams building change-data-capture pipelines can use Drasi to detect events from system logs and change feeds.
Developers deploying Drasi for Kubernetes can apply Sources, create Continuous Queries, and define Reactions.
Operators of delivery or ordering systems can match ready orders with available drivers and send alerts.
Community users can experiment with proofs of concept and report bugs or feature requests through GitHub issues.
Evaluators can follow the tutorial by applying a Source, creating Continuous Queries, and applying a Debug Reaction.

Getting started

Users start by following the Getting Started tutorial for Drasi for Kubernetes, which leads through applying a Source, creating Continuous Queries, and applying a Debug Reaction. The README also points to the documentation site, tutorials, and how-to guides for Drasi for Kubernetes.

When to use it — and when not to

Drasi fits projects that need change-data-capture and change-detection workflows that react to data changes without copying data to a central data lake, especially in a Kubernetes-based proof of concept. It is less suitable for teams that need a mature production release, because the README calls this an early release for community learning and experimentation and the repository lists open issues. The provided facts do not describe a hosted option, package manager, or non-Kubernetes deployment path.

project readme (upstream, from github) — read inline

Drasi

Drasi is a data processing platform that simplifies detecting changes in data and taking immediate action. It is a comprehensive solution that provides built-in capabilities to track system logs and change feeds for specific events, evaluate them for relevance, and automatically initiate appropriate reactions. Visit our documentation site at https://drasi.io for detailed information. Drasi is a CNCF Sandbox project

Overview

Drasi provides real-time actionable insights without the overhead of traditional data processing methods. It tracks system changes and events without the need to copy data to a central data lake or repeatedly query data sources. Drasi uses queries to continuously evaluate incoming data changes. When the changes match the criteria and conditions specified in these queries the result sets of these queries are updated. These updates then trigger context-aware reactions defined tuned to your specific requirements.

Drasi operates through three components:

Sources connect to data repositories within software systems to monitor logs and feeds to track changing data.
Continuous Queries interpret monitored changes by applying criteria and conditions to identify significant changes. In Drasi, these Continuous Queries are written using the Cypher Query Language.
Reactions trigger meaningful responses based on updates to the result sets of the Continuous Queries.

To illustrate how Drasi interprets events and triggers appropriate responses, consider a delivery system for an online ordering service. Orders are processed through an order management system, and delivery drivers need real-time notifications when orders are ready for pickup. Drasi automates this process by:

Configuring a Source to monitor the order management system for changes in order statuses and a second Source to detect when a driver becomes available for a delivery run.
Creating a Continuous Query that combines data from both Sources to match orders ready for pickup with available drivers.
Defining a Reaction to send alerts to drivers, notifying them to proceed to the pickup area. This streamlined setup ensures drivers are promptly informed, optimizing the delivery process through real-time data integration and automated responses.

Getting Started

Follow the Getting Started tutorial and try out Drasi for Kubernetes. The tutorial will lead you through:

Applying a Source representing the data source whose changes you want to observe.
Creating Continuous Queries to define the data to observe, conditions to assess changes, and the structure of the output.
Applying a Debug Reaction to view the output generated by one or more Continuous Queries.

Head over to our documentation site and visit the Tutorial and How To guides to learn more about Drasi for Kubernetes.

Release Status

This is an early release of Drasi for the community learn about the platform and experiment with in Proofs Of Concept. Please share your thoughts on Drasi and create GitHub issues for any bugs you may find or if you have feature requests that will help improve Drasi.

This repo contains everything you require to build a Drasi-based solution with Sources, Reactions, and tooling for development and testing.

Community

We hope you will join us and contribute to Drasi! Some of the ways to get started with contributing are participating in Issue discussions or joining us on our Discord server. Check out our Community repo for more information on the community, and guidance on contributing and development.

Contributing To Drasi

Please see the Contribution guide for information on contributing to Drasi.

Security

Please refer to our guide on reporting security vulnerabilities

Code of Conduct

Please refer to Drasi's Code of Conduct

License

This project is licensed under the Apache 2.0 license. Please see the LICENSE file.

Contact the Drasi Authors

Please join us on Discord to contact us and we will get back to you as soon as possible. You can also email us at [email protected].

Frequently asked questions

Is drasi-platform free to use?

drasi-platform is open source under the Apache-2.0 licence. There is no licence fee and no seat count — you can self-host it or, where the project offers one, pay a vendor for a managed version instead.

What does drasi-platform do?

The Data Change Processing platform

What is drasi-platform written in?

drasi-platform is primarily written in C#. Its source is publicly available at https://github.com/drasi-project/drasi-platform, and it has 1,250 GitHub stars.