[opensourcedex]

CrateDB is a free, open source internet of things (iot) project written in Java and released under Apache-2.0. It has 4,436 GitHub stars, 617 forks and 332 open issues, and was last pushed 5 hours ago. On this registry it ranks #2 of 3 tracked projects in Internet of Things (IoT), with 5 head-to-head comparisons available. It gained 1 stars over the last 6 tracked days.

What is CrateDB?

What it is

CrateDB is a distributed, scalable SQL database built for storing and analyzing massive amounts of data in near real-time, including complex queries. It is written in Java, licensed under Apache-2.0, and based on Lucene. The project combines the benefits of an SQL database with the scalability and flexibility typically associated with NoSQL databases, and it is PostgreSQL-compatible, speaking the PostgreSQL wire protocol as well as offering an HTTP API. It lives in the analytics, big-data, and IoT ecosystem, and the topic list places it alongside distributed SQL databases, Elasticsearch, industrial IoT, and IoT analytics.

The concrete problem it solves is the gap between relational querying and horizontal scale. Modest CrateDB clusters can ingest tens of thousands of records per second, and its distributed query execution engine parallelizes query workloads across the whole cluster, so ad-hoc queries run against standard SQL rather than against a specialized query language. Dynamic table schemas and queryable objects add document-oriented features on top of relational SQL, which means time-series data, real-time full-text search, and geospatial data types and search can be handled in one system instead of several. Clusters are horizontally scalable, highly available, and fault-tolerant, with no shared state, so they run well in virtualized and containerized environments.

Key capabilities

  • Standard SQL access through the PostgreSQL wire protocol or an HTTP API.
  • Dynamic table schemas and queryable objects, giving document-oriented features alongside relational SQL.
  • Support for time-series data, real-time full-text search, and geospatial data types and search capabilities.
  • Horizontally scalable, highly available, fault-tolerant clusters that run well in virtualized and containerized environments.
  • Extremely fast distributed query execution that parallelizes workloads across the cluster.
  • Auto-partitioning, auto-sharding, and auto-replication.
  • Self-healing and auto-rebalancing, plus user-defined functions (UDFs) for extending functionality.

Who uses it and how

  • IoT and industrial IoT deployments that ingest high volumes of records and need near real-time analytics over them.
  • Analytics workloads that require ad-hoc standard SQL queries across a distributed cluster rather than a single node.
  • Containerized deployments, with documented how-to guides for running CrateDB on Docker and on Kubernetes.
  • Cloud deployments on ephemeral virtual machines, with tutorials covering AWS and Azure and deployments spanning multi-region hybrid clouds and the edge.
  • Interactive work through the Admin UI sql console or the CrateDB shell CLI tool, alongside a documented list of recommended clients and tools.

Getting started

Run the official Docker image with docker run --publish 4200:4200 --publish 5432:5432 --env CRATE_HEAP_SIZE=1g crate '-Cdiscovery.type=single-node', or consult the installation documentation for all available download and install options. A fully managed CrateDB Cloud Database as a Service option is also offered by Crate.io.

When to use it — and when not to

Choose C

project readme (upstream, from github) — read inline

.. image:: docs/_static/crate-logo.svg :alt: CrateDB :target: https://cratedb.com

.. image:: https://github.com/crate/crate/actions/workflows/main.yml/badge.svg :target: https://github.com/crate/crate/actions?query=workflow%3A%22CrateDB+SQL%22

.. image:: https://img.shields.io/badge/docs-latest-brightgreen.svg :target: https://cratedb.com/docs/crate/reference/en/latest/

.. image:: https://img.shields.io/badge/container-docker-green.svg :target: https://hub.docker.com/_/crate/

|

Help us improve CrateDB by taking our User Survey! _

About

CrateDB is a distributed SQL database that makes it simple to store and analyze massive amounts of data in real-time.

CrateDB offers the benefits_ of an SQL database and the scalability and flexibility typically associated with NoSQL databases. Modest CrateDB clusters can ingest tens of thousands of records per second without breaking a sweat. You can run ad-hoc queries using standard SQL_. CrateDB's blazing-fast distributed query execution engine parallelizes query workloads across the whole cluster.

CrateDB is well suited to containerization, can be scaled horizontally using ephemeral virtual machines (e.g., Kubernetes, AWS, and Azure) with no shared state. You can deploy and run CrateDB on any sort of network — from personal computers to multi-region hybrid clouds and the edge_.

Features

  • Use standard SQL_ via the PostgreSQL wire protocol_ or an HTTP API_.

  • Dynamic table schemas and queryable objects provide document-oriented features in addition to the relational features of SQL.

  • Support for time-series data, real-time full-text search, geospatial data types and search capabilities.

  • Horizontally scalable, highly available and fault-tolerant clusters that run very well in virtualized and containerized environments.

  • Extremely fast distributed query execution.

  • Auto-partitioning, auto-sharding, and auto-replication.

  • Self-healing and auto-rebalancing.

  • User-defined functions_ (UDFs) can be used to extend the functionality of CrateDB.

Screenshots

CrateDB provides an Admin UI_:

.. image:: crate-admin.gif :alt: Screenshots of the CrateDB Admin UI

Try CrateDB

Run CrateDB via the official Docker Image_:

.. code-block:: console

sh$ docker run --publish 4200:4200 --publish 5432:5432 --env CRATE_HEAP_SIZE=1g crate '-Cdiscovery.type=single-node'

Or visit the installation documentation_ to see all the available download and install options.

Once you're up and running, head over to the introductory docs. To interact with CrateDB, you can use the Admin UI sql console or the CrateDB shell_ CLI tool. Alternatively, review the list of recommended clients and tools_ that work with CrateDB.

For container-specific documentation, check out the CrateDB on Docker how-to guide_ or the CrateDB on Kubernetes how-to guide_.

Contributing

This project is primarily maintained by Crate.io_, but we welcome community contributions!

See the developer docs_ and the contribution docs_ for more information.

Security

The CrateDB team and community take security bugs seriously. We appreciate your efforts to responsibly disclose_ your findings, and will make every effort to acknowledge your contributions.

If you think you discovered a security flaw, please follow the guidelines at SECURITY.md_.

Help

Looking for more help?

  • Try one of our beginner tutorials, how-to guides, or consult the reference manual_.

  • Check out our support channels_.

  • Crate.io_ also offers CrateDB Cloud, a fully-managed CrateDB Database as a Service (DBaaS). The CrateDB Cloud Tutorials will get you started.

.. _Admin UI: https://cratedb.com/docs/crate/admin-ui/ .. _AWS: https://cratedb.com/docs/crate/tutorials/en/latest/cloud/aws/index.html .. _Azure: https://cratedb.com/docs/crate/tutorials/en/latest/cloud/azure/index.html .. _beginner tutorials: https://cratedb.com/docs/crate/tutorials/ .. _benefits: https://cratedb.com/product#compare .. _clients and tools: https://cratedb.com/docs/crate/clients-tools/ .. _containerization: https://cratedb.com/docs/crate/tutorials/en/latest/containers/docker.html .. _contribution docs: CONTRIBUTING.rst .. _Crate.io: https://cratedb.com/company/team .. _CrateDB clients and tools: https://cratedb.com/docs/crate/clients-tools/ .. _CrateDB Cloud Tutorials: https://cratedb.com/docs/cloud/ .. _CrateDB Cloud: https://cratedb.com/product/pricing .. _CrateDB on Docker how-to guide: https://cratedb.com/docs/crate/tutorials/en/latest/containers/docker.html .. _CrateDB on Kubernetes how-to guide: https://cratedb.com/docs/crate/tutorials/en/latest/containers/kubernetes/index.html .. _CrateDB shell: https://cratedb.com/docs/crate/crash/ .. _developer docs: devs/docs/index.rst .. Docker image: https://hub.docker.com//crate/ .. _document-oriented: https://en.wikipedia.org/wiki/Document-oriented_database .. _Dynamic table schemas: https://cratedb.com/docs/crate/reference/en/master/general/ddl/column-policy.html .. _fulltext search: https://cratedb.com/docs/crate/reference/en/latest/general/dql/fulltext.html .. _geospatial features: https://cratedb.com/docs/crate/reference/en/master/general/dql/geo.html .. _how-to guides: https://cratedb.com/docs/crate/howtos/ .. _HTTP API: https://cratedb.com/docs/crate/reference/en/latest/interfaces/http.html .. _installation documentation: https://cratedb.com/docs/crate/tutorials/en/latest/basic/index.html .. _introductory docs: https://cratedb.com/docs/crate/tutorials/ .. _Kubernetes: https://cratedb.com/docs/crate/tutorials/en/latest/containers/kubernetes/index.html .. _multi-region hybrid clouds and the edge: https://cratedb.com/docs/cloud/en/latest/tutorials/edge/index.html .. _no shared state: https://en.wikipedia.org/wiki/Shared-nothing_architecture .. _PostgreSQL wire protocol: https://cratedb.com/docs/crate/reference/en/latest/interfaces/postgres.html .. _queryable objects: https://cratedb.com/docs/crate/reference/en/master/general/dql/selects.html#container-data-types .. _reference manual: https://cratedb.com/docs/crate/reference/ .. _relational: https://en.wikipedia.org/wiki/Relational_model .. _responsibly disclose: https://en.wikipedia.org/wiki/Coordinated_vulnerability_disclosure .. _scaled horizontally: https://stackoverflow.com/questions/11707879/difference-between-scaling-horizontally-and-vertically-for-databases .. _SECURITY.md: https://github.com/crate/crate/blob/master/SECURITY.md .. _sql console: https://cratedb.com/docs/crate/admin-ui/en/latest/console.html#sql-console .. _standard SQL: https://cratedb.com/docs/crate/reference/en/latest/sql/index.html .. _support channels: https://cratedb.com/support .. _time-series data: https://cratedb.com/docs/crate/howtos/en/latest/getting-started/normalize-intervals.html .. _user-defined functions: https://cratedb.com/docs/crate/reference/en/latest/general/user-defined-functions.html

Frequently asked questions

Is CrateDB free to use?

CrateDB is open source under the Apache-2.0 licence. There is no licence fee and no seat count — you can self-host it or, where the project offers one, pay a vendor for a managed version instead.

What does CrateDB do?

Scalable SQL database for real-time analytics

What is CrateDB written in?

CrateDB is primarily written in Java. Its source is publicly available at https://github.com/crate/crate, and it has 4,436 GitHub stars.