UStore
Modular 1 Multi-Modal 2 Transactional 3 Database
For Artificial Intelligence 4 and Semantic Search 5
- supports:
RocksDB
•
LevelDB
•
UDisk
•
UCSet
backends
2. can store:
Blobs
•
Documents
•
Graphs
•
🔜 Features
•
🔜 Texts
3: guarantees
Atomicity
•
Consistency
•
Isolation
•
Durability
4: comes with
Pandas
and
NetworkX
API
and 🔜
PyTorch data-loaders
5: brings
vector-search
integrated with
USearch
and
UForm
drivers:
Python
•
C
•
C++
•
GoLang
•
Java
packages:
PyPI
•
CMake
•
Docker Hub
Youtube intro • Discord chat • Full documentation
Quickstart
Installing UStore is a breeze, and the usage is about as simple as a Python dict.
$ pip install ukv
$ python
from ukv import umem
db = umem.DataBase()
db.main[42] = 'Hi'
We have just create an in-memory embedded transactional database and added one entry in its main collection.
Would you prefer that data on disk?
Change one line.
from ukv import rocksdb
db = rocksdb.DataBase('/some-folder/')
Would you prefer to connect to a remote UStore server? UStore comes with an Apache Arrow Flight RPC interface!
from ukv import flight_client
db = flight_client.DataBase('grpc://0.0.0.0:38709')
Are you storing [NetworkX][networkx]-like MultiDiGraph?
Or [Pandas][pandas]-like DataFrame?
db = rocksdb.DataBase()
users_table = db['users'].table
users_table.merge(pd.DataFrame([
{'id': 1, 'name': 'Lex', 'lastname': 'Fridman'},
{'id': 2, 'name': 'Joe', 'lastname': 'Rogan'},
]))
friends_graph = db['friends'].graph
friends_graph.add_edge(1, 2)
assert friends_graph.has_edge(1, 2) and \
friends_graph.has_node(1) and \
friends_graph.number_of_edges(1, 2) == 1
Function calls may look identical, but the underlying implementation can be addressing hundreds of terabytes of data placed somewhere in persistent memory on a remote machine.
Is someone else concurrently updating those collections? Bundle your operations to guarantee consistency!
db = rocksdb.DataBase()
with db.transact() as txn:
txn['users'].table.merge(...)
txn['friends'].graph.add_edge(1, 2)
So far we have only covered the tip of the UStore. You may use it to...
- Get C99, Python, GoLang, or Java wrappers for RocksDB or LevelDB.
- Serve them via Apache Arrow Flight RPC to Spark, Kafka, or PyTorch.
- Store Document and Graphs in embedded DB, avoiding networking overheads.
- Tier DBMS between in-memory and persistent backends under one API.
But UStore can more. Here is the map:
- Basic Usage:
- Advanced Usage for production, performance tuning, and administration:
- For contributors and advanced users looking to fork, extend, wrap, or distribute and, potentially, monetize alternative builds of UStore:
Basic Usage
UStore is intended not just as database, but as "build your database" toolkit and an open standard for NoSQL potentially-transactional databases, defining zero-copy binary interfaces for "Create, Read, Update, Delete" operations, or CRUD for short.
A few simple C99 headers can link almost any underlying storage engine to numerous high-level language drivers, extending their support for binary string values to graphs, flexible-schema documents, and other modalities, aiming to replace MongoDB, Neo4J, Pinecone, and ElasticSearch with a single ACID-transactional system.
[Redis][redis], for example, provides RediSearch, RedisJSON, and RedisGraph with similar objectives. UStore does it better, allowing you to add your favorite Key-Value Stores (KVS), embedded, standalone, or sharded, such as [FoundationDB][foundationdb], multiplying its functionality.
Modalities
Blobs
Binary Large Objects can be placed inside UStore. The performance will va