Big Data News Hubb
Advertisement
  • Home
  • Big Data
  • News
  • Contact us
No Result
View All Result
  • Home
  • Big Data
  • News
  • Contact us
No Result
View All Result
Big Data News Hubb
No Result
View All Result
Home News

Speedb’s Data Storage Engine Goes Open Source

admin by admin
November 16, 2022
in News


Speedb has announced it is going open source with its enterprise data storage engine billed as a drop-in replacement for RocksDB.

The company says making this technology available through open source brings an opportunity for the development community to take part in shaping the data architecture of the future. Key features from the enterprise version of Speedb are being made available to the open source community to support the needs of today’s data-intensive workloads, the company stated.

“We’re 100% committed to building a robust developer community around a modern data engine for modern applications, where data volumes often grow large and unwieldy,” said Adi Gelvan, co-founder and CEO of Speedb, in an email to Datanami. “We need developers to help us build the best storage engine we can. We want to ask the community what it wants, receive their pull requests, and refine options we may not even have considered. With the feedback we get from the community we can build something really special.”

The Speedb data engine is a key-value store, which is a NoSQL database composed of a key and an associated data value. Used in cases where horizontal scaling is required, key-value stores are especially hot in metadata management where a flood of metadata is impacting the performance of databases, streaming apps, and analytics.

RocksDB, an open source project since 2012, is the de facto standard embedded key-value store, according to Speedb. However, while working with RocksDB at storage vendor Infinidat, the founders of Speedb found it begins to stall when metadata storage approaches 50GB. Other RocksDB issues include code instability and write amplification, along with the need for sharding and performance tuning, says Speedb.

Speedb redesigned key parts of RocksDB including a compaction algorithm that reduces the Write Amplification Factor (WAF) from ~30 to ~5 – as well as redesigning its I/O and flow mechanisms. RocksDB users can embed the Speedb data engine by changing a few lines of code to boost metadata memory performance and reduce operational tasks, according to Speedb.

“We’ve been using Speedb internally for a while now, and we’re very excited for the promise it shows over RocksDB, in memory management, compaction, and configurability,” said Haikal Pribadi, CEO at Vaticle, creators of the strongly-typed database, TypeDB, and its query language, TypeQL. “Most importantly, having the dedicated support of Speedb in our development efforts on top of the storage infrastructure has been invaluable. Speedb already has made a huge difference to our team.”

A still from an introductory video shows the capabilities of Speedb. Source: Speedb

Speedb will continue selling its enterprise edition through Redis and other OEMs for those developers with larger and more complex applications. The company’s enterprise edition offers additional features, higher scalability, support services, and tuning. The company claims users of its enterprise edition can generate up to double throughput with half the latency and double the capacity compared to the same deployment with RocksDB.

By offering its data engine through open source, the company hopes to improve its product through collaboration with the developers who know it best. The Speedb open source community will serve as a hub for Speedb and RocksDB users to collaborate while sharing knowledge, issues, and solutions, the company says.

“By going open source, Speedb can help far more developers to remove the key barriers to data performance and take their hyperscale data operations to the next level, backed by a supportive developer community,” said CEO Gelvan. “We’ll be making ongoing additions of our enterprise functionality to the open-source edition and are looking forward to embracing the community assistance and collaborating with the many creative minds in the developer realm that are also searching for more efficient ways to rev their data engines.”

Speedb’s open source edition is available at GitHub and Speedb’s dev site.

Related Items:

Speedb Picks Up Where RocksDB Leaves Off

Second Half 2022 Tech Predictions for Data and AI

Speedb Secures $4M and Launches on Redis to Reveal New Data Engine Technology



Source link

Previous Post

A Brief Guide on Unstructured Text (Data)

Next Post

Your guide to streaming data & real-time analytics at re:Invent 2022

Next Post

Your guide to streaming data & real-time analytics at re:Invent 2022

Recommended

“State of AI in the Enterprise” Report, 5th Edition, Uncovers Four Key Actions to Maximize AI Value

October 19, 2022

A system for generating 3D point clouds from complex prompts

March 17, 2023

Choose Both: Data Fabric and Data Lakehouse

January 16, 2023

Don't miss it

News

Global Benefits of International Courier Services

March 21, 2023
Big Data

An early look at the labor market impact potential of large language models

March 21, 2023
Big Data

Unlocking Hidden Value From Production Line Data

March 20, 2023
Big Data

Fine-Tuning Large Language Models with Hugging Face and DeepSpeed

March 20, 2023
News

Accelerating revenue growth with real-time analytics: Poshmark’s journey

March 20, 2023
News

Big Data Career Notes: March 2023 Edition

March 20, 2023

big-data-footer-white

© 2022 Big Data News Hubb All rights reserved.

Use of these names, logos, and brands does not imply endorsement unless specified. By using this site, you agree to the Privacy Policy and Terms & Conditions.

Navigate Site

  • Home
  • Big Data
  • News
  • Contact us

Newsletter Sign Up

No Result
View All Result
  • Home
  • Big Data
  • News
  • Contact us

© 2022 Big Data News Hubb All rights reserved.