Open Source PostgreSQL on Steroids: Swarm64 database acceleration software to improve performance and analysis

Posted on

PostgreSQL is a big deal. The most common SQL open source database that you’ve never heard of, as ZDNet’s own Tony Baer called it. In addition to being the framework on which a number of commercial offers were built, PostgreSQL has its own user base. According to DB-engines, PostgreSQL is the 4th most popular database in the world.

Swarm64on the other hand is a small supplier. So small, in fact, that we have shared the stage with CEO Thomas Richter at a local Berlin Meetup a few years back. Back then, Richter was not CEO, and Swarm64 was even smaller. But its value proposition still sounded attractive: increase PostgreSQL’s performance for free. Swarm64 is an acceleration layer for PostgreSQL.

Of course, there is no such thing as a free lunch, so the “free” part is a speech. Swarm64 is a commercial supplier. Until recently, however, the real gotcha was hardware: Swarm64 Database Acceleration (DA) required a specialized chip called FPGA to be able to perform its PostgreSQL magic. With Swarm64 DA 4.0 released today, this is no longer the case.

Accelerating PostgreSQL

First, a little history. Why choose to speed up PostgreSQL? Specific, An open source database makes sense for many reasons, and PostgreSQL is popular. So to make it more concrete: why not MySQL, PostgreSQL’s rival, which is even more popular?

When the Swarm64 DA was initially developed, the issue was considered, Richter said. PostgreSQL was finally chosen, “because it is a wonderfully expandable database that helped us bring the Swarm64 DA to market before – as an extension to the database rather than a fork”.

Having been familiar with previous versions of the Swarm64 DA, the difference in this one stood out: no FPGA needed. Richter noted that Swarm64 DA 4.0 is a software-only solution that requires no hardware upgrades at all. So how does it work?

A primary feature of Swarm64 DA is its ability to add more parallel processing to PostgreSQL queries, and it can be done with or without FPGA, Richter said. In lab studies, he kept adding, only CPU acceleration at a 1 terabyte TPC-H the database is 10x; adding an FPGA nearly doubles that. The FPGA benefits increase relative to the CPU only when the database size or the number of concurrent users increases.

Before moving on, word on the benchmarks and use cases. The TPC-H mentioned by Richter is a benchmark designed to evaluate the performance of systems used for analytics-related queries. While Swarm64 is about improving performance, the release of DA 4.0 marks a focus on analysis.

The message seems to be that PostgreSQL plus Swarm64 DA 4.0 can be used as a data storage solution. Richter also noted that FPGAs allow people to use a single database to support data collection and reporting at the same time: “CPUs put data into DBMS while FPGA handles reporting and analysis. We often see this in IoT , GIS use cases, perform real-time analysis. ”

Richter went on to add that Swarm64 DA speeds up PostgreSQL by using multiple techniques. Column indexing is one, but it adds much greater parallel processing for query execution and data compression, which helps reduce time-consuming I / O. This release also includes accelerated SQL interconnections and text search:

“These are the best ingredients in most data warehouses or analytics databases, and we add them to PostgreSQL to help companies deploy systems with insight much easier and less expensive.”

It never hurts to help people cut costs

PostgreSQL is a relational database designed for operational, transactional workloads. While convenience is part of the reason people might want to stick to their PostgreSQL for analytics workloads, the cost is yet another.

The value proposition for Swarm64 appears to be “keep your open source, free PostgreSQL, get it running faster”. Swarm64 DA’s price point is $ 33 / vCore / month. This may sound compelling, but one must also wonder how many of Swarm64’s clients really only use PostgreSQL without additional support or other services. Including the cost of internal resources and expertise is real TCO is hidden and adds.

Richter noted that the real benefit of Swarm64 DA is allowing people to use free, open source PostgreSQL in multiple projects, especially in analytics and data storage, where its adoption has not been as strong historically:

“Using PostgreSQL for these projects helps people save lots of money on commercial DBMS licensing costs; the annual DBMS maintenance fee paid to a vendor such as Oracle can reach 6. Using PostgreSQL for free eliminates it – others Costs like DB administrators and tools are expenses that people incur, whether they use commercial DBs or free of charge.

The cost of using Swarm64 DA-accelerated PostgreSQL bleaches compared to commercial local or cloud-based data storage platforms – roughly one-third of the total cost of using Amazon Redshift over 3 years and almost 10x less than most older data storage platforms “.

This math is something to consider for any potential user with multiple use cases in data warehousing modernization, new machine learning and data science systems and real-time analytics (IoT, Time Series, GIS), which Richter said are the ones they encounter the most.

As Richter noted, it never hurts to help people cut costs, and cost reduction has always been a big reason for success in adopting an open source database. The interest in saving money in new ways with Swarm64 DA-accelerated PostgreSQL is very strong in the current climate, just as it was before the pandemic, he continued to add.


This release seems to have the potential to make the combination of PostgreSQL plus Swarm64 DA 4.0 a competitor for analytical workloads. Richter mentioned that Swarm64 is focused on PostgreSQL for the foreseeable future, but adding support to MariaDB and MySQL or other databases at some point is an option.

In addition, Swarm64 does not forget its roots and keep an eye on hardware. Swarm64 has a dual relationship with Intel and Xilinx: they are both partners and investors. In addition to common business development, Richter said, these partnerships provide Swarm64 with an inside track on new hardware innovations:

“There’s a lot going on in hardware – FPGAs of course, but also persistent RAM, computational storage and edge computing. These are potential game changers in analytics and IoT data management, and our hardware partnerships allow us to help customers to get competitive performance benefits “.

DA 4.0 offers triple the speed of Swarm64 DA 3.0, but Richter believes there is still room for improvement. In addition to finding new ways to make PostgreSQL faster, the focus is on ease of use, making it even easier for PostgreSQL users to take advantage of Swarm64 DA.

Swarm64 has also developed a few open source projects that helps PostgreSQL users accelerate, scale, and measure performance. Over the past year, patches for resilient scaling and data backup / recovery were contributed and the plan is to continue developing them.

Source link

Leave a Reply

Your email address will not be published. Required fields are marked *