Our mission at Prospective is to make it easier for anyone, at any organization, at any scale to make their data actionable. We set out to build something that could finally transform any company into a data-driven organization without the need for complex infrastructure. What if custom data products could be built directly in the browser? What if instead of setting up server after server to handle ingestion, scale, and bespoke BI tools, you could do everything within a web application? What if you could finally transform the data team into a revenue generating asset rather than a cost center?
For many of our customers, the challenges of working with real-time data prevent them from answering these questions. We have seen them build out large-scale Kafka implementations only to realize that Kafka itself, while an incredible piece of software, does not equal real-time data analytics. We wanted to change this, so we built a way to dramatically simplify working with Kafka with Prospective.
The goal was simple. Give our customers a way to interact directly with Kafka data streams.
Sounds great, but what does this actually mean? Think about data in a Kafka data stream like cars on a highway. These cars are continuously driving down the highway just as data events are continuously getting created. In current data systems, these data events get ingested in a separate workflow before they can be visualized or used by a user on a desktop. Or, sticking with this analogy, the data needs to stop for gas before it can reach its final destination. This gas station is the equivalent to a separate server. A server that requires continuous maintenance, engineering resources, and capital. We built our Kafka adapter to remove the need for this gas station, or intermediate server.
Again, sounds great, but why is this important? There are three main implications of using Prospective’s Kafka adapter:
- Data can be consumed directly off the highway. For our capital markets clients this means they have actual real-time visibility into the market signals their quant team generates. This means faster and more profitable decision making.
- Leveraging Kafka no longer requires setting up additional infrastructure. Many of our design partners have built complex infrastructure to ensure they get the performance benefits of Kafka. Now, instead of setting up one server for the ingestion of Kafka events and another for users to visualize these events, they simply connect Prospective to the Kafka data stream directly. This means less time on infrastructure and more time building revenue generating products.
- The benefits of Kafka are no longer reserved for organizations with sophisticated engineering teams and huge budgets. This means anyone using Prospective can get the real-time benefits of Kafka immediately [out of the box]. We have seen this benefit marketing teams implementing real-time discounting, online gaming companies tracking real-time spreads, and crypto firms tracking real-time market updates.
Our Kafka adapter both (1) reduces our customers data infrastructure footprint (and bill) and (2) directly increases their ability to execute on their core business priorities. We’ve been able to do this because of the early bet we made on the browser as the foundation for software delivery.
If you are building with Kafka, we’d love to chat with you about how we could simplify and enhance your existing user experience @ https://prospective.co/meet-eric
Thanks to Caitlin Lohrenz and Davis Silverman for their help on this post!
