About Me

My photo
Rohit is an investor, startup advisor and an Application Modernization Scale Specialist working at Google.

Thursday, September 12, 2019

How to chose between a DIY or vendor supported Kafka Streaming Platform ?

Streaming is the all the rage now both in movies and in application modernization :-) Have you wondered what capabilities developers need from a streaming data platform ? How do you evaluate a DIY or a Kafka streaming platform from all dimensions. In this blog post I present a simple 10 item checklist to evaluate your streaming platform be it DIY or an external vendor provided.

Here are the Top 10 Apache Kafka Capabilities Desired in Ascending Order
  1. Ability to use a Schema Registry
  2. Day 2 Operational support in Production and non-production environments at desired level of SLA @ scale
  3. Self-Service Support - Ability to self-provision topics for developers in DEV environments
  4. Monitor Kafka to gain insight. Developer dashboard like a Control Center user interface that gives visibility into Kafka topics. This improves developer productivity, observability and onboarding for newbies as Kafka command line can be intimidating for beginners.
  5. Dedicated Kafka support and consulting expertise for complex issues 
  6. Customizable retention period for messages. Backup and Restore of platform and data. 
  7. Support for Stateful Streaming including in-memory and persistent stores. Support for running RocksDB on PaaS or CaaS platforms.
  8. Kafka production topology satisfies Enterprise DR Requirements
  9. Multi-tenancy, security and namespacing of tenants in Kafka
  10. Path to prod for  apps from Dev, Test & Stage to Production Kafka environments
  11. Cost $$$ :-) Self Explanatory. 
Hopefully this Capability Matrix provides a guide to chose  a DIY or a vendor provided platform for Streaming.

No comments:

Post a Comment