Twitter | Search | |
Jay Kreps
CEO of (). Co-creator of . Sí se puede.
6,961
Tweets
654
Following
23,568
Followers
Tweets
Jay Kreps retweeted
James Watters Aug 22
Excited for SF next month. See you there.
Reply Retweet Like
Jay Kreps Aug 18
Replying to @rjurney
Totally agree. I do think there is a substantial set of things moving from batch to real time, but by no means everything.
Reply Retweet Like
Jay Kreps Aug 18
Replying to @rjurney
I think you need to use the normalized topic for Apache Spark not the English word “spark”. It’s hard to conclude much from that though as Spark has both stream processing and batch processing capabilities.
Reply Retweet Like
Jay Kreps Aug 18
The world is moving from batch to real-time.
Reply Retweet Like
Jay Kreps retweeted
Tyler Treat Aug 10
Use managed databases.
Reply Retweet Like
Jay Kreps Aug 9
My home town! Della Fattoria is good as is The Tea Room. Actually all the food is about 100x better these days. At some point (after I left) Sonoma County stopped being “cow country” and started being “wine country”. Gentrification has its perks, I guess.
Reply Retweet Like
Jay Kreps retweeted
Michael Drogalis Aug 8
Definitely the coolest bit of building Tutorials is the underlying build system. We devised a way to describe each tutorial using data structures, then use parallel tools to both render and test the site.
Reply Retweet Like
Jay Kreps retweeted
Confluent Aug 8
We’re excited to announce Tutorials for , a collection of common event streaming use cases, with each tutorial featuring an example scenario and several complete code solutions. Learn more in our latest blog post by :
Reply Retweet Like
Jay Kreps Aug 7
This is pretty cool. The folks at have built an unbundled database that uses or LMDB for indexes and as the commit log.
Reply Retweet Like
Jay Kreps Aug 6
Huge congrats!
Reply Retweet Like
Jay Kreps Aug 4
An old blog post I wrote in like 2012 resurfaced on Hacker News. It’s not wrong, distributed systems ops remains hard, but these days the solution is easier: just get your distributed systems as a service from the people who wrote them, and whose business is keeping them up.
Reply Retweet Like
Jay Kreps Aug 4
FWIW, you can get Kafka as a service where you pay for what you use with no minimum, starting at $0.11/GB read/written/stored, so no need to use a worse abstraction just to avoid the ops burden.
Reply Retweet Like
Jay Kreps Aug 4
That’s the key point: there is significant overlap so even if you embed it you end up operating, configuring, and securing two consensus systems, the native one in Kafka and the one used for metadata. Converging these a bit harder but a much simpler, more elegant end product.
Reply Retweet Like
Jay Kreps Aug 4
Replying to @jaykreps
Doesn’t mean it won’t be hard, just that a lot of the pieces are there already and have to work correctly, and having a consistent implementation between data and metadata logs will be a big simplification in managability and scalability.
Reply Retweet Like
Jay Kreps Aug 4
On the Kafka ZK removal some worry “isn’t consensus hard to get right?” It is! However the consistent log abstraction Kafka provides is already essentially a form of consensus. Making that “self hosting” and using it for metadata makes sense.
Reply Retweet Like
Jay Kreps Aug 3
Huh, I find there is a pretty large group of people who are remarkably interested and thoughtful about the internals of the systems they use and like to follow along.
Reply Retweet Like
Jay Kreps Aug 2
Replying to @criccomini @jake_reps
Uh oh, my evil alter ego is discovered.
Reply Retweet Like
Jay Kreps Aug 2
It’s cool how KIPs (feature design proposals) have a following much larger than the set of people writing code in Kafka. A great thing about open source is seeing the thought process behind the system, and having smart people show up to tell you how to do it better.
Reply Retweet Like
Jay Kreps Aug 1
Folks are working on what is probably the most requested feature in : removing ZooKeeper as a dependency. This makes the minimum footprint smaller, and makes it a lot easier to manage, secure, and scale. Lots of work to do to make it real.
Reply Retweet Like
Jay Kreps Jul 31
Replying to @apachekafka
If what you want is , but don’t want the ops overhead, Confluent Cloud provides that as a service in AWS/GCP/Azure for ~$0.11/GB of usage with no minimum, so you can start as small as you want and scale up when needed.
Reply Retweet Like