APISIX API Gateway provides rich traffic management features like load balancing, dynamic upstream, canary release, circuit breaking, auth, and observability.
Strimzi provides a way to run an Apache Kafka cluster on Kubernetes in various deployment configurations.
Apache APISIX is a dynamic, real-time, high-performance API gateway.
APISIX provides rich traffic management features such as load balancing, dynamic upstream, canary release, circuit breaking, authentication, observability, and more.
You can use Apache APISIX to handle traditional north-south traffic, as well as east-west traffic between services. It can also be used as a k8s ingress controller.
Apache Pulsar is an open-source distributed pub-sub messaging system originally created at Yahoo and now part of the Apache Software Foundation
Faust is a stream processing library, porting the ideas from Kafka Streams to Python.
It is used at Robinhood to build high performance distributed systems and real-time data pipelines that process billions of events every day.
Faust provides both stream processing and event processing, sharing similarity with tools such as Kafka Streams, Apache Spark/Storm/Samza/Flink,
It does not use a DSL, it’s just Python! This means you can use all your favorite Python libraries when stream processing: NumPy, PyTorch, Pandas, NLTK, Django, Flask, SQLAlchemy, ++
Faust requires Python 3.6 or later for the new async/await syntax, and variable type annotations.
Apache Mesos abstracts CPU, memory, storage, and other compute resources away from machines (physical or virtual), enabling fault-tolerant and elastic distributed systems to easily be built and run effectively.
An OS to build, deploy and securely manage billions of devices
Recent Posts
Stuff The Internet Says On Scalability For August 28th, 2015x
7 Strategies for 10x Transformative Change
Ask HighScalability: Choose an Async App Server or Multiple Blocking Servers?
Stuff The Internet Says On Scalability For August 21st, 2015
The Microsoft Take on Containers and Docker
Sponsored Post: Surge, Redis Labs, Jut.io, VoltDB, Datadog, MongoDB, SignalFx, InMemory.Net, Couchbase, VividCortex, MemSQL, Scalyr, AiScaler, AppDynamics, ManageEngine, Site24x7
How Autodesk Implemented Scalable Eventing over Mesos
Stuff The Internet Says On Scalability For August 14th, 2015
Why My Water Droplet Is Better Than Your Hadoop Cluster
How Google Invented an Amazing Datacenter Network Only They Could CreateAll Time Favorites
More Favorites...
YouTube Architecture
Plenty Of Fish Architecture
Google Architecture
How Twitter Stores 250 Million Tweets A Day Using MySQL
Scaling Twitter: Making Twitter 10000 Percent Faster
Flickr Architecture
Amazon Architecture
How I Learned to Stop Worrying and Love Using a Lot of Disk Space to Scale
Stack Overflow Architecture
Facebook
An Unorthodox Approach to Database Design : The Coming of the Shard
Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud
Are Cloud Based Memory Architectures the Next Big Thing?
Latency is Everywhere and it Costs You Sales - How to Crush it
How will memristors change everything?
DataSift Architecture: Realtime Datamining At 120,000 Tweets Per Second
Useful Scalability Blogs
Scaling Traffic: People Pod Pool of On Demand Self Driving Robotic Cars who Automatically Refuel from Cheap Solar
VoltDB Decapitates Six SQL Urban Myths And Delivers Internet Scale OLTP In The Process
The Canonical Cloud Architecture
Justin.Tv's Live Video Broadcasting Architecture
Why Are Facebook, Digg, And Twitter So Hard To Scale?
What The Heck Are You Actually Using NoSQL For?
Playfish's Social Gaming Architecture - 50 Million Monthly Users And Growing
The Updated Big List Of Articles On The Amazon Outage
More Favorites...advertise
Login
Register
All Time Favorites
Useful Products
Useful Papers
Useful Strategies
Useful Blogs
Useful Books
Useful Conferences
Book Store
High Scalability RSS
High Scalability Comments RSS« The Mother of All Database Normalization Debates on Coding Horror | Main | Can cloud computing smite down evil zombie botnet armies? »
ZooKeeper - A Reliable, Scalable Distributed Coordination System
Tuesday, July 15, 2008 at 12:45AM
ZooKeeper is a high available and reliable coordination system. Distributed applications use ZooKeeper to store and mediate updates key configuration information. ZooKeeper can be used for leader election, group membership, and configuration maintenance. In addition ZooKeeper can be used for event notification, locking, and as a priority queue mechanism. It's a sort of central nervous system for distributed systems where the role of the brain is played by the coordination service, axons are the network, processes are the monitored and controlled body parts, and events are the hormones and neurotransmitters used for messaging. Every complex distributed application needs a coordination and orchestration system of some sort, so the ZooKeeper folks at Yahoo decide to build a good one and open source it for everyone to use.
Apache ZooKeeper is the coolest technology I recently came across. I found it when I was doing a research about Solr Cloud features. I got very impressed by Solr’s distributed computing. You literately have to fire a new instance and it will automatically find its place in “the cloud”. It will assign itself to a particular shards and it will make a decision to become a leader or a replica. Later you can query any of the available servers and it will find you all required data even if it’s not on that server. If some of the servers fail the service will continue to work. Very dynamic, very clever, very cool.
Apache Kafka is publish-subscribe messaging rethought as a distributed commit log.
Fast
A single Kafka broker can handle hundreds of megabytes of reads and writes per second from thousands of clients.
Scalable
Kafka is designed to allow a single cluster to serve as the central data backbone for a large organization. It can be elastically and transparently expanded without downtime. Data streams are partitioned and spread over a cluster of machines to allow data streams larger than the capability of any single machine and to allow clusters of co-ordinated consumers
Durable
Messages are persisted on disk and replicated within the cluster to prevent data loss. Each broker can handle terabytes of messages without performance impact.
Distributed by Design
Kafka has a modern cluster-centric design that offers strong durability and fault-tolerance guarantees.
Apache Feather