Presentation name: A Beginner's Guide to Inter-Service Messaging
Speakers: Aurélien Bompard, Jeremy Cline
Description: Sending and receiving messages is a common task in software.
Fedora Infrastructure has many event-driven services, we'll discuss some of the problems we encountered and their solutions. Topics include:
- How to pick the right message protocol.
- How to design a "good message". How to recover when you inevitably
don't design a "good message" or when the meaning of the word "good" changes. - Networking: centralized or decentralized?
- How to deal with lost messages. Is it bad? Should you care?
Each topic contains plenty of concrete examples from Fedora Infrastructure.
At the end of this talk, the audience should be well-equipped to not repeat
the mistakes we made and instead go make new and interesting mistakes.
This talk does not require prior knowledge of any message protocols.
Apache Kafka is publish-subscribe messaging rethought as a distributed commit log.
Fast
A single Kafka broker can handle hundreds of megabytes of reads and writes per second from thousands of clients.
Scalable
Kafka is designed to allow a single cluster to serve as the central data backbone for a large organization. It can be elastically and transparently expanded without downtime. Data streams are partitioned and spread over a cluster of machines to allow data streams larger than the capability of any single machine and to allow clusters of co-ordinated consumers
Durable
Messages are persisted on disk and replicated within the cluster to prevent data loss. Each broker can handle terabytes of messages without performance impact.
Distributed by Design
Kafka has a modern cluster-centric design that offers strong durability and fault-tolerance guarantees.
Apache Feather
My adjustable height desk [1] can now measure it's current height and announce it via MQTT thanks to a little NodeMCU script I whipped up and an HC-SR04 ultrasonic sensor connected to the ESP8266. Using mqttwarn [2] that data gets pushed into a Google Docs spreadsheet where I can then plot it easily.
QTT is a machine-to-machine (M2M)/"Internet of Things" connectivity protocol. It was designed as an extremely lightweight publish/subscribe messaging transport. It is useful for connections with remote locations where a small code footprint is required and/or network bandwidth is at a premium. For example, it has been used in sensors communicating to a broker via satellite link, over occasional dial-up connections with healthcare providers, and in a range of home automation and small device scenarios. It is also ideal for mobile applications because of its small size, low power usage, minimised data packets, and efficient distribution of information to one or many receivers
Mosquitto is an open source (BSD licensed) message broker that implements the MQ Telemetry Transport protocol versions 3.1 and 3.1.1. MQTT provides a lightweight method of carrying out messaging using a publish/subscribe model. This makes it suitable for "machine to machine" messaging such as with low power sensors or mobile devices such as phones, embedded computers or microcontrollers like the Arduino. A good example of this is all of the work that Andy Stanford-Clark (one of the originators of MQTT) has done in home monitoring and automation with his twittering house and twittering ferry. Andy gave a talk on this at OggCamp that explains a bit about MQTT and how he uses it. The slides and audio are available online at slideshare
The bandwidth management with parent queue tree every client. Every queue tree clients to be a parent that covers download, streaming, browsing, and online g...