Unikraft powers the next-generation of cloud native applications by enabling you to radically customize and build custom OS/kernels, unlocking best-in-class performance, security primitives and efficiency savings.
Originally posted at https:/opensource.com/article/19/8/introduction-bpftrace.
bpftrace is a new open source tracer for Linux for analyzing production performance problems and troubleshooting software. It is used by and has had contributions from many companies including Netfilx, Facebook, Red Hat, Shopify, and others. It was created by Alastair Robertson, a talented UK-based developer who has previously won various coding competitions.
Linux already has many performance tools, but these are often counter-based and have limited visibility. For example, iostat(1), or a monitoring agent, may tell you your average disk latency, but not the distribution of this latency. Distributions can reveal multiple modes, or outliers, either of which may be the real cause of your performance problems. bpftrace is suited for this kind of analysis: decomposing metrics into distributions or per-event logs, and creating new metrics for visibility into blind spots.
This teaches you bpftrace for Linux in 12 easy lessons, where each lesson is a one-liner you can try running. This series of one-liners introduces concepts which are summarized as bullet points. For a full reference to bpftrace, see the Man page
Contributed by Brendan Gregg, Netflix (2018), based on his FreeBSD DTrace Tutorial.
This page describes Pressure Stall Information (PSI), how to use it, tools and components that work with PSI, and some case studies showing how PSI is used in production today for resource control in large data centers.
Overview
PSI provides for the first time a canonical way to see resource pressure increases as they develop, with new pressure metrics for three major resources—memory, CPU, and IO.
These pressure metrics, in conjunction with cgroup2 and other kernel and userspace tools described below, allow you to detect resource shortages while they’re developing, and respond intelligently—by pausing or killing non-essentials, reallocating memory in the system, load shedding, or other actions.
PSI stats are like barometers that provide fair warning of impending resource shortages, enabling you to take more proactive, granular, and nuanced steps when resources start becoming scarce.
See the PSI git repo for additional information.
bpftrace is a high-level tracing language for Linux enhanced Berkeley Packet Filter (eBPF) available in recent Linux kernels (4.x). bpftrace uses LLVM as a backend to compile scripts to BPF-bytecode and makes use of BCC for interacting with the Linux BPF system, as well as existing Linux tracing capabilities: kernel dynamic tracing (kprobes), user-level dynamic tracing (uprobes), and tracepoints. The bpftrace language is inspired by awk and C, and predecessor tracers such as DTrace and SystemTap. bpftrace was created by Alastair Robertson.
Interested in learning more about low-level specifics of the eBPF stack? Read on as we take a deep dive, from its VM mechanisms and tools, to running traces on remote, resource-constrained embedded devices.
BPF and eBPF are the same thing and the name/acronym doesn't mean much anymore because the project evolved way beyond its initial scope. BPF and eBPF are used interchangeably in the series.
Part 1 and 2 give a condensed in-depth introduction to eBPF for newcommers or those looking to further their knowledge by taking a deep dive into the lower part of the eBPF stack.
Part 3 is an overview of the userspace tooling meant to increase productivity, built on top of the low level VM mechanisms presented in parts 1 and 2.
Part 4 focuses on running eBPF programs on embedded systems which are resource constrained, where a full toolchain stack (BCC/LLVM/python/etc) is undesirable. We'll cross-compile and run eBPF programs on 32 bit ARM with smaller embedded-focused tools. Readers interested only in this part can probably skip the rest.
Part 5 is about tracing userspace: Up until now our efforts were focused on tracing the kernel, so it's about time we also examine other user processes.I was in Prague last month for the 2017 edition of the KVM Forum. There I gave a talk about some of the work that I’ve been doing this year to improve the qcow2 file format used by QEMU for storing disk images. The focus of my work is to make qcow2 faster and to reduce its memory requirements.
I was in Prague last month for the 2017 edition of the KVM Forum. There I gave a talk about some of the work that I’ve been doing this year to improve the qcow2 file format used by QEMU for storing disk images. The focus of my work is to make qcow2 faster and to reduce its memory requirements.
Snabb (formerly "Snabb Switch") is a simple and fast packet networking toolkit.
We are also a grassroots community of programmers and network engineers who help each other to build and deploy new network elements. We care about practical applications and finding simpler ways to do things.
EnhanceIO driver is based on EnhanceIO SSD caching software product
developed by STEC Inc. EnhanceIO was derived from Facebook's open source
Flashcache project. EnhanceIO uses SSDs as cache devices for
traditional rotating hard disk drives (referred to as source volumes
throughout this document).
EnhanceIO can work with any block device, be it an entire physical
disk, an individual disk partition, a RAIDed DAS device, a SAN volume,
a device mapper volume or a software RAID (md) device.
The source volume to SSD mapping is a set-associative mapping based on
the source volume sector number with a default set size
(aka associativity) of 512 blocks and a default block size of 4 KB.
Partial cache blocks are not used. The default value of 4 KB is chosen
because it is the common I/O block size of most storage systems. With
these default values, each cache set is 2 MB (512 * 4 KB). Therefore,
a 400 GB SSD will have a little less than 200,000 cache sets because a
little space is used for storing the meta data on the SSD.
EnhanceIO supports three caching modes: read-only, write-through, and
write-back and three cache replacement policies: random, FIFO, and LRU.
Read-only caching mode causes EnhanceIO to direct write IO requests only
to HDD. Read IO requests are issued to HDD and the data read from HDD is
stored on SSD. Subsequent Read requests for the same blocks are carried
out from SSD, thus reducing their latency by a substantial amount.
In Write-through mode - reads are handled similar to Read-only mode.
Write-through mode causes EnhanceIO to write application data to both
HDD and SSD. Subsequent reads of the same data benefit because they can
be served from SSD.
Write-back improves write latency by writing application requested data
only to SSD. This data, referred to as dirty data, is copied later to
HDD asynchronously. Reads are handled similar to Read-only and
Write-through modes.
Veröffentlicht am 21.12.2016
Demo from the USENIX/LISA 2016 talk: Linux 4.X Tracing Tools: Using BPF Superpowers. Full talk slides and official video will be available at https://www.usenix.org/conference/lis... . This video is the 15 (more like 18) minute demo that spans ftrace, perf, and BPF. LISA is a registered trademark of the USENIX Association and is used in this video with permission.
Published on Mar 2, 2016
Video: https://www.facebook.com/atscaleevents/videos/1693888610884236/