Using shared memory for low-latency, intra-node communication in AWS Batch | AWS Compute Blog
Crazy HPC-oriented feature in current AWS offerings: shared memory buffers over EFA:
HPC workloads use algorithms that require parallelization and a low latency communication between the different processes. The two main technologies used for the parallel communications are message-passing with distributed memory and shared memory. Message Passing Interface (MPI) is a message-passing standard used for the communication in a parallel distributed environment. Elastic Fabric Adapter (EFA) enables your MPI applications to use low-latency, inter-node communication. The shared memory paradigm allows multiple processors in the same system to communicate using a memory (RAM) portion that is shared between the processes. This method takes advantage of the high-speed memory bus.
(tags: shared-memory hpc mpi shmem ram coding efa aws ec2 low-latency)
Extreme HTTP Performance Tuning: 1.2M API req/s on a 4 vCPU EC2 Instance | talawah.io
This is very cool. Updating the old “C10K” problem space to C1.2M — the current state of Linux userspace networking — using libreactor and a whole load of up-to-date tweaks. Interesting to note that this scale is feasible to run in Docker (using –network=host, of course).
(tags: http servers c10k linux performance scalability ops tuning libreactor networking tcp)
-
Scaleway dealing with the storage-driven cryptocurrency:
This cryptocurrency is a few months old, and at Scaleway, we are already witnessing the impact of Chia on all our products, from Object Storage and instances, to dedicated servers. Speculative farmers are forcing their way in to make as much money as possible, as quickly as possible. [….] In order to service as many clients as possible, we have decided that from today: Chia plotting is forbidden on all SSD and NVMe powered instances, dedicated servers, RPN-SAN, BMaaS and Block Storage services. Chia plotting is extremely I/O intensive and destroys most SSDs in under a few weeks Important notice: Chia plotting engages client responsibility according to Section 9 of our contract. We will bill clients for any SSDs and NVMes destroyed due to Chia plotting activities.
Alex Steffen on the discontinuity in our future
To believe that our choices are the restoration of continuity or the breakdown of society paradoxically is to not take the planetary [climate] crisis seriously enough — and that’s terrible, because the unexpected boon of seriousness is awakening to possibility, to the capacities we gain amidst disruption and acceleration. Seen through 20th century eyes, everything is about to get really weird, really fast. But discontinuity is not just danger. Discontinuity means change in our selves and our societies. [….] It is too late to avoid huge losses, enormous suffering. But it is absolutely not too late to limit our losses to those we’ve already set in motion, and to seize our opportunities to build a better human world — indeed, quite possibly a better world than the one we have now.
(tags: climate-change biodiversity crisis future alex-steffen green)