Taming the tail utilization of ads inference at Meta scale

Tail utilization is a significant system issue and a major factor in overload-related failures and low compute utilization. The tail utilization optimizations at Meta have had a profound impact on model serving capacity footprint and reliability.  Failure rates, which are mostly timeout errors, were reduced by two-thirds; the compute footprint delivered 35% more work for [...]


Read More...


The post Taming the tail utilization of ads inference at Meta scale appeared first on Engineering at Meta.


http://dlvr.it/T9QxWF

Komentar

Posting Komentar

Postingan populer dari blog ini

Inside Meta’s first smart glasses

Gambar
What’s new: Meta is sharing the inside story of how it developed the Ray-Ban Stories smart glasses. Why it matters: Creating Ray-Ban Stories meant Meta’s engineers had to take on new challenges to build smart glasses that married complex engineering dynamics. How do you make something that features cameras, microphones, audio, and touch controls, all [...] Read More... The post Inside Meta’s first smart glasses appeared first on Engineering at Meta. http://dlvr.it/SjWknx
Baca selengkapnya

Maintaining large-scale AI capacity at Meta

Gambar
Meta is currently operating many data centers with GPU training clusters across the world. Our data centers are the backbone of our operations, meticulously designed to support the scaling demands of compute and storage. A year ago, however, as the industry reached a critical inflection point due to the rise of artificial intelligence (AI), we [...] Read More... The post Maintaining large-scale AI capacity at Meta appeared first on Engineering at Meta. http://dlvr.it/T8BcRq
Baca selengkapnya

Post-quantum readiness for TLS at Meta

Gambar
Today, the internet (like most digital infrastructure in general) relies heavily on the security offered by public-key cryptosystems such as RSA, Diffie-Hellman (DH), and elliptic curve cryptography (ECC). But the advent of quantum computers has raised real questions about the long-term privacy of data exchanged over the internet. In the future, significant advances in quantum [...] Read More... The post Post-quantum readiness for TLS at Meta appeared first on Engineering at Meta. http://dlvr.it/T7Flq2
Baca selengkapnya