Tulip: Modernizing Meta’s data platform

The technical journey discusses the motivations, challenges, and technical solutions employed for warehouse schematization, especially a change to the wire serialization format employed in Meta’s data platform for data interchange related to Warehouse Analytics Logging. Here, we discuss the engineering, scaling, and nontechnical challenges of modernizing  Meta’s exabyte-scale data platform by migrating to the new [...] Read More... The post Tulip: Modernizing Meta’s data platform appeared first on Engineering at Meta.
http://dlvr.it/ShTYlR

Komentar

Posting Komentar

Postingan populer dari blog ini

Inside Meta’s first smart glasses

Gambar
What’s new: Meta is sharing the inside story of how it developed the Ray-Ban Stories smart glasses. Why it matters: Creating Ray-Ban Stories meant Meta’s engineers had to take on new challenges to build smart glasses that married complex engineering dynamics. How do you make something that features cameras, microphones, audio, and touch controls, all [...] Read More... The post Inside Meta’s first smart glasses appeared first on Engineering at Meta. http://dlvr.it/SjWknx
Baca selengkapnya

Simulator-based reinforcement learning for data center cooling optimization

Gambar
We’re sharing more about the role that reinforcement learning plays in helping us optimize our data centers’ environmental controls. Our reinforcement learning-based approach has helped us reduce energy consumption and water usage across various weather conditions.   Meta is revamping its new data center design to optimize for artificial intelligence and the same methodology will be [...] Read More... The post Simulator-based reinforcement learning for data center cooling optimization appeared first on Engineering at Meta. http://dlvr.it/TD4BJ2
Baca selengkapnya

Improving machine learning iteration speed with faster application build and packaging

Gambar
Slow build times and inefficiencies in packaging and distributing execution files were costing our ML/AI engineers a significant amount of time while working on our training stack. By addressing these issues head-on, we were able to reduce this overhead by double-digit percentages.  In the fast-paced world of AI/ML development, it’s crucial to ensure that our [...] Read More... The post Improving machine learning iteration speed with faster application build and packaging appeared first on Engineering at Meta. http://dlvr.it/T22Qcp
Baca selengkapnya