Scaling data ingestion for machine learning training at Meta

Many of Meta’s products, such as search and language translations, utilize AI models to continuously improve user experiences. As the performance of hardware we use to support training infrastructure increases, we need to scale our data ingestion infrastructure accordingly to handle workloads more efficiently. GPUs, which are used for training infrastructure, tend to double in [...] Read More... The post Scaling data ingestion for machine learning training at Meta appeared first on Engineering at Meta.
http://dlvr.it/SWPMWQ

Komentar

Posting Komentar

Postingan populer dari blog ini

Inside Meta’s first smart glasses

Gambar
What’s new: Meta is sharing the inside story of how it developed the Ray-Ban Stories smart glasses. Why it matters: Creating Ray-Ban Stories meant Meta’s engineers had to take on new challenges to build smart glasses that married complex engineering dynamics. How do you make something that features cameras, microphones, audio, and touch controls, all [...] Read More... The post Inside Meta’s first smart glasses appeared first on Engineering at Meta. http://dlvr.it/SjWknx
Baca selengkapnya

Simulator-based reinforcement learning for data center cooling optimization

Gambar
We’re sharing more about the role that reinforcement learning plays in helping us optimize our data centers’ environmental controls. Our reinforcement learning-based approach has helped us reduce energy consumption and water usage across various weather conditions.   Meta is revamping its new data center design to optimize for artificial intelligence and the same methodology will be [...] Read More... The post Simulator-based reinforcement learning for data center cooling optimization appeared first on Engineering at Meta. http://dlvr.it/TD4BJ2
Baca selengkapnya

Improving machine learning iteration speed with faster application build and packaging

Gambar
Slow build times and inefficiencies in packaging and distributing execution files were costing our ML/AI engineers a significant amount of time while working on our training stack. By addressing these issues head-on, we were able to reduce this overhead by double-digit percentages.  In the fast-paced world of AI/ML development, it’s crucial to ensure that our [...] Read More... The post Improving machine learning iteration speed with faster application build and packaging appeared first on Engineering at Meta. http://dlvr.it/T22Qcp
Baca selengkapnya