Composable data management at Meta

In recent years, Meta’s data management systems have evolved into a composable architecture that creates interoperability, promotes reusability, and improves engineering efficiency.  We’re sharing how we’ve achieved this, in part, by leveraging Velox, Meta’s open source execution engine, as well as work ahead as we continue to rethink our data management systems.  Data is at [...]


Read More...


The post Composable data management at Meta appeared first on Engineering at Meta.


http://dlvr.it/T7Fm52

Komentar

Posting Komentar

Postingan populer dari blog ini

How Meta enforces purpose limitation via Privacy Aware Infrastructure at scale

Gambar
At Meta, we’ve been diligently working to incorporate privacy into different systems of our software stack over the past few years. Today, we’re excited to share some cutting-edge technologies that are part of our Privacy Aware Infrastructure (PAI) initiative. These innovations mark a major milestone in our ongoing commitment to honoring user privacy.  PAI offers [...] Read More... The post How Meta enforces purpose limitation via Privacy Aware Infrastructure at scale appeared first on Engineering at Meta. http://dlvr.it/TCRbJ9
Baca selengkapnya

Inside Meta’s first smart glasses

Gambar
What’s new: Meta is sharing the inside story of how it developed the Ray-Ban Stories smart glasses. Why it matters: Creating Ray-Ban Stories meant Meta’s engineers had to take on new challenges to build smart glasses that married complex engineering dynamics. How do you make something that features cameras, microphones, audio, and touch controls, all [...] Read More... The post Inside Meta’s first smart glasses appeared first on Engineering at Meta. http://dlvr.it/SjWknx
Baca selengkapnya

Fully Sharded Data Parallel: faster AI training with fewer GPUs

Gambar
Training AI models at a large scale isn’t easy. Aside from the need for large amounts of computing power and resources, there is also considerable engineering complexity behind training very large models. At Facebook AI Research (FAIR) Engineering, we have been working on building tools and infrastructure to make training large AI models easier. Our [...] Read More... The post Fully Sharded Data Parallel: faster AI training with fewer GPUs appeared first on Facebook Engineering. http://dlvr.it/S3ndNR
Baca selengkapnya