Common modules for RecSys, such as models and public datasets (Criteo & Movielens).Pipelining to overlap dataloading device transfer (copy to GPU), inter-device communications (input_dist), and computation (forward, backward) for increased performance.A planner which can automatically generate optimized sharding plans for models.A sharder which can partition embedding tables with a variety of different strategies including data-parallel, table-wise, row-wise, table-wise-row-wise, and column-wise sharding.Optimized RecSys kernels powered by FBGEMM, including support for sparse and quantized operations.Modeling primitives, such as embedding bags and jagged tensors, that enable easy authoring of large, performant multi-device/multi-node models using hybrid data-parallelism and model-parallelism.TorchRec was used to train a 1.25 trillion parameter model, pushed to production in January 2022. This new library provides common sparsity and parallelism primitives, enabling researchers to build state-of-the-art personalization models and deploy them in production. To recap, TorchRec is a PyTorch domain library for Recommendation Systems. We announced TorchRec a few weeks ago and we are excited to release the beta version today. TorchVision - Added 4 new model families and 14 new classification datasets such as CLEVR, GTSRB, FER2013.TorchText - Added beta support for RoBERTa and XLM-R models, byte-level BPE tokenizer, and text datasets backed by TorchData.
PROJECT CARS 3 1.11 FULL
TorchAudio - Added Enformer- and RNN-T-based models and recipes to support the full development lifecycle of a streaming ASR model.TorchRec, a PyTorch domain library for Recommendation Systems, is available in beta.These updates demonstrate our focus on developing common and extensible APIs across all domains to make it easier for our community to build ecosystem projects on PyTorch.
We are introducing the beta release of TorchRec and a number of improvements to the current PyTorch domain libraries, alongside the PyTorch 1.11 release.
0 kommentar(er)
