A research team from Google and the University of California, Berkeley calculates the energy use and carbon footprint of large-scale models T5, Meena, GShard, Switch Transformer and GPT-3, and identifies methods and publication guidelines that could help reduce their CO2e footprint.

Here is a quick read: Google and UC Berkeley Propose Green Strategies for Large Neural Network Training.

The paper Carbon Emissions and Large Neural Network Training is on arXiv.

Source link