New work by Tsinghua Zhu Jun team: Train Transformer with 4-bit integers to accelerate the arrival of AGI!
Transferred from XinzhiyuanEditor Aeneas RunQuantifying activation, weight, and gradient to 4 bits is expected to accelerate neural network training.However, existing 4-bit training methods require a custom number format, which modern hardware does not support...
Tech >>
2023-07-04 06:39:48