mrq/torchscale
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Update README.md

Browse Source
This commit is contained in:
Shuming Ma 2023-08-11 11:26:19 +08:00 committed by GitHub
parent 8b07f19ba0
commit 70e047a53b
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@ -155,7 +155,7 @@ We plan to provide more examples regarding different tasks (e.g. vision pretrain
### Stability Evaluation
<p align="center">
<img src="https://publicmodel.blob.core.windows.net/torchscale/pic/convergence.png" width="800"/>
<img src="https://publicmodel.blob.core.windows.net/torchscale/pic/convergence.png?sv=2020-04-08&st=2023-08-11T03%3A09%3A09Z&se=2053-08-12T03%3A09%3A00Z&sr=c&sp=rl&sig=3b6nDda%2Fu0vD6E%2BhoTO%2BHfNSnSlUfgvXFV%2FCNKquWjE%3D" width="800"/>
</p>
The training curve is smooth by using TorchScale, while the baseline Transformer cannot converge.
@ -163,7 +163,7 @@ The training curve is smooth by using TorchScale, while the baseline Transformer
### Scaling-up Experiments
<p align="center">
<img src="https://publicmodel.blob.core.windows.net/torchscale/pic/scaling_curve.png" width="800"/>
<img src="https://publicmodel.blob.core.windows.net/torchscale/pic/scaling_curve.png?sv=2020-04-08&st=2023-08-11T03%3A09%3A09Z&se=2053-08-12T03%3A09%3A00Z&sr=c&sp=rl&sig=3b6nDda%2Fu0vD6E%2BhoTO%2BHfNSnSlUfgvXFV%2FCNKquWjE%3D" width="800"/>
</p>
TorchScale supports arbitrary depths and widths, successfully scaling-up the models without pain.