Skip to content

Commit 5c67453

Browse files
pancettapancetta
committed
deploy: 1921f3f 
1 parent 1e794ff commit 5c67453

1 file changed

Lines changed: 171 additions & 0 deletions

File tree

‎references/index.html‎

Lines changed: 171 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -374,6 +374,82 @@ <h2 id="year2026" class="year">2026 <span class="count-stat"></span></h2>
374374
</div>
375375

376376

377+
</div>
378+
</li>
379+
<li><div class="bibtex-entry-container">
380+
<div>
381+
<div class="col-md-3 col-sm-12 bibtex-ref-meta hidden">
382+
<div class="row">
383+
<div class="col-md-12 ref-label label label-default">
384+
GaraiEtAl2026
385+
</div>
386+
</div>
387+
</div>
388+
389+
<div class="bibtex-ref-entry">
390+
<span id="GaraiEtAl2026">G. Garai and B. C. Mandal, “Domain decomposition-based space-time preconditioner for the optimal control problem with constraints imposed by the Cahn-Hilliard equation,” <i>International Journal of Computer Mathematics</i>, pp. 1–23, Jan. 2026, doi: 10.1080/00207160.2026.2614077. [Online]. Available at: <a href="http://dx.doi.org/10.1080/00207160.2026.2614077" target="_blank">http://dx.doi.org/10.1080/00207160.2026.2614077</a></span>
391+
</div>
392+
393+
394+
<div class="row mt-4">
395+
<div class="btn-group btn-group-xs btn-group-justified" role="group">
396+
<div class="btn-group btn-group-xs" role="group">
397+
398+
<button class="button" data-toggle="modal" data-target="#modalGaraiEtAl2026Bibtex" aria-expanded="false"
399+
aria-controls="modalGaraiEtAl2026Bibtex">
400+
BibTeX
401+
</button>
402+
403+
</div>
404+
<div class="btn-group btn-group-xs" role="group">
405+
406+
<button class="button" disabled="disabled" data-toggle="modal" data-target="#modalGaraiEtAl2026Abstract"
407+
aria-expanded="false" aria-controls="modalGaraiEtAl2026Abstract">
408+
Abstract
409+
</button>
410+
411+
</div>
412+
</div>
413+
</div>
414+
</div>
415+
416+
<div class="modal" id="modalGaraiEtAl2026Bibtex" tabindex="-1" role="dialog"
417+
aria-labelledby="modalGaraiEtAl2026BibtexTitle" aria-hidden="true">
418+
<div class="modal-background">
419+
<div class="modal-card">
420+
<div class="modal-content">
421+
<header class="modal-card-head">
422+
<div class="modal-card-head-content">
423+
<div>
424+
<p class="modal-card-title" id="modalGaraiEtAl2026BibtexTitle">
425+
BibTeX entry <code>GaraiEtAl2026</code>
426+
</p>
427+
</div>
428+
<button type="button" class="delete" data-dismiss="modal" aria-label="Close"><span
429+
aria-hidden="true">&times;</span></button>
430+
</div>
431+
</header>
432+
<section class="modal-card-body">
433+
<pre class="abstract">@article{GaraiEtAl2026,
434+
author = {Garai, Gobinda and Mandal, Bankim C.},
435+
doi = {10.1080/00207160.2026.2614077},
436+
issn = {1029-0265},
437+
journal = {International Journal of Computer Mathematics},
438+
month = jan,
439+
pages = {1–23},
440+
publisher = {Informa UK Limited},
441+
title = {Domain decomposition-based space-time preconditioner for the optimal control problem with constraints imposed by the Cahn-Hilliard equation},
442+
url = {http://dx.doi.org/10.1080/00207160.2026.2614077},
443+
year = {2026}
444+
}
445+
</pre>
446+
</section>
447+
</div>
448+
</div>
449+
</div>
450+
</div>
451+
452+
377453
</div>
378454
</li>
379455
<li><div class="bibtex-entry-container">
@@ -451,6 +527,101 @@ <h2 id="year2026" class="year">2026 <span class="count-stat"></span></h2>
451527
</div>
452528

453529

530+
</div>
531+
</li>
532+
<li><div class="bibtex-entry-container">
533+
<div>
534+
<div class="col-md-3 col-sm-12 bibtex-ref-meta hidden">
535+
<div class="row">
536+
<div class="col-md-12 ref-label label label-default">
537+
JiangEtAl2026
538+
</div>
539+
</div>
540+
</div>
541+
542+
<div class="bibtex-ref-entry">
543+
<span id="JiangEtAl2026">S. Jiang, M. Salvado, E. C. Cyr, A. Kopaničáková, R. Krause, and J. B. Schroder, “Layer-Parallel Training for Transformers,” arXiv:2601.09026v1 [cs.LG], 2026 [Online]. Available at: <a href="https://arxiv.org/abs/2601.09026v1" target="_blank">https://arxiv.org/abs/2601.09026v1</a></span>
544+
</div>
545+
546+
547+
<div class="row mt-4">
548+
<div class="btn-group btn-group-xs btn-group-justified" role="group">
549+
<div class="btn-group btn-group-xs" role="group">
550+
551+
<button class="button" data-toggle="modal" data-target="#modalJiangEtAl2026Bibtex" aria-expanded="false"
552+
aria-controls="modalJiangEtAl2026Bibtex">
553+
BibTeX
554+
</button>
555+
556+
</div>
557+
<div class="btn-group btn-group-xs" role="group">
558+
559+
<button class="button" data-toggle="modal" data-target="#modalJiangEtAl2026Abstract" aria-expanded="false"
560+
aria-controls="modalJiangEtAl2026Abstract">
561+
Abstract
562+
</button>
563+
564+
</div>
565+
</div>
566+
</div>
567+
</div>
568+
569+
<div class="modal" id="modalJiangEtAl2026Bibtex" tabindex="-1" role="dialog"
570+
aria-labelledby="modalJiangEtAl2026BibtexTitle" aria-hidden="true">
571+
<div class="modal-background">
572+
<div class="modal-card">
573+
<div class="modal-content">
574+
<header class="modal-card-head">
575+
<div class="modal-card-head-content">
576+
<div>
577+
<p class="modal-card-title" id="modalJiangEtAl2026BibtexTitle">
578+
BibTeX entry <code>JiangEtAl2026</code>
579+
</p>
580+
</div>
581+
<button type="button" class="delete" data-dismiss="modal" aria-label="Close"><span
582+
aria-hidden="true">&times;</span></button>
583+
</div>
584+
</header>
585+
<section class="modal-card-body">
586+
<pre class="abstract">@unpublished{JiangEtAl2026,
587+
author = {Jiang, Shuai and Salvado, Marc and Cyr, Eric C. and Kopaničáková, Alena and Krause, Rolf and Schroder, Jacob B.},
588+
howpublished = {arXiv:2601.09026v1 [cs.LG]},
589+
title = {Layer-Parallel Training for Transformers},
590+
url = {https://arxiv.org/abs/2601.09026v1},
591+
year = {2026}
592+
}
593+
</pre>
594+
</section>
595+
</div>
596+
</div>
597+
</div>
598+
</div>
599+
600+
601+
<div class="modal" id="modalJiangEtAl2026Abstract" tabindex="-1" role="dialog"
602+
aria-labelledby="modalJiangEtAl2026AbstractTitle" aria-hidden="true">
603+
<div class="modal-background">
604+
<div class="modal-card">
605+
<div class="modal-content">
606+
<header class="modal-card-head">
607+
<div class="modal-card-head-content">
608+
<div>
609+
<p class="modal-card-title" id="modalJiangEtAl2026AbstractTitle">
610+
Abstract for BibTeX entry <code>JiangEtAl2026</code>
611+
</p>
612+
</div>
613+
<button type="button" class="delete" data-dismiss="modal" aria-label="Close"><span
614+
aria-hidden="true">&times;</span></button>
615+
</div>
616+
</header>
617+
<section class="modal-card-body">
618+
We present a new training methodology for transformers using a multilevel, layer-parallel approach. Through a neural ODE formulation of transformers, our application of a multilevel parallel-in-time algorithm for the forward and backpropagation phases of training achieves parallel acceleration over the layer dimension. This dramatically enhances parallel scalability as the network depth increases, which is particularly useful for increasingly large foundational models. However, achieving this introduces errors that cause systematic bias in the gradients, which in turn reduces convergence when closer to the minima. We develop an algorithm to detect this critical transition and either switch to serial training or systematically increase the accuracy of layer-parallel training. Results, including BERT, GPT2, ViT, and machine translation architectures, demonstrate parallel-acceleration as well as accuracy commensurate with serial pre-training while fine-tuning is unaffected.
619+
</section>
620+
</div>
621+
</div>
622+
</div>
623+
</div>
624+
454625
</div>
455626
</li>
456627
<li><div class="bibtex-entry-container">

0 commit comments

Comments
 (0)