Gå offline med appen Player FM !
BAM! Just Like That: Simple and Efficient Parameter Upcycling for Mixture of Experts
Manage episode 434502671 series 3524393
BAM enhances Mixture of Experts by fully utilizing dense model parameters, improving efficiency and performance in large language models, surpassing baselines in perplexity and downstream tasks.
https://arxiv.org/abs//2408.08274
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
--- Support this podcast: https://podcasters.spotify.com/pod/show/arxiv-papers/support
1627 episoder
Manage episode 434502671 series 3524393
BAM enhances Mixture of Experts by fully utilizing dense model parameters, improving efficiency and performance in large language models, surpassing baselines in perplexity and downstream tasks.
https://arxiv.org/abs//2408.08274
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
--- Support this podcast: https://podcasters.spotify.com/pod/show/arxiv-papers/support
1627 episoder
Alla avsnitt
×Välkommen till Player FM
Player FM scannar webben för högkvalitativa podcasts för dig att njuta av nu direkt. Den är den bästa podcast-appen och den fungerar med Android, Iphone och webben. Bli medlem för att synka prenumerationer mellan enheter.