Me, too, but I will also need a 2 x B200 infiniband in my stocking

September 13, 2024

Me, too, but I will also need a 2 x B200 infiniband in my stocking. (I know this is a TPU fan zone, but it's obviously too much to ask Santa for a couple of v5p racks and I don't think they'd fit down my chimney anyway.) https://t.co/2stV67KU0K

xjdr@_xjdr

Dear Meta,
If you are listening, i'd really appreciate a L4 1T dense base model with a 1:8 global:local_sliding attention at bfloat16 and a similar sized RM trained on the same distribution for Christmas. kthxbye.
xoxo,
// xjdr