Conversation

Hillel

@hillelogram

·

Jul 25, 2021

Is memory/performance optimization a big thing in the ML world? Seems like it would be: these models take a long time to train and GPUs are expensive, so there should be a lot of money in make it faster/smaller. Or is this not a thing for whatever reason?

13

1

44

Nicholas Schiefer

@nschiefer

Replying to

@hillelogram

I think we put a lot of optimization into deployed models, even down to the hardware level. I think a lot of focus (at least on hardware design) is on inference rather than training; apparently inference cost is often ~10x training cost over the lifetime of the model.

6:42 PM · Jul 25, 2021·Twitter Web App