We could have NEG instead of SUB, but with the constant folding it's a wash. DIV... | Hacker News

Hacker News new | past | comments | ask | show | jobs | submit

login

georgehotz on Nov 4, 2022 | parent | context | favorite | on: Tinygrad: A simple and powerful neural network fra...

We could have NEG instead of SUB, but with the constant folding it's a wash. DIV is already an HLOP with reciprocal (used to use POW, but that was slower. And what would you implement RELU in terms of?

WithinReason on Nov 4, 2022 [–]

max(0,x)

georgehotz on Nov 4, 2022 | [–]

That's a ReduceOp right now, more annoying to reason about than a UnaryOp. But in the limit, yea. Or add an elementwise BinaryOp for max.

Submit a PR if you can improve something!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact