This might be decoupled from productivity anxiety, but my theory is that there are different types of "fun".

There's the "simple carbohydrate" fun of, say, an action game, which is immediate, easy, but low-nutrition and, enjoyed overlong, makes me feel empty inside.

Then there's the "complex carbohydrate" fun of learning something new or working on a project, which is more fulfilling and interesting but also work -- mentally taxing -- and, enjoyed overlong, burns me out.

My brain seems to demand a balanced diet of both.