Panamax specifications (320 m length, 32 m width) have been in effect since the opening of the canal in 1914. Some ships were already larger than that back then. The Titanic was about that size.
The ship that knocked down the bridge could weigh ("DWT") up to 115 000 tons.
Some ships from the 1960s and 1970s:
Universe Ireland: Launched in 1968, it had a DWT of over 300 000 tons.
Esso Atlantic (1977): DWT of over 500 000 tons.
But these were tankers. You're right that container ships were not as big back then.
One of the larger container ships when the bridge was built in 1977 was MV Hamburg Express. It was 280 meters long, 32 meters wide, and had a DWT of 50 000 tons.
Do you really think a 280 meter long ship weighing 50 000 tons would not have knocked down the bridge in exactly the same way?
Doesn't sound plausible to me. Very large Panamax ships were used in the 1970's when the bridge was built.