INS essentially was expensive and AFAIK once GPS became available started to drop off in use outside of military. And with GPS availability coinciding with switching to more modern integrated Flight Management System/Computer, a lot of planes simply don't have INS installed.
Your words are near to truth. Before GPS from nearly 1950s used LORAN navigation system, with similar to GPS principles, but used long waves and have relatively low precision - about kilometer at best.
Before LORAN, used radio beacon navigation and star navigation (from Newton time), and good human navigator could achieve about 50km precision.
You could easy see signs of star navigation on good preserved old planes - they all have some sort of fully glass dome, or blister, to provide good near semi-sphere view.
And sure, all those before-GPS era planes have separate navigator job position, sometimes shared with mechanic.
You could ask, how planes could fly with 50km precision? Answer is easy - at all plane routes built ground structures easy seen from air and last mile navigation become essentially visual flight, nothing more, nothing less.
On some places ground navigation structures preserved now, for examples:
LORAN was mostly long range over sea, on the ground we had NDB, DME, VOR, etc all ultimately linking into "airways" for higher altitude operations where earth might be not visible due to cloud cover for example
The INS unit is separate and often has its own set of gyros and has to be connected as separate input to FMS or other navigational computers, same as connecting GPSes or other radio nav components.
For example the current model of popular Universal Avionics UNS1 series of NCU (navigational computer part of FMS) come with built-in augmented GPS receiversz but do not mention INS functionality at all even in extended models. Don't have access to manuals at the moment, but I'd expect to see INS as optional to connect over one of the external connectors on the MCU, as it was on the older models without integrated GPS
I had assumed that the ADIRU’s inertial reference data from its gyroscopes and accelerometers would feed into the FMS to provide INS capabilities in case GNSS was unavailable.
ADIRU in its minimal common form (some vendors might make a beefier one, who knows) provides the equivalent of old pneumatic "air data central" + artificial horizon and turn indicator gyros.
GNSS becoming "reasonably easy" to add meant that there was way less push to integrate INS after early use in transoceanic flights
Many small planes don't have INS in typical meaning, but their pilot is INS computer, calculated approximate nav from air data (air speed + weather data + compass or radio compass).
