A bank is not a single entity, it's made of people converging after reaching consensus
There are a lot of banks, with different interests that usually compete with each other and work under regulations that have been discussed and approved after reaching consensus
There are several different central banks, that work together under regulations discussed and approved because they reached consensus
In the end you do trust the code, a computer running the code or some currency exchange.