Of course, where to start debugging a library which is officially encrypted? Well, in the distant past some people already wrote a decrypt program for the pci.library (the background story of the 'why' is also interesting, but maybe for some other time).
After a bit of digging the culprit was found (and accidentally, also the spider recognition code. Again, another story).
The Sonnet is a PCI card based on the MPC107 chip set, also known as the TSi107 chip set (at least mine is). This makes the Sonnet actually a non-transparent PCI-bridge. This means that the Amiga cannot directly communicate with the resources on the card.
To achieve this, these kind of PCI-bridges have address translation units (ATUs) on board so you can address the card through PCI memory. This is different from 'normal' PCI cards.
The Mediator treats the Sonnet as a normal PCI card. It writes all ones to the Base Address Registers (BARs) of all the PCI cards it can find. The PCI card responds with setting up a mask in the BAR to indicate how big the needed address space is. This will be allocated in PCI memory by the Mediator.
However, BAR0 of the Sonnet is actually an ATU. Writing all ones to this register does not result in what the Mediator expects. The value returned causes an infinite loop in the Mediator as it cannot allocate the appropriate size of memory it thinks it needs by reading back the ATU.
We insert some code to skip BAR0 if the PCI Vendor/ID matches the Sonnet and we add the approriate info to the vendors.txt file... et voila:
The Mediator reads all the values in one go and stores them inside a big array. As you can see, I inserted a dummy value for Memspace0 (BAR0) in this array.
Now we have the card detected without crashing, we are going to try to get it to execute code. The next task is to read up on how to program the MPC107 chip set. That's for another day.