I have an update:
I tried the Windows 98SE installation with lowered bus of 33MHz. Also switched back to the original 2800 Ami Bios. Then set the settings to Default, which seems to be ultra slow, I think cache write 2-2-2-2 and cache read 3-2-2-2. Still crashing at the beginning of the Win98SE installation. Omg. I replaced then the RAM with other modules, with 2 times 16MB or 2 times 32MB (but different modules) things started to work.
The Windows 98 installation finished. But it really needed those slow settings as it seems.
Today I tried with 40MHz again. Short summary: no dice!
Even with slowest (!) settings and L2 cache completely disabled => Not working.
Then I replaced the VLB graphics card with an ISA graphics card (so only one VL card in the system) => Not working.
I lowered the clock to 120MHz, still the same.
I checked the voltage on the voltage regulator when the board was running: 3.52 volts, that is good.
Last thing was changing both JP20 and 21 to 2-3, which is recommended for 40MHz (1 WS and Bus speed > 33) => Still the same.
The only explanation I have, that the board really does not like the VLB I/O controller at 40MHz.
I was thinking to try out 50MHz, but it makes no sense, when I use already the slowest timing settings at 40 and it still does not work. So did not make that test.
But as I wrote in the beginning, there are huge problems also at 33MHz. Horrific.
EDIT: I also switched also back to the Award Bios, using the original AMI Bios did not improve something.