I decided to build the same code for the Windows now, and launch the server locally. In this case all the umlauts are shown correctly. At least some progress, but I'm still not sure what can it be, maybe there is some bug or just a small detail for the Ubuntu/Linux build? Here's the Google VM's OS info:
Distributor ID: Ubuntu
Description: Ubuntu 22.04.5 LTS
Release: 22.04
Codename: jammy