When we are using the RemoteDB, compression is enabled. However, when looking at the data that is sent over the wire, we see that no compression is done from server to client. We looked at the server side code and see that a SameText is used that checks for 'deflate' in 'RemoteDB.Server.Module.pas' on line 517. When we look at the header value, we see that the value is 'deflate, gzip'.
When we changed this to be a 'Contains' instead of a 'SameText', the compression was performed and 'deflate' is present in the response content-encoding header. When reading the headers in the client, no content-encoding header is present. This results in an error on the client side. In short, compression does not work.
This lack of compression causes our customers to experience performance issues, so this has quite a high priority for us.
I wonder why is that? RemoteDB client requests should always come from a TRemoteDBDatabase component - it doesn't make sense to have other clients invoking the server.
And the client clearly adds only the deflate value to the accept-encoding header, as we can see in line 1171 of RemoteDB.Client.Database unit:
if FDB.Compress then
Req.Headers.SetValue('accept-encoding', 'deflate');
So I fail to see why your accept-encoding header is coming as deflate, gzip?
I have also been looking into this issue. You are right that the header is set differently in your code and for us it is also a surprise why it is received differently in the RemoteDB Server components. This should all be handled in the RemoteDB components and therefore we ask you they question why this is
Here you see the traffic captured by Wireshark which clearly indicates the header we are talking about.
To move on with this issue, we also added the compression middleware server side. To get this working we have to change some code in Sparkle.Middleware.Compress to make sure the response is compressed.
function TCompressMiddleware.IsCompressible(const MimeType: string): Boolean;
var
LowerMime: string;
begin
Result := True;
// LowerMime := LowerCase(MimeType);
// Result := StartsStr('text/', LowerMime) or EndsStr('+json', LowerMime) or EndsStr('+xml', LowerMime)
// or (LowerMime = 'application/javascript')
// or (LowerMime = 'application/json')
// or (LowerMime = 'application/xml')
// or (LowerMime = 'application/wasm');
end;
This seems to work fine if you look in the Wireshark response.
I don't know. Is this Wireshark detecting traffic in the client machine, or server machine? Do you have some kind of proxy, HTTP modifier anything that might be intercepting the call? Is the client just a regular, simple RemoteDB application?
We will.
The compression middleware is a generic one as you saw. It only compress data that is "compressible" (mostly text files and text formats like JSON, HTML, XML...). It can't keep compressing everything like JPG, PNG or any binary format it doesn't understand in advance. That's why RemoteDB server takes care of compressing its own response.
We first need to understand why the header is going with "deflate, gzip". That is wrong and is not being set by RemoteDB client. It should be working out of the box.
If you run the existing, default, RemoteDB demo that is provided in RemoteDB distribution, does the same thing happen in Wireshark?
If you run the existing, default, RemoteDB demo that is provided in RemoteDB distribution, does the same thing happen in Wireshark?
Of course, the same thing happens with the default RemoteDB demo (both running on the same machine, so Wireshark also runs on that machine). I have just tested it. Attached is the Wireshark output where you can clearly see that the header adds gzip and the output is not compressed.
Hi Laurens, I see that this is being added by WinHttp library itself.
The solution is just improve RemoteDB code indeed. In unit RemoteDB.Server.Module, function TCustomRemoteDBModule.ProvideResponse, you can change the very first line of the function to this:
This should solve the issue. Please confirm it's working for you there.
We will include such change in our official source code and next updates will already include it.
Hi Wagner, we already tried that (before creating this thread ;-)) and this gives issues in the client, see below. This was of course tested with the default TMS example code to exclude other influences. It seems that the whole compression mechanism is just broken.
First chance exception at $758598B2. Exception class EWinHttpClientException with message
'Could not perform WinHttp operation. Error: (-2147467260) Operation aborted'.
Process RemoteDBClient.exe (472)
Sorry, you're right. This was not as simple as it looked at first sight.
But we fixed the issue and in next RemoteDB release it will be resolved. Both client and server compilation will be required.
I have some clients using RemoteDB technology. If I need to send a new EXE (client) release I will need to send a new RemoteDB server because an error will happen or simply will continue working with no compression?
@Fernandez_Diego,
The changes made won't break existing clients. If you are using an "old" RemoteDB Server and deploy a new client with the existing code, it will keep working.
