Compress content on Client Request

Zelaya_Omar · September 26, 2021, 5:31pm

Hi,

I want to compress the client request content. I'm using the following code without success(Server throws a Json reader exception).
Any hints?

Thanks in Advance,
Omar Zelaya

XDataClient.HttpClient.OnSendingRequest :=
procedure(Req: THttpRequest)
var
OutStream : TBytesStream;
LZip: TZCompressionStream;
begin
Req.Timeout := READTIMEOUT;
Req.Headers.SetValue('Authorization','Bearer ' + FvToken);
if Req.ContentLength > 1000 then
begin
try
try
OutStream := TBytesStream.Create;
LZip := TZCompressionStream.Create(clDefault, OutStream);
LZip.Write(Req.ContentBuffer^, Req.ContentLength);
LZip.Free;
OutStream.Position := 0;
Req.SetContent(OutStream.Bytes);
Req.Headers.SetValue('content-encoding','gzip');
except
end
finally
FreeAndNil(OutStream);
end
end;
end;

wlandgraf · September 27, 2021, 5:22pm

Which JSON reader error you receive? Have you checked if your compressed content is a valid gzip content? Are you sure you are not generating a deflate content? First you must isolate the problem by making sure your content is correct.

Zelaya_Omar · September 27, 2021, 6:33pm

Hi,

I'm getting the following exception:
imagen

I'm using WireShark to chek data transfer betwen my client and the server, and it looks like Wireshark decompress ok the gzip stream.

Any hint?

Thanks in advance,
Omar Zelaya

Zelaya_Omar · September 27, 2021, 6:35pm

wlandgraf · September 27, 2021, 11:58pm

I don't know about Wireshark, maybe it's detecting the encoding format? Have you tried setting content-encoding to deflate? What is the JSON you are trying to send to the server? What is the JSON decoded by Wireshark? Please copy and paste the real text, not images, thank you.

Zelaya_Omar · September 28, 2021, 9:23pm

Hi Wagner,

WireShark is a network protocol analyzer. I’m calling a service operation using a TXDataClient and I’m using the OnSendingRequest method to compress the request content.

Where in Sparkle/XData source code can I set a breakpoint to check if the content of the request is decompressed ok.

This is the data captured by wireshak that is sent from the client to the server when calling the service method.

This is the data captured when triying to compress:

POST /INTEC/xdata/ICS/ICSService/LiquidaDeclaracion/ HTTP/1.1

Connection: Keep-Alive

Content-Encoding: gzip

Accept-Encoding: gzip, deflate

Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJ...

User-Agent: TMS Http/1.0

xdata-expandlevel: 3

Content-Length: 8192

Host: 127.0.0.1:81

This is the decompressed content(decompressed by Wireshark for display).

Thanks in advance,

Omar Zelaya

wlandgraf · September 29, 2021, 3:47pm

What I mean is that I don't know if Wireshark detects only based on content-encoding or if it tries to detect the content encoding automatically.
Again, have you tried changing the content-encoding to deflate and see what happen?

Zelaya_Omar · September 29, 2021, 4:12pm

Hi,

I get the same exception when using deflate. It is possible to set a breakpoint in source code of xdata/Sparke to check if content has been decompressed ok?

Thanks in advance,

Omar E. Zelaya

wlandgraf · September 29, 2021, 9:37pm

Actually, the server doesn't automatically decompress requests sent with a deflate or gzip content-encoding. You have to do it yourself. You can easily do that by adding a generic middleware to the XData server and adding a code like this:

procedure TServerContainer.XDataServerGenericRequest(Sender: TObject;
  Context: THttpServerContext; Next: THttpServerProc);
const
  BufSize = 65536;
var
  TempStream: TStream;
  Buffer: array[0..BufSize - 1] of Byte;
  BytesRead: Integer;
begin
  if Context.Request.Headers.Get('content-encoding') = 'deflate' then
  begin
    TempStream := TMemoryStream.Create;
    try
      // copy stream
      repeat
        BytesRead := Context.Request.ContentStream.Read(Buffer[0], BufSize);
        TempStream.Write(Buffer[0], BytesRead);
      until BytesRead <> BufSize;
      TempStream.Position := 0;
      Context.Request.ContentStream.Free;
      Context.Request.ContentStream := TZDecompressionStream.Create(TempStream, 15, True);
      TempStream := nil;
      Context.Request.Headers.SetValue('content-length', '-1');
    except
      TempStream.Free;
      raise;
    end;
  end;
  Next(Context);
end;