Perhaps a "dumb" question, but how do I process the access token - obtained from a Sphinx server - in an XData server?
I understand it with the JWT access token (in demo) but that's merely because the XData server generates the token and can compare. XData has no knowledge whatsoever of a token that was generated by Sphinx, so how can it be validated?
Is there any good demo code, as I cannot seem to find it on my installation.
It should have knowledge. You have two options:
If you use symmetric keys (sign the token with a secret), then the Sphinx server and XData server should share the secret. So Sphinx signs the JWT with the secret, and XData uses the same secret to validate.
You can use asymmetric keys. The Sphinx server signs the token with its private key, and then you use the respective public key from XData to validate the token.
We have demos for both situations. For 1, it's the regular Sphinx simple demo. Also, in XData demos you have JwtAuthDemo which shows both options in action. Finally the BIZ-Boilerplate project in GitHub also shows public/private key usage.
The documentation also mentions both approaches:
Hi Wagner,
Thanks for the info but the more I read, the more I seem to get confused. My main "struggle" at this point where to implement the signing:
In the XData server it appears that I must add the JWT middleware in order to use the GetSecretEx event for validating the signature.
Looking at the Sphinx Simple Demo, I see the TSphinxConfig.OnConfigureToken and the TSphinxConfig.OnGetSigningData events implemented. My understanding is that the former is used to "build" the token (payload etc.) and the latter to return the secret (equivalent of JWT GetSecretEx?)
Is my understanding of the above correct? If not, can you perhaps describe in a few simple steps the required workflow (for XData, Sphinx and a VCL client). Just some simple steps. I'll figure out the code myself from the many examples.
Thanks in advance!
PS: Due to circumstances it may take a while for me to respond further.
Yes, your understanding is correct, you got it right, that's all to it.
For clarity: does that also apply to point 1? I.e. I MUST include JWT middleware?
The JWT middleware is the one that decode requests sent with JWT and set the Request.User property based on what it found. It also rejects requests with invalid JWT and, if you forbid anonymous requests, it also rejects requests without JWT.
So, you do not must add a JWT to your server - it's up to you, of course. But if you don't, your server will be open to anyone.
Clear! I'll do some further trials. Thanks so far...
Another clarification (something that's not directly obvious to me); in the example code on
I see:
// Change KeyId if you recreate the private key
RSAKeyId = 'D00CD046-FEDA-4120-9258-391371649A32';
Is this an arbitrarily chosen value (e.g. a GUID) or is it based on/derived from something (e.g. the generated key files)?
I suppose the JWT secret is an arbitrary value (chosen by developer)?
It's an arbitrary value, just to uniquely identify the public key. So you know if a request with that specific ID comes, you should use the respective public key to validate the JWT.
It's probably me but I just don't succeed in getting this thing working. I have attached a project I have created so far (partly "stolen" from examples and partly "my own touch") and but it simply doesn't work. When attempting variations to this code, I keep getting different exceptions.
The idea is to run WellDoc_DataServer.exe (XData Server), SD_Sphinx.exe and then WellDoc_XData.exe (an XData client app). In WellDoc_XData click to Login button to test the authorization etc etc.
In SD_Sphinx.exe (Form.MainSphinx) I have added a TSparkleJwtMiddleware to the SphinxServer1 and implemented the following code:
const
SD_JWTSECRET = 'Super_Secret_0123456789_0123456789';
SD_KEYID = '2C8792EF-CB90-49AC-8E5C-9E2CCA7036BB';
SD_PRIVATE_KEYFILE = 'C:\Development\Projects\SD_Sphinx\SD-Sphinx_RSA256-private.key';
procedure TfrmMain.SphinxConfig1GetSigningData(Sender: TObject; Args: TGetSigningDataArgs);
begin
// This event is triggered when a JWT needs to be signed ??
Args.Data.Algorithm := 'RS256';
Args.Data.KeyId := SD_KEYID;
Args.Data.Key := TFile.ReadAllBytes(SD_PRIVATE_KEYFILE);
end;
procedure TfrmMain.SphinxServer1JWTGetSecretEx(Sender: TObject; const JWT: TJWT; Context: THttpServerContext;
var Secret: TBytes);
var
JWK: TJWK;
SigningKey: TArray<Byte>;
SigningAlgorithm: TJOSEAlgorithmId;
begin
SigningKey := TFile.ReadAllBytes(SD_PRIVATE_KEYFILE); // PRIVATE KEY !!
SigningAlgorithm := TJOSEAlgorithmId.RS256;
JWT.Header.KeyId := SD_KEYID;
JWK := TJWK.Create(SigningKey);
try
Secret := TJOSE.SerializeCompact(JWK, SigningAlgorithm, JWT, false);
finally
JWK.Free;
end;
end;
In the XDataServer (WellDoc_DataServer.exe, Module.WellDocServer) I have implemented:
procedure TServerContainer.XDataServerJWTGetSecretEx(Sender: TObject; const JWT: TJWT; Context: THttpServerContext;
var Secret: TBytes);
begin
// Do I need below commented code if I only intend to use RS256 (asymmetric keys)??
// if JWT.Header.Algorithm = 'HS256' then
// Secret := TEncoding.UTF8.GetBytes(SD_JWTSECRET)
// else
if JWT.Header.Algorithm = 'RS256' then
begin
if JWT.Header.KeyID = SD_KEYID then
Secret := TFile.ReadAllBytes(SD_PUBLIC_KEYFILE)
else
raise EJOSEException.CreateFmt('Unknown Key ID in JWT', [JWT.Header.KeyID]);
end
else
raise EJOSEException.CreateFmt('JWS algorithm [%s] is not supported', [JWT.Header.Algorithm]);
end;
Obviously I am going wrong somewhere or I am missing something. Unfortunately the documentation is rather sparse and various examples are using different approaches and thereby they become confusing (at least to me 
). For example, the boiler plate server example (see above) seems to use a completely different approach than the SimpleServer Sphinx demo.
So far I only got to use Aurelius (working fine). I never got to XData and Sphinx due to other projects. This is a pilot for my next project, but I am getting pretty frustrated and I'm about to give up. Therefore, I would appreciate it if you can have a look at the attached project and let me know what I need to do to get it working with an asymmetric key pair. Thanks in advance!
MyProject-SoFar.zip (1.4 MB)
*
PS: The current error message is "Missing parameter: AuthorizationEndpoint" and before that "Invalid JWS signature"...
Thanks for the project, but it has several projects that i'm not sure which one to run to reproduce the issue. Also it has 3rd-party components so I can't even compile it.
But at first sight from your code, I see that you have added JWT middleware to your Sphinx server. There is no need for that. In your Sphinx server you only use the GetSigningData event, as you are doing, providing the private key for the JWT to be signed.
Then, in your XData server, you should indeed add the JWT middleware as you are doing, and use GetSecretEx event to provide the public key for it to validate the JWT.
All is fine, just remove the JWT middleware from the Sphinx server, you don't need to protect it.
In any case, I provided you information about two existing projects that are compilable and are working fine. If you are having such difficulty, I suggest you follow those projects, or even copy and paste and continue from them. If you hit any problem, check what your project has different from the ones that are already working.
Apologies, I simply used my usual components and forgot they're 3rd party. I will update with standard components and add to this subject for future reference etc...
Removing the JWT from Sphinx seems to get me further but now I get the exception:
Project SD_Sphinx.exe raised exception class ESignException with message '[OpenSSL] Unable to load OpenSSL libraries'.
I placed libeay32.dll and ssleay32.dll in the Sphinx application folder, as well as the folder where the keys are stored (for the sake of it) but no luck. Both were downloaded today (GitHub) for another project I am working on. How to fix this issue?
For my understanding: if I have the following in OnGetSigningData
Args.Data.Algorithm := 'RS256';
Args.Data.KeyId := SD_KEYID;
Args.Data.Key := TFile.ReadAllBytes(SD_PRIVATE_KEYFILE);
Am I not just handing out the private key? Or should I implement another event handler in which I provide a 'secret' which is signed with this key (which makes more sense to me)?
Apologies for the lengthy request, but as you can see (hopefully 
) I keep finding these little "stumbling blocks" that are hard to figure out from the documentation/examples. So thanks again for your support - it's highly appreciated.
Yes, you should just put libeay32.dll and ssleay32.dll in the same folder as your executable. Make sure it has the same bitness (32-bit if you Sphinx is 32-bit).
The private key is already your secret that is used to sign the JWT.
Because I suspected that continuous experimenting could have resulted in erroneous settings here or there, I decided to restart from scratch. The sample project is attached with the keys and DLL's in the ..\Win32\Debug folder. Simply build and run XData Server, Sphinx Server and Client_VCL (in that order with the latter in DEBUG mode, but also note the last question under GENERAL below).
I have the apps using only standard Delphi components and it now works, but I have some questions remaining. Hopefully you can answer these so I finally get a good understanding.
(Note: you can find the questions also as comments in the source code; simply search for '// >>> #Q:').
Unit Module.XDataServer:
Line 65:
Registering BaseURL with TMSHttpConfig tool. Can this be just for Current User or should Network Service always be included (I can't seem to find anything in the docs)?
Line 68-70:
How do XDataServer.DefaultEntitySetPermissions and XDataServerJWT.ForbidAnonymousAccess work together? If I omit DefaultEntityPermissions then Anonymous- or Logged In access will still have no effect (i.e. not permitted)?
Note that the statement XDataServer.DefaultEntitySetPermissions := EntitySetPermissionsAll seems to have no effect. Only if I set them design-time, I can get entities from the server (otherwise 403). Any ideas?
Form.MainSphinx
Line 73:
What's the effect of RequirePKCE := true? Do we need to do anything with it?
Line 190:
Did not get to experiment with this, but I take it that here I must add claims to the JWT? In the TEntity implementations I then add EntityAuthorizeScopes attributes so I can control access in the XData Server using the claims I read from the JWT?
Line 206-208: See GENERAL below.
GENERAL
I keep struggling with the concept of JWT, secret and asymmetric signing. My understanding is (and probably still wrong ) that somewhere a JWT is issued (by Sphinx) that is signed (encrypted) with the PRIVATE key in Sphinx and decrypted with the PUBLIC key in the XData Server. However, nowhere I see something like JWT := TJWT.Create (as in the example TLoginService.Login on Authentication and Authorization | TMS XData documentation). Neither does the Sphinx server use the JWT middleware. Does it generate the JWT (implicitely) "under the hood" somewhere? Is there any requirement to set the XDataServerJWT.Secret property (Module.XDataServer)?
Typically, I run the XData Server, then the Sphinx Server and then Client_VCL (in debugging mode). This all works fine (assuming all is protected correctly) but if I run the VCL_Client before the Sphinx Server (now in debugging mode), I get an exception when I attempt to log in with the button on the Client_VCL app:
Project Sphinx_Server.exe raised exception class EJOSEException with message 'JWS signature is invalid: SzVt_-Uh66fo9P9QVU_dgdwJ9vYU6eTEaH0IB7fGaNc'.
What would/could cause this? Is this an exception I should trap to ensure Sphinx is running?
Thanks for you patience and support!
XData_Sphinx.zip (1.8 MB)
You need to register for the users that will run the server. For example, in your development machine, probably you only need to enable it for the current user, as you are going to run under your Windows user, while you are developing it.
If and when you deploy the application to another server, that app will also run under a user. If you make the app a Windows Service app, then it will be run as a service under a specific Windows user, it depends on how you configure the service to run. If it's under network service, or system user, etc,. you just reserve that URL for the Windows user running the app.
They are not related.
DefaultEntitySetPermissions and EntitySetPermissions are related to Aurelius CRUD endpoints. It's a high-level, automatic way to create CRUD endpoints in XData without having to explicitly code each of them. By just adding an Aurelius entity to your application, you can have CRUD endpoints being published for those entities.
By default, no CRUD endpoints are published, even when you add entities to the app. You control that by using EntitySetPermissions - for each entity (Customer, Country, Invoice, etc.) you can define which CRUD operations you want to be published. You might want to publish endpoints GET (select), POST (insert), PUT (update) for Customer, but no Delete. You might want to publish only GET for Invoice, etc.
The DefaultEntitySetPermissions is just a general property where you set such things for all entities at once.
Think of EntitySetPermissions as a "development phase", i.e., which endpoints you want to be made available by Aurelius.
Now, ForbidAnonymousAccess is about authentication and authorization, and it's at JWT Middleware level and it applies to the whole XData server - ALL endpoints, either Aurelius CRUD endpoints you published, or service operations.
The JWT middleware checks if the request has a JWT or not. If it does NOT has a JWT, then the request is anonymous (not authenticated). ForbidAnonymousAccess controls whether you want to allow non-authenticated requests to be passed forward or not.
If ForbidAnonymousAccess is true, then any request that doesn't have a JWT will be immediately rejected and returned to the client with a 401 error. If it's false, then the request will pass the JWT middleware and be moved forward in the pipeline. It doesn't mean it will be immediately accepted - it will then depend on other configurations and settings for the endpoint. It simply means the JWT middleware won't reject it.
It just increases security. This is from OAuth standard: https://blog.postman.com/what-is-pkce/
Yes, if you want to add custom claims. There are already some basic claims in the JWT, like the user id, expiration date, etc.
Correct. Almost correct, as "sign" and "encrypt" are different things. The JWT is signed with private key (not encrypted). Then the signature is verified using the public key.
Because Sphinx does it under the hood.
Because the JWT middleware is used to verify a JWT signature, to accept or reject a request based on the existing JWT it carries or not. It's used to protect a server from anonymous and/or unauthorized access. That's for your API (XDataServer), not for Sphinx. Sphinx is a public, anonymous server, you don't want to have a JWT middleware in it, it's the server that is going to give you a JWT, not to require you have a JWT to access it.
Yes.
I didn't understand this question.
I still can't compile your Client_VCL application, it still uses 3rd-party components.
After manually deleting and removing and any reference to them, if I run the Client_VCL without Sphinx server running and I client the "Log in" button I get this:
Which is expected, as the client couldn't find the Sphinx server, thus couldn't retrieve the authorization endpoint from the .well-known endpoint.
About the "XDataServerJWT.Secret property": I suppose this is only used for SharedKey signing, so not used in this scenario?
Regarding the Client_VCL: perhaps I accidently copied some unit names (DevExpress) from my previous example, as I'm sure I only used Delphi components. I'll fix the application and repost tomorrow.
The rest is clear. The "under the hood" thing in Sphinx is what kept confusing me.
Thanks again!
Please find attached the project with (hopefully now all) 3rd party components/units removed. If I still missed one and the component class or unit name starts with 'dx' or 'cx', simply remove it.
please compile all in Debug configuration; I have included ..\Win32\Debug with all necessary files (dll's etc.)
Everything is working but I run into two issues:
XDataServer.DefaultEntitySetPermissions := EntitySetPermissionsAll;
The strange thing here is that I can only retrieve entities if the DefaultEntitySetPermissions are set (to true) in the Object Inspector at design-time. If not, I get a 403 error. The above line is definitely executed (I set a breakpoint) but it appears to make no difference. What am I missing?
Project Sphinx_Server.exe raised exception class EJOSEException with message 'JWS signature is invalid: SzVt_-Uh66fo9P9QVU_dgdwJ9vYU6eTEaH0IB7fGaNc'.
Running this order with Sphinx also without debugging works! Any idea what may cause this because sometimes I need Sphinx in debug mode (so that it has to be last if the other two need to be running as well)?
Of course I am open to any comments on my code!
I also intend to write a 'XData/Sphinx for Dummies' that addresses everything I learnt here. Sort of a manual for people starting from scratch. Let me know if you're interest (maybe to publish for other users?) and I'll be happy to share in due time!
XData_Sphinx.zip (3.0 MB)
You have set SparkleHttpSysDispatcher1.Active property to True at design-time. When the application is launched, the data module is loaded, the dispatcher is set to active, the XData module is created, and everything is up. After that, the DataModuleCreate and ServerSetup is called. But then the ship has sailed.
Set Active property to False at design-time so you have time to set properties from code.
Because the browser is sending cookies from a previous session so the server can check if the user is already logged in, but the cookies are not valid. Those exceptions are raised because you are in debugging mode but they are ignored by Sphinx server.
Sure, anything that can be helpful for other users is welcome!