How to limit the fields in the select statement.

Hello,

I am new to TMS Aurelius.

I have modified the sample project MusicLibraryVCL.dpr and I have added some properties to the class of a entity :

  [Entity, Automapping]
  TArtist = class
  strict private
    FId: integer;
    FName: string;
    FField1: Nullable<Integer>;
    FField2: Nullable<Integer>;
    FField3: Nullable<Integer>;
    FField4: Nullable<Integer>;
    FField5: Nullable<Integer>;
    FField6: Nullable<Integer>;
    FField7: Nullable<Integer>;
    FField8: Nullable<Integer>;
    FField9: Nullable<Integer>;
  public
    constructor Create(const AName: string); overload;
    property Id: Integer read FId write FId;
    property Name: string read FName write FName;
    property Field1 : Nullable<Integer> read FField1 write FField1;
    property Field2 : Nullable<Integer> read FField2 write FField2;
    property Field3 : Nullable<Integer> read FField3 write FField3;
    property Field4 : Nullable<Integer> read FField4 write FField4;
    property Field5 : Nullable<Integer> read FField5 write FField5;
    property Field6 : Nullable<Integer> read FField6 write FField6;
    property Field7 : Nullable<Integer> read FField7 write FField7;
    property Field8 : Nullable<Integer> read FField8 write FField8;
    property Field9 : Nullable<Integer> read FField9 write FField9;
  end;

When I open the application, the select is :

SELECT A.ID AS A_ID, A.NAME AS A_NAME, A.ARTIST_ID AS A_ARTIST_ID, B.ID AS B_ID, B.NAME AS B_NAME, B.FIELD_1 AS B_FIELD_1, B.FIELD_2 AS B_FIELD_2, B.FIELD_3 AS B_FIELD_3, B.FIELD_4 AS B_FIELD_4, B.FIELD_5 AS B_FIELD_5, B.FIELD_6 AS B_FIELD_6, B.FIELD_7 AS B_FIELD_7, B.FIELD_8 AS B_FIELD_8, B.FIELD_9 AS B_FIELD_9
FROM ALBUM A
  LEFT JOIN ARTIST B ON (B.ID = A.ARTIST_ID)
ORDER BY  A.NAME Asc

What can I do to retrieve only the field NAME of the table ARTIST and ignore the fields field1 ... field9?.

For speed pourposes and because I am showing only the ARTIST.NAME field, I want this select :

SELECT A.ID AS A_ID, A.NAME AS A_NAME, A.ARTIST_ID AS A_ARTIST_ID, B.ID AS B_ID, B.NAME AS B_NAME
FROM ALBUM A
  LEFT JOIN ARTIST B ON (B.ID = A.ARTIST_ID)
ORDER BY  A.NAME Asc

Thank you.

Hello,

You could try to use projections: 
http://www.tmssoftware.biz/business/aurelius/doc/web/projections.html
http://www.tmssoftware.biz/business/aurelius/doc/web/property_value.html

Hello Wagner,

I thought that there would be an attribute for this purpose. Something like :

  [Entity, Automapping]
  TAlbum = class
  strict private
    FId: integer;
    FName: string;
    FArtist: TArtist;
    [ManyValuedAssociation([TAssociationProp.Lazy], CascadeTypeAllRemoveOrphan)]
    FTracks: Proxy<TList<TTrack>>;
    function GetTracks: TList<TTrack>;
  public
    constructor Create; overload;
    constructor Create(const AName: string; AArtist: TArtist); overload;
    destructor Destroy; override;
    property Id: integer read FId write FId;
    property Name: string read FName write FName;
    [SelectOnlyLookupFields('Name')]
    property Artist: TArtist read FArtist write FArtist;
    property Tracks: TList<TTrack> read GetTracks;
  end;

Meaning "SelectOnlyLookupFields" attribute that only the indicated fields should be selected from the Artists table ( in this case only the "Name" field ) when I make calls like :

FManager.Find<TAlbum>.OrderBy('Name');

Or that at run time would exists a select fields method to get only the fields of lookup tables needed but in a less complex way that your proposal. For example :

FManager.Find<TAlbum>.SelectOnlyLookupFields(TArtist,['Name']).OrderBy('Name');

Could it be improved in the future? Or does this proposal have design problems?.

Thank you.

It goes against the OOP and the entity state management performed by Aurelius. The concept is that you bring entities, and save them back, if needed. You do not think of SQL but objects.

If you only bring a few properties of the objects (artist for example), if you try to persist its state back to the database, all the properties that were not previously loaded will be erased in the database, since Aurelius doesn't know which was their previous values to compare.
It will lead to a full confusion.

Having said that, we're investigating ways to do what you mean (bring only a few properties of the entities) but that will require a more complex setup because of what was said above. Probably a stateless manager where you can't save data back to the database. 


Hello Wagner,

Perhaps becouse I am novice with your system I have not the knowledge to emit opinios about it, but I know that should to be a way to limit the fields of the lookup tables, because I have large secondary tables and depending of the options of the application that your are navigating to, I need initially some basic properties to be shown in the main grid and after selecting options of the application, I need more properties of the lookup tables, so for one property ( the name of the artist in this case ), I do not want to charge a lot of fields from the artist table in the initial select, it is as a mix between lazy load and eager. I only need the rest of the fields in other parts of the program, and I do not want to update them, thos properties are only to show them o to make or not to make somethings, so I do not know why the album table should be read-only. I want to make changes in the album table and be able to limit the fields of the artist table to be loaded in the select statement, but I do want to make changes to the album table. I say this by you comment about do not save them data back to the database.

Thank you.
Hello Wagner,

I have modified the code and instead of this :

Criteria := FManager.Find<TAlbum>.OrderBy('Name');

I have written this :

  Criteria := FManager.Find<TAlbum>
              .CreateAlias( 'Artist', 'b' )
              .Select( TProjections.ProjectionList.Add( Linq[ 'ID' ].As_( 'A_ID' ) ).Add( Linq[ 'NAME' ].As_( 'A_NAME' ) ).Add( Linq[ 'B.NAME' ].As_( 'B_NAME' ) ) );
  Criteria.OrderBy('NAME');

And the select now is :

SELECT A.ID As f0_, A.NAME As f1_, B.NAME As f2_
FROM ALBUM A
  LEFT JOIN ARTIST B ON (B.ID = A.ARTIST_ID)
ORDER BY  A.NAME Asc

What I am I doing bad?. If you see the fields of the select always are : as f0_, as f1_, and I am using As_('A_ID' ) ...  what can I do to get as : A.ID as A_ID, A.NAME as A_NAME ... etc.

Thank you.
Hello Wagner,

I have modified the code and instead of this :

Criteria := FManager.Find<TAlbum>.OrderBy('Name');

I have written this :

  Criteria := FManager.Find<TAlbum>
              .CreateAlias( 'Artist', 'b' )
              .Select( TProjections.ProjectionList.Add( Linq[ 'ID' ].As_( 'A_ID' ) ).Add( Linq[ 'NAME' ].As_( 'A_NAME' ) ).Add( Linq[ 'B.NAME' ].As_( 'B_NAME' ) ) );
  Criteria.OrderBy('NAME');

And the select now is :

SELECT A.ID As f0_, A.NAME As f1_, B.NAME As f2_
FROM ALBUM A
  LEFT JOIN ARTIST B ON (B.ID = A.ARTIST_ID)
ORDER BY  A.NAME Asc

What I am I doing bad?. If you see the fields of the select always are : as f0_, as f1_, and I am using As_('A_ID' ) ...  what can I do to get as : A.ID as A_ID, A.NAME as A_NAME ... etc.

Thank you.

Back to Top

That's what you should do. You should not worry about the internal SQL statement, it's performed by Aurelius. The As_('A_ID') is used in the TCriteriaResult, not in SQL: 

http://www.tmssoftware.biz/business/aurelius/doc/web/results_with_projections.html

See this piece of code:


 EstimateSum := Results[0].Values['EstimateSum'];





Thank you very much.

Now, I want to make a query over Track table. I have written this :

var
  LTrack : TList<TTrack>;
begin
    LTrack := FManager.Find<TTrack>.Where( Linq.Sql( '{TRACKS_ALBUM_ID} = ' +  ads.FieldByName( 'ID' ).AsString ) ).List;
    try
      adsTracks.Close;
      adsTracks.SetSourceList(LTrack);
      adsTracks.Open;
    finally
      LTrack.Free;
    end;
end;

But an exception is raised : EPropertyNotFound, "Property TRACKS_ALBUM_ID not found on class "TTrack".

How can I filter the records that has TRACKS_ALBUM_ID = xxxx?.

Thank you.

Hello Wagner,

Sorry by my question. I have seen the problem :

LTrack := FManager.Find<TTrack>.Where( Linq.Sql( 'TRACKS_ALBUM_ID = ' + adsAlbums.FieldByName( 'ID' ).AsString ) ).List;

Thank you.



Hello Wagner,

I
have been trying to follow your advice to use projections to limit
fields in the select statement but it break the benefits of using
Aurelius because for example I have not access to the TDataSetField
generated by the criteria and It force me to use AfterScroll in Album
dataset to show the track table. I have seen that the system is very
flexible, but when I go out of the design I loose capabilities.

Could
it be easy to create a new attribute called for example :
EagerModeFields that only would be applied to properties of type
entities and it would mean that we only want to select in eager mode the
indicated fields and that when the rest of the fields that have not
been indicated in the list are accessed for reading or writing, the
corresponding lazy load is made in the entity property?.

It would
not break the OOP and it would give speed to the querys executed by
Aurelius in many situations. You told me that could be a problem because
the entities would be partial filled with data. So, why not use another
structure to hold the eager data until the lazy load is done over the
entity?. It would not break the entities states.

Do not you think that the extra fields that the select execute can make the process slower in certain situations?.

IMHO,
the concept of ORM and eager/lazy loading of the associated entities is
very good, but for tables with a lot of fields in the associated
entities, it can make the select statement slow the system if you select
eager loading, and if you select lazy, the lookup can be slower than
eager with few fields. With my little knowns, I know that Aurelius and Sparkle is in the
base of XData. With Sparkle I do not see any problem ( using http.sys
is very quickly ), but I am not sure of the speed problems in the
Selects, thinking in scalability when the servers be serving a lot of
users.

The idea would be something like this :

  [Entity, Automapping]
  TAlbum = class
  strict private
    FId: integer;
    FName: string;
    EagerModeFields(['Name']);
    FArtist: TArtist;
    [ManyValuedAssociation([TAssociationProp.Lazy], CascadeTypeAllRemoveOrphan)]
    FTracks: Proxy<TList<TTrack>>;
    function GetTracks: TList<TTrack>;
  public
    constructor Create; overload;
    constructor Create(const AName: string; AArtist: TArtist); overload;
    destructor Destroy; override;
    property Id: integer read FId write FId;
    property Name: string read FName write FName;
    property Artist: TArtist read FArtist write FArtist;
    property Tracks: TList<TTrack> read GetTracks;
  end;

I think that the work done in Aurelius, Sparkle and XData is amazing.

Maybe I'm wrong on this subject. I would like to know your opinion about the topics that I have written here.

Thank you.

How would that work? It's not possible to change, at runtime, a property type that is defined at compile time. You can't have a field FArtist declared as TArtistEager and then later as TArtistLazy. It doesn't make sense, or maybe I didn't understand what you mean.

<strong>But</strong>, note that you can use that approach (some users do), I mean, you can simply declare a different class TAlbumClean that associated with a different class TArtistClean which declare only the properties you want to load, and simply use those classes instead of the "original" ones when it's convenient to you. Just declare them in separate models and tell Aurelius which model you want to use for that specific object manager.


An ORM by definition makes the process slower when compared to using raw SQL. You just weight the pros and cons of its use. Aren't you doing premature optimization here? 
Maybe you just can't have a cake and eat it too (best performance without losing Aurelius "benefits"), but still you have many options, as described: using Aurelius as it is (decrease performance); using projection (not able to use entity classes); using raw SQL (best performance, loses Aurelius benefits); using different classes that only declare the fields you want to load (a "middle" solution).

Sparkle is in the base of XData, Aurelius is not. Aurelius is just an option. You can write a full REST server with XData without using Aurelius, fine-tuned it for performance if you need to. 
And about performance and scalability, it's one of XData stronger points - it's very robust, stable and have good performance. But again, you have to know what you are using and doing. Maybe you are just doing premature optimization. Or maybe for your specific needs, you need to optimize, then you have all the options mentioned above.

[quote=Wagner R. Landgraf]


How would that work? It's not possible to change, at runtime, a property type that is defined at compile time. You can't have a field FArtist declared as TArtistEager and then later as TArtistLazy. It doesn't make sense, or maybe I didn't understand what you mean.

<strong>But</strong>, note that you can use that approach (some users do), I mean, you can simply declare a different class TAlbumClean that associated with a different class TArtistClean which declare only the properties you want to load, and simply use those classes instead of the "original" ones when it's convenient to you. Just declare them in separate models and tell Aurelius which model you want to use for that specific object manager.


An ORM by definition makes the process slower when compared to using raw SQL. You just weight the pros and cons of its use. Aren't you doing premature optimization here? 
Maybe you just can't have a cake and eat it too (best performance without losing Aurelius "benefits"), but still you have many options, as described: using Aurelius as it is (decrease performance); using projection (not able to use entity classes); using raw SQL (best performance, loses Aurelius benefits); using different classes that only declare the fields you want to load (a "middle" solution).

Sparkle is in the base of XData, Aurelius is not. Aurelius is just an option. You can write a full REST server with XData without using Aurelius, fine-tuned it for performance if you need to. 
And about performance and scalability, it's one of XData stronger points - it's very robust, stable and have good performance. But again, you have to know what you are using and doing. Maybe you are just doing premature optimization. Or maybe for your specific needs, you need to optimize, then you have all the options mentioned above.
[/QUOTE]

Hello Wagner,

You have resolved one of my main erroneous concepts about XData, your are right, I have the option to use or not to use Aurelius, so I am going to evaluate the options before using an alternative.I am optimizing things before value the benefits of the ORM approach. I'm very used to using the Delphi DataSet model and it's hard for me to get used to the ORM model, with persistent fields in the Delphi DataSet model I only see objects with properties and in the ORM model I see plain properties with attributes, more difficult to manage ( no fields editor for example and more things to write ). My main goal is XData, so maybe I am complaining for nothing. I am going to center in XData and see what are the options managing databases, seeing the pros and cons of Aurelius and classical components like Firedac with persistent fields for example.

Thank you very much for your patience and best regards.



Hello Wagner,

Can you give me a example about using TAlbumClean and TArtistClean classes applied to this issue?.

Thank you.
You just create those classes the same way you do with the previous classes.
The only difference is that you add a Model attribute on top of the class, e.g.:


[Entity, Automapping]
[Model('Simple')]
TAlbumClean = class


That code will add TAlbumClean to model "Simple". Then, when you create a TObjectManager instance to deal with entities, you create it using an overload of Create method where you indicate which model you want to use:


Manager := TObjectManager.Create(Connection, TMappingExplorer.Get('Simple'));
Album := Manager.Find<TAlbumClean>(1);




Hello Wagner,

It  works ok, but only a problem.

I have defined the following classes :

type
  [Entity, Automapping]
  [Model('Default')]
  [Model('Lookups')]
  TGenre = class
  strict private
    FId: integer;
    FName: string;
  public
    constructor Create(const AName: string); overload;
    property Id: Integer read FId write FId;
    property Name: string read FName write FName;
  end;

  [Entity, Automapping]
  [Model('Default')]
  [Model('Lookups')]
  TArtist = class
  strict private
    FId: integer;
    FName: string;
    //
    FField1: Nullable<Integer>;
    FField2: Nullable<Integer>;
    FField3: Nullable<Integer>;
    FField4: Nullable<Integer>;
    FField5: Nullable<Integer>;
    FField6: Nullable<Integer>;
    FField7: Nullable<Integer>;
    FField8: Nullable<Integer>;
    FField9: Nullable<Integer>;
  public
    constructor Create(const AName: string); overload;
    property Id: Integer read FId write FId;
    property Name: string read FName write FName;
    property Field1 : Nullable<Integer> read FField1 write FField1;
    property Field2 : Nullable<Integer> read FField2 write FField2;
    property Field3 : Nullable<Integer> read FField3 write FField3;
    property Field4 : Nullable<Integer> read FField4 write FField4;
    property Field5 : Nullable<Integer> read FField5 write FField5;
    property Field6 : Nullable<Integer> read FField6 write FField6;
    property Field7 : Nullable<Integer> read FField7 write FField7;
    property Field8 : Nullable<Integer> read FField8 write FField8;
    property Field9 : Nullable<Integer> read FField9 write FField9;
  end;

  [Entity, Automapping]
  [Model('Default')]
  [Model('Lookups')]
  TTrack = class
  strict private
    FId: integer;
    FName: string;
    FGenre: TGenre;
    FComposer: Nullable<string>;
    FMilliseconds: Nullable<integer>;
  private
    function GetDuration: string;
  public
    constructor Create(const AName: string); overload;
    constructor Create(const AName: string; AGenre: TGenre); overload;
    constructor Create(const AName: string; AGenre: TGenre; const AComposer: string); overload;
    constructor Create(const AName: string; AGenre: TGenre; const AComposer: string; AMilliSeconds: integer); overload;
    property Id: integer read FId write FId;
    property Name: string read FName write FName;
    property Genre: TGenre read FGenre write FGenre;
    property Composer: Nullable<string> read FComposer write FComposer;
    property Milliseconds: Nullable<integer> read FMilliseconds write FMilliseconds;
    property Duration: string read GetDuration;
  end;

  [Entity, Automapping]
  [Model('Default')]
  [Model('Lookups')]
  TAlbum = class
  strict private
    FId: integer;
    FName: string;
    //
    [Association([TAssociationProp.Lazy], CascadeTypeAllButRemove)]
    [JoinColumn('ARTIST_ID', [])]
    FArtist: Proxy<TArtist>;
    //
    [ManyValuedAssociation([TAssociationProp.Lazy], CascadeTypeAllRemoveOrphan)]
    FTracks: Proxy<TList<TTrack>>;
    //
    function GetArtist: TArtist;
    procedure SetArtist(const Value: TArtist);
    function GetTracks: TList<TTrack>;
  public
    constructor Create; overload;
    constructor Create(const AName: string; AArtist: TArtist); overload;
    destructor Destroy; override;
    property Id: integer read FId write FId;
    property Name: string read FName write FName;
    property Artist: TArtist read GetArtist write SetArtist;
    property Tracks: TList<TTrack> read GetTracks;
  end;

  [Entity, Automapping]
  [Model('Lookups')]
  [Table('Artist')]
  TArtist_L = class
  private
    FId: Integer;
    FName: string;
  public
    property Id: Integer read FId write FId;
    property Name: string read FName write FName;
  end;

  [Entity, Automapping]
  [Model('Lookups')]
  [Table('Album')]
  TAlbum_L = class( TAlbum )
  strict private
    [JoinColumn('ARTIST_ID', [])]
    FArtist_L : TArtist_L;
  public
    property Artist_L : TArtist_L read FArtist_L write FArtist_L;
  end;

And in View.Albuns.pas I have written :

    Criteria := FManagerL.Find<TAlbum_L>;
    if Term <> '' then
      Criteria.CreateAlias('Artist', 'a').Add( Linq['Name'].Upper.Contains(Term) or Linq['a.Name'].Upper.Contains(Term) );
    Criteria.OrderBy('NAME');

And now, I get the following sql :

SELECT A.ID AS A_ID, A.NAME AS A_NAME, A.ARTIST_ID AS A_ARTIST_ID, B.ID AS B_ID, B.NAME AS B_NAME, A.ARTIST_ID AS A_ARTIST_ID
FROM Album A
  LEFT JOIN Artist B ON (B.ID = A.ARTIST_ID)
ORDER BY  A.NAME Asc

As you can see, the ARTIST_ID is selected twice in the select statement and is occupying more network bandwith.

I think that should to be a more easy way to deal with it.

Thank you and best regards.

How would that work? It's not possible to change, at runtime, a property type that is defined at compile time. You can't have a field FArtist declared as TArtistEager and then later as TArtistLazy. It doesn't make sense, or maybe I didn't understand what you mean.

 [/QUOTE]

You are right, sorry by my mistake, I was thinking more in DataSet classes with persistent fields than in plain classes with properties and attributes.

I think that this could be possible :

  [Entity, Automapping]
  TAlbum = class
  strict private
    FId: integer;
    FName: string;
    //
    Lookup(TArtist, 'Name');
    FArtist_Name : string;
    //
    Lookup(TArtist, 'Country.Name');
    FArtist_Country_Name : string;
    //
    [Association([TAssociationProp.Lazy], CascadeTypeAllButRemove)]
    FArtist: Proxy<TArtist>;
    //
    [ManyValuedAssociation([TAssociationProp.Lazy], CascadeTypeAllRemoveOrphan)]
    FTracks: Proxy<TList<TTrack>>;
    //
    function GetArtist: TArtist;
    procedure SetArtist(const Value: TArtist);
    function GetTracks: TList<TTrack>;
  public
    constructor Create; overload;
    constructor Create(const AName: string; AArtist: TArtist); overload;
    destructor Destroy; override;
    property Id: integer read FId write FId;
    property Name: string read FName write FName;
    // This property is not created as a field in the database table when UpdateDatabase method is called because it is defined with the Lookup attribute.
    // The attribute tell that it must be included in the sql select statement that is generated by Aurelius.
    // This property is theorically read-only but it need to be writeable because the manager need to write a value to it when the initial select is done or the value of the ARTIST_ID change.
    // This property is excluded from the sql Update and/or Insert statements.
    property Artist_Name : string read FArtist_Name write FArtist_Name;
    property Artist_Country_Name : string read FArtist_Country_Name write FArtist_Country_Name;
    // The Artist data is loaded in lazy mode ( has too much fields and normally I only need them when I want to insert or update an album ).
    property Artist: TArtist read GetArtist write SetArtist;
    property Tracks: TList<TTrack> read GetTracks;
  end;

Also it should to exists a method to select at runtime what properties can act as lookup. I could to define several properties as lookup with the Lookup attribute and be able to select or not select them at runtime depending of the application option that be running the user. If I deselect at runtime the property to work with the Lookup attribute it will behave like a Transient ( it is never persisted ).

A real example about the usage of a Lookup attribute is the following :

  [Entity, Automapping]
  TInvoiceDetail = class
  strict private
    FId: integer;
    FQuantity: Currency;
    FPrice : Currency;
    FAmount : Currency;
    //
    Lookup(TProduct, 'Name');
    FProduct_Name : string;
    //
    [Association([TAssociationProp.Lazy], CascadeTypeAllButRemove)]
    FProduct: TProduct;
    //
  public
    property Id: integer read FId write FId;
    property Quantity: Currency read FQuantity write FQuantity;
    property Price : Currency read FPrice write FPrice;
    property Amount : Currency read FAmount write FAmount;
    //
    property Product_Name : string read FProduct_Name write FProduct_Name;
    //
    property Product: TProduct read FProduct write FProduct;
  end;

- Suppose that we have an option in our application with the title : Consult an invoice.
  When we select this option, the application get a TInvoiceDetail object list and show them in a grid or in a web table. The columns showed are :   quantity, price, amount and product name ( I want this property to be selected in eager mode ), but I do not need to show the remain properties of the product property ( they will be loaded is lazy mode when be accessed ).
- When I select a detail item of the invoice and I click on the delete button, I do not need the Product entity, I only question the user if he wants to delete the record.
- When I select a detail item of the invoide and I click on the modify button, perhaps I need to check if the price can be changed and I need to read the Product entity property to see if the allowchanges property of the product entity is true ( perhaps I need to read other properties of the product entity property ).
- When I insert a select item of the Product table then I need always the Product property.

Saying that, perhaps a user enter in the consult invoice option and he only see the details and do not modify nothing. Thinking in this case I was trying to limit the number of fields that the sql select of the entity is selecting automatically for me.

I know that I could change the design of the entity and include the product_name as a real field of the database table, but for legacy applications could be necessary to deal with this needment.

The same example could to be applied to order detail, budget detail ... etc.

What do you thing about it?. Am I in a big mistake of design?.

Thank you and best regards.

TAlbum_L should not inherit from TAlbum. The two classes should be completely separated and put in different Models. Either you should use one or another.


Regarding your suggestions, I think you are constantly coming back to SQL way of thinking. The strong point of ORM is working with objects. Adding all those lookups and scalar duplicated, repeated properties will make the model to become a mess, in my opinion.