The attached test project demonstrates the subject issue. Running in Debug mode using FireFox Developer Edition with the developer tools showing it behaves as expected: When first run the IDB is confirmed empty. After clicking on the Refresh Table button it is filled with 15000 records and they are displayed in the TWebDBGrid. Repeated button clicks simply refill the IDB with 15000 records. If the project is restarted, the same 15000 records are confirmed present in the console and displayed in the grid.
If Chrome or Edge browsers are used, the console and the grid both show 15000 records after the button click. However, when the project is re-run or restarted the grid shows only a fraction of the 15000 records, and the console reports a different fraction. Repeated button clicks eventually cause the UI to freeze, but no errors are reported in the console.
IDB_Issue_Demo.zip (8.9 KB)
P.S.: I tried inserting a WebIndexedDBClientDataSet1.Refresh at the end of the button-click method (where the commented Close is) and that caused a brief delay, but did not correct the issue.
I ran this like 10x after each other in Chrome but saw 15000 records every time.
If you see something different in the browser console; I can only suspect this is a browser related storage limitation, not something related to our components.
To make sure that we're on the same page: I also see 15000 records in the grid and the console after each button click. It's only after restarting the page via ctrl-R or exiting and rerunning the browser where Chrome and Edge show <15000 retained records, while FF shows the full 15000.
This smells like a buffer overrun and, yes, it's probably a browser buffer. If you still see in Chrome the full 15000 records reported after a ctrl-R reset, how do I configure my Chrome like yours?
P.S.: Are you testing with the 9/26 beta RC or your internal update??
Here is a screen shot showing the result of an Edge run (in Release mode). It began showing that only some 2k records were retained from the previous run. Then the first button click refreshed the grid to 15000 records and the method concluded ("Finished LoadWIDBCDS" in the console). After the second button click the method failed to reach the final log message and the grid indicates that the EndUpdate stalled after record 9673. No error message!
As I mentioned, I cannot see this being related to our code but to how the browser behaves.
Bruno, I've continued to work on this issue and I've got a pretty good work around, although I don't understand the delay that occurs when re-enabling controls on the WebIndexedDbClientDataSet after it is emptied and refilled with controls disabled. I get the least delay by closing the WIDBCDS, after refilling it, with controls still disabled and then enabling controls before re-opening it. The delay (~20 sec.) then occurs during TAwait.ExecP<Boolean>(WIDBCDS.OpenAsync).
OTOH, if I enable controls before closing the data set there is a very long delay (more than a couple minutes) for await(WIDBCDS.EnableControls).
Initially opening the data set takes only ~3 sec.
Does this behavior make sense to you? Am I still overlooking a necessary step?
I have an update on this issue. I have been first emptying and then refilling, in a for loop with Append calls, the WIDBCDS records. I found that with most browsers this entails a substantial delay compared with simply opening the WIDBCDS at form creation.
I tried just now leaving out the emptying step and using a for loop of the following form that combines Append and Edit calls to create the new WIDBCDS. Finalizing the update still requires that the data set be closed and reopened after filling it, but there is essentially no difference in the reopening time compared with the initial opening time.
WIDBCDS.First;
for j := 1 to BufferGrid.RowCount - 1 do
begin
if WIDBCDS.Eof then WIDBCDS.Append else WIDBCDS.Edit;
{ define new fields }
WIDBCDS.Post;
end;
P.S.: Clearly, this code needs to be amended to handle the case of the new data set being smaller than the original one.
As far as I can interpret from your last message, the delay seems to come from IndexedDB operations. Have you tried to perform these operations when no DB-aware controls are connected at all, to confirm this assumption?
If a problem persists and there is a reason to suspect the DB binding, please isolate and send a sample source app with which this can be reproduced here.
By "no DB-aware controls" do you mean just none bound to the WIDBCDS? Or do I need to more directly access the IDB itself somehow?
nothing bound to this dataset
Attached here is a small project to demonstrate the effects that I see. There may be nothing more to this than an enormous difference in the efficiency with which Firefox handles the IDB compared with Chrome and Edge. On my system, Firefox zips through the two refresh methods on datasets of 20,000 or more records, while the other two take more than a minute on anything over about 8000. Firefox does the DB edit test about twice as fast as the delete & refill test, while the difference is over 3x in Edge with the intermediate DB close (i.e., DoClose defined) and about 5x without it. It seems suspiciously like some sort of coding error to me.
IDB_Issue_Demo.zip (11.8 KB)
I could see that Firefox indeed manages this significantly faster to insert large arrays of data.
I found this topic
but even when avoiding to use onsuccess, I could not see an improvement in Chrome. So, I'm not sure if we can do something about this.
Thanks, Bruno. I think my Edit/Append update method will be fast enough for my current purpose, but it would be even better if you can speed IDB writes up more on non-FF browsers.
